One year anniversary, giveaway, issue 52 follow-up, working on a better method to create penalty tables for layout optimization models.
1 year anniversary
It's crazy but I've been doing this project exactly for a year. How did this all start?
Coming back to r/mk after a long hiatus, I felt I missed something. And there was nowhere I could look for a sumup/recap of innovative ideas, cools projects, etc. published in the previous weeks or months.
In 2018, r/mk made it possible to build my first custom keyboard from scratch, without any background in electronics or firmware programming. I found pretty much all the info necessary on r/mk or in blog posts or other resources shared on r/mk.
Then I was done with my build and I stopped visiting r/mk for months.
Checking back after this hiatus I had to realize that the sub turned into something totally different. As it went from 200k members to 800k in only two years, it became a hard-to-follow platform flooded by spam.
Nevertheless, I kept bookmarking valuable content occasionally and thought it were useful to post them as a snapshot of the best of r/mk - without the annoying and distracting stuff.
So I posted my first "quick links" in a test sub I kept for historical reasons.
(I'm still waiting for my MBK Legends featured in that post... :S)
The rest is history. :D You may check the timeline part on my about page for some milestones:
https://kbd.news/about#milestones
Giveaway
Ugh. Managing this giveaway turned out to be much harder than expected. Now I understand GB runners much better. This is my elaborated CRM system to keep track of everything prize-related: :D
At the end I will have sent about 100-200 emails I guess. (To contact the winners and sponsors, to clear things up and to take care of everything.)
Btw, I did the raffle on Saturday and I'm quite glad the 27 prizes go to 13 countries. (I think I posted 14 countries on Twitter but I clearly cannot count.)
To minimize shipping costs, taxes and custom fees, I approached the sponsors with the idea to offer prizes for they region. (I had some bad experiences with stuff coming to the EU from the US lately.)
So most of the prizes were offered for US based participants. But looking at the list of winners, I'm happy to see they come from 13 different countries.
I simply exported the google form data in CSV, opened it in Excel and included a new column with RAND() in each cell.
Then I ordered all the entries by this random value after 5 recalcs. Volilá! :)
I started contacting winners on Saturday, continued yesterday, but there are still 5 winners not announced yet. I have their emails but I'm waiting for clarification from two sponsors yet.
So feel free to check your inbox and spam folder, but don't be surprised if you get an email from me only in the next few days.
Issue #52 follow-up
Newsletter opening rates are back over 50%, and clickthrough rates over 25%. (That's still pretty good: according to mailchimp, my "peers performance is 37% and 6.2%, respectively.)
A new member! After contacting Aleks to ask for additional material about his cyberpunk OLED animation, he signed up as a member. Welcome and thanks!
And of course thanks to all of you who support this project or supported it occasionally during this past first year.
Logical layout design
I try to get back to my logical layout design stuff, and with a quite ambitious plan.
My main goal is to be publish some tools with you can optimize/generate your layout without any knowledge in genetic programming or evolutionary models. A few clicks and the layout optimized for your typing habits is ready and served.
(And I have an even more ambitious plan, but more on this later.)
Having a few blank spots, missing puzzle pieces, and having identified some unreasonably subjective steps and solutions in the grand scheme of this addicting game called genetic optimization and logical layout design, I'm working on a better method for creating penalty tables and especially to fill them with proper data.
TLDR; You don't need the penalty table at all when doing proper modeling, i.e. using bigrams and trigrams. And if you insist on one, you should approach the task way more objectively and scientifically.
So a penalty table is a simple matrix of penalty points you use when calculating your overall score doing the evaluation part of your optimization model. In simple models this table takes care of putting the most frequent characters on the home row or easy to reach positions.
This is what I'm talking about. Or this.
Colemak Mod D-H penalty table / effort grid (source).
Do you press a key on the home row under your index finger? No or minimal penalty. (Why is there any penalty?!) Do you move to the upper row? Bang! A few points of penalty. Pinky involved? Much more points added to the score in the case above. Inner or outer column? Even more penalty. (So eventually, "bad" layouts collect more penalty points and the layout with the least score will be the best one.)
Alright, alright, this is usually done slightly more sophisticated. (Hopefully.) Assigning an integer value to each key position is only the first step, but the thing is: those values aren't really objective. Just look at those numbers. They are too round numbers, in many penalty tables they are integers -- what's quite suspicious.
It seems, when optimizing their layouts, people often tend to choose numbers intuitively.
And that approach introduces a basic flaw, a subjective set of values in an otherwise objective model, right at the beginning.
Well, my next goal is to create such a penalty table more scientifically.
Even better, to offer a tool to create such penalty tables. (However, as you will see, this is harder for people typing in English and this is the only or one of the rare cases when people cursed by national and accented characters invading the keyboard have an advantage.)
As a first step, I have to finish 3 small sub-projects:
A basic firmware for test purposes - without any modifiers and double function keys, yet usable for typing tests (Bye-bye home row mods and MO() in general.) On-hold functions would cause a bias in the measurement part because QMK waits for key release. (I was surprised to see Space is one of my slowest keys/positions until I found out it's a layer switch and QMK sends the keycode to Windows only when I release it.)
A dedicated corpus generator - A script generating texts with desired letter frequencies (instead of natural frequencies). This should make test typing sessions shorter.
A tester - a script and interface to measure keypress times with some tweaks, like mirroring and aggregating data. My theory is that using symmetric keyboards (most ergo splits) makes it possible to collect more data on infrequently used keys by mirroring the same positions on the two halves.
Well, that's what I want to accomplish during the next few weeks.
Thanks for reading, and thanks for your support!
Cheers,
Tamás