Bounded-Error Log

theoretical and applied randomness by betaveros

Rankk Solving Statistics

2013-07-18 (1139 words) filed under CS

Funny, I go on a trip to Penghu followed by a four-day science camp and also get dragged into drawing classes and some sort of movie advising joint, and this is what I decide to blog about.

Since it’s summer, I went back to Rankk and solved stuff. This is lots of fun if you’re good with computers, plus a little math, cryptography, and general puzzling. I’m still stuck on level 8… oh well. Since the levels didn’t seem very indicative of difficulty to me, I decided to do some analysis.

New challenges have been added to Rankk over time, so my metric of difficulty is the number of solvers divided by the time from release to now. Of course this is far from perfect; for example, a challenge’s author doesn’t always seem consistently counted as a solver, problems with lower numbers and problems that will help level up are more likely to get checked out by new rankkers, and so on. But this is just for fun.

Disclaimer: I don’t think this goes against the spirit of Rankk since I say absolutely nothing about what the actual contents of challenges are and all these statistics are publicly available (in one place under “stats > By challenges”). If you are somebody important and disagree then please tell me; I’m sure you can find my contact information :P . I can find virtually no precedent for posting about this stuff, but the FAQ encourages people to “Blog about Rankk or your Rankk journey”, so here it is.

The sorted data is here. It uses the =NOW() function because I’m lazy, so the metric might be affected even if nobody solves any more challenges. Plus, given so few solvers for the top challenges, this metric is extremely volatile and everything below might be invalidated in a few days or sooner (I wouldn’t be surprised if it might even cause its own invalidation). But again this is just for fun. What does it say about difficulty?

edit: Sphinx linked to this! Yeah so it’s probably ok. I went back and, I hope, future-proofed a copy of the sheet as of time of original post; apparently not as much has changed as I feared.

edit: …and I just realized I subtracted by the wrong column facepalm Still luckily, not much has changed. I’ve only had to swap a few challenges to retroactively correct this.

ETA 2015-02-18: Since I wrote this post, I’ve future-proofed it and copied the statistics several different times, and I’ll probably keep doing this whenever I remember because it takes about four minutes to do. It’s interesting to see the change. The stats are in the same place linked above.

8/2 ToeJam (7 solved), which people like to refer to as the prototypical “hard challenge”, is only 11th hardest.

The top 10 hardest (parenthetical numbers are solver counts) are: 4/30 Dice rolls (2), 4/31 Byte swapping (3), 4/47 Disguise (1), 4/26 Incarnation (4), 5/25 Dice rolls II (2), 6/13 Time for church (2), 4/45 Meteoroid (2), 4/37 Letterboxes (5), 5/15 Stronger algorithm (6), 4/40 Signal to noise (4). This might suggest that the metric overestimates the difficulty of more recent challenges, since fewer solvers will see them or will put extra effort into them in trying to advance to the next level. Alternatively, maybe it just says that new challenges are usually added to middle levels to preserve the higher levels’ selectiveness, even if they’re really hard. But at least 5/15 Stronger algorithm seems definitely hard, since it was only released four months after ToeJam and still has fewer solvers.

Huh, now I’m even more curious if there really is a lot more to 4/30 than I’m seeing.

The hardest challenge from each level, sorted from hardest to easiest: 4/30 Dice rolls (2), 5/25 Dice rolls II (2), 6/13 Time for church (2), 8/2 Meet ToeJam (7), 3/67 Holy Rankk Transform (4), 2/49 IRC II (13), 9/1 The showdown (26), 7/2 Mummified (27), 1/43 Nothing like home (53).

Yes, every level except for (edit:) 1 and 7 has a harder challenge than the final Level 9 challenge. For more comparison: 1/43 is right above 6/2 SQL injection (71) and slightly above 7/3 Eliza speak (71). No wonder I can’t solve it.

The path of least resistance an average rankker would take to become Geb (i.e. the 10 - n easiest challenges for each level n) would be:
  1. 1/1 Really elementary, 1/2 Easy Javascript, 1/4 Python 101, 1/13 True or false, 1/6 The nth term, 1/3 Hieroglyphics math, 1/15 Basecamp, 1/22 Easy stegano, 1/25 Weak password
  2. 2/3 View me, 2/15 Sheep and goats, 2/1 Shhh, 2/10 HTML encode, 2/4 Hit the dots!, 2/19 Nonograms I, 2/67 Let’s Program 1A, 2/24 Python 102
  3. 3/10 Sudoku I, 3/11 Guess the number, 3/19 Nonograms II, 3/12 Eight commands, 3/36 Regex revisit, 3/28 Squeeze me not, 3/1 Camel programming
  4. 4/1 A special browser, 4/18 Regex final, 4/4 Move it, 4/5 Crypto entrance test, 4/19 Weak algorithm II, 4/6 Sudoku II
  5. 5/2 Too slow!, 5/8 Averages, 5/12 Rotation, 5/10 Convert and count, 5/11 Best product
  6. 6/3 Linked list, 6/5 Ascii art, 6/4 All but one, 6/6 Hangman
  7. 7/5 Invisible captcha, 7/1 Proxy hunting, 7/6 The returning knight
  8. 8/3 The odd one, 8/1 Formless
  9. 9/1 The showdown

There’s a cute mix of challenges for levels 1 to 4; after that, programming becomes very clearly the best means of proceeding.

Stuff about myself:

The hardest challenge I have solved is 4/37 Letterboxes, followed by 4/33 Primeval painting and then 5/28 Cyclic Redundancy Chase. I find this… pretty funny. 4/37 is harder than ToeJam. What.

The easiest challenge I have not solved seems to be 3/28 Squeeze me not, followed by 1/57 Challenge tube. I really can’t. Don’t tell me.

For some comparison, my level-advancement challenges were:
  1. 1/1 Really elementary, 1/2 Easy Javascript, 1/6 The nth term, 1/13 True or false, 1/34 Lisp I, 1/18 Number of ways, 1/21 Frame of mind, 1/15 Basecamp, 1/41 Bye intruder
  2. 2/1 Shhh, 2/3 View me, 2/6 Don’t blink, 2/7 Be alert, 2/12 Hackers’ Camp 1A, 2/13 Intersection, 2/9 Into the future, 2/24 Python 102
  3. 3/2 Brackets rulez!, 3/11 Guess the number, 3/1 Camel programming, 3/42 Weak algorithm, 3/19 Nonograms II, 3/10 Sudoku I, 3/18 Labourer tag
  4. 4/1 A special browser, 4/18 Regex final, 4/13 Gray matter, 4/33 Primeval painting, 4/25 Ancient math, 4/19 Weak algorithm II
  5. 5/6 Chamber door, 5/9 Pinch mark, 5/2 Too slow!, 5/8 Averages, 5/16 Baa baa blacksheep
  6. 6/3 Linked list, 6/7 Traversal, 6/9 Amazing, 6/4 All but one
  7. 7/8 Gen8:17, 7/7 Jackpot, 7/9 26!
  8. 8/3 The odd one, (???)
  9. (???)

The biggest abnormality is in level 7 — the three of my chosen challenges were the second- to fourth-hardest level 7 challenges, and are all harder than 8/1 Formless. I’m weird.

…I need to start my summer homework…

(note: the commenting setup here is experimental and I may not check my comments often; if you want to tell me something instead of the world, email me!)