In Vim, in Insert mode, if you type Ctrl-K followed by two
characters, you can insert a special Unicode character corresponding to
those two characters. The two-character combo is often easier to
remember than other codes. These are called digraphs.
For example, Ctrl-K12 will insert
½.
Everything below this is unnecessary detail.
What are all the digraphs?
I used to think the big table :help
digraphs-table at the bottom of Vim’s help page for digraphs
was exhaustive. As the page explains, the mnemonics are based on RFC1345, an
Internet standard that provides two-character mnemonics for various
characters.
I’m still doing interpretability at Anthropic. This year, among many
other research updates, we scaled
sparse autoencoders to Claude 3 Sonnet. There’s not a whole lot I
have to add. I like to think I improved at various clichéd abstract work
skills that I don’t have anything insightful to say about, the most
front-of-mind being communication and prioritization. It’s good to, uh,
communicate what everybody is prioritizing and communicate all the
information everybody needs to prioritize things. Thanks for coming to
my TED talk.
In my personal life, I went to more in-person social events than ever
before. I took a bunch of improv classes and might actually end up
performing soon™. I went to a Jacob Collier concert and a Bear Ghost
concert. I scored 141/150 plus two beers on the AMC 12. I attended three
separate furry conventions, one of which I believe I have to credit with
indirectly motivating me to hit the gym semiregularly for the first time
in my life. (Also I went to Tural for
summer vacation.) I’m pretty happy with all that, but it also doesn’t
really add up to exciting reflections.
You may also know about some of the considerations went into the
colormap’s design, chiefly perceptual uniformity: equally distant colors
on the colormap should “look equally different” to humans.
I won’t discuss color theory much here; the talk introducing viridis
(below) is quite good, and there are many other resources online, such
as Jamie Wong’s From
Hexcodes to Eyeballs or Bartosz Ciechanowski’s Color Spaces.
Instead, I nerd-sniped myself with a different question: How is
viridis, specifically, defined?
This seems like it should be easy to answer. Just go into matplotlib
and find the source
code for viridis, right? Unfortunately, that source code is just a
list of 256 RGB triples along which colors are interpolated. This makes
sense for efficiency and even portability because (as we’ll see) the
formula for producing viridis is incredibly complicated, but it isn’t a
very satisfying answer. How did those triples originally come to be?
In theory, the idea here is similar to when I was learning React/Redux
and diving into SQL
selects. In practice, I think most of D3’s complexity isn’t exactly
in a direction that is elucidated by writing down types for everything,
so the title is a mere personal snowclone. I’m just writing things out
to an arbitrary amount of detail until I understand them and can refer
to what I wrote here later.
Background
D3 is “a JavaScript library for
visualizing data”. It has a lot of sublibraries that interoperate
well but could be used separately — for example, it has utilities for
manipulating colors, time, and SVG paths. Of the various concepts,
though, I think D3 selections are the most distinctive and
fundamental, so they are the focus of this post.
At a high level, D3 selections feel like jQuery. You run some code
and it goes into the DOM and adds, deletes, and mutates a bunch of
elements. The docs even endorse monkeypatching
d3.selection to add custom helpers. However, D3 has data binding and
batch operations that make it easy to change the DOM in a way that
resembles reconciliation in a framework like React.
A D3 selection holds an array of arrays of
nullable1 DOM elements. The intermediate
arrays are called groups. Additionally, each group in a
selection is associated with a parent node. During
basic D3 usage, you might only ever work with selections with a single
group and ignore parent nodes.
When relevant, I will call the index of an element inside its group
the “within-group index” and the index of a group among all groups in a
selection the “across-group index”.
…it’s still you. Looking at yourself in the window. The second
afternoon after you finally get COVID for the first time.
As previously reported, I left Zoom late last year and spent a bit of
time unemployed, traveling for some of it but mostly staying home. In
the process, I got COVID, though not with a particularly interesting
story. Would not recommend.
Then I started work at Anthropic doing interpretability research —
moving way back into my comfort zone in a way by returning to my web dev
roots to create many of the visualizations we cared about, and way out
of it in another by jumping into the deeply theoretical end of research,
in a field where my total experience is one college course and one
casual reading group. Still, I figured some things out and we published
Towards
Monosemanticity in early October.
I don’t have much to add to the research results in the paper, though
I can share some trivial, mildly entertaining anecdotes about the
process:
I want to add a second word in the title, something like “Koans” or
“Vignettes”, but I don’t know a word with the right connotations.
I realized recently that I have been walking around for a long time
with some confusion and unknown unknowns about how concurrency works in
various settings, and decided to write about it until I stopped being
confused. This post doesn’t therefore have much of a “point”.
Concurrency and Parallelism
Wikipedia, as of time of writing:
Concurrency is the ability of different parts or
units of a program, algorithm, or problem to be executed out-of-order or
in partial order, without affecting the outcome.
There are two broad reasons concurrency is useful. One is for
performance: if you want your computer to perform as many floating point
operations as possible by lunchtime, you want all CPUs/GPUs/etc. to be
performing operations simultaneously. Another is that you’re in a
problem domain where you simply can’t predict the order of events:
you’re writing a user interface, and the user can click on any of
multiple buttons in any order; or you’re writing a web server, and any
number of clients can request any pages in any order. These reasons are
not mutually exclusive.
Passwords. It’s 2023 and we still have to deal with them.
Many people know that, per the canonical xkcd, sequences of randomly
chosen words such as
soak-science-wander-pew-goldfish-xray-speed-consult
or get the list as .txt or a
standalone generator (if my JavaScript were working the above would be a
random password and you wouldn’t be seeing this message)
make relatively memorable but hard-to-crack passwords. One popular
strategy for randomly choosing words is Arnold Reinhold’s Diceware™, a
list of 65 = 7776 “words” that you can randomly sample from
by rolling five dice (analog or digital). (I won’t go into topics like
how to calculate the entropy of passwords and how long a password you
should try to have here, since most Diceware overviews already discuss
them at length.)
Somehow this image was already on my blog without being used in any
post, but if there’s a post it belongs in, it’s surely this one
A few people have iterated on the concept since then: probably most
notably, the Electronic Frontier Foundation published their own word
list in 2016, with words chosen to be more well-known and memorable,
at the cost of taking longer to type. I’m a fast typer and prefer the
EFF’s wordlist over the original, and am very grateful to them for
creating it, but after generating quite a few passwords with it over the
last few years, I began to feel that it still had a lot of room for
improvement.
“shouldn’t this have been published a few months ago?” yeah,
probably. I even considered submitting it to the AoC
contest. time is a real beast.
The title is clickbait. I did not design and implement a programming
language for the sole or even primary purpose of leaderboarding on
Advent of Code. It just turned out that the programming language I was
working on fit the task remarkably well.
I can’t name just a single reason I started work on my language, Noulith, back in July
2022, but I think the biggest one was even more absurdly niche: I solve
and write a lot of puzzlehunts, and I
wanted a better programming language to use to search word lists for
words satisfying unusual constraints, such as, “Find all ten-letter
words that contain each of the letters A, B, and C exactly once and that
have the ninth letter K.”1 I have a folder of ten-line scripts
of this kind, mostly Python, and I thought there was surely a better way
to do this. Not necessarily faster — there is obviously no way I could
save time on net by optimizing this
process. But, for example, I wanted to be able to easily share these
programs such that others could run them. I had a positive experience in
this with my slightly older golflang Paradoc, which I had
compiled into a WASM blob and put online and,
just once, experienced the convenience of sharing a short
text processing program through a link. (Puzzle: what does this
program do?) I also wanted to write and run these programs while booted
into a different operating system, using a different computer, or just
on my phone.
As I worked on it, I kept accumulating reasons to keep going. There
were other contexts where I wanted to quickly code a combinatorial brute
force that was annoying to write in other languages; a glib phrasing is
that I wanted access to Haskell’s list monad in a sloppier language. I
also wanted an excuse to read Crafting
Interpreters more thoroughly. But sometimes I think the best
characterization for what developing the language “felt like” was that I
had been possessed by a supernatural creature — say, the dragon from the
Dragon
Book. I spent every spare minute thinking about language features
and next implementation steps, because I had to.
The first “real program” I wrote in Noulith was to brute force
constructions for The Cube,
for last year’s Galactic Puzzle Hunt in early August, and it worked
unexpectedly well. I wrote a for loop with a 53-clause
iteratee and the interpreter executed it smoothly. Eventually I realized
that the language could expand into other niches in my life where I
wanted a scripting language. For example, I did a few Cryptopals challenges in them. It
would take a month or two before it dawned on me that the same
compulsion that drove me to create this language would drive me to do
Advent of Code in it. That’s just how it has to be.
This post details my thought process behind the design of this
language. Some preliminary notes:
My seventh year doing Mystery Hunt with ✈✈✈ Galactic Trendsetters
✈✈✈, and after a hiatus it was in person again! This also makes it my
first in-person Mystery Hunt as an alumnus, where I flew in and stayed
at a hotel. How time marches on… I appreciated getting to see everybody
on Galactic, as well as quite a few internet puzzlers at the location
where all the cool people always go, Flour Bakery and Cafe.
Campus hadn’t changed too much. There were more card readers, but
also fancy kiosks where ID cards could be printed on demand (via the
official 1.2/5★-rated app). I set aside a little time before kickoff to
try to locate a working kiosk to print my ID, but the two kiosks I found
west of Mass Ave, in W20 and W35, were both out of order; only much
later did I print a card in 16. But I am a card-carrying alumnus now.
Galactic had two classrooms in 4-2 and lots of masks and tests. One of
my teammates brought their dog. It was a fun time.
As typical nowadays, the hunt announcement and kickoff began with a
facade theme of a museum. However, the twist was handled a bit
differently — kickoff had an additional diegetic level: normally the
story is followed by an out-of-character talk about health/safety and
policies, but this year that talk, while still in a different universe
from the museum, was intertwined with an introduction to MATE, the AI
who had ostensibly been writing all the puzzles. Over the course of the
hunt, instead of discovering a possibly predictable secret plan or
betrayal by MATE, we instead found ourselves on its side because (in the
outer fictional diegetic level) teammate had shut off some other “overly
creative” AIs and overworked MATE.
One day I’m going to run out of the energy to find barely adequate
allusions for the titles and thematic music videos for the openings of
these end-of-year posts, and they’ll just be called “2095 in Review” or
whatever. Or maybe I’ll just stop making them. But not today.
Good song. Good animation. Incredibly out of place on its YouTube
channel, in the most inspiring chaotic good way.
I closed out last year by saying that I wanted to accomplish a “big
milestone” this year. I actually had a specific milestone in mind that I
did not actually achieve and will not reveal, but I made good progress
towards it, and a lot of other things happened, enough that I think I’ll
count that as achieved.
The big thing is that I left my job at Zoom to have some time for
myself and family… though not before helping to give feedback on a draft
internet standard, publish a cryptography research paper
(on which I’m the “first author”, strictly due to the vagaries of the
English alphabet), and launch end-to-end
encrypted email. It was a productive year! I feel like I should have
more to say about all this, but it’s hard to think of anything that I
didn’t already write about last
year and also doesn’t require a blockbuster-length list of
prerequisites. However, if you ever want to hear about the difficulties
of actually getting end-to-end encryption into production in
excruciating detail, invite me to a cocktail party with a lot of
whiteboards.