AI × how models work01
Image: Distill
Looking inside a network, neuron by neuron
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter / Distill
The opening paper of the circuits programme argues that neural networks can be studied like cells under a microscope. Features are directions in activation space, circuits are the weighted connections between them, and both can be read off a trained vision model. Curve detectors and high-low frequency detectors are traced in detail. It is the founding document of mechanistic interpretability, and it stays concrete throughout.
For your interest in how models work. — Serius
Zoom In: An Introduction to Circuits ↗
Original · 40 min read · summary reads in about a minute
Tech × software craft02
Image: Strange Loop
Simple is not the same as easy
Rich Hickey / Strange Loop
Hickey separates two words we use interchangeably. Simple is objective — one fold, one braid, nothing twisted together. Easy is relative to you: near at hand, familiar, already installed. Most of what we choose in software is chosen for easiness, and the complexity we pay for it is not visible until the system is large. He then goes construct by construct — state, objects, inheritance, ORMs, conditionals — naming what each one complects. The talk that put 'complect' into working vocabularies.
For your interest in software craft. — Serius
Simple Made Easy ↗
Original · 62 min watch · summary reads in about a minute
AI × how models work03
Image: Dwarkesh Podcast
Search, planning, and the limits of scaling
Dwarkesh Patel and Demis Hassabis / Dwarkesh Podcast
The head of Google DeepMind on why scaling is still, in his words, an art form rather than an engineering discipline, and on what happens when you put AlphaZero-style search and planning on top of a language model. Patel presses him on timelines, on whether an intelligence explosion is a thing you can steer, on keeping model weights out of the hands of states that want them, and on how Gemini was actually trained. A transcript is posted alongside if you would rather read it.
Demis Hassabis — Scaling, superhuman AIs, AlphaZero atop LLMs, AlphaFold ↗
Original · 62 min listen · summary reads in about a minute