The Hitchhiker’s Guide to Rebranding Machine Learning (And a Shoutout to Geoff Hinton!)

Thu, 12 Dec 2024 00:00:00 +0000

There’s a funny thing that happens when you spend long enough inside the vocabulary of machine learning. Words begin to feel like worn shoes. Loss, gradient, training, inference. After a while you stop hearing them. And then one day a physicist wanders into the room and starts describing the very same things you do for a living, only the words coming out of their mouth sound like they belong on a chalkboard in a faculty lounge somewhere in Göttingen, circa 1925.

It’s the same machinery, of course. Just rotated by ninety degrees, so the light hits it differently.

A small guide to rebranding machine learning

I’ve been keeping a little Rosetta stone of these translations for a while, partly as a joke and partly because I think they genuinely help. Every time you reach for the physics word instead of the engineering one, some quiet door opens in your head and a hallway of intuitions you didn’t know you had comes spilling out. So here it is, the same dictionary I keep in the back of my own head, set out plainly so you can take what you like from it.

What an engineer says	What a physicist would have said
Machine learning	Statistical mechanics
Loss function	Energy functional
Optimizing the model	Minimizing free energy
Trained model	Equilibrium distribution
KL divergence	Free energy difference
Gaussian noise	Thermal fluctuations
Random step	Brownian motion
SGD	Directed Brownian motion
GPU	Simulated particle accelerator
Diffusion models	Langevin dynamics
LLMs	High-order discrete Markov chains
NLP	String theory (the strings being, well, strings)
Reinforcement learning	Control theory
Robotics	Physical computation
Audio learning	1D signal processing
Image learning	2D signal processing
Video learning	3D signal processing
Multimodal models	n-dimensional signal processing

There’s a real point hiding inside the silliness here, which is that the borders between fields are mostly historical accidents. Two communities looked at the same elephant from opposite sides of the room and gave it two different names. Noticing that is, I think, one of the small pleasures of being alive at a moment when the disciplines are quietly merging back into each other.

And while we’re here, a word about Geoff Hinton

I can’t talk about any of this without pausing for a moment of genuine awe at Geoff Hinton. As of this year he’s the second person ever to hold both a Turing Award and a Nobel Prize. The first was Herbert A. Simon, who picked up his Nobel in Economics. Two people, in the entire history of these prizes, have crossed that particular bridge. Both of them were thinking about minds, and about what it means for a physical system to have one. I find that detail quietly beautiful.

So the next time someone at a dinner party asks what it is you actually do, you have my permission to look them dead in the eye and tell them, in your most serious voice, that you spend your days minimizing free energy in high-order discrete Markov chains using simulated particle accelerators. It happens to be true. It also happens to be the kind of sentence that makes the universe sound a little more like itself.

Science | Mahyar's world 🌏

The Hitchhiker’s Guide to Rebranding Machine Learning (And a Shoutout to Geoff Hinton!)

A small guide to rebranding machine learning

And while we’re here, a word about Geoff Hinton