2024-01-04
If you were a scale believer over the last few years, the progress we’ve been seeing would have just made more sense. There is a story you can tell about how GPT-4’s amazing performance can be explained by some idiom library or lookup table which will never generalize. But that’s a story that none of the skeptics pre-registered.
As for the believers, you have people like Ilya, Dario, Gwern, etc more or less spelling out the slow takeoff we’ve been seeing due to scaling as early as 12 years ago.
It seems pretty clear that some amount of scaling can get us to transformative AI - i.e. if you achieve the irreducible loss on these scaling curves, you’ve made an AI that’s smart enough to automate most cognitive labor (including the labor required to make smarter AIs).
But most things in life are harder than in theory, and many theoretically possible things have just been intractably difficult for some reason or another (fusion power, flying cars, nanotech, etc). If self-play/synthetic data doesn’t work, the models look fucked - you’re never gonna get anywhere near that platonic irreducible loss. Also, the theoretical reason to expect scaling to keep working are murky, and the benchmarks on which scaling seems to lead to better performance have debatable generality.
So my tentative probabilities are: 70%: scaling + algorithmic progress + hardware advances will get us to AGI by 2040. 30%: the skeptic is right - LLMs and anything even roughly in that vein is fucked.
From Dwarkesh Patel. This is the piece that I've been waiting for someone to right. It doesn't matter if he is right, just the thought exercise of thinking through where the bottlenecks might be is really useful.
2024-01-01
Mild spoilers ahead
A fictional account of a military history about a war between China and the US and allied forces over Taiwan set in 2028.
I generally enjoyed this book (the narrative was compelling) and found it to be an easy way to develop a feel for the general geography and challenges that a come with a Taiwan conflict.
The device for this book is really clever. The actual book is obviously about events that haven't yet happened and are in the future. But within the book, the author mentions the decision to write this as a narrative history a la Killer Angels as a way of bringing the "historical" people within it to life for a new generation. This confusing to recount but is quite clever within the book.
As of the end of 2023 (and I guess early 2024), the book feels remarkably current. There's lots in it about Ukraine and Russia that feels like it could've been written ~a week ago.
One of the most interesting aspects of the book to me was the degree to which fooling the other side's AI through feeding it false data is the key to victory. You can't beat the algorithm, but you can misdirect it.
Space Force features prominently. I wonder how true to life this part is.
For about the first ~1/3 of the book there's a lot of discussion about "bespoke algorithms", which made me chuckle. What does that even mean?
I wish I had as much faith in US industrial capacity as the author does!
The ending felt like the author decided he was done. It made me think that the most likely equilibrium for such a conflict was a stalemate where China can't quite be pushed off the island but also can't quite control it.
2023-12-29
Two things I took away from Brian Potter's recap of the Apollo Program:
- The mixture between "complicated" innovation and "brute force" innovation; to make the second stage rocket booster light enough to be effective took both totally new design concepts and simply shaving off weight wherever it could be taken off.
Not every effort at weight reduction was solved through clever (if complicated) ideas like cold-strengthened aluminum or the common bulkhead. Much of the effort was achieved by pure brute force: parts would be fabricated, tested until failure, and then redesigned to be slimmer until they broke at exactly the required load (scaled by an appropriate safety factor).
- The interaction between new designs, new materials, and new techniques. New designs almost always require a new material or a new technique to be used.
Worth reading!
2023-12-26
The furnace proved that coke made from the nearby Connellsville seam of bituminous coal was uniquely able to be used in the blast furnaces that transformed iron ore into pig iron. For decades, coke made with coal sourced elsewhere proved unusable — giving southwestern Pennsylvania an enviable competitive advantage as the second industrial revolution powered explosive demand for iron and then steel.
That technical innovation gave coal-rich Pittsburgh, which was already a successful region for energy-intensive industries like iron and glass, an overwhelming advantage in ore-based steel production.
via Chris Briem. Despite growing up in Pittsburgh, I found this explanation enormously helpful.