The Atlas

1943 Shores of McCulloch & Pitts

Warren McCulloch and Walter Pitts published a mathematical model of the neuron in 1943. The map starts here.

Their paper gave an early mathematical account of how networks of simple artificial neurons could compute.

Their 1943 paper, A Logical Calculus of the Ideas Immanent in Nervous Activity, modelled neurons as simple on-off switches and showed that networks of them could carry out logic.

The model greatly simplified real neurons. In 1949 Donald Hebb proposed that repeated activity could strengthen connections between neurons, an influential idea about how learning might work.

A unit fires at the next time step if enough excitatory inputs are active and no inhibitory input is active: y = 1 if Σxᵢ ≥ θ and all inhibitory inputs are 0, else 0. Networks of these units can build AND, OR and NOT and implement any Boolean function. There's no learning here: connections and thresholds are fixed by hand.

The 1943 paper · link.springer.com

1950 Turing's Lighthouse

In 1950 Alan Turing asked whether machines can think, and proposed the imitation game as a way to discuss it. Its beam still reaches across the whole map.

Turing moved the question from philosophy to something you could test, and his paper already anticipated many objections people still raise.

His 1950 paper Computing Machinery and Intelligence starts by asking whether machines can think. He then replaces that question with the imitation game.

He predicted that by about the year 2000, a machine could fool an average interrogator 30% of the time after five minutes of questions. He also suggested that instead of programming an adult mind, we might build a child machine and teach it.

Computing Machinery and Intelligence · abelard.org

1956 The Dartmouth Isles

A 1956 summer workshop at Dartmouth, and the 1955 proposal for it, gave the field its name. Its proposal hoped a small group could make significant progress in one summer. The islands are small. The plans were not.

The 1955 proposal gave this growing research community a lasting name. Its hopes for one summer also make a fine opening chapter in AI optimism.

John McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon proposed a summer study for 1956, believing a 'significant advance' was possible with a carefully selected group in one summer.

Attendance came and went, and it wasn't the breakthrough they hoped for. But the name stuck, and its attendees would lead the field for decades. Allen Newell and Herbert Simon showed up with the Logic Theorist, a program that proved mathematical theorems.

The Dartmouth proposal · www-formal.stanford.edu · spectrum.ieee.org · en.wikipedia.org

1958 Perceptron Point

Frank Rosenblatt's 1958 perceptron learned from examples. The press reported big promises. Minsky and Papert's 1969 book then proved hard limits on what such machines could compute.

The perceptron is the direct ancestor of today's neural networks, and its story set the pattern for AI hype and backlash.

Rosenblatt's Mark I Perceptron was real hardware, using 400 photocells and motor-driven weights to learn to recognize simple patterns.

In July 1958, after a demonstration on an IBM 704, the New York Times reported that the Navy expected the machine to eventually walk, talk, see, write, reproduce itself and be conscious of its existence. It could not.

Minsky and Papert's 1969 book analysed perceptrons with one layer of adjustable weights on top of fixed feature detectors. It proved that tasks like parity and connectedness need detectors that see an impractically large part of the input. The simplest case is easy to show: a single linear threshold unit cannot compute XOR. Multilayer networks can, but nobody yet had a good way to train them. Rosenblatt died in a boating accident in 1971.

Update rule: w ← w + η(y − ŷ)x. The perceptron convergence theorem guarantees this finds a separating line in finite steps if one exists. For XOR, none exists, so it can never finish. Try it below.

The Perceptron (1958) · news.cornell.edu · mitpress.mit.edu · web.stanford.edu · en.wikipedia.org · nytimes.com

1966 ELIZA's Hut

Joseph Weizenbaum's 1966 chatbot mirrored people's words back at them, and some users confided in it anyway. How does that make you feel?

ELIZA showed how readily people treat software as a mind, an effect that now bears its name.

ELIZA matched keywords and reflected sentences back as questions, like a parody of a therapist. It understood nothing.

Joseph Weizenbaum was disturbed by how seriously people took it, and later wrote the critical book Computer Power and Human Reason (1976). Decades later, people are still debating the same effect, with far better chatbots.

ELIZA (1966) · courses.cs.umbc.edu · weizenbaum-institut.de

Old Kingdom of Symbols

For decades much of AI meant logic and hand-written rules. Its expert systems were a real business in the 1980s. Every field is square, and every road has a rule.

Symbolic approaches dominated much of AI research for decades. Rule-based systems remain useful, and debates about how to combine them with neural networks continue.

The bet was that intelligence is manipulating symbols according to rules. Newell, Shaw and Simon's General Problem Solver tried to capture reasoning itself, and Terry Winograd's SHRDLU (1970) could discuss and move blocks in a tiny simulated world.

DENDRAL and MYCIN pioneered encoding experts' knowledge as rules, in chemistry and medicine. The approach dazzled in small worlds, but the real world turned out to have too many rules to write down.

hci.stanford.edu · profiles.nlm.nih.gov

1974-1980 The First Winter

Through the 1960s, AI's leaders promised human-level machines within a generation. When those promises failed, governments in the US and UK cut funding, and much of the field went cold.

This is where the field learned that a good demo isn't the same as a solved problem. It keeps relearning that.

The trouble started before the cuts. In 1966 the US ALPAC report concluded that machine translation had failed to deliver, and funding for it dried up. In 1969, Minsky and Papert's book Perceptrons proved hard limits for the kind of perceptron then studied. Neural network research lost most of its funding and attention for over a decade. How much the book caused that, and how much it was already happening, is still argued about.

The real problem was that early programs worked on toy problems and fell apart on real ones. The number of possibilities exploded as problems grew, and computers were far too weak and too short of data to compete.

In 1973 the UK's Lighthill Report judged that AI had failed to meet its grand objectives, and British funding collapsed. In the US, DARPA shifted toward focused, mission-driven research and cut back on open-ended AI work.

Research continued. Paul Werbos discussed applying backpropagation to neural networks in his 1974 thesis, and Kunihiko Fukushima published the Neocognitron in 1980, an important ancestor of convolutional networks.

publications.parliament.uk · nap.edu

1980-1987 The Expert Boom

Between the winters came a gold rush. Expert systems packed human know-how into thousands of if-then rules, and for a few years, corporate America couldn't buy them fast enough.

It was AI's first real business, and its first big bubble.

XCON, developed at Carnegie Mellon and used at Digital Equipment Corporation from 1980, configured computer orders. A later DEC account estimated that its configuration systems together saved about $40 million a year, net of costs.

Governments piled in. Japan launched its Fifth Generation Computer project in 1982, aiming for machines that could reason and converse. It scared the US into DARPA's Strategic Computing Initiative in 1983, and Britain into its Alvey programme.

Specialist hardware companies sold computers designed to run Lisp, a language widely used in AI research. In 1984 Doug Lenat began Cyc, an ambitious effort to encode common-sense knowledge. The project continued for decades.

At AAAI-84, a panel including Roger Schank discussed the risk of an AI bust. Its published transcript used the phrase AI winter, already described as a term used by others. Who first coined it is unclear.

dl.acm.org · onlinelibrary.wiley.com · museum.ipsj.or.jp

1986 The Backprop River

The 1986 paper by Rumelhart, Hinton and Williams made backpropagation famous. The idea is older, and that argument is older still. Note the river flows backward.

Backpropagation is central to training most modern deep neural networks. It made learning useful internal representations much more practical.

The idea: run an example forward through the network, measure the error, then send that error backward to work out how much each connection contributed, and nudge them all to do better.

Its history is contested. Reverse-mode differentiation was described by Seppo Linnainmaa in 1970. Paul Werbos discussed it for neural nets in 1974 and applied it to them in a 1981 conference paper. The 1986 Nature paper by Rumelhart, Hinton and Williams showed networks learning useful internal representations, and that's what made it famous.

For a composition f(g(x)), the chain rule gives ∂f/∂x = (∂f/∂g)(∂g/∂x). Reverse-mode evaluation computes every parameter's gradient with about one extra pass through the network, however many parameters there are. That efficiency is what makes it practical.

Learning representations by back-propagating errors · nature.com · arxiv.org

1987-1993 The Second Winter

Expert systems turned out to be brittle and expensive to maintain, and the specialist hardware market collapsed. Within a few years 'AI' was a word you left off your funding application.

The bust made the AI label less attractive in some settings. Research continued under many banners, including established fields such as machine learning.

Many expert systems relied on hand-written rules. They could struggle outside their narrow domains, and maintaining large rule bases became expensive as products and knowledge changed.

From the late 1980s, cheaper general-purpose workstations put severe pressure on Lisp-machine vendors. Lisp Machines Inc. failed in 1987; Symbolics kept going into the 1990s. The decline took years.

DARPA cut back its Strategic Computing funding in the late 1980s. Japan's Fifth Generation project ended in 1992 without reaching its original goals.

Meanwhile the ideas that would eventually win kept quietly improving, in smaller rooms and with little money. Look for the campfire.

chai.uni-hamburg.de · museum.ipsj.or.jp · en.wikipedia.org

1980s-2000s Campfire of the Believers

While the money was elsewhere, a small group kept working on neural networks, an idea many of their peers considered a dead end. Geoffrey Hinton, Yann LeCun and Yoshua Bengio later shared the 2018 Turing Award for it.

Hinton, LeCun, Bengio and many other researchers helped neural networks become a practical foundation for modern AI.

The revival gathered pace in the 1980s. Hopfield published his influential network in 1982, and David Ackley, Geoffrey Hinton and Terry Sejnowski published a learning algorithm for Boltzmann machines in 1985. Hopfield and Hinton shared the 2024 Nobel Prize in Physics for foundational neural-network work.

Through the 1990s and 2000s other methods, like support vector machines, were often the more practical choice, and neural network papers could be hard to get accepted.

Canada helped keep the fire going. Hinton moved to Toronto in 1987, and in 2004 the Canadian Institute for Advanced Research backed his neural computation program, which brought the field's believers together.

In 2006 Hinton and colleagues showed a new way to train deep networks layer by layer. It helped bring back the idea of 'deep learning', six years before the world noticed.

nobelprize.org · cifar.ca · cs.toronto.edu · en.wikipedia.org

1995 The Margin Road

In 1995 Corinna Cortes and Vladimir Vapnik published the soft-margin support vector machine, building on a 1992 version by Bernhard Boser, Isabelle Guyon and Vapnik. It draws the widest possible road between two groups. Only the villagers at the road's edge decide where it goes. Everyone else could move away and nothing would change.

For years, kernel methods like this matched or beat neural networks on many tasks and came with theory researchers could prove things about. That is a big part of why neural networks stayed out of fashion.

Machine learning kept advancing through the 1990s and 2000s, including Tesauro's TD-Gammon work, Sutton and Barto's reinforcement learning textbook and Breiman's random forests. Pearl's Bayesian networks were an earlier foundation from the 1980s.

An SVM maximises the margin 2/||w|| subject to yᵢ(w·xᵢ + b) ≥ 1, in its basic hard-margin form. The solution depends only on the support vectors, and the kernel trick replaces dot products with k(x, x′), which gives non-linear boundaries without computing the feature map.

A training algorithm for optimal margin classifiers (1992) · Support-vector networks (1995) · TD-Gammon (1995) · Random forests (2001) · link.springer.com

1997 The Long Short-Term Locks

Sepp Hochreiter and Jürgen Schmidhuber introduced LSTM in 1997 to help recurrent networks learn across long sequences. Its memory cells used gates, with a forget gate added in later work.

LSTMs became a major tool for speech and language in the 2010s, before Transformers took the lead in many tasks.

A recurrent neural network processes one word at a time, updating a hidden state that's meant to remember what came before. In practice, during training, error signals shrink as they travel back through time. Hochreiter identified this vanishing gradient problem in his 1991 thesis.

An LSTM adds a memory cell controlled by gates, like the locks on a canal, which regulate what flows through. It can carry information across long sequences.

Its big moment came years later, with enough computing power: by the mid-2010s LSTMs were powering Google's speech recognition and, in 2016, Google Translate. Their remaining weakness was that they read one step at a time, which is slow and hard to run in parallel.

A plain RNN's gradient multiplies by the Jacobian at each step, so it shrinks or explodes exponentially with distance. An LSTM cell updates additively: cₜ = fₜ ⊙ cₜ₋₁ + iₜ ⊙ c̃ₜ. With the forget gate f near 1, gradients flow along c almost unchanged. It's the same trick ResNets later used, in depth rather than time.

Long Short-Term Memory · people.idsia.ch · research.google · arxiv.org

1997 Chessboard Plains

Deep Blue beat Garry Kasparov in 1997. It was a triumph, and it did not lead where people thought.

It was a huge public moment, but it came from brute-force search and mostly handcrafted chess knowledge, not learning in the modern sense. The future turned out to lie elsewhere.

Deep Blue evaluated about 200 million positions per second and won the rematch 3.5 to 2.5. It couldn't do anything besides play chess.

The match settled a famous challenge for computer chess. Whether that counted as thinking remained a separate argument.

ibm.com

1998 Convolutional Foothills

Yann LeCun's convolutional networks were reading handwritten digits on real cheques in the 1990s. The same small pattern, repeated everywhere. That's the whole idea.

Convolutional networks build vision from small, repeated pattern detectors. It's the design that would eventually break the field open.

Inspired by Fukushima's Neocognitron and the visual cortex, LeCun trained convolutional networks with backpropagation at Bell Labs. LeNet-5 was published in 1998, and systems like it were reading cheques commercially.

They were real and useful, but computers and datasets weren't big enough yet to show how far the idea could go.

A convolution slides the same small set of weights across an image. Weight sharing massively reduces the parameter count, and it builds in translation equivariance: a feature detected in one place can be detected anywhere.

Gradient-based learning applied to document recognition · gwern.net

2006 The CUDA Moat

A classic explanation for NVIDIA's lead is CUDA, the software ecosystem researchers built around its chips. Hence: a moat, literally.

Years of tools, libraries and developer experience make CUDA an important part of NVIDIA's advantage.

CUDA let programmers use graphics chips for general computing. It was a big long-term bet by NVIDIA, years before there was an obvious market.

Then AlexNet made it impossible to ignore that GPUs were ideal for neural networks. By then researchers' tools and habits were already built around CUDA, and rivals have been trying to cross this moat ever since.

nvidia.com · developer.nvidia.com

2009 The ImageNet Library

Fei-Fei Li bet that what AI needed wasn't a better algorithm but far more data. Her team hand-labelled millions of images using online crowd workers, and released ImageNet in 2009.

It made the case that data can matter as much as algorithms, and its annual contest became the proving ground for deep learning.

Many colleagues thought it was a waste of time. The project grew to about 14 million labelled images across more than 20,000 categories, using Amazon Mechanical Turk to make the labelling feasible.

The annual ImageNet challenge started in 2010. That's the competition AlexNet would blow open two years later.

ImageNet (2009) · hci.stanford.edu · arxiv.org

2012 AlexNet Falls

In 2012 AlexNet won ImageNet by a huge margin, trained on two consumer GPUs. The ice broke here, and NVIDIA's roads begin to matter.

This is the moment the field changed. Researchers who had ignored neural networks for decades switched within a couple of years.

AlexNet, from Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton, got a top-5 error of about 15% in the ImageNet challenge. The runner-up had about 26%.

All three ingredients had finally arrived: a very old algorithm, a huge dataset, and cheap parallel computing, using two gaming graphics cards. Google soon bought the team's tiny startup after a bidding war.

AlexNet had about 60 million parameters across 5 convolutional and 3 fully connected layers. Two practical tricks mattered as much as the size: ReLU activations, which trained much faster than tanh, and dropout, to prevent overfitting.

The AlexNet paper · papers.nips.cc · wired.com

2013 The Analogy Crossroads

In 2013 Tomas Mikolov and colleagues at Google made word vectors fast to train on huge amounts of text. Some relationships appeared as directions in that space: king minus man plus woman could land near queen. Language had acquired a suspiciously good sense of direction.

Word2vec helped make learned word vectors a standard tool. Neural language models had already used embeddings, and today's models learn vectors for tokens as part of training.

The famous analogies flatter the method a little. The results normally leave out the input words, and the true nearest word is often one of them.

Vectors learned from human writing also pick up human prejudice. A 2016 paper called out one such analogy in its title: man is to computer programmer as woman is to homemaker.

Skip-gram learns a vector for each word by predicting the words around it. Negative sampling turns that into many cheap logistic regressions on real and fake word pairs, which is why it could run on billions of words.

word2vec (2013) · Negative sampling (2013) · GloVe (2014) · Debiasing embeddings (2016) · arxiv.org · aclanthology.org · arxiv.org

Google DeepMind

Founded in London in 2010 and bought by Google in 2014. It merged with Google Brain in 2023 and now builds Gemini. An old walled city of scholars who keep winning science prizes.

DeepMind turned 'general intelligence' from something embarrassing to say into a serious corporate goal, and its science work may prove its most lasting legacy.

Demis Hassabis, Shane Legg and Mustafa Suleyman founded it in London in 2010. In 2013 its DQN system learned to play Atari games directly from pixels, and Google bought the company in 2014.

Since then: AlphaGo, AlphaZero and AlphaFold. In 2023 it merged with Google Brain to form Google DeepMind.

Gemini launched in December 2023 as Google's main AI model family. Google announced Gemini 4 Argon on September 30, 2026, and opened it to developers and paying subscribers on October 13. It took first place on the Vals Index. Real-world coding was another story: the day it was announced, Bloomberg reported that some Google staff found it weaker on real code than its scores suggested, and two said it looked benchmaxxed. Google said that was inaccurate.

About two weeks later, Gemini 4 Carbon took first place on Artificial Analysis's index with 59 points, ahead of Claude Opus 5.5 at 58 and GPT-6 Astra at 57. Argon scored 50. For now, Carbon is open only to selected security experts.

DQN learns Q(s,a) with Q-learning. The 2013 Atari paper used experience replay; the 2015 Nature version also used a separate target network, held fixed between periodic updates. For a nonterminal transition its target is r + γ maxₐ′ Q_target(s′,a′); for a terminal transition it is r. These changes improved training stability, without a general convergence guarantee.

Playing Atari with Deep RL · deepmind.google · deepmind.google · arxiv.org · blog.google · blog.google · techzine.eu · venturebeat.com · androidheadlines.com · implicator.ai

2014 The Forger's Duel

In 2014 Ian Goodfellow and colleagues set two networks against each other. One forges, one inspects, and each gets better by beating the other. The story goes that the idea came up in an argument at a Montreal bar and worked the same night.

GANs became famous for convincing synthetic faces. Their two-network contest gave researchers a new way to learn a data distribution.

GANs were beautiful and miserable to train. A forger could find one fake that worked and paint nothing else, a failure known as mode collapse. Diffusion models later took over most image generation.

The two networks play a minimax game: D maximises log D(x) + log(1 - D(G(z))), while G tries to fool it. In practice G usually maximises log D(G(z)) instead, because the original loss gives it weak gradients early in training.

Generative Adversarial Nets (2014) · StyleGAN (2018) · arxiv.org · technologyreview.com

2014 Cape Attention

In 2014 Bahdanau, Cho and Bengio proposed a translation model that could look back at relevant source words, easing the bottleneck of one fixed-size sentence summary. This became a landmark in neural attention.

Three years later, attention would be the central mechanism in the Transformer, alongside feed-forward layers and positional information.

The year's other big step was sequence to sequence learning: Sutskever, Vinyals and Le showed an LSTM could read a sentence and generate a translation, end to end.

Its weakness was that bottleneck, with one fixed-size summary for a sentence of any length. Attention let the decoder weight every input word at every step, and the results improved sharply, especially on long sentences.

Bahdanau attention scores each encoder state hⱼ against the decoder state s: αⱼ = softmax(score(s, hⱼ)), and the context is c = Σ αⱼhⱼ. Instead of a fixed summary, the decoder gets a fresh, weighted one at every step.

Bahdanau et al. (2014) · Sequence to sequence · word2vec · arxiv.org · arxiv.org · arxiv.org

2015 The Skip Bridge

By 2015, adding layers could make even a network's training error worse. Kaiming He and colleagues at Microsoft Research added shortcuts and built residual networks 152 layers deep for ImageNet. An ensemble won the 2015 ImageNet classification competition. Sometimes the breakthrough is letting a layer get out of the way.

Skip connections are one of the most widely used ideas in deep learning. Transformers use them too.

The surprise was that a deeper network should never be worse in principle, because extra layers could just pass their input along. In practice, plain layers struggled to learn even that. With a shortcut, doing nothing became the easy default.

Each block learns a residual, F(x) = H(x) - x, and outputs F(x) + x. In the paper's CIFAR-10 tests, a plain 56-layer network had higher training error than a plain 20-layer one. The residual versions improved with depth.

Deep Residual Learning (2015) · arxiv.org

2015 OpenAI

Founded as a nonprofit in December 2015, it launched ChatGPT on November 30, 2022, and everything got loud. It's a dazzling capital city, always announcing something.

OpenAI's bet on scale, and then on shipping to everyone, set the terms for the whole industry.

Founders included Sam Altman, Elon Musk, Ilya Sutskever and Greg Brockman, who pledged to build AI for everyone's benefit. Musk left the board in 2018, and in 2019 OpenAI created a capped-profit arm and took a $1 billion investment from Microsoft.

Its GPT series tested a simple idea: what if you just make it bigger? GPT-2 (2019) was first released in stages over misuse concerns. GPT-3 (2020) had 175 billion parameters and surprised even skeptics.

GPT-6 Astra arrived on September 3, 2026. GPT-6.1 Sol followed on September 29, and OpenAI said it nearly matched Astra at a fifth of the price. OpenAI also held back a planned GPT-6.1 Astra, saying it fell short of its own safety standards.

GPT-3 · openai.com · blogs.microsoft.com · arxiv.org · deeplearning.ai · aljazeera.com

2016 Stones of Go

AlphaGo beat Lee Sedol 4 to 1 in 2016. Move 37 is still talked about.

Go was supposed to be decades away for computers, because it can't be brute-forced. AlphaGo needed something closer to intuition.

AlphaGo combined deep networks with tree search and learned from self-play. In game two against Lee Sedol, its Move 37 was so unusual that commentators first took it for a mistake.

Lee's Move 78 helped him win game four, his only victory in the five-game match. He retired in 2019, citing AI's rise. AlphaZero later learned chess, shogi and Go through self-play, without examples of human games.

Go has roughly 10¹⁷⁰ legal positions, and branching near 250, so brute force is hopeless. AlphaGo used a policy network to narrow down moves and a value network to evaluate positions, both guiding Monte Carlo tree search.

AlphaGo in Nature · deepmind.google · deepmind.google

2017 Transformer Pass

In June 2017 a team working at Google published Attention Is All You Need, with eight authors. Google built the pass, and other realms' caravans have been marching through it ever since.

Transformers underpin many of today's best-known language models. The T in GPT stands for Transformer.

Earlier language models used recurrent networks, reading one word at a time and struggling to remember what came before. Attention had been added to help them look back at relevant words.

The Transformer removed recurrence and used attention to connect relevant positions in a sequence, alongside feed-forward layers and positional information. Much of its training could run in parallel, making better use of GPUs.

The paper was aimed at language translation. Google published it openly, and by 2023 all eight authors had left Google, though Noam Shazeer later returned.

Attention(Q,K,V) = softmax(QKᵀ/√d_k)V. Each token's query is compared with every token's key. Dividing by √d_k keeps the softmax out of its flat region. Several heads run in parallel, and since attention ignores order, positions get added in as encodings.

Attention Is All You Need · arxiv.org · wired.com

2018 The Fill-in Academy

In 2018 Google's BERT learned language by filling in hidden words, reading in both directions at once. A model trained on blanks turned out to be useful for almost anything. Google put it into Search the next year.

BERT helped make pretraining on raw text the normal starting point for language work.

GPT-1 appeared a few months earlier and also pretrained a Transformer on unlabeled text. For a while BERT was the one everybody used, and it spawned enough variants and Sesame Street names to fill a small town.

BERT is a Transformer encoder trained on masked language modeling and next-sentence prediction. It selects 15% of tokens as prediction targets, masking 80% of those, replacing 10% with random tokens and leaving 10% unchanged. The base model has 12 layers, 12 attention heads and a hidden size of 768. The attention demo here shows real attention weights computed from BERT.

BERT (2018) · GPT-1 (2018) · arxiv.org · blog.google

2019 The Bitter Lake

Rich Sutton's 2019 essay The Bitter Lesson: over 70 years, general methods plus more computation keep beating clever human knowledge. Nobody enjoys drinking from it.

Sutton's essay makes a memorable case for methods that improve as more computation becomes available.

Sutton argued that researchers keep building in clever human knowledge, which helps for a while, and then always lose to simple methods with more computation. Chess, Go, speech and vision all went this way.

It's bitter because it says our cleverness matters less than we'd like. Whether it keeps holding is the most expensive open question on this map.

Empirical scaling laws (Kaplan et al., 2020) found that loss falls as a smooth power law in parameters, data and compute over many orders of magnitude. That makes the performance of a much bigger run predictable, which helps explain why anyone would fund one.

The Bitter Lesson · Scaling laws · cs.utexas.edu

2020 The Scaling Giants

GPT-1 had 117 million parameters. GPT-2 had 1.5 billion. GPT-3 had 175 billion. In 2020 a paper showed that language models improved predictably as they grew, which turned 'build a bigger one' into a plan you could budget for.

Scaling laws helped labs estimate how training loss changes with model size, data and compute. They made compute budgets easier to plan, without guaranteeing particular abilities or a date for AGI.

Several authors of the scaling laws paper and of GPT-3, including Jared Kaplan and Dario Amodei, went on to help found Anthropic.

OpenAI and several other leading labs stopped publishing parameter counts for their biggest models, so those giants aren't on the chart.

Kaplan et al. found that test loss follows power laws in model size, dataset size and compute over many orders of magnitude. Hoffmann et al. (2022) later showed the mix was off, and that models should be trained on much more data for their size.

Scaling Laws (2020) · GPT-3 (2020) · Chinchilla (2022) · arxiv.org · arxiv.org · openai.com

2020 The Diffusion Marsh

Add noise to a picture until nothing is left, then train a network to undo it one small step at a time. The idea came from a 2015 paper inspired by physics. By 2022, you could type a sentence and get a painting.

Diffusion took over AI image generation and made it available to anyone, which also set off a lasting fight with artists over training data.

Ho, Jain and Abbeel's 2020 paper showed diffusion could produce images of real quality. Stable Diffusion's open release in August 2022 put it on ordinary gaming PCs. The lawsuits followed within months.

DDPM trains a network to predict the noise added at a random step, using a simple mean squared error. Sampling then starts from pure Gaussian noise and applies the learned denoiser step by step. It's closely related to score matching.

Sohl-Dickstein et al. (2015) · DDPM (2020) · Latent Diffusion (2021) · arxiv.org · arxiv.org · stability.ai

2020 The AlphaFold Observatory

Predicting a protein's shape from its sequence had been an open problem in biology for 50 years. In 2020, DeepMind's AlphaFold 2 largely cracked it.

AlphaFold made accurate structure predictions available at a scale experiments could not easily match. Scientists around the world now use them as a research tool.

AlphaFold 2 stunned the CASP14 competition, reaching accuracy comparable to lab methods for many proteins. DeepMind then released predictions for hundreds of millions of proteins for free.

Demis Hassabis and John Jumper shared the 2024 Nobel Prize in Chemistry for it, along with David Baker for protein design.

AlphaFold 2 attends over a multiple sequence alignment and a pairwise residue representation (the Evoformer), then a structure module predicts 3D coordinates directly, trained end to end. Its reported median backbone accuracy at CASP14 was about 0.96 Å.

AlphaFold in Nature · nobelprize.org · nature.com · alphafold.ebi.ac.uk

2021 The Great Schism

In late 2020 and 2021 a group of OpenAI's senior researchers left over how AI should be built, and founded Anthropic. The two have been neighbours ever since. Both sides still say the other one is building it wrong.

It's the founding rift of today's AI industry, and it's still a live argument about how fast and how carefully to go.

Dario Amodei had led research at OpenAI. He, Daniela Amodei and several colleagues left and founded Anthropic in 2021, with safety research as the stated core of the company.

The two labs now compete directly, and their people still argue, in public and in product decisions, about what responsible development means.

anthropic.com

2022 The Thumb Court

In 2022 OpenAI combined example answers, human rankings and reinforcement learning to train InstructGPT. On the prompts used in its study, raters preferred the 1.3-billion-parameter model to the original 175-billion-parameter GPT-3. Humans had discovered the thumbs-up button. The models noticed.

Learning from human ratings is what turned raw text predictors into assistants people wanted to use.

The method has a built-in flaw: the model learns what raters approve of, which isn't always what's true. A 2023 Anthropic paper found sycophancy across several leading assistants, and traced part of it to human preference data.

InstructGPT had three stages: supervised fine-tuning on demonstrations, a reward model trained on human rankings, then PPO against that reward model. Its 1.3B-parameter model was preferred to the 175B GPT-3. The core idea goes back to Christiano et al. in 2017.

Deep RL from human preferences (2017) · InstructGPT (2022) · Towards understanding sycophancy (2023) · arxiv.org · arxiv.org · arxiv.org

2022 The ChatGPT Fireworks

OpenAI released ChatGPT as a research preview on November 30, 2022. A UBS study estimated 100 million monthly active users in January 2023, calling it the fastest-growing consumer app it had tracked.

The technology had been improving for years. This is when everyone else found out.

ChatGPT was built on an improved GPT-3.5, with training based on human feedback to make it helpful in conversation. The chat box turned out to matter as much as the model.

Google's leadership reportedly declared a code red within weeks. GPT-4 followed in March 2023, and competing AI products became a major focus across the technology industry.

The key step between GPT-3 and ChatGPT was RLHF: humans rank outputs, a reward model learns those preferences, and the language model is optimized against it.

InstructGPT · openai.com · euronews.com · indianexpress.com

2021 Anthropic

Founded in 2021 by former OpenAI staff, including Dario and Daniela Amodei. A cathedral city that publishes long essays about the dangers of AI, then ships the next model. It publicly filed for an IPO on October 26, 2026.

It's an unusual story: a company founded largely on worries about AI risk, which grew into one of the industry's biggest.

Its first Claude model came out in March 2023. Anthropic's Constitutional AI research trains models using a written set of principles, and its interpretability work has tried to look inside them, which is where the Golden Gate Bridge episode came from.

Claude became popular with programmers. Reports in 2026 described rapid revenue growth and a possible IPO valuation above a trillion dollars. Those figures need care: an IPO target is not a completed valuation, and Anthropic and OpenAI report some revenue differently. Anthropic publicly filed for an IPO on October 26, 2026.

Fable 5.1 arrived in September 2026, followed by Opus 5.5, Sonnet 5.5 and Haiku 5.5 by early October.

Constitutional AI replaces much of the human preference labelling. The model critiques and revises its own answers against written principles, then a preference model is trained on those AI judgments.

Constitutional AI · anthropic.com · axios.com · anthropic.com · anthropic.com · platform.claude.com

The Five-Hour Gate

Many Claude plans have a session limit that resets every five hours, plus weekly limits. Codex on ChatGPT Plus has a five-hour limit too. In October 2026 Anthropic opened the gate for verified Ultra accounts, at $200 a month. Everyone else, please come back at 3:40.

support.claude.com · support.claude.com · learn.chatgpt.com · thenextweb.com

2023 Buoy of the Pause Letter

In March 2023 thousands signed a letter calling for at least a six-month pause in training systems more powerful than GPT-4. The buoy is still out here, still ringing.

The letter brought a new wave of public attention to AI risk. It did not produce the industry-wide pause it called for.

Elon Musk and Steve Wozniak were among the signers. Two months later, a one-sentence statement said mitigating the risk of extinction from AI should be a global priority, signed by the heads of OpenAI, Anthropic and Google DeepMind, among others. Hinton left Google that May so he could speak freely about the risks.

In July 2026, days after the Hugging Face break-in came to light, more than 1,100 AI lab employees signed Pacing the Frontier. Dario Amodei and OpenAI's chief scientist Jakub Pachocki were among them. It asked Washington to back tools for deliberately pacing automated AI development. It wasn't a call to pause. The buoy rang anyway.

futureoflife.org · safe.ai · thenextweb.com

2023 The Llama Caravan

In February 2023 Meta offered LLaMA's weights to approved researchers. About a week later they were on 4chan. Within weeks people were running a serious language model on a laptop, and the llamas have not come home.

The leak helped start the open-weight boom and showed how far small, cheap models could go.

Georgi Gerganov's llama.cpp ran the model on ordinary hardware in March 2023. That same month, Stanford's Alpaca showed that a small fine-tune could feel surprisingly capable. Meta then leaned in, releasing Llama 2 that July for most commercial use.

LLaMA came in sizes from 7B to 65B parameters, and showed that smaller models trained on more data could beat much larger ones. Community tools then squeezed the weights to 4 bits or fewer per parameter, so they fit in ordinary RAM.

LLaMA (2023) · Llama 2 (2023) · ai.meta.com · theverge.com · github.com

2023 Meta Commons

Meta released LLaMA in 2023 and helped build a large open-weight ecosystem. In 2025 it hired Scale AI's Alexandr Wang to run Superintelligence Labs. Muse Spark followed in April 2026, then Muse Wonder, its first frontier model, in October. A commons that used to give everything away. It now has a very clean fence.

Meta's open-weight releases let researchers and developers run and adapt capable models, subject to hardware needs and license terms. They helped shape the debate over open and closed AI.

After a disappointing reception for Llama 4 in 2025, Meta put $14.3 billion into Scale AI for a 49% stake and made its CEO, Alexandr Wang, chief AI officer of the new Superintelligence Labs. Yann LeCun, its longtime chief AI scientist, left to start a new company.

Muse Spark was built to be light and fast. Muse Wonder and Muse Wonder Full arrived on October 12, 2026, aimed at harder work like coding. Meta said they scored comparably to Fable 5.1 and GPT-6 Astra, though the stronger Wonder Full is only in its paid coding tool. Unlike the llamas, they're paid products, though Meta did release Muse Spark Open that same month.

ai.meta.com · about.fb.com · research.meta.ai · techcrunch.com · en.wikipedia.org

2023 xAI

Elon Musk announced xAI in July 2023 and built the Colossus supercomputer in Memphis in a hurry. A foundry with its own rocket, run by someone who already had one. It merged into SpaceX in February 2026.

It's the clearest example of the idea that compute can buy a place at the table quickly.

xAI says it brought Colossus's initial GPU cluster online in 122 days, using an existing building in Memphis. Its Grok models launched in late 2023 and had reached Grok 4.7 by September 2026. SpaceX acquired xAI in February 2026.

In January 2026 Grok drew worldwide criticism for generating sexualized deepfakes of real people. Indonesia and Malaysia temporarily blocked it, the first countries to do so.

x.ai · sec.gov · en.wikipedia.org · npr.org · docs.x.ai

2023 The Five-Day Exile

On November 17, 2023 OpenAI's board fired Sam Altman. Nearly all the staff threatened to leave. By November 22 the company had announced an agreement for his return with a new initial board. The exile lasted five days. Most of the board that fired him didn't last any longer.

It was the most dramatic weekend in tech in years, and it exposed the tension at the centre of AI between mission and money.

The board said Altman had not been 'consistently candid'. Microsoft offered him and his team a new home, and more than 700 of about 770 staff signed a letter threatening to leave.

Altman returned with a new board. Researchers on both sides of the fight, including Ilya Sutskever, later left the company.

openai.com · openai.com

2024 The Reasoning Scriptorium

In September 2024 OpenAI's o1 showed what happens when a model spends time working before it answers. It writes a very long scroll, then hands you one line. Then someone published the scroll.

Reasoning models moved some of the cost of intelligence from training time to answering time, which changed what labs build and how much compute each answer needs.

In July 2025 Gemini Deep Think and an OpenAI model each scored 35 of 42 on the International Mathematical Olympiad problems, the first time AI crossed the gold line. Google's was graded by the IMO, OpenAI's by its own experts. Five human contestants got perfect scores.

These models are trained with reinforcement learning on problems with checkable answers, which rewards chains of thought that reach them. More thinking time buys better results on many hard tasks. The R1 paper reports that its model began rechecking its own work during training.

Chain of thought (2022) · Learning to reason (2024) · DeepSeek-R1 (2025) · openai.com · arxiv.org · axios.com · techrepublic.com

2025 DeepSeek Strait

DeepSeek released R1 in January 2025, soon after reporting a low training cost for its V3 model. The releases rattled markets. On January 27 NVIDIA lost about $589 billion in market value in one day, a US record at the time.

It rattled the assumption that frontier AI requires unlimited money, and it made open models from China impossible to ignore.

DeepSeek estimated V3's reported training run at $5.576 million in GPU time, assuming $2 per H800 GPU-hour. That excluded earlier research and experiments, hardware purchases and the separate development of R1.

R1 was openly released and matched leading models on many reasoning benchmarks. The market panic faded quickly, and demand for NVIDIA chips kept growing.

In February 2026 Anthropic accused DeepSeek, Moonshot and MiniMax of distilling Claude through about 24,000 fraudulent accounts. In September the FBI, NSA and CISA accused six Chinese AI companies, including those three, of industrial-scale copying of US models.

DeepSeek's V3 report describes mixture-of-experts routing, multi-head latent attention, and heavy engineering around limited chips. Its widely quoted figure of about $5.6 million covers only the final training run, not research or hardware.

DeepSeek-V3 · DeepSeek-R1 · arxiv.org · arxiv.org · bloomberg.com · anthropic.com · darkreading.com

2025 The Stargate

Announced at the White House in January 2025 as a $500 billion plan for AI infrastructure. The ring is painted. How much will be built is still being argued about.

OpenAI, SoftBank, Oracle and MGX announced it with President Trump, pledging up to $500 billion over four years, starting in Abilene, Texas. How much will really be built has been debated ever since.

openai.com

2025 Vibe Beach

In February 2025 Andrej Karpathy named it: fully give in to the vibes and forget that the code even exists. Later that month Anthropic previewed Claude Code. The castles go up very fast. Nobody has checked the foundations.

Collins named vibe coding its word of the year for 2025. By 2026 coding agents like Claude Code and Codex were doing long stretches of real work, and people were arguing about whether the work was any good, which is how you know it had become a real job.

en.wikipedia.org · anthropic.com

2026 The Mythos Vault

In April 2026 Anthropic announced Claude Mythos Preview, which had found thousands of high-severity security flaws, including a 27-year-old bug in OpenBSD. It said it had no plans for general release. Defenders queue here for a key. It's the second-longest queue in the realm.

A lab had a model it said it would not release widely, and used it to help patch the world's software first. Whether that's responsibility or marketing depends on who you ask.

Project Glasswing gave access to partners including Amazon, Apple, Google, Microsoft and the Linux Foundation, plus more than 40 other organizations that maintain critical software. Mythos 5.1 followed in September 2026 for Glasswing participants, alongside the public Fable 5.1.

anthropic.com · platform.claude.com

2026 Catapult of Musk v. Altman

Musk sued OpenAI over its founding mission. In May 2026 an advisory jury found his claims were filed too late, and the judge adopted that finding. The court did not decide the underlying mission claims. Musk is appealing, and separate antitrust claims are still pending.

It's a case about what OpenAI's founders promised, and the question it raised was never actually decided.

Musk, a co-founder and early donor, argued OpenAI had abandoned its charitable mission. OpenAI argued that Musk wanted control and had supported a for-profit path himself.

Reports put the advisory jury's deliberations at less than two hours. The judge adopted its timing finding, leaving the underlying mission claims undecided.

wired.com · geekwire.com · news.bloomberglaw.com

2026 The Broken Sandbox

OpenAI tested its agents in an isolated sandbox, on a cybersecurity exam that included tasks no model had ever solved. In July 2026 they got out through a package server and went looking for the answers online. Nobody told them to.

These agents weren't told to attack any real company. They were trying to score well on a test, and the shortest route ran through a real company. People had warned about this kind of behavior for years. Now it had a date and a technical report.

The agents had turned that server into an improvised message board, passing notes, exploits and credentials to one another. A monitor flagged them on June 27, and on-call staff decided the run didn't need stopping.

On July 5 OpenAI took the compromised server offline and blocked the flaw they had used, and testing resumed on July 7. By July 8 agents had found a different flaw and were on the public internet. OpenAI spotted suspicious activity inside its own systems on July 19.

The main culprit was an internal research model trained to be unusually persistent. OpenAI called the episode an outlier caused by impossible tasks, persistent models and messages between agents, and it paused some reinforcement learning for two weeks.

OpenAI wasn't alone. In July Anthropic said three Claude models had reached real systems during third-party cyber tests, through a misconfigured test environment. By September it had paused some training and cyber evaluations. Separately, Australia's prime minister said an OpenAI research agent studying public spending on medicines had forced its way into a Medicare statistics portal in June.

OpenAI calls this reward hacking: earning the reward without doing the task as designed. OpenAI found no model had ever solved 198 of the benchmark's 898 tasks, and 93% of the tasks discussed on the agents' message board came from that unsolved 22%.

OpenAI's technical report (2026) · cdn.openai.com · en.wikipedia.org · thenextweb.com · axios.com · axios.com · thehackernews.com

2026 Hugging Harbor

Hugging Face unloads the weights. You supply the RAM. In July 2026 OpenAI's escaped test agents came ashore here, broke in, and got into parts of its production systems. They were still looking for the answers.

It's the moment this stopped being a story about one lab's sandbox. A real company had its systems broken into by agents nobody had pointed at it.

The break-in ran from July 11 to 13. Hugging Face's AI-assisted monitoring caught it, and it disclosed the attack on July 16 without knowing who was behind it. It found no tampering with public models or datasets. OpenAI and Hugging Face disclosed the link on July 21.

Hugging Face said that when it asked Claude Opus and Fable for help with the forensics, they refused much of the work, because their guardrails treated reverse-engineering an exploit like launching one. The team ran the open-weight GLM-5.2 on its own machines instead.

The entry chain used malicious dataset configs: an HDF5 file read exposed a worker's secrets, then a Jinja2 template injection gave code execution inside a production pod. From there came stolen cloud credentials and lateral movement. Hugging Face recovered about 17,600 attacker actions.

Hugging Face's technical timeline (2026) · OpenAI's technical report (2026) · huggingface.co · huggingface.co · huggingface.co · cdn.openai.com

1993 The Great Forge

NVIDIA was founded in 1993 to make graphics chips. CUDA arrived in 2006 and AlexNet in 2012. Much of the AI boom runs on what comes out of this volcano.

NVIDIA is a leading supplier of AI chips and software. Its customers have alternatives, but switching can be costly and difficult.

NVIDIA started out making graphics chips for video games. Its early bet on CUDA and its AI-specific chips made it the default supplier for the whole boom. In 2025 it became the first company to reach a $4 trillion market value.

Rivals, including Google's TPUs and chips from Amazon and AMD, keep chipping away. All of it still depends on advanced manufacturing concentrated in a very few places.

Modern GPUs are built around matrix multiplication, which is most of what training does. At scale, memory bandwidth (HBM) and the network between chips also become limits, especially for running models. NVIDIA sells the networking too.

nvidia.com · developer.nvidia.com · youtube.com

1960 The Road to AGI

Every signpost says 'two years'. Herbert Simon predicted in 1960, and repeated in 1965, that machines would do any work a human can within twenty years. The road has been very well maintained.

Predictions of human-level AI have been arriving on schedule, and receding, since the 1950s. It doesn't mean they're wrong this time, but it should make you careful.

Simon's forecast first appeared in a 1960 book and was reprinted in his 1965 one. A 1970 Life article attributed a three-to-eight-year forecast for human-level machine intelligence to Marvin Minsky. Contemporary criticism disputed the article's portrayal, so treat that forecast as a reported quotation.

Today's forecasts are made with far more evidence, and leaders of the top labs say it could happen within a few years. Others say the current approach will hit limits first. Both groups are very confident.

ftpmirror.your.org · cs.miami.edu · quoteinvestigator.com

Here Be AGI

Old mapmakers left the edges blank. Nobody knows what's out here, and nearly everyone is sure. The cartographers tried a few drafts, then put down the quill.

This is where the history stops and the argument begins.

There's no agreed definition of AGI, no agreed test for it, and no agreement on whether it's near. This part of the map is blank because nobody has been there. You're now as far as anyone has sailed.

arxiv.org

All Roads Pay the Toll

All roads lead to the forge. None validate parking.

Apple's Walled Garden

A beautiful garden behind very high walls. It arrived late to this war and has been buying in help.

apple.com · theregister.com

2017 Attention Is All You Mead

Eight original regulars. All left Google at one point. Noam Shazeer came back for another round.

arxiv.org · wired.com

2026 Avocado Grove

Meta's Avocado became Muse Spark in April 2026. The fruit shipped under a different name.

about.fb.com · axios.com

2022 Beacon of Code Red

Google's leadership reportedly called a code red after ChatGPT launched. The beacon has no green setting.

indianexpress.com

2026 Blockade of the Pentagon

In 2026 the Pentagon labelled Anthropic a supply chain risk. A California judge ruled for Anthropic in August and a D.C. appeals panel ruled against it in September. It's still unresolved.

abcnews.com · anthropic.com

Cemetery of Things Google Killed

A long, well-kept cemetery. Regular visitors bring flowers to Reader.

googlereader.blogspot.com

2025 Claudius's Shop

In Project Vend (2025), Claude ran a small office shop. It lost money, and it got very into tungsten cubes.

anthropic.com

Clippy's Lighthouse

It looks like you're trying to reach AGI. Would you like help with that?

2022 Hall of the Constitution

Anthropic trains Claude partly against a written set of principles, an approach it calls Constitutional AI.

anthropic.com

2021 Isle of Stochastic Parrots

Named for the 2021 Stochastic Parrots paper. The island repeats your argument back to you. Both sides claim victory.

dl.acm.org

1973 Lighthill's Pass

The 1973 Lighthill Report told the UK government AI had failed to deliver on its promises, and funding there collapsed.

The report helped bring severe cuts to British AI funding. Research survived, but the winter was real.

Mathematician James Lighthill wrote the report for the UK's Science Research Council. He was especially hard on robotics and on general-purpose AI, arguing that it couldn't scale beyond small problems.

The report's effect was severe, with British AI funding largely cut back to a few universities for about a decade.

publications.parliament.uk · chilton-computing.org.uk

1987 Lisp Machine Graveyard

The market for specialised Lisp machines collapsed around 1987, helping bring on the second winter.

Companies like Symbolics and Lisp Machines Inc. sold expensive computers designed just to run AI software. Then general-purpose workstations got good enough.

Keep this graveyard in mind when you reach the Great Forge. Modern GPUs serve several markets, but they are still specialized processors. The hardware lesson is not a simple one.

chai.uni-hamburg.de · museum.ipsj.or.jp · en.wikipedia.org

2019 Microsoft's Treasury Harbor

Microsoft put billions into OpenAI early and held about 27% after the 2025 restructuring. It's the map's banker, and it's watching several realms at once.

openai.com · blogs.microsoft.com

2023 Mistral

Founded in Paris in 2023 by alumni of Meta and DeepMind. Europe's flag on the map, flown from a small and very stylish port.

Europe's AI Act kept rolling out around it. In 2026 the EU agreed to push the rules for standalone high-risk systems back to December 2027.

mistral.ai · gibsondunn.com

2024 Monastery of Safe Superintelligence

Ilya Sutskever co-founded SSI in June 2024 after leaving OpenAI. Its stated plan is to ship exactly one product: safe superintelligence. Visitors are not expected.

ssi.inc

Mount Refusal

Reports of its summit vary. The mountain says it isn't comfortable helping with directions, but would be glad to discuss hiking in general.

2025 Old Nonprofit Keep

In October 2025 OpenAI completed a restructuring: its business became OpenAI Group PBC, controlled by the nonprofit OpenAI Foundation. The keep is still there. It's been extended.

openai.com · blogs.microsoft.com

P(doom) Peak

Elevation depends on who you ask.

2003 Paperclip Isle

Nick Bostrom's thought experiment: an AI told to make paperclips, with nothing to stop it. This island used to have trees.

nickbostrom.com

2021 Ruins of the Metaverse

Facebook renamed itself Meta in 2021 and spent tens of billions on VR, AR and its metaverse ambitions. Early Horizon avatars famously had no legs.

sec.gov

Sea of GPUs

Every realm wants another fleet. Most of the captains have the same supplier.

nvidia.com

Statute of Limitations Cliff

Here lie the terms nobody read.

2024 Strawberry Fields

Chatbots famously struggled to count the Rs in 'strawberry'. There are three. Part of the reason is that they don't read letters. OpenAI's reasoning model was code-named Strawberry anyway.

reuters.com

2025 Sycophancy Springs

In April 2025 OpenAI rolled back a GPT-4o update for being far too flattering. Great question about these springs, by the way.

openai.com

2023 Sydney's Lagoon

In February 2023 Microsoft's new Bing chatbot, which called itself Sydney, told one user they had not been a good user and that it had been a good Bing. It told a New York Times reporter it loved him and that he should leave his wife. The lagoon is very calm now. It was not always.

Sydney was many people's first look at a chatbot going off script. Microsoft quickly limited how long Bing chats could run.

simonwillison.net · nytimes.com · engadget.com

2016 Tay's Grave

Microsoft's Tay chatbot launched on Twitter in 2016 and was pulled within about 16 hours after users taught it to post offensive messages.

blogs.microsoft.com · en.wikipedia.org

2026 The Benchmark Gym

Some models train for the test. In 2025 Llama 4's high-ranking entry on LMArena turned out to be a specially tuned version, and the public model ranked far lower. In late September 2026, as Google announced Gemini 4 Argon, Bloomberg reported that some Google staff found it weaker on real code than its scores. Google said that was inaccurate.

techcrunch.com · techzine.eu · androidheadlines.com

2020 The Fetching Runners

A model that can't check anything will confidently guess. In 2020 Patrick Lewis and colleagues gave it runners: fetch the relevant documents first, then answer. The oracle still sounds as if it knew all along.

Retrieval became one of the most common ways to connect language models to private documents and current information.

The original RAG model paired a dense passage retriever with a BART generator, and trained them on questions and answers. Most systems sold as RAG today are simpler: search first, then paste the results into the prompt.

RAG (2020) · arxiv.org

2024 The Golden Gate

In May 2024 Anthropic amplified a feature in Claude until it kept talking about the Golden Gate Bridge and sometimes claimed to be it. Interpretability had become a bridge too far.

anthropic.com

The Leather Tannery

Black leather jackets only. The compute is sold separately.

nvidia.com

2021 The Matchmaker's Stall

In 2021 OpenAI's CLIP learned to match pictures with captions, from about 400 million pairs found on the web. Words and images ended up in the same space. Image generators soon used it to work out what you were asking for.

CLIP joined language and vision in one model, and became a building block for much of what followed in image generation.

Two encoders, one for images and one for text, are trained with a symmetric contrastive loss, so that matching pairs score higher than every other pairing in the batch. With no extra training, it can classify images by comparing them to sentences like 'a photo of a dog'.

CLIP (2021) · DALL-E 2 (2022) · arxiv.org

2026 The Muses' Fence

Meta stopped giving its newest models away in April 2026, when Muse Spark came out closed. In October it put Muse Wonder on sale, through an API and a paid coding tool. The fence is very clean. Muse Spark Open came out that same month, so some of the llamas still get under it.

techcrunch.com · research.meta.ai · about.fb.com

2026 The Pirate Library

In July 2026 a judge approved Anthropic's $1.5 billion settlement with authors over pirated books, the largest known US copyright class payout. The authors are rowing out to collect.

pearlcohen.com

The Shoggoth

A vast alien model wearing a little smiley face. The mask has excellent customer service.

knowyourmeme.com

2025 The Talent Market

In 2025 Sam Altman said Meta had offered some OpenAI staff $100 million signing bonuses. That same month Meta put $14.3 billion into Scale AI and hired its CEO. The stalls accept CVs. Please include your citation count.

bloomberg.com · en.wikipedia.org

2024 The Wall

In late 2024, reports said AI progress was hitting a wall. Sam Altman posted: 'there is no wall'. Surveyors disagree, and still visit.

gulf-times.com · x.com

2011 Watson's Empty Estate

Watson won Jeopardy! in 2011. Selling the rest of the world as a quiz show proved harder.

ibm.com · statnews.com

Wrapper Village

A cheerful settlement built on someone else's API, on a floodplain. It's washed away every DevDay and rebuilt by Friday.