"Do you think you could build an agent with a one-token context window? We did for years, and then as language models got better, they ate our agent's stack, one byte at a time. I'll show you how it happened because I think your stack is next. In 2016, we were shipping agents with language models that read exactly one token, every word on its own. The embarrassing thing is that it mostly worked. It worked because the model had one tiny job, turn the message into numbers and match it to one of your 50 intents. Everything else we wrote by hand. Then Transformers came along, and the model took its first real bite. It stopped counting words and started reading them, in order, in context. That was BERT. But everything the model produced was just labels. Deciding what to do with them was still on us as developers. The next bite was whole conversations. This is where we hit the limits of supervised learning. Google and others tried sequence-to-sequence models. At Rasa, we built TED. Both hit the same problem. The only way to steer your agent was to curate thousands of training dialogues. Then GPT-3 came along, and with it, in-context learning. No data set to label and maintain. You steer the model by editing its input. This honestly felt like magic. First, a single token. Then a sentence. Then a whole conversation. Then your tools and your memory. Every time the context window grew, language models swallowed another layer of the stack we'd built around them. This is what our systems used to look like. A lot of handwritten code. First, the model swallowed dialogue state, because the context window holds the whole transcript now. Then the policy. The tool specs go straight into the prompt, and the model decides. Then the templates. You stop writing what the agent says, you write who it is. Even memory. Results feed right back in, and the model acts again. A model using tools in a loop. That's an agent. That's all that's left. So did the model eat all the work? Just look at your agent stack today. There's still plenty of code. Retrieval, re-ranking, guardrails, evals. You write the machine that assembles the context window. People call that context engineering. Does this look familiar? It's our 2016 diagram with better parts. So yes, language models will eat this stack too. Good. That means we get to go and solve the next set of problems."