The token casino: A practitioner's guide to risk, reward, and AI

If you're a developer, PM, designer, analyst, or any other type of knowledge worker who uses tools like Copilot or ChatGPT on a daily basis, this probably feels both exciting and slightly unreal. These systems bring real value for you—faster drafts, working code, new ideas—but they also sometimes create bugs, hallucinations, and tech debt you only discover later.

With the persistence of those chasing AGI, and the investment of billions of dollars in its development, the public discourse has become greatly skewed. The way these tools are marketed and discussed often makes them sound like some kind of digital awakening approaching sentience. Framed like this, it is hard not to believe that these technologies are nothing short of magic.

Most people using ChatGPT today, including many of us who are integrating its use at work, do not think twice about how any of these things work. The maths and engineering under the hood are genuinely complex, and concepts like backpropagation or modern reinforcement learning research sit far outside what you need to know to build features, write docs or analyse data. The good news is, you don't need extensive academic rigour to use these tools effectively, but you do need a shared and understood vocabulary for thinking about them. Following Feynman's approach, this post is an attempt to offer exactly that: a practical way to talk about and reason about these systems, so you can make better use of them in your day-to-day work.

Large Language Models are just that: models. What they model is a bit abstract, but you can think of it as an approximation of a brain. With that approximation, an astronomical amount of information is thrown at the brain. The brain is then 'tuned' to respond in the 'right way' to a specific set of uses based on the context and parameters. The quality of these recent models means they are generally more useful across a very broad array of uses.

Models work with tokens, and depending on the type of model being used, its training method, purpose, and the data fed into it, tokens can represent a significant amount. The multitude of tokenisers out there varies greatly in their subtle uniqueness. A token could be whole or parts of words, punctuation, symbols (&) or even parts of images or videos.

Despite the varied forms they can take, their function is generally the same. You supply tokens in the front, and the model can predict for you what the likely next token(s) should be. The cool thing about these things is their variability. You can put way more than one token in, and get a wide range of outputs. You can even tune how the output is going to come out in the model itself!

The vast array of training data and model tuning strategies available means that the models can perform quite a wide range of tasks. The ability to move in and out of languages is one such example, and the languages that these models are adept in cross over from general communication to the domain of programming languages. In effect, this is how one would Vibe Code: taking words and ideas and asking a model to return functioning code.