Owen Park
Episode three. How generative models work, how you pick one, and how you deploy it. This is the 'identify A.I. model components and configurations' objective, almost word for word.
All episodes
Episode 03 · Identify AI concepts
How generative models work
Owen draws the next-token picture. Mira maps it to Foundry: chat versus embeddings versus multimodal versus image generation, serverless versus managed compute, temperature and max tokens.
- How generative AI models work
- Choose a model by capability
- Deployment options
- Configuration parameters
0:00—
Transcript
On the mics
Mira Cole
Start with a picture. A large language model is a very well-read autocomplete. You give it a sequence of tokens, which are pieces of words. It scores what token is likely next. Then it does that again. The fluent paragraph is a chain of guesses.
Owen Park
That is why it can sound sure and still be wrong. It is not looking up a fact in a binder unless you ground it. Completion means the generated continuation. Prompt means everything you sent in.
Mira Cole
Context window is the size of the binder it can hold in one go. If a policy document is longer than the window, the model cannot 'just read the whole P.D.F.' You retrieve pieces, which is a later episode, or you pick a model with a bigger window.
Owen Park
Embeddings are a different shape. An embedding model turns text, or sometimes images, into a list of numbers, a vector. Similar meanings sit near each other. That is how search and retrieval work. Chat models generate language. Embedding models measure sameness. Do not deploy an embedding model and expect a poem.
Mira Cole
The Foundry Models catalog is a store with more than ten thousand models from Microsoft, OpenAI, Anthropic, Meta, and others. The exam will not ask you to name ten thousand. It will ask you to match a capability to a model type.
Owen Park
Need a conversation? A chat or completion model. Need to search a knowledge base? Embeddings plus retrieval. Need to understand a photo in the same prompt as a question? A multimodal model. Need a new picture from a description? An image-generation model. Need audio in or out? Speech or a multimodal model that accepts audio.
Mira Cole
Capability first, cost second. A tiny cheap model that cannot see images will never win a 'describe this X-ray' scenario, no matter how pretty the price is.
Owen Park
Two deployment stories to keep straight. Serverless, sometimes called a token-billed endpoint: you pay for what you use, Foundry hosts the capacity, great for prototypes and spiky traffic. Managed compute: you provision dedicated capacity and pay for it whether it is busy or idle. Use that when you need reserved performance or isolation.
Mira Cole
Think of serverless as a metered taxi. Managed compute is leasing a car. The taxi is cheaper until you ride it all day.
Owen Park
Configuration knobs. Temperature and top-p steer variety. Higher temperature, more surprise, more risk of wandering. Lower temperature, more deterministic, better for extraction and classification. Max tokens caps how long the answer can be, which also caps cost.
Mira Cole
System prompt versus user prompt belongs with this objective even though the skills list also repeats it under Foundry implementation. System prompt is the standing job description: you are a claims assistant, never invent policy numbers. User prompt is today's question. Standing rules go in the system prompt so they are not overwritten by one enthusiastic user.
Owen Park
Zero-shot is just asking. Few-shot is showing two or three examples of the format you want. Chain-of-thought is asking the model to reason step by step on a multi-hop problem. Technique beats paragraph length.
Mira Cole
Question one. A generative model produces a confident biography of a person who does not exist. What best explains that behavior?
Owen Park
It predicts likely next tokens from patterns, not a guaranteed lookup of truth. Unless you ground it, fluency is not a citation.
Mira Cole
Question two. You must find the ten support tickets most similar to a new one. Which model capability do you need first?
Owen Park
Embeddings, so you can compare vectors. A chat model can summarize after you retrieve. It is not the similarity engine.
Mira Cole
Question three. An app has unpredictable student traffic and the team does not want idle virtual machines. Which deployment option fits?
Owen Park
A serverless, token-billed endpoint. Managed compute would sit there billing even at three in the morning.
Mira Cole
Question four. You need JSON with a fixed schema, same field order every time. What configuration change helps most?
Owen Park
Lower the temperature, and put the schema in the system prompt, maybe with a few-shot example. Do not crank temperature 'to make it creative.' Creative JSON is a bug.
Mira Cole
Question five. A scenario needs the model to answer questions about a photograph the user uploads. What do you choose?
Owen Park
A multimodal model that accepts image plus text. Not a text-only chat model, and not an image-generation model, which creates pictures instead of reading them.
Mira Cole
Next episode we put those models to work as workloads: generative, agentic, text analysis, speech, vision, information extraction. The exam loves 'here is a business story, name the workload.'
Owen Park
Bring a sorting hat. See you in four.