Introduction to AI Engineering, How LLMs Work & Generative AI Fundamentals

AI engineering is the craft of building working software on top of large language models (LLMs) and other generative AI systems. This tutorial gives you the mental model an FDE needs: what the field is, what actually happens inside an LLM, and the core ideas behind generative AI.

Table of Contents

• What Is AI Engineering?
• Where AI Engineering Sits in an FDE's Toolkit
• Generative AI Fundamentals
• What an LLM Actually Is
• How LLMs Work: Tokens and Prediction
• Training vs. Inference
• Context Windows and Parameters

What Is AI Engineering?

AI engineering is the discipline of designing, building, and shipping applications where an LLM or other AI model does real work inside the system — answering questions, writing code, summarizing documents, or triggering actions.

It is not the same as training models. Most FDEs never train a model from scratch; they compose existing models with prompts, tools, data, and guardrails to solve a customer's problem. The skill is knowing how to make a model reliable enough to put in front of a real user.

Where AI Engineering Sits in an FDE's Toolkit

An FDE already knows programming, APIs, and customer discovery. AI engineering plugs into that toolkit rather than replacing it.

The trap is using AI where a if statement would do. AI engineering is for fuzzy, language-heavy problems — not for arithmetic or rigid rules.

Generative AI Fundamentals

Generative AI is a branch of AI that creates new content — text, images, code, audio — instead of just classifying or predicting a single label. An LLM generating an email is generative; a spam filter labeling an email as "spam" is not.

Why useful: generative models turn a natural-language request into a finished artifact, which is exactly what customers want — "draft the reply," "summarize the ticket," "write the migration script."

The two families an FDE meets most:

• Text generation (LLMs) — chat, summarization, code, extraction.
• Other modalities — image, speech, and embedding models (covered later in this category).

Note: "Generative" says what the model produces. It does not mean the output is correct — that gap is the entire subject of evaluation and guardrails (see Tutorial 9).

What an LLM Actually Is

A Large Language Model is a giant statistical function trained on huge amounts of text. Feed it some text, and it returns the most likely text that should follow.

It has no database, no beliefs, and no memory between calls. Everything it "knows" lives in the numerical weights it learned during training. When you chat with it, you are prompting that function — not querying a brain.

How LLMs Work: Tokens and Prediction

LLMs work by predicting one token at a time. A token is a chunk of text — roughly 4 characters or ¾ of a word in English. "Learning" is 1 token; "unbelievable" might be 2.

The model does this:

1. Take the input text (your prompt).
2. Predict the next most likely token.
3. Append that token and repeat until it stops.

Input:  "The capital of France is"
Model predicts next token:  " Paris"
Now input becomes: "The capital of France is Paris"
Model predicts next: "."  ... and so on.

Explanation: this is next-token prediction, run in a loop. The model never "decides the full sentence up front" — it commits to one token, then re-reads everything including that token to pick the next.

Best Practice: Remember the model predicts, it doesn't retrieve. If you need facts it wasn't trained on, you must supply them (that's what RAG does — Tutorial 6).

Training vs. Inference

Two phases, often confused:

• Training — the expensive, one-time process where the model learns patterns from massive text. Done by labs, not FDEs.
• Inference — the cheap, per-request process of running the trained model to get a prediction. This is what you do every time you call an API.

Training:  read billions of documents  ->  produce model weights
Inference: prompt + weights           ->  generated text (what you pay per call)

Common Mistake: Assuming the model "remembers" what you told it last week. It doesn't — each API call is independent unless you send the history back in.

Context Windows and Parameters

Two numbers you'll see on every model card:

• Context window — how many tokens (prompt + response) the model can see at once. A 200k-token window fits a small library; a 4k window fits a few pages. Overflow and the earliest text gets dropped.
• Parameters — the count of learned numbers (e.g. 7B, 70B). Roughly, more parameters = more capability but more cost and latency.

Model: "Sonnet"    Context: 200,000 tokens    Used: prompt(1,200) + reply(300) = 1,500 tokens billed

Note: Bigger context is not free memory. The model still predicts token-by-token; a huge context just means more text to read each step, which costs time and money.