9. ChatGPT & Large Language Models

Generative AI in Practice

What are LLMs?

Large Language Models (LLMs) are neural networks with billions of parameters trained on massive text corpora. They can generate coherent text, answer questions, translate languages, write code, and more — all by predicting the next token autoregressively.

ChatGPT Training Pipeline

1. Pre-training — GPT model trained on internet text (next token prediction). This is the most expensive stage, consuming thousands of GPUs for weeks.
2. Supervised Fine-Tuning (SFT) — Fine-tune on high-quality (prompt, response) pairs crafted by human labelers
3. Reward Modeling — Train a reward model on human preferences (rankings of outputs)
4. RLHF (PPO) — Optimize the SFT model against the reward model with a KL penalty for stability

Prompt Engineering

Techniques to get better outputs from LLMs:

• Zero-shot — Just ask without examples ("Summarize this article: ...")
• Few-shot — Provide 2-3 examples in the prompt before the real query
• Chain-of-Thought (CoT) — Ask the model to think step-by-step, improving reasoning
• System Prompts — Set the behavior, tone, and constraints at the start of the conversation
• Role Prompting — "Act as an expert programmer..." improves code quality

# Prompt template
prompt = """You are a helpful coding assistant.
Explain the following code in simple terms:

```
def fibonacci(n):
    if n <= 1: return n
    return fibonacci(n-1) + fibonacci(n-2)
```
"""

Capabilities

• Text generation, summarization, translation, question answering
• Code generation and explanation (Python, JS, SQL, etc.)
• Mathematical reasoning and problem solving
• Creative writing (poems, stories, scripts)
• Role-playing and conversation

Limitations

• Hallucinations — Confidently generating incorrect information
• Context Window — Limited attention span (4K-200K tokens depending on model)
• Outdated Knowledge — Training cutoff means no awareness of recent events
• No True Understanding — Statistical patterns, not genuine comprehension
• Bias — Inherits biases from training data