AI Beyond Fundamentals › Module 0 › Lesson

Module 0 · Setup

Setup and the Course Repo

This module doesn't ship a feature — it builds the infrastructure every later module stands on. The folder layout, the .env convention, and above all shared/llm_client.py (one function that calls any provider and logs every call) aren't throwaway setup steps; they're the shared architecture that Module 1's cost meter, Module 4's budget alerts, and every project after that will import and extend instead of rebuilding. Get this module right and the rest of the course gets easier, not harder.

Everything below assumes Python 3.11+ and a terminal. Unless a step says otherwise, run every command from the ai-beyond-fundamentals/ repo root you create in Lab 1, with its venv activated — that's what makes relative paths like chatbot/three_providers.py resolve correctly. Work through the concepts once, then do the labs in order — each one builds on the last.

StepApprox. time
Lab 1 · Scaffold the repo (includes creating three API keys)1 to 2 hours
Lab 2 · Three providers, one prompt30 to 60 min
Lab 3 · Run a model locallyabout 18 to 36 min
Lab 4 · Build the loggerabout 13 to 26 min
Project · One client, four providers (with the README)about 17 to 34 min
Module totalabout 2 to 4 hours

The low end is a timed first run through the module with Python and Ollama already installed. The high end doubles it for a first-time learner who has to install tools, create accounts and read the code as they go.

Concepts

Keep API keys in environment files, never in code

A provider API key is a bearer credential: whoever holds the string can spend money on your account. If it's typed directly into a .py file, it ships the moment that file is committed to git — including to a public GitHub repo, a screen share, or a zip you email someone. Leaked keys get scraped by bots within minutes and run up real charges.

The fix is a .env file at the repo root holding NAME=value pairs, loaded at runtime by a library like python-dotenv. The file never gets committed — it's listed in .gitignore before it ever holds a real key. Collaborators instead get a .env.example with the same variable names and blank or placeholder values, committed normally, so they know what to fill in without ever seeing yours.

Python environments with venv and pip, in a monorepo layout

venv is Python's built-in tool for creating an isolated environment so this project's packages don't collide with anything else installed on your machine. pip installs into it.

If you write anthropic in a requirements file with no version attached, pip install grabs whatever the newest release happens to be on the day it runs — today that might work, three months from now a new major version could change the SDK's method names and break every lab that imports it, with no code change on your end to explain why. "Pinning" means locking each package to the exact version number you tested with, so installing never silently changes what you're running.

Neither venv nor pip tracks those exact versions for you automatically (this is what uv or poetry would do instead). You do it by hand with pip freeze, which inspects everything currently installed in the active environment and prints each package with an exact == version:

$ pip freeze anthropic==2.3.0 openai==3.37.0 google-genai==2.33.0 python-dotenv==1.2.4 ...

Redirecting that output into requirements.txt and committing the file turns it into the pin list. From then on, pip install -r requirements.txt installs exactly those versions, so your setup, your collaborator's, and the one this checklist describes all run identical code.

Prefer uv or poetry? This course uses venv and pip because they ship with Python and need nothing extra installed. uv and poetry are popular alternatives that create the environment, install packages and record exact versions in a lockfile for you. If you already use one, it works for every lab: translate the install commands and skip the manual pip freeze step.

A monorepo here just means one repository, one pyproject.toml, and one lockfile for every project this course builds — the chatbot, the extraction tool, the RAG app, the eval suite, the agent, and the RAG-vs-fine-tune comparison — each in its own folder, plus a shared/ folder for code every project reuses. One environment to install instead of seven, and one place to fix a dependency problem instead of seven.

ai-beyond-fundamentals/ (one repo, one venv, one lockfile) shared/ llm_client.py, calls.jsonl chatbot/ extraction/ rag/ evals/ agent/ ragvsfinetune/ import Root files .env (ignored by git) · .env.example · .gitignore · requirements.txt
One environment to install, one place to fix a dependency problem, one shared logger.

A shared client wrapper that logs every call

Every lab from here on calls a model. If each lab calls the provider SDK directly, cost and latency tracking has to be rebuilt seven times — or, more likely, never gets built at all. Instead, this module produces one function, call(prompt, model), that every later lab imports from shared/. It hides which provider you're talking to and silently appends a row — model, tokens in, tokens out, milliseconds, dollars — to a shared log file on every call.

That log file is what makes Module 1's cost meter, Module 4's budget alerts, and Module 4's regression tests possible without extra plumbing. You're building the instrument panel before you need to read it.

Your lab code chatbot/ rag/ agent/ call(prompt, model) shared/llm_client.py Anthropic OpenAI Gemini Ollama (local) One log row per call model, tokens in/out, ms, $ shared/calls.jsonl
One function in front of every provider: the labs never import a provider SDK directly.

Hands-on Labs

  1. Create the repo and one virtual environment at the root — every project folder below shares it, so you activate it once per terminal session, not once per folder.
    mkdir ai-beyond-fundamentals cd ai-beyond-fundamentals
    macOS / Linuxpython3 -m venv .venv source .venv/bin/activate
    Windows (PowerShell)python -m venv .venv .venv\Scripts\Activate.ps1
    What does python3 -m venv .venv do?

    It creates an isolated Python environment inside a folder named .venv. Breaking the command down:

    • python3 runs your system's Python interpreter.
    • -m venv tells Python to run a standard-library module as a program. The module is venv, which ships with Python, so there is nothing to install.
    • .venv is the folder where the new environment goes. The leading dot hides it in Finder and in a plain ls; that is just convention.

    What ends up inside .venv/:

    • bin/python, a private link to the interpreter. This is what runs once you activate.
    • bin/pip, an installer that puts packages in .venv/lib/.../site-packages instead of your system Python.
    • bin/activate, the script source .venv/bin/activate runs. It puts .venv/bin first on your PATH, so typing python or pip now means the private ones.

    Why bother: packages installed here never touch your system Python or other projects, other projects can't break this one by upgrading a shared package, and the pinned versions in requirements.txt stay exactly what you tested with.

    Creating the venv does not activate it. Run source .venv/bin/activate in every new terminal. You'll see (.venv) at the start of your prompt when it's on. Check with which python: the path should end in .venv/bin/python, not /usr/bin/python3. .venv is listed in .gitignore because it is large and machine-specific; anyone else rebuilds it from requirements.txt.

  2. Build the project skeleton. Deliberately no shared/__init__.py here — every file inside shared/ imports its siblings with bare names like from llm_log import log_call, which only resolves if shared/ itself sits directly on sys.path. Adding __init__.py turns shared into a real package, which changes how pytest resolves its imports in Lab 4 and breaks exactly that.
    macOS / Linuxmkdir -p shared chatbot extraction rag evals agent ragvsfinetune
    Windows (PowerShell)mkdir shared, chatbot, extraction, rag, evals, agent, ragvsfinetune
    Why does macOS use mkdir -p but PowerShell doesn't?

    They are two different programs that happen to share a name.

    macOS / Linux: mkdir -p

    • mkdir is a small program from the Unix toolset, and it takes flags.
    • By default it makes one folder at a time, and it errors if the parent is missing or the folder already exists.
    • -p (“parents”) changes that in two ways: it creates any missing parent folders, and it does not error if the folder already exists.
    • -p is also what makes listing several folders in one command work, as this step does.

    Windows PowerShell: mkdir shared, chatbot, ...

    • In PowerShell, mkdir is a built-in function that wraps New-Item -ItemType Directory.
    • That command already creates missing parents and already tolerates existing folders, so there is nothing for a -p flag to turn on.
    • PowerShell takes multiple names as a comma-separated list, because it passes them as a list of values rather than as separate words.

    One more difference: PowerShell commands usually take full-word parameters like -Path or -Force, not single-letter Unix flags. Typing mkdir -p in PowerShell would likely fail or be read as an abbreviated parameter name.

  3. Install the packages every later module needs, then freeze the exact versions so the repo pins what you tested with.
    pip install anthropic openai google-genai python-dotenv pyyaml pytest pip freeze > requirements.txt
  4. Now three separate files, all at the repo root (ai-beyond-fundamentals/, next to .venv). Start with .gitignore, so the next two are already covered by it the moment they exist.
    macOS / Linuxcat > .gitignore <<'EOF' .env .venv __pycache__/ *.log *.jsonl EOF
    Windows (PowerShell)@' .env .venv __pycache__/ *.log *.jsonl '@ | Set-Content .gitignore
  5. .env.example — the committed template, values left blank, so a collaborator knows which keys to set without ever seeing yours.
    macOS / Linuxcat > .env.example <<'EOF' ANTHROPIC_API_KEY= OPENAI_API_KEY= GEMINI_API_KEY= EOF
    Windows (PowerShell)@' ANTHROPIC_API_KEY= OPENAI_API_KEY= GEMINI_API_KEY= '@ | Set-Content .env.example
  6. Get a key from each provider's dashboard before you can fill in .env. Each needs its own account; Anthropic and OpenAI also require billing to be set up before a new key will actually work.

    Each key is shown to you once. Copy it somewhere safe immediately — if you lose it, the dashboard can only let you revoke it and create a new one, not show it to you again.

  7. .env — your real keys. Copy the template, then open the copy directly from the terminal — a filename starting with a dot is hidden by default in Finder and File Explorer, so double-clicking to find it in a file browser often won't work.
    macOS / Linuxcp .env.example .env code .env # VS Code # or: nano .env (terminal editor, Ctrl+O to save, Ctrl+X to exit) # or: open .env (macOS — opens in whatever app handles plain text)
    Windows (PowerShell)Copy-Item .env.example .env code .env # VS Code # or: notepad .env
    Paste each key in after its =, one per line, no quotes and no spaces around the =, so it reads:
    ANTHROPIC_API_KEY=sk-ant-api03-... OPENAI_API_KEY=sk-proj-... GEMINI_API_KEY=AIza...
    Save the file. It's never committed — .gitignore already excludes it.
  8. Commit the skeleton.
    git init git add . git commit -m "Scaffold course repo"
Using a coding agent instead? Hand it this:
Scaffold a Python monorepo called ai-beyond-fundamentals with a single venv at the repo root, created with whichever Python command is correct for my OS. Create these folders: shared, chatbot, extraction, rag, evals, agent, ragvsfinetune. Install these dependencies with pip and freeze them into requirements.txt: anthropic, openai, google-genai, python-dotenv, pyyaml, pytest. Create a .gitignore that excludes .env, .venv, __pycache__ and log files, then a .env.example with blank ANTHROPIC_API_KEY, OPENAI_API_KEY and GEMINI_API_KEY entries so I know which keys to fill in. Do not create a .env file yourself — you don't have my real API keys, so leave that file for me to create by copying .env.example and filling it in by hand.
Check: git status shows .env as ignored, not untracked. If it shows up as untracked, fix .gitignore before continuing — don't just avoid committing it this once.
Check before you paste

Model names below can go stale within weeks — providers rename, deprecate and retire models on their own schedule, not this lesson's. Writing this exact lab, we hit both a deprecated Claude model and a Gemini model that 404'd for new accounts. Check each provider's current model list first (Anthropic, OpenAI, Gemini) — or skip the docs and ask the SDK directly, the trick that found this lesson's working Gemini model after gemini-2.5-flash 404'd on us:

from google import genai client = genai.Client(api_key="...") for m in client.models.list(): print(m.name)

If a model name anywhere in this module throws a 404 or a deprecation warning, swap in whatever's current and keep going — nothing in the lesson's logic depends on the exact name.

  1. Create a new file at chatbot/three_providers.py (from the repo root: code chatbot/three_providers.py for VS Code, nano chatbot/three_providers.py for a terminal editor, or touch chatbot/three_providers.py on macOS/Linux / New-Item chatbot\three_providers.py on Windows then open it in your editor — code and nano both create the file themselves, no touch needed first). Paste in one function per provider. Each prints the model's actual reply — not just its usage — so a wrong or empty answer is visible immediately instead of hiding behind a token count that looks fine on its own.
    import json import os from datetime import datetime from dotenv import load_dotenv load_dotenv() PROMPT = "Explain what a token is, in three sentences." def call_anthropic(): from anthropic import Anthropic client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"]) resp = client.messages.create( model="claude-haiku-5-5", max_tokens=200, thinking={"type": "disabled"}, # otherwise Haiku can spend the whole token budget thinking and return no text messages=[{"role": "user", "content": PROMPT}], ) # a "thinking" response can put a ThinkingBlock before the text block, # so pick the block that actually is text rather than assuming index 0 text = next(b.text for b in resp.content if b.type == "text") return {"text": text, "usage": resp.usage.model_dump(mode="json", exclude_none=True)} def call_openai(): from openai import OpenAI client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) resp = client.chat.completions.create( model="gpt-5.4-mini", messages=[{"role": "user", "content": PROMPT}], ) return { "text": resp.choices[0].message.content, "usage": resp.usage.model_dump(mode="json", exclude_none=True), } def call_gemini(): from google import genai from google.genai import types client = genai.Client( api_key=os.environ["GEMINI_API_KEY"], # milliseconds; without a timeout the SDK keeps retrying an overloaded model http_options=types.HttpOptions(timeout=20000), ) resp = client.models.generate_content(model="gemini-flash-lite-latest", contents=PROMPT) return { "text": resp.text, "usage": resp.usage_metadata.model_dump(mode="json", exclude_none=True), } if __name__ == "__main__": results = {} for name, call in [("anthropic", call_anthropic), ("openai", call_openai), ("gemini", call_gemini)]: try: results[name] = call() except Exception as e: # one provider failing (a 503, say) shouldn't lose the other two answers results[name] = {"error": f"{type(e).__name__}: {e}"} output = json.dumps(results, indent=2) print(output) filename = f"lab2-output-{datetime.now():%Y%m%d-%H%M%S}.json" # a new file every run with open(filename, "w") as f: f.write(output + "\n") print("saved to", filename)
  2. Run it. The script prints the results as JSON and also saves them to a new timestamped file such as lab2-output-20261025-125400.json (every run keeps its own file, so earlier results are never overwritten), so you can open a file in your editor (VS Code folds and colors JSON) or compare runs. Read all three answers, then the three usage objects. If a provider fails, its entry shows an error instead of the answer, and the other two still come through.
    python chatbot/three_providers.py
    If Gemini returns a 503 ("high demand") or a 504 (deadline exceeded), or the run seems stuck, that's the model being overloaded, not your code. The timeout turns a stuck call into an error entry in the JSON; rerun after a few seconds. We use gemini-flash-lite-latest because the larger flash models were overloaded when we wrote this lab and the lite model answered in under a second. Swap the name if a model is busy.
  3. Each SDK names its fields differently. At the bottom of chatbot/three_providers.py, add a comment recording all three shapes exactly as they appear in the JSON — you'll be back in this file for Lab 4, writing the code that normalizes them into one shape.
    # usage field names, from the lab2-output-*.json file: # anthropic: input_tokens / output_tokens # openai: prompt_tokens / completion_tokens # gemini: prompt_token_count / candidates_token_count
Check: all three answers actually explain what a token is (not an error message or an empty string), and each usage object shows a non-zero input token count. A 401 means the key name in .env doesn't match what the code reads, or the key is stale: load_dotenv() does not override a key already exported in your shell, so a terminal window opened before you rotated a key keeps using the old one. Open a fresh terminal window (or run source ~/.zshrc) and retry; a 429 means you're rate-limited — wait a minute and retry.
  1. Install Ollama.
    macOS, via Homebrewbrew install ollama ollama serve # leave running in its own terminal
    Linuxcurl -fsSL https://ollama.com/install.sh | sh ollama serve # leave running in its own terminal
    Windows# Download the installer from ollama.com/download and run it — # it installs Ollama and starts it running in the background automatically.
  2. Pull a small model and sanity-check it from the command line. llama3.2 downloads about 2 GB (it defaults to the 3B variant); check you have that much free disk space first, or pull the smaller llama3.2:1b tag instead (about 1.3 GB) if you're tight on room. This can be run from any directory — Ollama stores pulled models once in your home directory (~/.ollama on macOS/Linux, %USERPROFILE%\.ollama on Windows), not inside the project repo, so it doesn't matter where your terminal's cwd is for this step.
    ollama pull llama3.2 ollama run llama3.2 "Explain what a token is, in three sentences."
  3. Back in chatbot/three_providers.py, add a fourth function, call_ollama(), alongside the three from Lab 2 — same SDK as call_openai, pointed at a different base_url so it talks to your local Ollama server instead of OpenAI's. Add ("ollama", call_ollama) to the list in the __main__ loop so it lands in the output file with the other three. It returns usage as None if your local server sends none, instead of crashing.
    def call_ollama(): from openai import OpenAI local = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") # key is unused but required by the SDK resp = local.chat.completions.create( model="llama3.2", # or whichever model `ollama list` shows on your machine messages=[{"role": "user", "content": PROMPT}], ) usage = resp.usage.model_dump(mode="json", exclude_none=True) if resp.usage else None return {"text": resp.choices[0].message.content, "usage": usage}
Check: the local call returns a real answer with no network call to a provider — turn off Wi-Fi and rerun it to prove it. Note whether resp.usage comes back populated; some local setups return None, which the logger in Lab 4 needs to handle without crashing.
  1. Create a new file at shared/llm_log.py (code shared/llm_log.py, nano shared/llm_log.py, or touch shared/llm_log.py / New-Item shared\llm_log.py then open it) and paste in:
    import json, time, pathlib LOG_PATH = pathlib.Path(__file__).parent / "calls.jsonl" def log_call(provider, model, input_tokens, output_tokens, elapsed_ms, cost_usd): row = { "ts": time.time(), "provider": provider, "model": model, "input_tokens": input_tokens or 0, "output_tokens": output_tokens or 0, "elapsed_ms": round(elapsed_ms, 1), "cost_usd": round(cost_usd, 6), } with LOG_PATH.open("a") as f: f.write(json.dumps(row) + "\n")
  2. Open chatbot/three_providers.py again. Because log_call lives in shared/, a sibling folder, add these two lines near the top of the file (below your other imports) so Python can find it:
    # top of chatbot/three_providers.py, added in Lab 4 import sys, pathlib, time sys.path.insert(0, str(pathlib.Path(__file__).resolve().parent.parent / "shared")) from llm_log import log_call
    Then change each of the four functions the same way: start a timer, call the SDK, normalize that provider's usage object into plain ints using the field names from your Lab 2 comment, and call log_call(...) with a flat cost_usd=0 for now (the project below replaces that with a real number). Keep the existing return — logging is in addition to saving the answer to the JSON file, not instead of it. Here's call_anthropic before and after — apply the same change to call_openai, call_gemini and call_ollama, each with its own field names:
    # before (Lab 2) def call_anthropic(): from anthropic import Anthropic client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"]) resp = client.messages.create( model="claude-haiku-5-5", max_tokens=200, thinking={"type": "disabled"}, # otherwise Haiku can spend the whole token budget thinking and return no text messages=[{"role": "user", "content": PROMPT}], ) text = next(b.text for b in resp.content if b.type == "text") return {"text": text, "usage": resp.usage.model_dump(mode="json", exclude_none=True)} # after (Lab 4) def call_anthropic(): from anthropic import Anthropic client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"]) start = time.time() resp = client.messages.create( model="claude-haiku-5-5", max_tokens=200, thinking={"type": "disabled"}, # otherwise Haiku can spend the whole token budget thinking and return no text messages=[{"role": "user", "content": PROMPT}], ) elapsed_ms = (time.time() - start) * 1000 text = next(b.text for b in resp.content if b.type == "text") log_call("anthropic", "claude-haiku-5-5", resp.usage.input_tokens, resp.usage.output_tokens, elapsed_ms, cost_usd=0) return {"text": text, "usage": resp.usage.model_dump(mode="json", exclude_none=True)}
    call_ollama needs one extra guard. Ollama's OpenAI-compatible endpoint can return resp.usage as None instead of a real usage object (unlike OpenAI itself, which always fills it in) — and reading resp.usage.prompt_tokens straight off a None crashes with AttributeError: 'NoneType' object has no attribute 'prompt_tokens'. Your Lab 3 code already guards the model_dump call for this reason; the logging needs the same guard. Check resp.usage before reading a field off it, and fall back to 0 when it's missing. X if condition else Y is Python's inline if/else — it does that check in one line instead of a separate if block:
    # call_ollama, with the Lab 4 timer/logging added def call_ollama(): from openai import OpenAI local = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") start = time.time() resp = local.chat.completions.create( model="qwen3-vl:4b-instruct", # the model you pulled in Lab 3 messages=[{"role": "user", "content": PROMPT}], ) elapsed_ms = (time.time() - start) * 1000 input_tokens = resp.usage.prompt_tokens if resp.usage else 0 output_tokens = resp.usage.completion_tokens if resp.usage else 0 log_call("ollama", "qwen3-vl:4b-instruct", input_tokens, output_tokens, elapsed_ms, cost_usd=0) usage = resp.usage.model_dump(mode="json", exclude_none=True) if resp.usage else None return {"text": resp.choices[0].message.content, "usage": usage}
  3. Create shared/test_logger.py the same way, and paste in a checkpoint test.
    # shared/test_logger.py import json from llm_log import LOG_PATH def test_all_providers_logged(): rows = [json.loads(line) for line in LOG_PATH.read_text().splitlines()] providers = {row["provider"] for row in rows} assert providers == {"anthropic", "openai", "gemini", "ollama"}
  4. Run everything and then the test.
    python chatbot/three_providers.py pytest shared/test_logger.py
Check: cat shared/calls.jsonl shows four lines, one per provider, and pytest passes.

Project

Project · One client, four providers

Combine the four labs into one file, shared/llm_client.py: a single call(prompt, model) function that looks up the provider for a given model name in a config file, dispatches to the right SDK, times the call, computes real cost, and logs it through log_call from Lab 4. The five steps below build it piece by piece.

  1. Create shared/models.yaml (code shared/models.yaml, nano shared/models.yaml, or touch shared/models.yaml / New-Item shared\models.yaml then open it) and paste in one entry per model. These prices are what the providers' pricing pages listed in October 2026 — prices change, so check each page (Anthropic, OpenAI, Gemini) before trusting them.
    # shared/models.yaml claude-haiku-5-5: provider: anthropic price_in: 0.10 # $ per 1M input tokens (prompts up to 100k tokens) price_out: 0.50 # $ per 1M output tokens gpt-5.4-mini: provider: openai price_in: 0.75 price_out: 4.50 gemini-flash-lite-latest: provider: gemini price_in: 0.30 # the page lists Gemini 3.5 Flash-Lite at these rates; the alias target is not stated price_out: 2.50 # Gemini's output price includes thinking tokens qwen3-vl:4b-instruct: provider: ollama price_in: 0 # runs on your machine: no per-token charge price_out: 0
    Use whatever model name you actually pulled in Lab 3 and have been calling since — if you pulled something other than qwen3-vl:4b-instruct there, use that same name here and in every file below, not the name printed in this lesson.
  2. Create shared/llm_client.py the same way, and paste in the client itself.
    # shared/llm_client.py import os, time, random, pathlib, yaml from llm_log import log_call CONFIG = yaml.safe_load((pathlib.Path(__file__).parent / "models.yaml").read_text()) RATE_LIMIT_ERRORS = () # fill in with each SDK's RateLimitError class def call(prompt, model, max_attempts=5): cfg = CONFIG[model] start = time.time() for attempt in range(max_attempts): try: text, usage = _dispatch(cfg["provider"], model, prompt) break except RATE_LIMIT_ERRORS: if attempt == max_attempts - 1: raise time.sleep(2 ** attempt + random.random()) elapsed_ms = (time.time() - start) * 1000 cost = (usage["input_tokens"] / 1_000_000 * cfg["price_in"] + usage["output_tokens"] / 1_000_000 * cfg["price_out"]) log_call(cfg["provider"], model, usage["input_tokens"], usage["output_tokens"], elapsed_ms, cost) return text def _dispatch(provider, model, prompt): # one branch per provider, reusing the call_* functions from Lab 2 & 3 — # each must return (text, {"input_tokens": int, "output_tokens": int}) ...
  3. Fill in _dispatch. Copy the logic from the four call_* functions in chatbot/three_providers.py (Labs 2 & 3) into one if provider == "..." branch each — you can leave the originals in three_providers.py alone, they still work as a standalone reference. Two changes on the way in: drop the timer/log_call/print lines you added in Lab 4 (this file's call() already times and logs centrally, so keeping both would log every call twice), and return the actual reply text alongside a plain usage dict instead of printing.
    # shared/llm_client.py — _dispatch, filled in def _dispatch(provider, model, prompt): if provider == "anthropic": from anthropic import Anthropic client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"]) resp = client.messages.create( model=model, max_tokens=200, thinking={"type": "disabled"}, messages=[{"role": "user", "content": prompt}], ) # same ThinkingBlock gotcha as Lab 2 — don't assume index 0 is the text block text = next(b.text for b in resp.content if b.type == "text") return text, { "input_tokens": resp.usage.input_tokens, "output_tokens": resp.usage.output_tokens, } if provider == "openai": from openai import OpenAI client = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) resp = client.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], ) return resp.choices[0].message.content, { "input_tokens": resp.usage.prompt_tokens, "output_tokens": resp.usage.completion_tokens, } if provider == "gemini": from google import genai from google.genai import types client = genai.Client( api_key=os.environ["GEMINI_API_KEY"], http_options=types.HttpOptions(timeout=20000), # milliseconds, as in Lab 2 ) resp = client.models.generate_content(model=model, contents=prompt) return resp.text, { "input_tokens": resp.usage_metadata.prompt_token_count, "output_tokens": resp.usage_metadata.candidates_token_count, } if provider == "ollama": from openai import OpenAI local = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") resp = local.chat.completions.create( model=model, messages=[{"role": "user", "content": prompt}], ) usage = resp.usage return resp.choices[0].message.content, { "input_tokens": usage.prompt_tokens if usage else 0, "output_tokens": usage.completion_tokens if usage else 0, } raise ValueError(f"Unknown provider: {provider}")
  4. Fill in RATE_LIMIT_ERRORS at the top of the file, above CONFIG, with the real rate-limit exception class from each SDK.
    # shared/llm_client.py — add near the top, replacing the placeholder from anthropic import RateLimitError as AnthropicRateLimitError from openai import RateLimitError as OpenAIRateLimitError from google.genai.errors import ClientError as GeminiClientError RATE_LIMIT_ERRORS = (AnthropicRateLimitError, OpenAIRateLimitError, GeminiClientError) # GeminiClientError covers other 4xx errors too (bad request, auth), not # just 429 — check e.code == 429 inside the except block if you want to be # precise. Good enough to make the retry loop fire for this exercise.
    The agent prompt below can do this step for you if you'd rather not hunt through each SDK's docs for the exact class names.
  5. Test it end to end. Write a small test script, shared/try_client.py, that calls call(prompt, model) once for each of the four models. call() returns the reply instead of printing it — that's the point, so other code can use the answer instead of just displaying it — so this script's own print(...) is the only place any of these four answers actually reaches your terminal. If you want to see what a provider is really responding with, this is where you look; nothing upstream in _dispatch prints anything.
    # shared/try_client.py — a quick smoke test: one call to each model from dotenv import load_dotenv load_dotenv() from llm_client import call for model in ["claude-haiku-5-5", "gpt-5.4-mini", "gemini-flash-lite-latest", "qwen3-vl:4b-instruct"]: print(model, "->", call("Give me five different greetings, one per line.", model))
    python shared/try_client.py cat shared/calls.jsonl
    Read the four printed replies first — five greetings back from each model, not an error string or an empty reply — then confirm calls.jsonl has four new rows with a real, non-zero cost_usd for every model except llama3.2, which should show 0.
Worth knowing

You just hand-built a small version of what LiteLLM is as a library: one call() across 100+ providers, with cost tracking, retries and logging already built in. A real production codebase would likely reach for LiteLLM (or something like it) instead of this file. You built it by hand here so that choice is a shortcut once you understand what it's hiding, not a black box you're trusting blind.

Stuck on the retry/backoff branch? Hand your agent this:
Here is my shared/llm_client.py (paste the file). It should retry on a 429 rate-limit error from any of the anthropic, openai, or google-genai SDKs with exponential backoff, and give up after 5 attempts by re-raising the original error. Fill in RATE_LIMIT_ERRORS with the correct exception classes for all three SDKs and fix _dispatch so it calls the right one based on cfg["provider"]. Don't change the function signatures in llm_log.py or models.yaml.

Done when:

  • Swapping providers means changing one line in models.yaml, not code.
  • pytest shared/test_logger.py passes, showing a log row for all four providers.
  • Optional: a simulated 429 (raise the error manually in a test) triggers a retry with increasing delay, and a 6th consecutive failure still raises.
  • A README in the repo root explains the folder layout well enough that Module 1 can start from it without asking you a question.
← Back to the course map