Track 01
AI Engineer
After 12 hours, an attendee can take an LLM agent from a working prototype to a production candidate: wrapped in a harness, traced and evaluated against a test suite, scoped and cost-capped with human oversight where it matters, and red-teamed with its findings closed. Each step is proved by a command that passes.
For engineers whose product is the AI itself. When the model misbehaves, the product misbehaves, and your users see it.
Who it is for
Engineers, mid to senior (2+ years shipping software), whose product depends on an LLM: they build agents, RAG pipelines or LLM features, own a codebase, and get paged when the model misbehaves. Comfortable in Python and a terminal.
Not sure it's your track?
If you use AI to write software faster but your product itself is not AI, Track 02 is where you belong. If your job is deciding what to build, what it costs and whether it clears the EU AI Act, look at Track 03.
What you build
A production agent that is evaluated, observable and hardened.
One customer-support agent for a supplied case company, built on by all four classes: class 1 builds it, class 2 traces and evaluates it, class 3 scopes and cost-caps it, class 4 attacks it and closes the findings.
AI Engineer Track · Certificate of Completion
Awarded for completing the four classes. No exam.
The four classes
1. Harness Engineering
By the end, the attendee can build the control loop around a model (a typed tool registry, file-backed memory, one sub-agent, and a permission interceptor that sees every tool call) and run the case-company support agent through five tickets with each tool call logged to SQLite.
You leave with: The running, deliberately over-capable agent (about 10 tools, permissive interceptor, memory, one sub-agent, audit log), tagged harness-v1. Check: one ticket resolved with at least two tool calls, every call logged, three supplied tests pass.
2. Evals & Observability for Agents
By the end, the attendee can trace every model and tool call of the class 1 agent with OpenTelemetry, build an eval suite of at least 20 cases (unit, trajectory and LLM-as-judge) that runs in under two minutes, and turn a failing trace into a new eval case.
You leave with: The eval suite and the trace viewer, tagged evals-v1. Check: `make evals` runs 20 or more cases in under two minutes; the attendee opens one trace and points at the tool call behind a failing case; one new case added from a real failure. This suite is the regression gate for classes 3 and 4.
3. AI Agents in Production
By the end, the attendee can turn the over-capable agent into one that can only do its job (tools cut to the task, god-mode credential replaced by a task-scoped expiring one, an irreversible permission tier, cost and action-rate ceilings, approvals routed by risk) and prove it with an automated governance audit while the class 2 eval suite still passes.
You leave with: The hardened agent, tagged hardened-v1. Check: the automated auditor scores at least 8/10, evals still green, one out-of-scope call blocked, one irreversible action stopped for approval, one ceiling tripped. Director's working title: 'AI Agents in Production: Control, Security & Costs'.
4. AI Security & Red Teaming
By the end, the attendee can run a structured red-team against their own hardened agent (direct and indirect prompt injection, tool-result poisoning, exfiltration, jailbreaks), write the findings up with severity and reproduction, and close each one with a guardrail that the eval suite and governance audit confirm did not break the agent.
You leave with: Red-team report plus closed findings, tagged final-v1. Check: report lists 5 or more findings with severity and reproduction, 3 or more closed with a guardrail, the runner shows before and after, evals and auditor still pass. CHECKS.md submitted.
Stack
Python, the Anthropic SDK behind a provider adapter, Pydantic, SQLite and OpenTelemetry. No agent framework: you build the loop by hand.
Before you come
- You write code regularly (Python or TypeScript ideal)
- You're comfortable in a terminal
- A laptop you can install software on: macOS, Linux or Windows 10+, 8 GB RAM, admin rights to install Python 3.11, Git and dependencies
- API credits for Anthropic or OpenAI, about 10 USD for the two days. A chat-only subscription (Claude Pro, ChatGPT Plus) is not enough
- recommendedYou've called an LLM API from your own code
- recommendedYou've deployed AI solutions to production
- recommendedYou've fine-tuned or served your own model
Questions about this track
Do I need an API key?
Yes. Every class calls a model from code, so you need about 10 USD of API credits with Anthropic or OpenAI. A chat subscription such as Claude Pro or ChatGPT Plus gives no API key.
Which agent framework do we use?
None. You build the agent loop by hand in Python with the Anthropic SDK behind a thin adapter, so the same code runs against an OpenAI-compatible endpoint with one config change.
Do I work on my own agent or on a case?
On a supplied case: a support agent for a B2B software company with 15 customers and 18 tickets. The last class closes with how to transfer each layer to your own agent.
the other tracks