AI Engineering Lab

I’ve spent ten years building web platforms: Angular frontends, Node.js microservices, serverless backends. More and more products now ship with an LLM somewhere in the loop, and I wanted to understand that layer as well as I understand the rest of the stack. Calling an API in a tutorial isn’t enough for that. So I wrote myself a curriculum and I’m working through it, alongside a full-time job.

What I can build today

Six modules in, this is what I’ve built and measured myself, not just read about:

• A resilient multi-provider LLM client in TypeScript: Gemini, local models through Ollama, and OpenRouter, with retries, backoff with jitter, streaming, cancellation, cost estimates and model routing. 129 automated checks, offline and live.
• Hybrid search on PostgreSQL + pgvector: vector and full-text search fused with Reciprocal Rank Fusion, with recall@10 and MRR measured on a test set.
• Agentic retrieval: a multi-step loop where a model judges what it found and searches again for what’s missing, plus Corrective RAG (grading every retrieved chunk) and Self-RAG (checking the answer after writing it).
• Tool calling written by hand: an assistant that answers HR questions by combining company policies with the user’s own calendar, balances and expense claims, and exposes the same tools through an MCP server.

How the path is built

Sixteen modules, from LLM foundations through retrieval, tool calling and MCP, agents, evaluation, observability, guardrails, serving, fine-tuning and LLMOps. A few rules keep it honest:

• Every exercise produces a measurement. Latency, token counts, pass rates, cost. The output of a module is working code plus a CSV, not a certificate.
• By hand before the framework. Wherever a framework shows up, the first version is written without it. I use LangChain only as a thin client for the models; the loops, the retrieval and the decisions are my own code.
• TypeScript by default, Python where the ecosystem demands it (for example fine-tuning). After module 03 I rebuilt everything so far in TypeScript with no AI help, to check that the knowledge had really stuck.
• Declared AI assistance. Each exercise states what I can delegate to a coding assistant (boilerplate, HTTP clients, charts) and what I can’t (parameter choices, prompt and tool design, reading the results). Every log records who wrote what.
• Notes written without AI. Every module ends with questions I answer from memory. If I can’t explain it without help, the module isn’t done.
• Cut the volume, keep the measurements. In October I trimmed each module down to the exercises that measure something or produce a piece I’ll reuse. Scoping is part of the job too.

The risk isn’t using an assistant. The risk is finishing with three working projects and no transferred skill.

What the numbers said so far

Foundations and prompting

Tokenization isn’t language-neutral. The same sentence costs 10 tokens in English and 15 in Italian, and rare Italian words break into up to seven sub-tokens. Anyone building for Italian users pays that difference on every request.

Temperature 0 means deterministic, not correct. Asked about the original members of the European Union, the model included Austria (which joined in 1995) in all five runs at temperature 0. Greedy decoding makes the output consistent. It doesn’t make it true.

Prompt order is a latency decision. With a ~6,200-token prompt on a local Qwen, putting a changing value at the start forces a full prefill on every request: about 30 s. With the stable prefix first, prefill drops to 0.2–0.4 s. Same model, same hardware; only the order of the prompt changed.

A schema that validates can still say nothing. Extracting fields into a strict schema with small free models, every response validated, and almost every field was not_determinable. The schema had given the model a valid way to give up. Fill rate has to be measured next to validation rate.

Production plumbing

Streaming doesn’t fix a long prompt. With a 5,700-token document, the time to the first token was 98% of the total. When most of the wait happens before the first token, streaming barely helps; caching or less context does.

SDK defaults are behaviour. Reading the SDK sources during the TypeScript recap, I found that the openai client silently retries twice on its own. Stacked on my retry layer, one failing request became up to nine calls, and my latency numbers included retries I couldn’t see. The same SDK ends a cancelled stream without an error, so my summary reported “completed” when the user had pressed Ctrl+C. Neither shows up in a quick demo. Both show up in production.

Retrieval

Hybrid search isn’t automatically better. On 20 test queries over a company-policy corpus, lexical search alone put the right document first 20% of the time, and vector search 60%. Hybrid with RRF also scored 60%, with the same perfect recall@10. On this corpus it added cost without adding accuracy. It’s a small set, so I read it as a direction, but it’s exactly the measurement that should come before adding complexity.

The agent was only as good as its judge. An agentic retrieval loop used 4–5× the tokens of single-shot RAG. With a local 8B model as the judge it answered “I don’t know” on three multi-step questions, even though the right passage was already in its context. With a stronger judge model, the same loop solved them. The bottleneck wasn’t the loop design; it was the model deciding when to stop.

Graders need grading too. For Corrective RAG I compared two models scoring each retrieved chunk. A small local grader rated almost everything close to 1.0, and in another setup threw away the one chunk that held the answer (score 0.02). A hosted grader gave that chunk 0.97 and discarded 23 of 65 irrelevant chunks. The pattern is only as reliable as the model running it.

Tool calling

Security goes in the design, not in the prompt. None of the assistant’s tools accepts a user ID. Identity comes from the logged-in session, so no prompt, however clever, can make the model read a colleague’s data. Each tool also returns errors written for the model, saying what was wrong and how to fix it:

// Translated from Italian. The user comes from the session, never from the model.
async function getUserCalendar({ from, to }, config) {
  const days = (Date.parse(to) - Date.parse(from)) / 86_400_000 + 1;
  if (days > MAX_DAYS) {
    // An error the model can act on: what's wrong, and what to do instead
    throw new Error(
      `Range of ${days} days, maximum ${MAX_DAYS}. Split it into several calls, e.g. one month at a time.`,
    );
  }
  const { calendar } = readEmployee(config.context.userId);
  return calendar.filter((e) => e.from <= to && e.to >= from);
}

Across 10 test requests, the model hit a tool error twice and fixed itself both times. Asked for “everything until the end of the year”, it got the 31-day error and split the request into three monthly calls. The more interesting failures were the tools it didn’t call: twice it answered without checking the calendar and filled the gap with a plausible date it made up. Good error messages fix bad calls. They do nothing for missing ones, and that’s what the evaluation module is for.

Where it’s going

Next up is module 07: an agent loop from scratch, then human-in-the-loop approval for actions that change something. After that come evaluation with LLM-as-judge and regression tests in CI, tracing and cost per request, guardrails against prompt injection, local model serving, fine-tuning with LoRA, and LLMOps.

Instead of three throwaway capstones, I want to finish by turning these pieces into one or two public open-source projects, so the code is there for anyone to read. Meanwhile I’m applying what I learn to real projects: the Personal Expenses Tracker uses the same structured-output ideas on real bank statements.