The Agent–Data Flywheel

One-Line Summary: AI agents are only as good as the data they can reach at inference time, so the real product moat is the pipeline that keeps that data fresh, clean, and shaped for retrieval.

Prerequisites: Familiarity with what an LLM agent is (a model + tools + a control loop) and basic SQL.

What's the Concept?

An agent without a data layer is a clever conversationalist with no facts. Hook the same model up to a well-engineered pipeline — current orders, latest product specs, fresh support tickets — and it stops hallucinating and starts answering. The interesting work shifts from prompt engineering to the engineering of the data the prompt is allowed to consult.

The flywheel idea is simple. Agents read from a refined data layer. Users interact with agents. Those interactions produce new signals — questions, corrections, tool calls — which become inputs to the next pipeline run. Better data → better answers → more usage → more signal → better data. The faster you can turn that loop, the wider the moat.

How It Works

There are three layers worth naming clearly because they show up over and over in this course:

┌──────────────────────────────────────────────────────────────┐
│  EXTERNAL SOURCES                                            │
│  APIs · webhooks · databases · file drops · streams          │
└──────────────────┬───────────────────────────────────────────┘
                   │  ingest (Module 02)
                   ▼
┌──────────────────────────────────────────────────────────────┐
│  RAW LAKE  —  Cloud Storage (GCS)                            │
│  schema-on-read · partitioned · immutable                    │
└──────────────────┬───────────────────────────────────────────┘
                   │  refine (Module 04)
                   ▼
┌──────────────────────────────────────────────────────────────┐
│  REFINED LAYER  —  BigQuery + Vector Search                  │
│  conformed · modeled · embedded · queryable as a tool        │
└──────────────────┬───────────────────────────────────────────┘
                   │  retrieve (Module 05)
                   ▼
┌──────────────────────────────────────────────────────────────┐
│  AGENT CONTEXT                                               │
│  tools call BigQuery / vector search · LLM reasons over it   │
└──────────────────────────────────────────────────────────────┘

The pipeline never serves raw data to the agent. The agent always reads from the refined layer, where someone — a person, a dbt model, a Dataflow job — has already decided what counts as a fact in this system.

Why It Matters

Hallucination is mostly a data problem. Models confabulate when no relevant retrieval is in context. A pipeline that surfaces the right rows turns "make something up" into "read and summarize."
Freshness is a feature. A support agent that quotes yesterday's pricing is a liability. The ingestion cadence is part of the product spec, not an infra detail.
Shape determines what's askable. An agent can only answer questions that the refined layer can express. The data model is the agent's vocabulary.

Key Technical Details

Production agent stacks typically read from 2–6 different refined datasets per request (structured + semantic).
End-to-end freshness — from external event to agent-visible row — runs from seconds (CDC + streaming) to hours (batch). Pick deliberately per use case.
Embedding regeneration is the most expensive recurring cost in this whole pipeline; we'll come back to that in Module 07.

Common Misconceptions

"RAG solves it." Retrieval-augmented generation is just the last six inches of the pipe. The other 100 metres — ingestion, deduplication, conforming, modeling, embedding refresh — is data engineering, and that's where most production failures happen.

"The model will figure it out from raw data." It won't, and you don't want it to. Pre-shaping the data is what makes retrieval cheap, predictable, and auditable.

Connections to Other Concepts

02-from-warehouse-to-agent-context.md — How the classic warehouse pattern adapts when an LLM is the downstream consumer.
03-the-medallion-pattern-bronze-silver-gold.md — The naming convention we'll use throughout the course.
04-the-gcp-data-stack-at-a-glance.md — The specific GCP services that implement each layer.