The generate helper — _shared/gemini.ts

Embeddings are done (Step 3, all local). The only external model call in the whole build is generation, and _shared/gemini.ts is the one file that makes it — here it is in full:

const GEMINI_API_KEY = Deno.env.get("GEMINI_API_KEY")!;
const BASE = "https://generativelanguage.googleapis.com/v1beta/models";
const GEN_MODEL = "gemini-2.5-flash";
 
/** Ask Gemini to write the grounded answer. */
export async function generate(prompt: string): Promise<string> {
  const res = await fetch(`${BASE}/${GEN_MODEL}:generateContent?key=${GEMINI_API_KEY}`, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      contents: [{ parts: [{ text: prompt }] }],
      generationConfig: { temperature: 0.2 },
    }),
  });
  if (!res.ok) throw new Error(`generate failed: ${res.status} ${await res.text()}`);
  const data = await res.json();
  return data.candidates?.[0]?.content?.parts?.[0]?.text ?? "";
}

gemini-2.5-flash for generation — fast and cheap, which is what a grounded RAG answer wants. temperature: 0.2 keeps it close to the facts: RAG isn't a creative-writing task, so we turn the dial down toward deterministic.

The query function

supabase/functions/query/index.ts is the read phase, and it maps one-to-one onto the four steps from Step 1.

type Match = { id: number; source: string; content: string; similarity: number };
 
Deno.serve(async (req) => {
  if (req.method === "OPTIONS") return new Response("ok", { headers: corsHeaders });
 
  try {
    const { question } = await req.json();
    if (!question?.trim()) throw new Error("`question` is required");
 
    // 1. Embed the question (same model + vector space as the stored chunks).
    const queryEmbedding = await embed(question);
 
    // 2. Retrieval: ask Postgres for the closest chunks. This RPC is the tool
    //    the answer is grounded on.
    const { data, error } = await db.rpc("match_documents", {
      query_embedding: queryEmbedding,
      match_count: 5,
    });
    if (error) throw error;
    const matches = (data ?? []) as Match[];

Step 1 — embed the question. The exact same embed() helper as the write phase — gte-small has no query/document distinction, so one call puts the question in the same vector space as the stored chunks, and "nearest vector" actually means "closest in meaning."

Step 2 — retrieve. A single .rpc("match_documents", …) call runs the SQL function from Step 2, which orders by cosine distance over the hnsw index and hands back the top 5 chunks with a similarity score. This RPC is the retrieval tool — the one piece an agentic version would call from a loop.

Grounding: the prompt that stops hallucination

    // 3. Build a grounded prompt with numbered sources.
    const context = matches
      .map((m, i) => `[${i + 1}] (source: ${m.source})\n${m.content}`)
      .join("\n\n");
 
    const prompt = [
      "You are a precise assistant. Answer the question using ONLY the context below.",
      "Cite the sources you use inline like [1], [2]. If the answer isn't in the",
      "context, say you don't know — do not make anything up.",
      "",
      "# Context",
      context || "(no documents found)",
      "",
      "# Question",
      question,
    ].join("\n");

This is the heart of grounding. Each retrieved chunk becomes a numbered block — [1] (source: …) — and the instructions are strict: answer ONLY from the context, cite inline, and say "I don't know" rather than invent. The model never sees your raw documents, only these five chunks, so its answer is anchored to real retrieved text. The context || "(no documents found)" fallback means an empty knowledge base produces an honest "I don't know," not a hallucination.

Return the answer and its sources

    // 4. Ask Gemini, and hand back the sources so the UI can show them.
    const answer = await generate(prompt);
 
    return json({
      answer,
      sources: matches.map((m, i) => ({
        n: i + 1,
        source: m.source,
        similarity: Number(m.similarity.toFixed(3)),
      })),
    });
  } catch (e) {
    const message = e instanceof Error ? e.message : String(e);
    return json({ error: message }, 400);
  }
});

The response is { answer, sources[] }. The [1], [2] markers in the answer text line up with the n in each source entry — so the UI (Step 5) can render the citations against the list of documents they came from, each with its similarity score. That alignment is the entire "with sources" feature: the same numbering the model cites is the numbering the UI lists.

Step 5 builds the front-end that calls both of these functions.