Shortcut: if you'd rather not type out the scaffold, the companion repo is the exact code this step produces. Clone it and skip ahead to "The .env File."

Install uv (or skip)

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh
 
# Windows (PowerShell)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

uv is Astral's fast Python package manager — one binary replaces pip + venv + pip-tools. If you'd rather use pip, every uv add X below maps to pip install X after running python -m venv .venv && source .venv/bin/activate.

Scaffold the Project

mkdir rag-on-gcp
cd rag-on-gcp
 
uv init --python 3.12

Add the four dependencies we need:

uv add \
  "cloud-sql-python-connector[pg8000]" \
  google-cloud-aiplatform \
  google-genai \
  fastapi \
  "uvicorn[standard]" \
  python-dotenv

What each one does:

PackageRole
cloud-sql-python-connector[pg8000]Connects to Cloud SQL using IAM auth. pg8000 is the pure-Python Postgres driver it speaks through.
google-cloud-aiplatformVertex AI client — used for text-embedding-005.
google-genaiThe unified Google GenAI SDK — used to call Gemini.
fastapi + uvicornThe web framework + the ASGI server we'll run in Cloud Run.
python-dotenvLoads a .env file in dev. Cloud Run uses env vars directly.

Project Layout

mkdir -p src data sql scripts
touch src/__init__.py src/config.py src/db.py
touch sql/001_init.sql
touch .env .env.example

You should end up with:

rag-on-gcp/
├── data/                     # Drop .txt files here later
├── sql/
│   └── 001_init.sql          # Schema reference (same SQL from Step 3)
├── scripts/
├── src/
│   ├── __init__.py
│   ├── config.py             # Loads env config
│   └── db.py                 # Cloud SQL connection helper
├── .env
├── .env.example
└── pyproject.toml

The .env File

Three values. Two come from Step 2 and 3.

# .env
GOOGLE_CLOUD_PROJECT=your-project-id-from-step-2
GOOGLE_CLOUD_REGION=us-central1
INSTANCE_CONNECTION_NAME=your-project-id:us-central1:rag-db
DB_NAME=rag
DB_USER=your-email@gmail.com

Copy the same structure into .env.example but with placeholder values — that one gets committed; .env does not.

Quick .gitignore:

cat >> .gitignore <<'EOF'
.env
.venv/
__pycache__/
*.pyc
EOF

src/config.py — One Place for Env

"""Read environment variables once; expose them as a typed object."""
 
from dataclasses import dataclass
import os
from dotenv import load_dotenv
 
load_dotenv()
 
 
@dataclass(frozen=True)
class Config:
    project: str
    region: str
    instance_connection_name: str
    db_name: str
    db_user: str
 
 
def load_config() -> Config:
    def _required(key: str) -> str:
        value = os.environ.get(key)
        if not value:
            raise RuntimeError(f"Missing required env var: {key}")
        return value
 
    return Config(
        project=_required("GOOGLE_CLOUD_PROJECT"),
        region=_required("GOOGLE_CLOUD_REGION"),
        instance_connection_name=_required("INSTANCE_CONNECTION_NAME"),
        db_name=_required("DB_NAME"),
        db_user=_required("DB_USER"),
    )

src/db.py — The Connector

This is the file that does the magic. The Cloud SQL Python Connector uses your Application Default Credentials (set up in Step 2) to authenticate to Cloud SQL over Google's private network. No public IP, no proxy binary, no password.

"""Cloud SQL connection helper using the Python Connector + IAM auth."""
 
from contextlib import contextmanager
from google.cloud.sql.connector import Connector, IPTypes
import pg8000.dbapi
 
from .config import load_config
 
_config = load_config()
_connector = Connector(refresh_strategy="lazy")
 
 
def _connect() -> pg8000.dbapi.Connection:
    return _connector.connect(
        _config.instance_connection_name,
        "pg8000",
        user=_config.db_user,
        db=_config.db_name,
        enable_iam_auth=True,
        ip_type=IPTypes.PUBLIC,
    )
 
 
@contextmanager
def get_conn():
    """Yield a Postgres connection. Closes on exit."""
    conn = _connect()
    try:
        yield conn
    finally:
        conn.close()

Two things worth flagging:

  • enable_iam_auth=True — tells the connector to fetch a short-lived OAuth token from ADC and use it as the password. This is why we never set a Postgres password.
  • ip_type=IPTypes.PUBLIC — we created the instance with a public IP. The connector still tunnels through Google's network; "public" here is about the addressing, not the routing. When we deploy to Cloud Run in Step 7, we'll switch this to PRIVATE over a VPC connector for production.

Smoke Test

Drop a tiny test script next to src/:

# scripts/test_connection.py
from src.db import get_conn
 
with get_conn() as conn:
    cur = conn.cursor()
    cur.execute("SELECT 1, current_user, current_database()")
    print(cur.fetchone())

Run it:

uv run python -m scripts.test_connection

You should see something like:

(1, 'your-email@gmail.com', 'rag')

That's the database confirming: you (by your IAM identity) connected to the rag database and ran a query. No password ever changed hands.

Common First-Run Failures

ErrorWhat it means
google.auth.exceptions.DefaultCredentialsErrorYou skipped gcloud auth application-default login in Step 2. Run it.
pg8000.exceptions.DatabaseError: ... password authentication failedThe IAM user wasn't created or doesn't match your email exactly. Check gcloud sql users list --instance=rag-db.
Connection refusedThe instance was stopped. gcloud sql instances patch rag-db --activation-policy=ALWAYS.
403 Cloud SQL Admin API has not been usedAPI isn't enabled. Re-run gcloud services enable sqladmin.googleapis.com.

sql/001_init.sql — Keep the Schema in the Repo

Drop the schema from Step 3 into the repo so you can re-create it from scratch if needed:

-- sql/001_init.sql
CREATE EXTENSION IF NOT EXISTS vector;
 
CREATE TABLE IF NOT EXISTS chunks (
  id           BIGSERIAL PRIMARY KEY,
  source       TEXT NOT NULL,
  chunk_index  INTEGER NOT NULL,
  content      TEXT NOT NULL,
  embedding    vector(768) NOT NULL,
  created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
 
CREATE INDEX IF NOT EXISTS chunks_embedding_hnsw
  ON chunks
  USING hnsw (embedding vector_cosine_ops);

We won't ship this to Cloud Run — it's a one-time bootstrap. But future-you (or a teammate) will thank you.

What You Have Now

  • A Python project with the four packages we need
  • A typed Config object that reads env vars
  • A get_conn() context manager that authenticates as your Google identity
  • A passing smoke test that proves end-to-end connectivity

Next: ingest some text.

Reference: Cloud SQL Python Connector · pg8000 driver · uv quickstart