Building the Agent, Part One: Two Clouds, No Secrets

There is a terminal on the landing page of denizyilmaz.cloud. A visitor types a question. The request leaves the browser, enters AWS in London, crosses into Google Cloud, retrieves from a search index built over my blog posts, and a Gemini model writes an answer grounded entirely in what I have published. The answer comes back with numbered citations linking to the source posts. The visitor can ask a follow-up and the agent remembers the conversation.

The request path runs through two clouds. There is no API key, no service account JSON, and no secret stored anywhere in the system. Every credential in the chain is machine-issued at runtime and expires within the hour.

This post documents how it is built. It is part one of two. Part two covers the frontend, the terminal, the commands, and the suggestion system.

---

The Requirement

The agent had to do four things. Answer natural language questions about me using only my published blog posts. Support follow-up questions with conversation context. Cite the source post for every answer. Run on a public landing page without exposing credentials or an unbounded bill.

This is the second agent generation on my sites. The first generation was a Bedrock agent on digitalden.cloud that searched my RSS feed with keyword matching. It matched strings. This generation retrieves meaning. A question like “how strong is Deniz” finds my strength snapshot post and quotes the actual numbers, because the retrieval layer is a real search index over full post content rather than a feed parser looking for matching words.

I had already compared the grounding options on Google Cloud using these same blog posts. Agent Search, then named Vertex AI Search, won that comparison decisively. The retrieval infrastructure behind it produced deeper, better synthesised answers than a hand-built RAG pipeline, and it indexes straight from my live site with no upload step. That evaluation made the data layer decision for this build. The index lives on Google Cloud.

The rest of my infrastructure lives on AWS. The site is served from S3 and CloudFront, the contact form runs on Lambda, and everything sits in eu-west-2. The backend for the agent belongs there with it. So the design question became how an AWS Lambda calls a Google API safely, and the answer to that question turned out to be the most interesting part of the build.

---

The Architecture

flowchart TB
  A[Terminal UI on denizyilmaz.cloud]
  -->|POST /agent| B[Amazon API Gateway<br/>HTTP API, eu-west-2]
  --> C[AWS Lambda<br/>agent broker]
  C -->|Federated token exchange| D[Google STS and<br/>IAM Credentials]
  C -->|Answer API call| E[Agent Search engine]
  E --> F[(Indexed blog posts)]
  E -->|Grounded answer,<br/>citations, session| C
  C --> B --> A

Each cloud does the thing it is genuinely best at. Google holds the search index and the grounded generation. AWS holds the cheap, locked-down serverless plumbing in the region where the rest of my stack already runs. The Lambda in the middle is a stateless broker. It accepts untrusted questions from the public internet on one side and makes privileged calls on the other, and the entire security model of the system lives in how it earns the right to make those calls.

---

The Grounding Layer

Google rebranded this product line in 2026. Vertex AI Search is now Agent Search, part of the Gemini Enterprise Agent Platform. The APIs kept their legacy names for backward compatibility, which is why everything below calls discoveryengine.googleapis.com and the resource names in my project still carry the vertex prefix.

The Data Store

The Agent Search data store crawls denizyilmaz.cloud through its sitemap with advanced website indexing enabled. URL exclusion patterns keep navigation pages out of the knowledge base, so tags, categories, archives, and pagination never appear in answers.

denizyilmaz.cloud/*              included
denizyilmaz.cloud/tags/*         excluded
denizyilmaz.cloud/categories/*   excluded
denizyilmaz.cloud/archives/*     excluded
denizyilmaz.cloud/page*          excluded

The exclusion patterns are editorial control over the agent. The index is the knowledge base, and the patterns decide what the agent is allowed to know. One operational rule worth stating plainly. The index is a snapshot, not a live view of the site. If a post moves or gets removed, the old content keeps serving from the index until you exclude it explicitly. Manage the index like a database, not like a cache.

The Answer API

The engine on top of the data store exposes the Answer API, and this single method carries the whole product. Each call runs retrieval against the index, hands the matching chunks to a Gemini model, and generates an answer constrained to that material. The grounding is the point. The model never answers from its general training, which designs out the hallucination class the first agent generation suffered from, where a stale model confidently answered questions my content never covered.

Gemini’s role in this pipeline needs stating precisely, because it confused me until I tested it. The model is the writer, not the source. Its general training shapes the language and the fluency, but it is not permitted to contribute facts, every claim in an answer has to come from a retrieved chunk. Ask the agent whether a book is good before I have logged a verdict and it declines, even though the model behind it could review that book from memory. Google offers a different API for that behaviour, generateContent, where Gemini answers from its full training with the data store attached as one source among everything it knows. That is what Agent Studio uses and why Studio conversations feel so free. I chose the constrained API deliberately. A public endpoint with a model that can freelance is a free general chatbot running on my bill, and an agent that can speak beyond my posts can also be wrong about me.

The request the Lambda sends looks like this.

payload = {
    "query": {"text": question},
    "session": session_name,
    "answerGenerationSpec": {
        "ignoreAdversarialQuery": True,
        "ignoreNonAnswerSeekingQuery": False,
        "ignoreLowRelevantContent": False,
        "includeCitations": True,
        "promptSpec": {"preamble": PREAMBLE},
        "modelSpec": {"modelVersion": "stable"},
    },
}

The preamble shapes the agent’s delivery, and tuning it turned out to be a real design decision. The first version told the model to refuse anything the posts did not directly answer, which produced an agent that stonewalled open questions like “tell me something interesting” even when retrieval had found relevant posts. The shipped version flips the default. Answer generously from whatever retrieval found, synthesize across posts, handle open questions by picking real highlights, and only decline when the posts genuinely contain nothing relevant. Two request flags work alongside it. Setting ignoreNonAnswerSeekingQuery to false stops the API skipping conversational phrasings, and ignoreLowRelevantContent set to false lets it attempt answers from weaker retrieval matches, the right trade for a personal corpus where even weak matches are still about me. The one rule that never loosens is the architecture itself. The model can only speak from the posts, so generosity never becomes invention.

Sessions

Conversation memory lives on the Google side. The first request in a conversation passes a session path ending in a dash, which tells Agent Search to create a session and return its ID. The Lambda hands that ID back to the browser, the browser sends it with the next question, and Agent Search sees the full conversation when generating the follow-up answer. Neither the browser nor the Lambda holds any conversation state. The only thing that travels is the ID, which means the Lambda stays stateless and scales to zero.

This is why a follow-up like “how many times a week does he do it” works. The session resolves the pronoun from the previous turn about training and answers with the training frequency.

The Model

The stable model specification is an alias, not a model name. It currently resolves to a question answering variant of Gemini 2.5 Flash, and Google rotates the alias as models retire. Pointing at the alias instead of pinning a version means the agent survives model retirements with zero code changes, at the cost of answer style shifting slightly when the alias moves. For a personal site that trade is correct. For a regulated product you would pin and manage migrations deliberately. The version under the alias at the time of this build retires within weeks of publication, which is exactly why the alias was the right call.

---

Identity Without Credentials

The Lambda runs on AWS and needs to prove its identity to Google. The conventional answer is a service account key, a JSON file you download once and protect forever. It works, and it is also a permanent password that can leak from a laptop, a repo, or a secrets store, and that someone has to rotate for the life of the system.

This build uses Workload Identity Federation instead. Both clouds already operate cryptographic identity systems for their own workloads. Federation connects them, so the Lambda’s native AWS identity becomes acceptable proof to Google. There is no key because there is nothing to store.

The Trust Chain

Three resources on the Google side define the trust.

gcloud iam workload-identity-pools create aws-lambda-pool \
  --location=global \
  --display-name="AWS Lambda"

gcloud iam workload-identity-pools providers create-aws aws-provider \
  --workload-identity-pool=aws-lambda-pool \
  --account-id=AWS_ACCOUNT_ID \
  --location=global

gcloud iam service-accounts add-iam-policy-binding \
  vertex-agent-proxy@PROJECT_ID.iam.gserviceaccount.com \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-lambda-pool/attribute.aws_role/arn:aws:sts::AWS_ACCOUNT_ID:assumed-role/vertex-agent-lambda-role"

The pool is a container for external identities. The provider declares that Google will accept identity proofs signed by my AWS account, because AWS cryptographically signs every credential it issues and Google can verify those signatures. The final binding is the lock. It grants exactly one AWS identity, the Lambda’s execution role, permission to act as the vertex-agent-proxy service account. Nothing else in my AWS account qualifies, and nothing outside it gets near.

The service account itself holds a single role, Discovery Engine Viewer. Even fully compromised, the identity the Lambda borrows can only read a search index.

The Runtime Exchange

At runtime the flow is mechanical. Lambda starts and AWS injects temporary credentials for its execution role into the environment, rotated constantly and never visible to me. The google-auth library presents those credentials to Google’s token service. Google verifies the AWS signature, checks the binding, and issues a GCP access token impersonating the service account. The token lives for one hour and refreshes automatically. The Lambda uses it to call the Answer API.

The only artifact this requires is a credential configuration file, generated once.

gcloud iam workload-identity-pools create-cred-config \
  projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/aws-lambda-pool/providers/aws-provider \
  --service-account=vertex-agent-proxy@PROJECT_ID.iam.gserviceaccount.com \
  --aws \
  --output-file=gcp-wif-config.json

The file contains pool paths and a token URL. It holds no secret of any kind, which means it ships inside the Lambda deployment package and can sit in a private repo without ceremony. The proof of identity arrives fresh from AWS on every cold start.

If you take one pattern from this post, take this one. Cross-cloud calls do not require stored credentials anymore. Federation costs a few extra setup commands and removes an entire category of risk, rotation work, and secrets infrastructure for the life of the system.

---

The Lambda

The Lambda is a broker with four responsibilities. Validate untrusted input, build the authenticated Answer API request, reshape Google’s verbose response into a minimal contract, and own every string a visitor can ever see.

The credentials initialise at module level, outside the handler, so the token exchange runs once per cold start and warm invocations reuse it.

credentials, _ = google.auth.default(
    scopes=["https://www.googleapis.com/auth/cloud-platform"]
)
authed = AuthorizedSession(credentials)

Owning the Contract

The browser never sees Google’s response format. The Lambda returns three fields and nothing else.

{
  "answer": "...",
  "references": [{"title": "...", "uri": "..."}],
  "sessionId": "..."
}

The references list is filtered before it leaves. The raw API cites whatever documents contributed to retrieval, which includes index pages and the homepage. A regex keeps only real post URLs, so every citation a visitor clicks lands on an actual article.

POST_URL_RE = re.compile(
    r"^https://denizyilmaz\.cloud/posts/(?!page\d+/?$)[^/]+/?$"
)

This contract is the seam in the system. The frontend knows nothing about Agent Search, Gemini, or Google Cloud. The entire backend could be rebuilt on a different retrieval stack tomorrow and the terminal would not change a line.

Owning Every String

When the Answer API decides a question is outside the corpus, it skips generation and returns stock text about search results that the terminal has no way to render sensibly. A production proxy does not pass upstream internals to visitors. The Lambda detects the skip through the structured answerSkippedReasons field and substitutes a message in the agent’s own voice that redirects toward what the posts do cover. The refusal itself is correct behaviour. The grounding contract means the agent only speaks where my writing speaks, and the suggested questions under the input recover the visitor in one click.

---

The Front Door

API Gateway provides the public endpoint, and the HTTP API flavour rather than REST. It is cheaper per request, CORS is native configuration instead of hand-built OPTIONS plumbing, and stage throttling is built in. REST APIs earn their extra cost when you need usage plans with API keys, which do nothing useful for an anonymous public endpoint.

aws apigatewayv2 create-api \
  --name vertex-agent-api \
  --protocol-type HTTP \
  --cors-configuration 'AllowOrigins=https://denizyilmaz.cloud,http://127.0.0.1:4000,AllowMethods=POST,OPTIONS,AllowHeaders=content-type'

The stage throttles at 2 requests per second with a burst of 5, the Lambda permission is scoped to this exact API and route, and the integration pins payload format 1.0 so the handler’s event shape stays stable. CORS allows the production domain plus local Jekyll, which means the live endpoint is testable from bundle exec jekyll serve before anything ships.

---

The Terminal

The frontend is the terminal UI from the first agent generation with new wiring. Vanilla JavaScript, no libraries, no framework. It captures input, POSTs the question with the current session ID, renders the answer, and renders references as numbered links under it. The session ID starts as null, the backend mints it on the first answer, and the clear command resets it so clearing the terminal genuinely starts a fresh conversation.

One design rule came out of testing and it applies to any session-based agent. Suggested question buttons fire on cold sessions, so they must be self-contained. A button reading “How did he get into cloud engineering?” fails on a fresh session because there is no prior turn to resolve the pronoun against, while the same question with the name answers perfectly every time. Free text follow-ups can rely on session context for pronouns. Buttons cannot, because buttons are openers. Every suggested question on the site carries the name.

The questions themselves are chosen to ambush. “How strong is Deniz?” looks like banter and returns a full strength table at 78kg bodyweight with lift numbers and citations. “What does Deniz bake?” comes back with potato buns, Finnish blueberry pie, and Swedish kladdkaka, each linked to its post. A suggested question is only worth its button if you have tested it and the answer delivers.

---

Cost and Containment

The expected monthly cost of this system is approximately zero. Agent Search includes 10,000 free queries per month and one visitor question is one query. The HTTP API costs about a dollar per million requests and Lambda’s permanent free tier covers a million invocations. A personal site does not approach any of these numbers.

The engineering question is not the expected cost. It is what bounds the bill if something goes wrong, and the honest answer requires knowing what each control actually does.

Layer	Control	What it does
Lambda	Input validation, 500 char cap	Rejects garbage before it costs an Agent Search call
API Gateway	2 rps throttle, burst 5	Chokes request floods with 429s
Lambda	Reserved concurrency of 3	Hard ceiling on simultaneous Agent Search calls
GCP quota	LLM query requests per minute	Platform-level rate ceiling on answer generation
CloudWatch	Alarm above 200 invocations per hour	Email within the hour on abnormal traffic
GCP budget	£20 monthly with threshold alerts	Detection at half spent and fully spent

The budget deserves a blunt note because the point is widely misunderstood. A GCP budget alert notifies. It does not cap or stop anything. The throttle, the concurrency limit, and the quota are what slow a runaway. The budget guarantees you hear about one. Security spend should match what is at stake, and for a system whose worst realistic month is a few pounds, layered rate limits plus detection is the proportionate posture.

---

Observability

Every conversation turn lands in CloudWatch as a pair of JSON lines. The Lambda prints the question on the way in and the answer on the way out, with the citation count and the session ID tying the pair together.

{"asked": "What does Deniz bake?", "session": "new"}
{"answered": "Deniz bakes a few different things! He's made potato buns, which he perfected for smash burgers...", "refs": 3, "session": "12341476159908524641"}

Reading a week of conversations is one command.

aws logs tail /aws/lambda/vertex-agent --region eu-west-2 \
  --since 7d --format short | grep -E '"asked"|"answered"'

The refs count is a quality signal on its own. An answer carrying zero references is either the redirect for a question the posts do not cover or the model stretching a thin retrieval, and both are worth a look. The session field separates opening questions from follow-ups, which shows who clicked one button and left versus who got pulled into a real conversation.

Google keeps its own record. Sessions store every turn server side, the same mechanism that powers follow-up questions, so the engine can be queried directly for conversation history.

curl -s -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "x-goog-user-project: PROJECT_ID" \
  "https://discoveryengine.googleapis.com/v1/projects/PROJECT_ID/locations/global/collections/default_collection/engines/ENGINE_ID/sessions?pageSize=50"

Each turn in a session carries the question text plus a resource name pointing at the stored answer, and fetching that resource name returns the full generated response with its citations. So the same conversation exists twice, the CloudWatch log for daily reading and the session store on the Google side as the source of truth.

Worth stating plainly as the operator. The terminal keeps what strangers type, which is standard practice for any public endpoint, and something to be aware of rather than something to apologise for.

---

The Mechanical Result

A stranger lands on denizyilmaz.cloud, clicks a suggested question or types their own, and gets an answer written from my posts with links to the sources. They ask a follow-up with a pronoun and the agent resolves it from conversation context. They ask something my writing does not cover and the agent says so and points at what it can answer. They check the page source and find a public API endpoint with nothing worth stealing behind it, because the system holds no credential anywhere.

Two clouds, one request path. AWS absorbs the public internet and Google answers from my writing, with a one-hour borrowed identity carrying every call across the boundary.

Try it at denizyilmaz.cloud.

---

Future Enhancements

The index refreshes automatically as I publish, so the agent already grows with the blog without any feeding. These are the enhancements the architecture supports when I want them, none of them are commitments and none of them require touching the design.

Inline citations are the strongest candidate. The Answer API returns sentence-level citation spans with every response, and the Lambda currently keeps only the source list. Rendering numbered markers inside the answer text at the exact sentences they support would turn the references from a footer into claim-by-claim attribution. The data is already in every response, so this is a Lambda mapping change and a small frontend renderer, no new infrastructure.

Question analytics is the most interesting one strategically. The logging above already captures every question, so the remaining piece is a small scheduled job that summarises a week of them into themes. What strangers genuinely ask feeds two things, the suggested questions under the terminal and the decision of what to write next. The agent becomes a feedback loop into the blog itself.

Scope growth is the cheapest. The engine currently indexes one site. Agent Search engines can attach multiple data stores, so future sites and content sources can join the same agent with a console change. The Lambda, the federation, and the terminal would not know anything happened.

Each of these lands without touching the architecture. That was the test the design had to pass.

---

Documented June 2026.