Expose the resolved File Search retrieved chunks (not only cited ones) on the file_search_result step

[pullely-samuel]

Environment: google-genai 2.10.0 · Python 3.13 · macOS · Gemini API (not Vertex) · model gemini-3.5-flash with the file_search tool.

Summary

When using File Search via the Interactions API, there is no way to inspect the full set of chunks the retriever returned and injected into the model's context. The retrieved chunk text is observable only indirectly, via model_output → file_citation.source annotations — i.e. only for chunks the model ends up citing. This blocks reliability/eval work and debugging "what context did the model actually see."

What's observable today

• file_citation annotations expose source (chunk text), document_uri, file_name, custom_metadata, start_index/end_index. Only for cited chunks. (Side note: the source field appears undocumented — the public File Search docs still describe annotations as metadata-only.)
• The file_search_result step contains only call_id, type, and an opaque base64 signature (~16KB, encrypted) — no readable retrieved content.
• file_search_stores.documents.get/list return metadata only; download_media on a document returns 403 PERMISSION_DENIED.
• Token usage doesn't help: total_input_tokens stays flat and total_tool_use_tokens is ~constant regardless of top_k, so the retrieved-context size isn't reflected anywhere.

Why it matters

Rigorous RAG faithfulness/groundedness evaluation must judge an answer against the actual retrieved context, not the full source document. Today we can only approximate it with the cited chunks. Empirically, at the default top_k the model cites every retrieved chunk (cited count tracks top_k 1:1 up to ~5), so cited ≈ retrieved in that regime — but this is inferred, not guaranteed, and it breaks down at higher top_k (e.g. top_k=10 → 8 cited, top_k=20 → 16 cited), where retrieved-but-uncited chunks become invisible. (Same concern raised re: server-side context being a black box: https://x.com/_philschmid/status/2069458803074986376)

Repro

store = "fileSearchStores/<your-store>"
it = client.interactions.create(
    model="gemini-3.5-flash",
    input="<a question your store can answer>",
    tools=[{"type": "file_search", "file_search_store_names": [store], "top_k": 20}],
    store=True,
)
full = client.interactions.get(it.id)
# file_search_result step: only call_id / type / signature — no retrieved chunks.
# Retrieved chunk text is reachable only via model_output file_citation.source,
# i.e. only the CITED chunks; with a large top_k the cited count is < top_k, so
# the retrieved-but-uncited chunks are invisible.

Request

Surface the resolved File Search retrieval on the file_search_result step (or via an opt-in flag): the full set of retrieved chunks — text + document/source ref + relevance score, including uncited ones. This makes the retrieved context inspectable for evaluation and debugging, and lets developers trim/manage it as suggested in the GA guidance.

[Venkaiahbabuneelam]

Hi @pullely-samuel, thanks for reaching out to us.

To help us investigate, could you please share the full code snippet and the associated error logs?

Thanks

[pullely-samuel]

Hi @Venkaiahbabuneelam, thanks for following up. I've put together a minimal, self-contained reproduction (single file, PEP 723 inline deps — runs with just uv run repro_filesearch_chunks.py, no project setup):

https://gist.github.com/pullely-samuel/5cccd95e9ab0844f1a951a04fc6b0f8e

It takes a GEMINI_API_KEY + a FILE_SEARCH_STORE (any File Search store with documents) and sweeps top_k. For reference, my store contains only plain-text (.txt) documents, and I only ever issue text input queries (the query is embedded and used for semantic search) — so although File Search supports multimodal embeddings/queries, this repro exercises the text-only path. Verified output (google-genai 2.10.0, gemini-3.5-flash, AI Studio):

 top_k | cited_chunks | < top_k?
----------------------------------
     1 |            1 |       no
     3 |            3 |       no
     5 |            4 |      YES
    10 |            8 |      YES
    20 |           11 |      YES

--- file_search_result step ---
{'type': 'file_search_result', 'call_id': '...', 'signature': '<69320 base64 chars, opaque/encrypted>'}
retrieved chunk text on this step? False

So at higher top_k the cited count falls well below top_k, and the retrieved-but-uncited chunks are unobservable — file_citation.source only exposes cited chunks, and the file_search_result step carries just an opaque encrypted signature, no readable retrieved content. (Counts wobble run-to-run at temperature=1.0, but the direction is stable.)

Two clarifications/additions to the original report:

1. Parent-document retrieval (not clearly called out in the issue). Beyond the retrieved-vs-cited gap, there's no supported way to go from a chunk back to its full parent source document. file_search_stores.documents.get(...) returns metadata only (size_bytes, mime_type, custom_metadata) — no text/content field — and file_search_stores.download_media(...) requires a media_id of the form fileSearchStores/<store>/media/<blob_id> that store documents never surface. For evaluation/debugging it would help to fetch a chunk's full parent document text.

2. Correction on download_media. My original report said download_media on a document returns 403 PERMISSION_DENIED. In 2.10.0 it's actually a client-side ValueError: Invalid media_id format (the method lives on file_search_stores, not documents, and there's no usable blob id for store documents) — so the document content is effectively unreachable either way.

Net request: surface the resolved File Search retrieval on the file_search_result step (or via an opt-in flag) — the full set of retrieved chunks (text + source ref + relevance score, including uncited ones) — plus a supported way to fetch a chunk's full parent document text.

Happy to provide anything else that helps. Thanks!