Knowledge with files and Azure AI Search

Concept 5 showed how prompts and model settings shape what a model says. Knowledge tools answer a different question: where the facts come from. Without them, an agent guesses from general training data. With them, it can pull passages from documents you control and weave those passages into the reply.

Microsoft Foundry offers two main paths for document knowledge. File search is for files you upload into a vector store inside your project. The Azure AI Search tool is for content that already lives in an Azure AI Search index. Both ground the agent in your material and can attach citations so a human can verify the source.

File search lets a Foundry agent retrieve relevant chunks from documents you provide, then use those chunks when it generates an answer. Where files live depends on agent setup: standard setup stores uploads in Azure Blob Storage you connect during setup and builds vector stores on your connected Azure AI Search resource; basic setup stores uploads in Microsoft-managed storage and uses a Microsoft-managed search resource for vector stores.

• Create or pick a test file that contains a fact you can check later (for example a short product FAQ).
• Upload the file and add it to a vector store. SDK helpers can upload and poll until ingestion finishes; otherwise poll the vector store until no file remains in the in_progress state.
• Enable file search on the agent—either attach a file search tool with your vector store IDs, or add file search to a toolbox and attach that toolbox to the agent.
• Ask a question only your file can answer, and confirm the reply quotes or paraphrases that content with a file citation when search ran.

Before you upload, confirm your identity has the Foundry User role on the project and the Storage Blob Data Contributor role on the project storage account. You need a basic or standard agent environment and a model deployment in the same project. File search bills separately from model tokens, so treat uploads and retrieval as their own line item when you estimate cost.

When you add a file to a vector store, the service parses the document, splits it into chunks, generates embeddings, and stores the result in a database that supports both keyword and semantic search. When a user asks a question, file search rewrites the query for search, may split complex questions into parallel searches, runs hybrid search across the vector store, reranks hits, and passes the best chunks to the model.

• Each vector store can hold up to 10,000 files. You can attach at most one vector store to an agent and at most one vector store to a conversation.
• Maximum file size is 512 MB. Each file should contain no more than 5,000,000 tokens (computed when you attach the file).
• Vector stores created through conversation helpers expire seven days after they were last active; expired stores cause runs on that conversation to fail until you recreate the store and reattach the same files.
• Wait until the vector store status is completed before you rely on search results; a run may wait up to 60 seconds for conversation uploads still processing, but that fallback does not apply to the agent’s vector store.

The code path for file search is the same for basic and standard agent setups. The difference is where files and vector stores live. Basic setup uses Microsoft-managed storage and a Microsoft-managed search resource for vector stores. Standard setup uses Azure Blob Storage and Azure AI Search resources you connect when you configure the agent environment.

• Choose file search when you upload PDFs, Word files, code, or markdown directly and want the service to handle parsing and indexing.
• Choose the Azure AI Search tool when the collection is large, already indexed, or you need query types and filters on an existing index.
• Use the web search tool when you need current public web content; combine multiple tools when one query needs more than one source.

The Azure AI Search tool retrieves documents from an index you already maintain. The model then answers with inline citations, often including a URL when your index exposes one. The tool can target only one index per configuration. Your Foundry project and the search service must be in the same tenant.

• Configure the index for vector search: at least one searchable, retrievable Edm.String field; at least one searchable Collection(Edm.Single) vector field; a retrievable text field for content the agent should cite; a retrievable field with a source URL (and optionally a title) for link citations.
• Supported query_type values besides the default are simple, vector, semantic, and vector_simple_hybrid.
• For keyless access, assign Search Index Data Contributor and Search Service Contributor on the search service to the system-assigned managed identity of the Foundry account that contains your project.
• On a private virtual network, use Microsoft Entra project managed identity in the Azure AI Search connection. Key-based authentication is not supported there.

If the connection does not exist yet, open the Microsoft Foundry portal, choose your Foundry project, select Manage, then Project details. Open the Connected resources tab, select Add connection, pick Azure AI Search, browse to your search service, choose an authentication type, and select Add connection. SDK samples resolve the connection name to a connection ID when you create the agent.

Imagine a small shop that sells kitchen goods. Support gets the same questions about returns and shipping every week. The team wants a prompt agent that answers from official policy text, not from memory.

• File-search path: export returns_policy.pdf and shipping_faq.md, upload them into a vector store named ShopPolicies, attach file search with that store ID, and instruct the agent to search files before answering policy questions.
• Azure AI Search path: if policies already sit in a corporate index with title and url fields, create a project connection to the search service, point the Azure AI Search tool at index shop-policies with query_type vector_semantic_hybrid, and instruct the agent to always cite sources.
• Test with known questions: “How many days do I have to return a blender?” and “Do you ship to PO boxes?” Verify citations reference the correct document or URL.
• If answers ignore the files, confirm file_search is on the agent, vector_store_ids includes your store, and indexing finished; for search-index answers, set tool_choice to required during tests so the agent must call the search tool.

Once grounding works, tighten instructions (tone, when to escalate to a human) in Concept 5’s style, then add other tools in Concept 7. Knowledge tools supply facts; they do not replace clear instructions about when to use them.

• File search supports specific file formats and UTF-8, UTF-16, or ASCII text encodings; unsupported types fail at upload or indexing.
• A basic Foundry resource with basic agent deployments cannot use private Azure AI Search resources or search services with public access disabled and a private endpoint; use standard agent setup with virtual network injection for that pattern.
• If the agent runs without virtual network injection, the Azure AI Search tool may fail to reach a search service that only exposes a private endpoint even when public network access is enabled.
• Features and availability vary by region for file search; check Microsoft Foundry region support before you depend on a tool in production.
• When file search runs through a toolbox on a hosted agent, user isolation is not supported—all users in the project share access to vector stores referenced in the tool or supplied at runtime.

Budget for file search charges on top of model token usage. Watch vector store size and reindexing when policies change, and delete test files and stores when experiments end so storage and search fees do not linger.

• 401 or 403 errors usually mean a missing or expired token, or missing Foundry User / storage roles—refresh credentials and confirm role assignment finished propagating.
• Empty or irrelevant search results often mean indexing is incomplete or the query is too broad—poll until the vector store is completed and ask narrower questions.
• Missing citations mean the model skipped the tool or the index lacks url/title fields—instructions should require citations, and index schemas should include retrievable URL metadata.
• DNS or “cannot connect to host” errors with private networking and API keys mean the connection must switch to Microsoft Entra project managed identity instead of key-based authentication.

You can ground answers in uploaded files or in an existing Azure AI Search index. Concept 7 adds more agent tools—function calling, code interpreter, and patterns for choosing which tool runs when—so your shop assistant can look up policy and then call an order API in one flow.

• Upload files, vector stores, chunk defaults, and troubleshooting: File search tool for agents
• Index requirements, connections, query types, and private networking: Connect an Azure AI Search index to Foundry agents
• Pick vector stores per request without new agent versions: Customize agent behavior at runtime with structured inputs
• When to call each tool and how to validate tool usage: Tool best practices for Microsoft Foundry Agent Service