Azure AI Foundry Tutorial: Build Enterprise AI Apps

Enterprise teams no longer ask whether to use generative AI — they ask how to ship it safely, measurably, and on Azure. Scattered notebooks, ad-hoc API keys, and prompt copy-paste in Slack do not pass security review. Azure AI Foundry consolidates model access, prompt engineering, agents, evaluation, and deployment into one platform — and this Azure AI Foundry tutorial walks you from empty subscription to production-ready patterns.

Whether you are a developer prototyping a RAG chatbot, an architect designing a governed AI landing zone, or a CTO evaluating build-vs-buy, you will learn what AI Foundry provides, how its pieces connect, and what enterprises do differently from demo-day clickthroughs.

Why Azure AI Foundry Matters in 2026

Microsoft repositioned its AI developer story around Foundry because enterprises needed more than a model endpoint. They needed:

• A model catalog with consistent deployment and billing across providers
• Prompt flow for visual and code-based orchestration with versioning
• Agents combining tools, retrieval, and multi-step reasoning
• Evaluation frameworks to gate releases with metrics, not gut feel
• Governance tying projects to Azure RBAC, policies, and private networking

This Azure AI Foundry tutorial focuses on practical implementation — the same capabilities Emerrank deploys for clients building copilots, support agents, and document intelligence on Azure.

What Is Azure AI Foundry?

Azure AI Foundry (formerly Azure AI Studio) is Microsoft's cloud platform for the full generative AI lifecycle: explore models, design prompts and flows, build agents, run evaluations, and deploy endpoints consumed by your applications.

It sits above raw Azure OpenAI resources and connects to Azure AI Search, Storage, Key Vault, Application Insights, and Microsoft Entra ID — so your AI app is part of your Azure estate, not a shadow IT experiment.

Azure AI Foundry Portal
├── Hub (governance, connections, policies)
│   └── Project (prompts, flows, agents, data)
│       ├── Model deployments
│       ├── Prompt flow graphs
│       ├── Agent configurations
│       └── Evaluation runs
└── Connected Azure services (OpenAI, Search, Storage)

Prerequisites for This Azure AI Foundry Tutorial

• Azure subscription with Contributor access (or custom RBAC for AI Foundry)
• Approved region with model quota (verify GPT-4o or your chosen model)
• Azure CLI and Python 3.10+ optional for SDK workflows
• Enterprise: policy approval for external model providers if required by compliance

Step 1: Create a Hub and Project

Start in the Azure AI Foundry portal.

1. Select Create a hub — choose subscription, resource group, region, and storage
2. Enable connections to Azure OpenAI and other model providers as needed
3. Inside the hub, Create a project for your application team
4. Assign RBAC — developers on the project, readers on shared hubs, security on policy

Step 2: Deploy a Model from the Catalog

AI Foundry's model catalog lists Azure OpenAI models (GPT-4o, GPT-4, embeddings) and partner models depending on region and agreement.

1. Open Model catalog → select a model → Deploy
2. Choose deployment name, capacity (TPM), and target AI Services resource
3. Wait for provisioning — failed deploys usually indicate quota or naming issues
4. Copy endpoint URL and authentication details for application integration

# Call your deployment from Python (Azure OpenAI SDK)
from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-06-01"
)

response = client.chat.completions.create(
    model="gpt-4o-deployment-name",
    messages=[{"role": "user", "content": "Summarize our refund policy in three bullets."}]
)
print(response.choices[0].message.content)

Production apps should use managed identity instead of API keys — grant the app identity Cognitive Services OpenAI User on the resource.

Step 3: Build a Prompt Flow

Prompt flow is AI Foundry's orchestration layer — chain prompts, Python nodes, LLM calls, and conditional logic visually or in YAML.

Tutorial: Simple RAG Flow

1. Create Azure AI Search index with your documents (see our RAG applications guide)
2. In AI Foundry project → Prompt flow → Create → Chat flow template
3. Add retrieval node pointing to your search index connection
4. Add LLM node with system prompt instructing grounded answers only from context
5. Test in playground with sample questions; inspect retrieved chunks

# Simplified retrieval + generation concept in a flow node
def rag_node(question: str, search_client, openai_client) -> str:
    chunks = search_client.search(question, top=5)
    context = "\n".join(c["content"] for c in chunks)
    prompt = f"Answer using context only:\n{context}\n\nQuestion: {question}"
    return openai_client.chat.completions.create(
        model="gpt-4o-deployment-name",
        messages=[{"role": "user", "content": prompt}]
    ).choices[0].message.content

Step 4: Build an AI Foundry Agent

Agents extend chat with tools — Azure AI Search, Azure Functions, custom OpenAPI tools, code interpreter. The model decides when to invoke tools based on user intent.

Enterprise agent example: internal HR assistant with tools search_policy_docs, get_leave_balance (read-only API), and create_hr_ticket (write, requires approval in host app).

User question
    ↓
Foundry Agent (orchestrator model)
    ├── tool: search_policy_docs → Azure AI Search
    ├── tool: get_leave_balance → HR API
    └── synthesize answer with citations

Connect agent concepts with Model Context Protocol (MCP) when exposing tools across multiple hosts beyond Foundry.

Step 5: Evaluate Before Production

Demos lie; evaluations don't. AI Foundry includes built-in evaluators for groundedness, relevance, coherence, and safety.

1. Prepare a test dataset — question/answer pairs or golden conversations
2. Run evaluation against your prompt flow or agent version
3. Compare metrics across prompt iterations
4. Block promotion if groundedness score drops below threshold

Step 6: Deploy to Production

Deploy prompt flows and agents as managed endpoints within AI Foundry or export orchestration to your app using SDKs. Typical production architecture:

Web / mobile app
    ↓ HTTPS + Entra ID
Azure API Management
    ↓
App Service / Container Apps (your API)
    ↓ managed identity
AI Foundry endpoint + Azure OpenAI + AI Search

Enable Application Insights for request tracing, token counts, and latency. Align with our Azure OpenAI Solutions delivery patterns for enterprise hardening.

Azure AI Foundry vs Alternatives

Many teams combine Semantic Kernel in application code with models and search deployed through AI Foundry — not either/or.

Enterprise Architecture Patterns

AI Foundry in a Landing Zone

• Hub-spoke networking with private endpoints to OpenAI and AI Search
• Central AI Foundry hub per business unit; projects per product
• Azure Policy denying public network access on AI resources
• Cost management budgets and alerts per project subscription

Content Safety and Responsible AI

Enable Azure AI Content Safety filters on inputs and outputs. Configure blocklists for regulated terms. Log blocked requests for review. Document known limitations for legal and compliance stakeholders.

Security Best Practices

• Use managed identity everywhere; rotate remaining keys via Key Vault
• Scope RBAC at project level — not Owner on entire subscription for developers
• Disable public access; use Private Link for OpenAI, Search, Storage
• Redact PII in logs; avoid storing full prompts with sensitive data
• Implement API Management policies — rate limits, JWT validation, IP restrictions
• Review third-party model data processing terms for partner models in catalog

Performance and Cost Optimization

• Right-size model — GPT-4o-mini for classification; GPT-4o for complex reasoning
• Cache frequent retrieval results at API layer where freshness allows
• Batch embedding generation for index updates
• Set max tokens and timeouts on flows to prevent runaway costs
• Monitor TPM utilization; request quota increases before launch spikes
• Use provisioned throughput for predictable high-volume workloads

CI/CD for Prompt Flows and Agents

Treat prompts and flow YAML as code in Git. Pipeline stages:

1. Lint and unit test Python nodes
2. Run automated evaluation against staging project
3. Deploy flow to staging endpoint; run integration tests
4. Manual or automated promotion to production with approval gate

Reference: Azure AI Foundry documentation (Microsoft Learn).

Managing Hub Connections and Shared Assets

AI Foundry hubs centralize connections to Azure OpenAI, Azure AI Search, Storage accounts, and Key Vault. Projects inherit connections instead of each team storing credentials separately — a major enterprise win for rotation and audit.

When setting up connections in this Azure AI Foundry tutorial path, use managed identity on the hub where supported and document which projects may consume each connection. Read-only search indexes for production RAG should not share write keys with experimental dev projects.

• Register each connection with owner team and escalation contact
• Use separate search indexes per environment (dev/test/prod)
• Store API secrets in Key Vault; reference from connection configuration
• Review connection usage quarterly — remove orphaned links to retired resources

Monitoring, Observability, and FinOps

Production AI Foundry solutions require telemetry beyond the portal playground. Integrate Application Insights to capture dependency calls to OpenAI, search latency, and custom events for tool invocations.

FinOps teams should allocate token spend by project tags and subscription. Set Azure Cost Management alerts at 50%, 80%, and 100% of monthly AI budget. Unexpected spikes often trace to runaway agent loops or missing max-token limits.

Responsible AI and Human Oversight

Azure AI Foundry includes responsible AI tooling — content filters, transparency notes, and evaluation dimensions for harmful content. Enterprise programs should publish an AI use policy specifying:

• Approved use cases and prohibited data classes
• Human review requirements for high-risk outputs (medical, legal, financial advice)
• Feedback channels when users report incorrect or biased responses
• Incident response when safety filters fail or are bypassed

Azure AI Foundry vs Azure Machine Learning

Teams familiar with Azure ML ask how Foundry relates. Azure Machine Learning focuses on traditional ML — training custom models, MLOps for tabular and vision workloads, compute clusters for training jobs. Azure AI Foundry focuses on generative AI — prompt engineering, agents, foundation model deployment, RAG.

Many enterprises use both: AML for custom forecasting or computer vision models; AI Foundry for LLM applications and copilots. Data from AML pipelines can feed features into Foundry agents through APIs without merging the platforms into one confusing workspace.

Multi-Team Collaboration Model

Scale AI Foundry adoption with clear roles:

• Platform team — Owns hubs, policies, connections, approved model list
• AI engineers — Build flows, agents, evaluations within projects
• Application teams — Integrate Foundry endpoints into web and mobile apps
• Security / compliance — Review data flows, model terms, audit logs
• Operations — On-call for endpoint outages, quota, cost incidents

Weekly office hours reduce shadow experiments — teams show work-in-progress flows and get architecture feedback before production cutover.

Step-by-Step Walkthrough: First Agent in 30 Minutes

This condensed Azure AI Foundry tutorial sequence gets a working agent from zero to demo:

1. Sign in to ai.azure.com with an account that has Contributor on the target subscription.
2. Create hub — Choose region with GPT-4o availability; enable managed identity.
3. Create project — Name it support-agent-poc; link to the hub.
4. Deploy model — Azure OpenAI → Deployments → Add → select GPT-4o mini for cost-effective iteration.
5. Build agent — Agents → Create → define system prompt, attach Azure AI Search connection for knowledge base.
6. Test in playground — Ask three questions your support team receives daily; note failures.
7. Add evaluation — Import five golden Q&A pairs; run batch eval; fix prompt or retrieval settings.
8. Export endpoint — Copy REST URL and authenticate with managed identity from a test Azure Function.

Document each step with screenshots for internal runbooks. Teams that skip documentation repeat the same setup mistakes on every new project.

Integration Patterns for Line-of-Business Apps

Foundry rarely replaces your entire stack — it augments existing applications:

• Embedded copilot panel — React or Blazor widget calling Foundry agent API with streaming SSE responses
• Backend orchestration — .NET API mediates between CRM and Foundry; enforces business rules before LLM sees data
• Event-driven enrichment — Service Bus triggers Function that summarizes tickets via Prompt Flow batch node
• SharePoint / Teams — Microsoft Copilot extensibility or custom bot using same Foundry project backend

Keep the LLM behind your API gateway — never expose raw keys to browsers. Use short-lived tokens and per-user rate limits.

Common Mistakes in Azure AI Foundry Projects

• One project for everything — Dev prompts overwrite production experiments
• No evaluation dataset — Prompt tweaks regress quality silently
• API keys in source control — Immediate rotation and incident response
• Over-tooling agents — Too many tools confuse model routing
• Ignoring region data residency — Legal blockers late in program
• Skipping cost alerts — Viral internal demo burns monthly budget in days
• RAG without chunking strategy — Retrieval returns irrelevant context

Troubleshooting Tips

• 403 on model call — Check RBAC, deployment name, API version mismatch
• Empty RAG answers — Verify index has documents, skillset ran, query uses correct index
• Agent loops — Limit max tool calls; improve tool descriptions
• Slow flows — Profile each node; parallelize retrieval and safety checks where safe
• Evaluation failures — Confirm dataset format matches evaluator requirements

Real-World Scenario: Document Intelligence Portal

A logistics company used this Azure AI Foundry tutorial pattern to ship a contract Q&A portal in eight weeks. They created a hub with shared content safety policies, three projects (dev/test/prod), deployed GPT-4o with Azure AI Search over 40,000 PDFs, and gated releases with groundedness evaluation > 0.85. API Management exposed the endpoint to their .NET portal with Entra ID auth.

Outcome: legal review time for standard contract questions dropped 45%, with every answer citing retrieved clauses for auditability.

Related Topics Teams Search With This Tutorial

Azure AI Studio migration, AI Foundry hub vs project, prompt flow tutorial, Azure model catalog deployment, generative AI on Azure, AI agent Foundry, Azure AI Search RAG, content safety Azure, LLM evaluation pipeline, Azure OpenAI enterprise, AI Foundry pricing, responsible AI Azure, and Azure AI Foundry tutorial for beginners and architects.

Next Steps in Your Azure AI Foundry Tutorial Journey

After completing this walkthrough:

1. Add a formal evaluation dataset from real user questions (redacted)
2. Integrate managed identity and private endpoints before wider rollout
3. Connect observability dashboards for tokens, latency, and safety blocks
4. Explore AI agent development for multi-step autonomous workflows
5. Engage platform engineering to register your project in the corporate AI catalog

Conclusion

This Azure AI Foundry tutorial showed how hubs, projects, model deployments, prompt flows, agents, and evaluations fit together on Azure — the same building blocks enterprises use to move from prototype to governed production.

AI Foundry is not a magic deploy button. Success requires RBAC, networking, cost controls, and evaluation discipline. Teams that invest in those foundations ship AI features faster on the second and third project because reusable patterns already exist.

Need help standing up AI Foundry hubs, RAG architectures, or agent platforms on Azure? Contact Emerrank Consultancy Services for Azure OpenAI, AI development, and data engineering expertise.