Azure OpenAI Service provides REST API access to OpenAI's powerful language models including GPT-4o, GPT-4 Turbo, GPT-3.5, text-embedding models, and DALL-E for image generation. The service is Microsoft's exclusive cloud provider for OpenAI models, offering the same capabilities as the OpenAI API with additional Azure features — enterprise-grade security, managed identity authentication, virtual network integration, private endpoints, responsible AI content filtering, and compliance certifications. Azure OpenAI models are deployed within Azure data centers, meaning your data stays within the Azure ecosystem and never leaves the Microsoft trust boundary.
On the AI-102 exam, Azure OpenAI appears in the "Implement Azure OpenAI" domain, which was added in the 2024 exam update. You should understand the available models, how to provision and deploy them, the prompt engineering fundamentals, content filtering, and responsible AI practices. The exam tests both conceptual understanding and practical implementation — expect scenario-based questions about model selection, prompt design, token management, and content safety.
Azure OpenAI resources are provisioned like other Azure AI services. You create an Azure OpenAI resource in the Azure portal, specifying a region and a pricing tier. The service is available in select regions — East US, South Central US, West US, West Europe, France Central, Canada East, Japan East, and others. Model availability varies by region — GPT-4o has broader deployment availability while GPT-4 Turbo is available in fewer regions. After creating the resource, you deploy specific models through the Azure AI Studio or the Azure CLI. Each deployment selects a model, version, and a capacity unit (in thousands of tokens per minute, or TPM).
Quota is managed at the subscription level per region per model family. When you create a deployment, you specify the TPM capacity. The total TPM across all deployments in a region cannot exceed the subscription quota for that model family. You can view current quota and request increases through the Azure portal's Quota blade or through Azure AI Studio. For production workloads, ensure you have sufficient quota before deploying. The pricing model is pay-per-token — you are billed for both input tokens (the prompt) and output tokens (the completion). Token counts are reported in the API response and can be monitored through Azure Monitor metrics.
# Deploy a model using Azure CLI
az cognitiveservices account deployment create `
--resource-group my-rg `
--name my-openai `
--deployment-name gpt-4o-deployment `
--model-name gpt-4o `
--model-version "2024-11-20" `
--model-format OpenAI `
--sku-name "GlobalStandard" `
--sku-capacity 100
# Available SKUs: Standard, GlobalStandard, ProvisionedManagedThe AI-102 exam covers three deployment SKUs. Standard provides pay-per-token pricing with regional capacity. GlobalStandard routes requests across multiple Azure regions for higher throughput and resilience. ProvisionedManaged reserves dedicated throughput capacity with predictable latency — this is the most expensive option and is used for high-volume production workloads with strict latency requirements.
Azure OpenAI offers several model families, each designed for different use cases. GPT-4o (omni) is the flagship multimodal model that accepts text and image inputs and produces text outputs. It is the fastest and most capable GPT-4 model, with support for vision, structured outputs, and function calling. GPT-4 Turbo is the previous-generation high-capacity model with a 128K token context window. GPT-3.5 Turbo offers lower cost and faster response times for simpler tasks. Text-Embedding-3 models convert text to vector embeddings for semantic search and RAG applications. DALL-E 3 generates images from text descriptions. Whisper provides speech-to-text capabilities.
| Model | Max Tokens | Input | Best For |
|---|---|---|---|
| GPT-4o | 128K | Text, images | General reasoning, vision, chat, code, structured outputs |
| GPT-4 Turbo | 128K | Text | Complex reasoning, long documents, advanced analysis |
| GPT-3.5 Turbo | 16K | Text | Simple Q&A, chat, cost-sensitive applications |
| text-embedding-3-large | N/A | Text | Semantic search, clustering, RAG vector stores |
| text-embedding-3-small | N/A | Text | Cost-effective embeddings, high throughput |
| DALL-E 3 | N/A | Text prompt | Image generation from natural language |
Prompt engineering is the practice of designing inputs to language models to produce desired outputs. The quality of the output is directly tied to the quality of the prompt. The Azure OpenAI chat completion API uses a structured conversation format with three message roles: system, user, and assistant. The system message sets the behavior and persona of the assistant ("You are a helpful customer support agent for a technology company"). User messages contain the actual requests or questions from the user. Assistant messages represent the model's responses, and can also be used to provide few-shot examples in the prompt.
Effective prompt engineering starts with a clear and specific system message. Instead of "Answer questions," use "You are an AI assistant that answers questions about Azure AI services. Provide concise answers with code examples when relevant. If you do not know the answer, say 'I do not have that information' rather than guessing." The user message should be direct and include necessary context. For complex tasks, break the instruction into steps — "1) Analyze the code, 2) Identify bugs, 3) Propose fixes, 4) Explain the reasoning."
Model parameters control the randomness and structure of the generated output. Temperature (range 0-2) controls randomness — lower values (0.0-0.3) produce focused, deterministic outputs ideal for factual Q&A and code generation; higher values (0.7-1.0) produce more creative, varied outputs suitable for brainstorming or creative writing. Top P (nucleus sampling) is an alternative to temperature — it limits the model to the top P probability mass of tokens. Setting top_p to 0.1 means the model only considers the top 10% most likely tokens. Typically you adjust either temperature or top_p but not both.
Max tokens limits the length of the generated response. This includes both the prompt tokens and the completion tokens — if the model exceeds the limit, the response is truncated. You can detect truncation by checking the finish_reason field in the response — stop means the model finished naturally, length means it reached the max token limit. Frequency penalty (range -2 to 2) discourages the model from repeating the same tokens — higher values reduce repetition. Presence penalty (range -2 to 2) encourages the model to talk about new topics — higher values increase topic diversity. Both penalties work by subtracting a penalty amount from the logit score of tokens that have already appeared (frequency) or have appeared at all (presence).
# Python - chat completions with parameter tuning
import os
from openai import AzureOpenAI
client = AzureOpenAI(
azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
api_key=os.environ["AZURE_OPENAI_KEY"],
api_version="2024-10-21"
)
response = client.chat.completions.create(
model="gpt-4o-deployment",
messages=[
{
"role": "system",
"content": "You are an Azure AI expert. Answer questions "
"concisely and provide code examples in Python."
},
{
"role": "user",
"content": "How do I analyze an image with Azure AI Vision?"
}
],
temperature=0.3,
max_tokens=500,
frequency_penalty=0.5,
presence_penalty=0.0
)
print(response.choices[0].message.content)
print(f"Token usage: {response.usage.total_tokens} "
f"(prompt: {response.usage.prompt_tokens}, "
f"completion: {response.usage.completion_tokens})")// C# - chat completions
using Azure;
using Azure.AI.OpenAI;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
var key = Environment.GetEnvironmentVariable("AZURE_OPENAI_KEY");
var client = new AzureOpenAIClient(
new Uri(endpoint),
new AzureKeyCredential(key));
var chatClient = client.GetChatClient("gpt-4o-deployment");
var response = chatClient.CompleteChat(
[
new SystemChatMessage("You are an Azure AI expert."),
new UserChatMessage("How do I analyze an image with Azure AI Vision?")
],
new ChatCompletionOptions
{
Temperature = 0.3f,
MaxTokens = 500
}
);
Console.WriteLine(response.Value.Content[0].Text);GPT models excel at code-related tasks. You can use Azure OpenAI to explain existing code, write new code from specifications, debug by providing error messages, refactor legacy code, generate unit tests, and translate between programming languages. The key to good code generation is a precise, complete specification — include the programming language, frameworks, input/output expectations, edge cases, and constraints. For debugging, include the full error message, the relevant code context, and what you expected to happen. For refactoring, specify the target style (e.g., "convert this synchronous code to use asyncio" or "refactor this to use dependency injection").
# Prompt: "Write a Python function that uses Azure AI Document Intelligence
# to extract invoice data from a PDF and returns a dictionary with
# vendor_name, invoice_date, and total fields."
def extract_invoice_data(pdf_path: str) -> dict:
from azure.core.credentials import AzureKeyCredential
from azure.ai.formrecognizer import DocumentAnalysisClient
import os
endpoint = os.environ["DOC_INTELLIGENCE_ENDPOINT"]
key = os.environ["DOC_INTELLIGENCE_KEY"]
client = DocumentAnalysisClient(
endpoint=endpoint,
credential=AzureKeyCredential(key)
)
with open(pdf_path, "rb") as f:
doc = f.read()
poller = client.begin_analyze_document("prebuilt-invoice", doc)
result = poller.result()
invoice = result.documents[0].fields
return {
"vendor_name": invoice.get("VendorName").value
if invoice.get("VendorName") else None,
"invoice_date": str(invoice.get("InvoiceDate").value)
if invoice.get("InvoiceDate") else None,
"total": invoice.get("InvoiceTotal").value.amount
if invoice.get("InvoiceTotal") else None
}Azure OpenAI includes built-in content filtering that screens both input prompts and model outputs for harmful content across four categories: hate, sexual, violence, and self-harm. Each category is scored at four severity levels (safe, low, medium, high). Content filtering runs automatically on every API call and cannot be disabled on Standard deployments. You can configure the severity thresholds per category — for example, allowing low-severity hate content while blocking medium and high. The filtering system also includes jailbreak detection (prompt shields) and protected material detection that checks if the model output matches copyrighted content.
Microsoft's Responsible AI principles guide the deployment and use of Azure OpenAI. The key principles are fairness (models should not discriminate), reliability and safety (models should function consistently and safely), privacy and security (data should be protected), inclusiveness (services should empower everyone), transparency (users should know they are interacting with AI), and accountability (organizations should take responsibility for their AI systems). For the exam, understand that you must implement transparency — your application should disclose when users are interacting with AI. You should also monitor model behavior, test for harmful outputs, and have a human-in-the-loop for high-stakes decisions.
The AI-102 exam tests your ability to choose the right model and configuration for a scenario. When the scenario requires low latency and low cost, choose GPT-3.5 Turbo. When it requires complex reasoning or long document processing, choose GPT-4o with the 128K context window. When image understanding is needed, choose GPT-4o (the only model that accepts image inputs). For token management, remember that the max token limit applies to both the prompt and the completion — a long prompt reduces the available space for the response. Use the max_tokens parameter to cap response length. Monitor token usage through the API response's usage field. For pricing, you are billed for input tokens and output tokens at different rates — output tokens are typically more expensive than input tokens. Prompt injection prevention is a key exam topic — always use a system message to establish behavior boundaries, validate and sanitize user inputs before including them in prompts, and consider using a separate moderation API (Content Safety) to scan user inputs for injection attempts.
Build a chat assistant using Azure OpenAI that answers technical questions about Azure AI services.
Azure OpenAI Service provides access to OpenAI's GPT-4o, GPT-4 Turbo, GPT-3.5, embedding, and DALL-E models within the Azure ecosystem. Provisioning involves creating a resource, deploying models with TPM capacity, and managing quota. Prompt engineering uses system, user, and assistant messages to guide model behavior. Parameters like temperature, top_p, max_tokens, frequency_penalty, and presence_penalty control output characteristics. Content filtering screens prompts and completions for harmful content across four categories. Microsoft's Responsible AI principles require fairness, reliability, privacy, inclusiveness, transparency, and accountability in AI applications. The exam focuses on model selection, prompt design, token management, and content safety. The next chapter covers advanced Azure OpenAI patterns including DALL-E image generation, function calling, embeddings, RAG, and fine-tuning.