Securing Azure AI solutions requires attention to three layers: network security, authentication and authorization, and key management. Network security controls which traffic can reach your service. Authentication ensures the caller is who they claim to be. Authorization determines what actions the caller can perform. A production-ready AI solution implements all three layers and combines them with monitoring to detect anomalous activity and cost governance to prevent budget overruns.
The AI-102 exam tests your ability to design secure AI solution architectures. You should understand when to use private endpoints versus service endpoints, how to configure managed identities for Azure services, and which RBAC roles grant the appropriate level of access. You should also understand how to detect security issues through diagnostic logs and metrics.
By default, Azure AI services are accessible from any internet-connected client that possesses the endpoint URL and authentication key. For many applications, this is too permissive. Virtual network integration restricts incoming traffic to only those requests originating from specified virtual networks, effectively making the service private to your Azure environment.
There are two approaches to network isolation: service endpoints and private endpoints. Service endpoints extend your virtual network identity to the Azure service over the Microsoft backbone network. They are easier to configure but do not remove the service from the public internet entirely. Private endpoints assign a network interface with a private IP address from your virtual network to the Azure service. All traffic to the service stays entirely within the Microsoft network and never traverses the public internet. Private endpoints also support network policies like network security groups and route tables.
To configure a private endpoint for an Azure AI service, you create a Private Endpoint resource in the Azure portal, specify the Azure AI service as the target, and select the virtual network and subnet. After creation, you must disable public network access on the AI service resource itself. The following Azure CLI commands accomplish this:
# Disable public network access
az cognitiveservices account update \
--name cv-ai102-study \
--resource-group rg-ai102-study \
--public-network-access Disabled
# Create a private endpoint
az network private-endpoint create \
--name pe-cv-ai102-study \
--resource-group rg-ai102-study \
--vnet-name vnet-ai102 \
--subnet default \
--private-connection-resource-id $(az cognitiveservices account show \
--name cv-ai102-study \
--resource-group rg-ai102-study \
--query id -o tsv) \
--group-id cognitiveservices \
--connection-name conn-cv-ai102-studyWhen you disable public network access, all clients outside the configured virtual network lose access to the service, including the Azure portal. Ensure your development machine is connected to the virtual network through VPN or ExpressRoute before disabling public access.
Managed identities provide Azure services with an automatically managed identity in Azure Active Directory. Instead of storing keys in connection strings or configuration files, your application uses the managed identity to obtain Azure AD tokens. This approach eliminates credential management from your code and improves security by removing long-lived secrets.
To use managed identity, you enable it on the Azure resource that calls your AI service (such as an Azure Function, App Service, or Virtual Machine), then grant that identity the appropriate RBAC role on the AI service. There are two types of managed identities. System-assigned identities are tied to a single resource and are deleted when the resource is deleted. User-assigned identities are independent resources that can be assigned to multiple Azure resources and have their own lifecycle.
The following example shows how to use managed identity to authenticate to a Language service from an Azure Function using the DefaultAzureCredential class:
// C# - using managed identity
using Azure.Identity;
using Azure.AI.TextAnalytics;
var credential = new DefaultAzureCredential();
var endpoint = Environment.GetEnvironmentVariable("LANGUAGE_ENDPOINT");
var client = new TextAnalyticsClient(new Uri(endpoint), credential);
DocumentSentiment sentiment = client.AnalyzeSentiment(
"Azure AI services are powerful and easy to use.");
Console.WriteLine($"Sentiment: {sentiment.Sentiment}");
Console.WriteLine($"Positive: {sentiment.ConfidenceScores.Positive:F2}");
Console.WriteLine($"Neutral: {sentiment.ConfidenceScores.Neutral:F2}");
Console.WriteLine($"Negative: {sentiment.ConfidenceScores.Negative:F2}");# Python - using managed identity
from azure.identity import DefaultAzureCredential
from azure.ai.textanalytics import TextAnalyticsClient
import os
credential = DefaultAzureCredential()
endpoint = os.environ["LANGUAGE_ENDPOINT"]
client = TextAnalyticsClient(endpoint=endpoint, credential=credential)
docs = ["Azure AI services are powerful and easy to use."]
for result in client.analyze_sentiment(docs):
print(f"Sentiment: {result.sentiment}")
print(f"Positive: {result.confidence_scores.positive:.2f}")
print(f"Neutral: {result.confidence_scores.neutral:.2f}")
print(f"Negative: {result.confidence_scores.negative:.2f}")When you cannot use managed identity, store your API keys in Azure Key Vault instead of configuration files or environment variables. Key Vault provides secure storage for secrets, keys, and certificates, with access policies that control who can read or modify them. Applications retrieve secrets at runtime using the Key Vault SDK, eliminating hardcoded secrets from your source code.
The integration pattern follows three steps. First, store the AI service key as a secret in Key Vault. Second, grant the application's identity read access to the Key Vault secrets using an access policy. Third, modify the application to retrieve the secret from Key Vault instead of reading from configuration.
// C# - retrieving secrets from Key Vault
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
var vaultUrl = new Uri("https://kv-ai102-study.vault.azure.net/");
var client = new SecretClient(vaultUrl, new DefaultAzureCredential());
// Retrieve the AI service key from Key Vault
KeyVaultSecret secret = client.GetSecret("vision-service-key");
string visionKey = secret.Value;
// Use the key to create the Vision client
var credential = new AzureKeyCredential(visionKey);
var endpoint = Environment.GetEnvironmentVariable("VISION_ENDPOINT");
var imageClient = new ImageAnalysisClient(new Uri(endpoint), credential);# Python - retrieving secrets from Key Vault
from azure.identity import DefaultAzureCredential
from azure.keyvault.secrets import SecretClient
vault_url = "https://kv-ai102-study.vault.azure.net/"
credential = DefaultAzureCredential()
client = SecretClient(vault_url=vault_url, credential=credential)
# Retrieve the AI service key from Key Vault
vision_key = client.get_secret("vision-service-key").value
# Use the key to create the Vision client
from azure.ai.vision import ImageAnalysisClient
from azure.core.credentials import AzureKeyCredential
endpoint = os.environ["VISION_ENDPOINT"]
image_client = ImageAnalysisClient(
endpoint=endpoint,
credential=AzureKeyCredential(vision_key)
)RBAC for Azure AI services lets you assign granular permissions to users, groups, service principals, and managed identities. Microsoft defines several built-in roles for Cognitive Services:
| Role Name | Permissions | Use Case |
|---|---|---|
| Cognitive Services Contributor | Full access to create, read, update, and delete resources | AI administrators and DevOps engineers |
| Cognitive Services User | Read keys and endpoints, use the service | Application developers and runtime identities |
| Cognitive Services Metrics Reader | Read-only access to metrics | Monitoring dashboards and read-only auditors |
| Cognitive Services Custom Vision Contributor | Full access to Custom Vision projects and training | Data scientists training custom models |
| Cognitive Services Custom Vision Labeler | Access to label images in Custom Vision | Labeling teams |
| Cognitive Services Custom Vision Trainer | View, train, and publish Custom Vision models | Model training role |
| Cognitive Services Custom Vision Deployment | Publish and unpublish Custom Vision models | Release management |
You can also create custom roles when the built-in roles do not provide the exact permissions you need. Custom roles define a set of allowed actions (e.g., Microsoft.CognitiveServices/accounts/read) and can be scoped to a subscription, resource group, or individual resource.
Exam questions frequently present a scenario with multiple personas (developer, data scientist, operator) and ask which role to assign to each. Remember that the Cognitive Services User role is sufficient for application code that calls the APIs, while the Contributor role is needed for creating and managing the resources themselves. The Custom Vision roles are more granular than the general Cognitive Services roles.
Azure Monitor aggregates metrics and logs from Azure resources, including AI services. For each AI service, Azure Monitor automatically collects platform metrics such as Total Calls, Successful Calls, Failed Calls, Data In, Data Out, and Latency (Response Time). These metrics are available in the Azure portal under the "Metrics" blade of the AI service resource. You can chart metrics over time, pin them to dashboards, and create alert rules that trigger when a metric crosses a threshold.
Metric alerts are critical for production AI solutions. Common alert rules include:
The following Azure CLI command creates a metric alert that fires when the number of failed calls exceeds 10 in a five-minute window:
az monitor metrics alert create \
--name "FailedCallsAlert" \
--resource-group rg-ai102-study \
--scopes $(az cognitiveservices account show \
--name cv-ai102-study \
--resource-group rg-ai102-study \
--query id -o tsv) \
--condition "Total Failures > 10" \
--window-size 5m \
--evaluation-frequency 1m \
--action-groups "/subscriptions/.../actionGroups/my-ag"While metrics provide aggregated data, diagnostic logs provide detailed records of individual requests. Enabling diagnostic logging requires configuring diagnostic settings on the AI service resource. You choose which log categories to collect and where to send them.
The available log categories for Azure AI services include:
You can send logs to three destinations. Log Analytics Workspace enables rich Kusto Query Language queries across logs from multiple resources. Azure Storage provides low-cost archival for compliance. Azure Event Hub streams logs to third-party monitoring tools like Splunk or Datadog.
To configure diagnostic settings using the Azure CLI:
az monitor diagnostic-settings create \
--name "ai102-logs" \
--resource $(az cognitiveservices account show \
--name cv-ai102-study \
--resource-group rg-ai102-study \
--query id -o tsv) \
--logs '[{"category":"Audit","enabled":true}]' \
--workspace /subscriptions/.../workspaces/log-analytics-wsOnce diagnostic logs flow into a Log Analytics workspace, you can query them using Kusto Query Language (KQL). The following query retrieves the most recent 100 calls, showing the timestamp, caller IP, operation, and HTTP status code:
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| project TimeGenerated, CallerIpAddress, OperationName, HttpStatusCode
| order by TimeGenerated desc
| take 100Use more specific queries to identify patterns:
// Top 10 callers by volume
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| summarize CallCount = count() by CallerIpAddress
| top 10 by CallCount desc
// Failed requests in the last hour
AzureDiagnostics
| where ResourceProvider == "MICROSOFT.COGNITIVESERVICES"
| where HttpStatusCode >= 400
| where TimeGenerated > ago(1h)
| project TimeGenerated, CallerIpAddress, OperationName, HttpStatusCode, ResultSignature
| order by TimeGenerated descEvery Azure AI service enforces rate limits at the resource level. For the Free F0 tier, the limit is typically 20 calls per minute. For the Standard S0 tier, the limit varies by service and region, ranging from 10 to several thousand calls per second. When you exceed the limit, the service returns HTTP 429 (Too Many Requests) with a Retry-After header indicating the number of seconds to wait before retrying.
Your application should implement retry logic with exponential backoff when it receives a 429 response. The Azure SDKs include built-in retry policies that handle this automatically. You can configure the retry policy parameters such as the maximum number of retries, delay between retries, and the retry mode (fixed or exponential).
// C# - configuring retry policy in the SDK
using Azure;
using Azure.AI.TextAnalytics;
var options = new TextAnalyticsClientOptions
{
Retry =
{
Mode = RetryMode.Exponential,
MaxRetries = 5,
Delay = TimeSpan.FromSeconds(1),
MaxDelay = TimeSpan.FromSeconds(30)
}
};
var client = new TextAnalyticsClient(
new Uri(endpoint),
new AzureKeyCredential(key),
options);Cost management for AI services involves three activities: setting budgets, creating cost alerts, and analyzing spending patterns. Azure Cost Management provides a unified view of spending across all services.
Create a budget in the Azure portal under "Cost Management + Billing" with a monthly amount and alert thresholds at 50%, 90%, and 100% of the budget. Configure action groups to send email notifications when thresholds are crossed. For production applications, create budgets at the resource group level to track AI spending separately from other resources.
To analyze costs, use the Cost Analysis view. Filter by service name (e.g., "Cognitive Services") to see the breakdown by resource, region, and pricing tier. The Cost Analysis data also shows trends over time, helping you identify unexpected spikes before they become budget problems.
commitment tiers offer significant savings for predictable workloads. When you purchase a commitment tier, you commit to a specific monthly throughput and receive a lower per-transaction rate. The commitment tier pricing applies 24 hours after purchase. If your traffic drops below the committed amount, you still pay for the full commitment.
Different regions have different data residency requirements. Some organizations require that all AI processing occurs within specific geographic boundaries. Azure AI services are available in most Azure regions, but not all services are available in all regions. You should check the Azure Products by Region page to confirm availability for the specific AI services your solution needs.
For data residency, pay attention to where your data is processed and stored. When you call an Azure AI service in a specific region, your data is processed in that region. For container deployments, all processing occurs locally within the container, which addresses data residency concerns because data never leaves the host machine. However, the container still requires periodic billing updates to Azure.
The exam may present scenarios where you must choose between cloud APIs and containers based on data residency, connectivity, or compliance requirements. Containers are always the right choice when the data must not leave the premises.
The following checklist summarizes the security measures you should implement for production AI solutions:
Using the AI service you provisioned in Chapter 1 or Chapter 2, perform the following steps. First, navigate to the Diagnostic settings blade and create a new diagnostic setting. Enable the Audit log category and send it to a new Log Analytics workspace. Second, navigate to the Alerts blade and create a new metric alert rule that fires when the total number of failed calls exceeds 5 in any 5-minute period. Configure the alert to send an email to your own address. Third, induce a failure by calling the service with an invalid key using curl or a REST client. Wait for the alert to trigger and verify you received the email notification.
Security, monitoring, and governance form the operational backbone of any Azure AI solution. Network isolation with private endpoints, authentication via managed identities or Key Vault, and granular RBAC permissions protect your services from unauthorized access. Azure Monitor metrics and diagnostic logs provide visibility into service health and usage patterns. Budgets and cost alerts prevent financial surprises. Together, these practices ensure that your AI solution is secure, observable, and cost-effective. The next chapter begins our deep dive into specific AI capabilities, starting with Azure AI Vision for image analysis.