AI enrichment transforms unstructured content into searchable, structured data during the indexing process. When you connect Azure AI Search to blob storage containing PDFs, images, or Office documents, the raw content is just binary data. AI enrichment applies cognitive skills that extract, infer, and restructure information — turning a scanned PDF into a searchable document with recognized entities, extracted key phrases, detected languages, and translated text. The enrichment pipeline sits between the data source and the search index, executing a skillset that processes each document as it flows through the indexer.
Enrichment is defined through a skillset — a collection of skills that execute sequentially or in parallel. Each skill has inputs (values from the document's in-memory enrichment tree), a context (the node in the enrichment tree where the skill executes), and outputs (values written back to the enrichment tree). The enriched content can flow into a search index, a knowledge store in Azure Storage, or both. On the AI-102 exam, skillsets appear prominently in the "Knowledge Mining" domain. You should understand the built-in skills, how to chain them, how to create custom skills, and how to project enriched data to a knowledge store.
Azure AI Search provides over a dozen prebuilt skills that cover text analytics, image processing, and document transformation. These skills are backed by Azure AI Services (Vision and Language) and require an attached Azure AI Services multi-service resource in the same region as your search service.
| Skill | Category | What It Extracts |
|---|---|---|
| OCR | Image | Printed and handwritten text from images and PDFs |
| Image Analysis | Image | Tags, captions, objects, brands, and celebrities from images |
| Entity Recognition | Text | Named entities: people, organizations, locations, dates, quantities |
| PII Detection | Text | Personally identifiable information: emails, phone numbers, SSNs |
| Key Phrase Extraction | Text | Key talking points and important phrases |
| Language Detection | Text | Primary language of the text |
| Sentiment Analysis | Text | Positive/negative/neutral sentiment scores per document |
| Text Translation | Text | Translation from detected source language to target language |
| Merge | Utility | Merges multiple text fields into a single field |
| Split | Utility | Splits large text into manageable chunks (pages, sections) |
| Conditional | Utility | Conditionally assigns values based on expressions |
| Document Cracking | Utility | Extracts text and metadata from PDF, Office, HTML, CSV files |
| Shaper | Utility | Restructures the enrichment tree into a custom shape |
Document cracking is the first skill that executes automatically — the indexer extracts text and metadata from source documents before any cognitive skills run. PDFs are cracked into pages, Office documents into their text content and embedded images, and CSV files into rows. Images embedded in documents are extracted and passed to the OCR or Image Analysis skills. The cracking process preserves document structure while making the content available for downstream enrichment.
A skillset is defined in JSON and contains an array of skills. Each skill specifies its @odata.type (the skill type identifier), context (the enrichment node path, such as /document or /document/pages/*), inputs (source data from the enrichment tree), and outputs (target names that become new enrichment nodes). Skills can be chained — the output of one skill becomes the input of another. The context path determines whether a skill runs once per document (/document) or once per item in a collection (/document/pages/*).
The following example shows a skillset that runs OCR on images, then extracts entities and key phrases from the OCR output:
{
"name": "document-enrichment-skillset",
"skills": [
{
"@odata.type": "#Microsoft.Skills.Vision.OcrSkill",
"context": "/document/normalized_images/*",
"defaultLanguageCode": "en",
"detectOrientation": true,
"inputs": [
{ "name": "image", "source": "/document/normalized_images/*" }
],
"outputs": [
{ "name": "text", "targetName": "ocrText" }
]
},
{
"@odata.type": "#Microsoft.Skills.Text.MergeSkill",
"context": "/document",
"inputs": [
{ "name": "text", "source": "/document/content" },
{ "name": "itemsToInsert", "source": "/document/normalized_images/*/ocrText" }
],
"outputs": [
{ "name": "mergedText", "targetName": "mergedText" }
]
},
{
"@odata.type": "#Microsoft.Skills.Text.EntityRecognitionSkill",
"context": "/document",
"categories": ["Person", "Organization", "Location"],
"defaultLanguageCode": "en",
"inputs": [
{ "name": "text", "source": "/document/mergedText" }
],
"outputs": [
{ "name": "persons", "targetName": "persons" },
{ "name": "organizations", "targetName": "organizations" },
{ "name": "locations", "targetName": "locations" }
]
},
{
"@odata.type": "#Microsoft.Skills.Text.KeyPhraseExtractionSkill",
"context": "/document",
"defaultLanguageCode": "en",
"inputs": [
{ "name": "text", "source": "/document/mergedText" }
],
"outputs": [
{ "name": "keyPhrases", "targetName": "keyPhrases" }
]
}
]
}When the built-in skills do not meet your needs, you can create custom skills that call your own Web API endpoints. A custom skill is an Azure Function, a Logic App, or any publicly accessible HTTPS endpoint that accepts a JSON request body and returns a JSON response. The request contains the skill's inputs in a standardized format — a values array where each entry has a recordId and data object with the input values. The response must follow the same structure, returning a values array with matching recordId entries and the computed data outputs.
Custom skills are ideal for domain-specific transformations — classify documents into custom taxonomies, extract custom entities not covered by the built-in entity recognition, validate data against business rules, or call third-party APIs for specialized analysis. The skill runs as part of the enrichment pipeline, with the same failure handling and retry logic as built-in skills.
// C# - custom skill as an Azure Function
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Newtonsoft.Json;
public static class CustomEntitySkill
{
[FunctionName("CustomEntitySkill")]
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequest req)
{
string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
var request = JsonConvert.DeserializeObject<SkillRequest>(requestBody);
var response = new SkillResponse();
foreach (var record in request.Values)
{
var text = record.Data["text"] as string;
var entities = ExtractCustomEntities(text);
response.Values.Add(new SkillResult
{
RecordId = record.RecordId,
Data = new Dictionary<string, object>
{
{ "entities", entities }
}
});
}
return new OkObjectResult(response);
}
private static List<string> ExtractCustomEntities(string text)
{
// Custom entity extraction logic
var entities = new List<string>();
var patterns = new Dictionary<string, string>
{
{ @"\bPO-\d{5}\b", "PurchaseOrder" },
{ @"\bINV-\d{6}\b", "InvoiceNumber" },
{ @"\b[A-Z]{3}\d{6}\b", "PartNumber" }
};
foreach (var pattern in patterns)
{
var matches = System.Text.RegularExpressions.Regex.Matches(text, pattern.Key);
foreach (System.Text.RegularExpressions.Match match in matches)
entities.Add($"{pattern.Value}: {match.Value}");
}
return entities;
}
}
public class SkillRequest
{
public List<SkillRecord> Values { get; set; }
}
public class SkillRecord
{
public string RecordId { get; set; }
public Dictionary<string, object> Data { get; set; }
}
public class SkillResponse
{
public List<SkillResult> Values { get; set; } = new();
}
public class SkillResult
{
public string RecordId { get; set; }
public Dictionary<string, object> Data { get; set; }
public List<string> Errors { get; set; }
public List<string> Warnings { get; set; }
}# Python - custom skill as an Azure Function
import azure.functions as func
import json
import re
def main(req: func.HttpRequest) -> func.HttpResponse:
body = req.get_json()
values = body.get("values", [])
results = []
for record in values:
text = record.get("data", {}).get("text", "")
entities = extract_custom_entities(text)
results.append({
"recordId": record["recordId"],
"data": {"entities": entities}
})
return func.HttpResponse(
json.dumps({"values": results}),
mimetype="application/json"
)
def extract_custom_entities(text):
entities = []
patterns = [
(r"\bPO-\d{5}\b", "PurchaseOrder"),
(r"\bINV-\d{6}\b", "InvoiceNumber"),
(r"\b[A-Z]{3}\d{6}\b", "PartNumber")
]
for pattern, label in patterns:
for match in re.finditer(pattern, text):
entities.append(f"{label}: {match.group()}")
return entitiesA knowledge store is a destination in Azure Storage where the enriched document tree is projected after the skillset executes. Unlike the search index, which is optimized for full-text search queries, the knowledge store preserves the complete enrichment tree in a structured format that can be analyzed with Power BI, Azure Data Factory, or other data tools. It supports three projection types: table projections (relational tables in Azure Table Storage or Azure SQL), object projections (structured JSON in Blob Storage), and file projections (binary files like extracted images or normalized images).
Table projections store enriched data in rows and columns, with foreign key references between tables to preserve the document hierarchy. Object projections store the full enrichment tree as a JSON blob, preserving the nested structure. File projections store extracted binary content — such as images extracted from PDFs — in a blob container. You can define multiple projections in a single knowledge store, targeting the same or different storage accounts. The knowledge store is defined within the skillset JSON under the cognitiveServices section.
{
"name": "document-enrichment-skillset",
"skills": [...],
"knowledgeStore": {
"storageConnectionString": "DefaultEndpointsProtocol=https;AccountName=mystorage;AccountKey=...;",
"projections": [
{
"tables": [
{
"tableName": "Documents",
"generatedKeyName": "DocumentId",
"source": "/document"
},
{
"tableName": "KeyPhrases",
"generatedKeyName": "PhraseId",
"source": "/document/keyPhrases",
"sourceContext": "/document/keyPhrases/*",
"inputs": [
{ "name": "phrase", "source": "/document/keyPhrases/*" }
]
}
],
"objects": [
{
"storageContainer": "enriched-json",
"source": "/document"
}
],
"files": [
{
"storageContainer": "extracted-images",
"source": "/document/normalized_images/*"
}
]
}
]
}
}The AI-102 exam tests your understanding of when to use a knowledge store versus a search index. A search index is optimized for millisecond full-text search queries with scoring, faceting, and filtering. A knowledge store preserves the complete enrichment tree for downstream analytics, reporting, and data science workloads. Knowledge stores are not searchable through the Azure AI Search query API — they are accessed through Azure Storage tools. Use the index when you need a searchable UI or API. Use the knowledge store when you need to run analytical queries in Power BI, reprocess enriched data with Azure Data Factory, or keep a durable, queryable copy of the enriched content. You can generate both from the same skillset, which is the most common pattern.
Incremental enrichment is a feature that tracks which documents have changed and only re-processes those documents through the skillset, rather than re-enriching the entire corpus on every indexer run. It works by caching the enrichment output for each document in Azure Storage. When a document is updated, the cache for that document is invalidated, and only that document passes through the skillset again. Documents that have not changed retain their cached enrichment output, saving compute time and Azure AI Services costs.
To enable incremental enrichment, you configure an Azure Storage connection string on the indexer's cache property. When the cache is enabled, the indexer also writes the intermediate enrichment state to blob storage, allowing complex skillset pipelines to resume from the point of failure. This is especially valuable for long-running enrichment pipelines with expensive skills like OCR and translation. Without incremental enrichment, every indexer run re-processes every document, even if the skillset definition has not changed. With the cache, only documents that have changed or documents affected by a skillset change are reprocessed.
Skillset debugging can be challenging because the enrichment pipeline is a complex data flow with multiple processing steps. Start by checking indexer execution history in the Azure portal — the Indexer History tab shows each run, the number of processed documents, and any errors or warnings. Common errors include connection failures to the Azure AI Services resource (check that the resource is in the same region as the search service), skill timeouts (the default timeout for custom skills is 30 seconds), and input field path errors. Use the enriched document viewer (available through the Azure portal for indexes with enrichment) to inspect the enrichment tree for a specific document. This viewer shows the full JSON path of every enrichment node, making it easy to verify that skills are producing the expected outputs.
When a skill fails, the indexer continues processing other documents unless the maxFailedItems threshold is exceeded. Check the errors and warnings arrays in the indexer execution result — warnings indicate non-critical issues (such as a missing field), while errors cause the document to be skipped. For custom skills, enable Application Insights logging on the Azure Function and check the function logs for request/response payloads. Most custom skill issues are caused by incorrect response format — the response must have a values array with recordId matching the request, and each value must have data, errors, and warnings properties.
This exercise brings together all the components — data source, index, indexer, and skillset — to build a complete AI search pipeline that processes scanned PDFs and extracts entities and key phrases.
metadata_storage_path to the id field using a base64 encoding function.ShaperSkill at the end of the skillset to restructure the output into a flattened table projection for the knowledge store. Add a knowledge store definition and verify the projected tables in Azure Storage.AI enrichment in Azure AI Search applies cognitive skills to transform unstructured content into structured, searchable data during indexing. The built-in skills cover OCR, image analysis, entity recognition, key phrase extraction, language detection, sentiment analysis, translation, and text utility operations. Custom skills extend the pipeline with domain-specific logic through Azure Functions or any HTTPS endpoint. The skillset definition specifies skill dependencies through context paths, inputs, and outputs. Knowledge stores project enriched data to Azure Storage for analytics and reporting. Incremental enrichment caches results to avoid reprocessing unchanged documents. Debugging involves checking indexer execution history, the enriched document viewer, and skill response format validation. On the exam, focus on skillset design, custom skill development, and the knowledge store vs. index distinction. The next chapter covers Azure AI Document Intelligence for extracting structured information from forms and documents.