Azure AI Search (formerly Azure Cognitive Search) is a cloud-based search-as-a-service solution that delivers full-text search, faceted navigation, and AI-powered enrichment over your content. It acts as a search index that sits on top of your data stores — you push content into it or pull it from supported data sources, and it returns ranked search results with millisecond latency. The service is designed for building search experiences over heterogeneous content, including documents, databases, and blob storage.
Azure AI Search is a key topic in the AI-102 exam under the "Knowledge Mining" domain. You will be tested on index design, data source connections, indexer configuration, query syntax, and how to apply relevance tuning. The service competes with Elasticsearch and Algolia in the broader search market, but its deep integration with Azure data services and its AI enrichment pipeline give it a unique position in the Azure ecosystem.
The service is organized around four core components: search indexes (the schema that defines what fields can be searched, filtered, or sorted), data sources (connections to your content stores), indexers (automatic data import pipelines), and queries (the requests that retrieve and rank results). Understanding how these components interact is essential for both the exam and real-world knowledge mining solutions.
You create an Azure AI Search resource through the Azure portal, CLI, or ARM templates. During provisioning you select a pricing tier that determines storage limits, index count, query throughput, and available features. The Free (F) tier is limited to 50 MB of storage, 3 indexes, and 15,000 documents. It is suitable for prototyping and small demos but not for production workloads. The Basic (B) tier offers 2 GB of storage, 15 indexes, and supports higher query volumes with a shared replica. The Standard (S1, S2, S3) tiers scale from 25 GB up to 1.4 TB per partition, with support for multiple replicas and partitions for high availability and throughput. The Storage Optimized (L1, L2) tiers prioritize large document volumes over query throughput, offering up to 24 TB per partition.
| Tier | Storage | Max Indexes | Replicas | Use Case |
|---|---|---|---|---|
| Free (F) | 50 MB | 3 | 1 (shared) | Prototyping and demos |
| Basic (B) | 2 GB | 15 | 3 | Small production workloads |
| Standard S1 | 25 GB | 50 | 12 | General purpose production |
| Standard S2 | 100 GB | 50 | 12 | Large-scale production |
| Standard S3 | 1.4 TB | 50 | 12 | Massive scale workloads |
| Storage Opt L1 | 1 TB | 10 | 6 | Document-heavy workloads |
| Storage Opt L2 | 24 TB | 10 | 6 | Extreme document volumes |
Scalability is managed through replicas and partitions. Replicas are copies of the search service that handle query load — adding replicas improves query throughput and provides high availability. Partitions divide the index across multiple storage units, allowing larger indexes and faster indexing throughput. For production, Microsoft recommends at least two replicas for SLA coverage (99.9% for Standard tiers) and two partitions for indexes larger than 50 GB.
A data source tells Azure AI Search where your content lives and how to connect to it. The supported data source types include Azure SQL Database, Azure Cosmos DB (SQL, MongoDB, and Gremlin APIs), Azure Blob Storage, Azure Table Storage, Azure Files, and Azure Data Lake Storage Gen2. Each data source is configured with a connection string, authentication credentials (key-based, managed identity, or SQL authentication), and optional container or query parameters. For Cosmos DB, you specify a collection and optionally a query that filters or projects the documents to index. For Blob Storage, you specify a container and optionally a file extension filter to limit which blobs are indexed.
// Creating a data source via REST API
POST https://{search-service}.search.windows.net/datasources?api-version=2024-07-01
Content-Type: application/json
api-key: {admin-key}
{
"name": "azure-sql-products",
"type": "azuresql",
"credentials": {
"connectionString": "Server=tcp:myserver.database.windows.net;Database=myproductsdb;User ID=myuser;Password=mypass;Trusted_Connection=False;Encrypt=True;"
},
"container": {
"name": "dbo.Products"
}
}When using managed identity authentication, you enable a system-assigned or user-assigned managed identity on the search service and grant it read access to the data source. This avoids embedding connection strings in the data source configuration and is the recommended approach for production. You can create a data source through the Azure portal, REST API, Azure SDK, or the Azure CLI.
A search index is the core schema that defines the structure of your searchable content. Each field in the index has a name, a data type (Edm.String, Edm.Int32, Edm.Double, Edm.Boolean, Edm.DateTimeOffset, Edm.GeographyPoint, or Collection types), and a set of attributes that control how the field participates in search queries. The five key attributes are searchable, filterable, sortable, facetable, and retrievable.
| Attribute | Description | Used In |
|---|---|---|
| searchable | Full-text searchable via Lucene analysis | Keyword and phrase searches |
| filterable | Can be referenced in $filter expressions | Exact match filtering |
| sortable | Can be used as a sort criterion | Ordering results by field |
| facetable | Can be used in faceted navigation | Category drill-down UI |
| retrievable | Returned in search results | Field selection in responses |
Fields that are searchable are analyzed through a Lucene analyzer that tokenizes the text, applies stemming and stop-word removal, and builds an inverted index for fast lookups. Non-searchable fields are stored as-is and can only be used for filtering, sorting, or faceting. Every index must have exactly one key field of type Edm.String that uniquely identifies each document. Complex types allow nested data structures, such as an "address" object containing street, city, and postalCode subfields.
{
"name": "product-index",
"fields": [
{ "name": "id", "type": "Edm.String", "key": true, "retrievable": true },
{ "name": "name", "type": "Edm.String", "searchable": true, "sortable": true, "retrievable": true },
{ "name": "description", "type": "Edm.String", "searchable": true, "retrievable": false },
{ "name": "category", "type": "Edm.String", "filterable": true, "facetable": true, "retrievable": true },
{ "name": "price", "type": "Edm.Double", "filterable": true, "sortable": true, "facetable": true, "retrievable": true },
{ "name": "rating", "type": "Edm.Double", "sortable": true, "filterable": true, "retrievable": true },
{ "name": "tags", "type": "Collection(Edm.String)", "filterable": true, "facetable": true, "retrievable": true },
{ "name": "lastUpdated", "type": "Edm.DateTimeOffset", "sortable": true, "filterable": true, "retrievable": true }
]
}You can create an index through the Azure portal by navigating to your search service and selecting "Add index" from the Overview blade. The portal provides a form-driven interface where you define field names, types, and attributes. For programmatic access, the REST API offers full control over index definition, as shown in the following Python example using the Azure SDK for Python:
from azure.core.credentials import AzureKeyCredential
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
SearchIndex, SearchField, SearchFieldDataType,
SimpleField, SearchableField
)
endpoint = "https://mysearch.search.windows.net"
admin_key = "YOUR_ADMIN_KEY"
client = SearchIndexClient(endpoint, AzureKeyCredential(admin_key))
fields = [
SimpleField(name="id", type=SearchFieldDataType.String, key=True),
SearchableField(name="name", type=SearchFieldDataType.String, sortable=True),
SearchableField(name="description", type=SearchFieldDataType.String),
SimpleField(name="category", type=SearchFieldDataType.String,
filterable=True, facetable=True),
SimpleField(name="price", type=SearchFieldDataType.Double,
filterable=True, sortable=True, facetable=True),
SimpleField(name="rating", type=SearchFieldDataType.Double,
filterable=True, sortable=True),
]
index = SearchIndex(name="product-index", fields=fields)
result = client.create_index(index)
print(f"Index '{result.name}' created successfully")// C# - creating an index with Azure SDK for .NET
using Azure;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Indexes.Models;
var endpoint = "https://mysearch.search.windows.net";
var credential = new AzureKeyCredential("YOUR_ADMIN_KEY");
var client = new SearchIndexClient(new Uri(endpoint), credential);
var index = new SearchIndex("product-index")
{
Fields =
{
new SimpleField("id", SearchFieldDataType.String) { IsKey = true },
new SearchableField("name") { IsSortable = true },
new SearchableField("description"),
new SimpleField("category", SearchFieldDataType.String)
{ IsFilterable = true, IsFacetable = true },
new SimpleField("price", SearchFieldDataType.Double)
{ IsFilterable = true, IsSortable = true, IsFacetable = true },
new SimpleField("rating", SearchFieldDataType.Double)
{ IsFilterable = true, IsSortable = true }
}
};
client.CreateIndex(index);
Console.WriteLine("Index created successfully");When creating indexes, you cannot add new searchable fields or change field attributes after the index is populated. To modify the schema of an existing index with data, you must either create a new index and rebuild it, or add new fields with the "allowIndexingOnNewFields" flag set on the index. Plan your schema carefully before indexing production data.
An indexer connects a data source to an index and automates the document ingestion process. It handles field mapping (mapping source fields to index fields), scheduled refresh (incremental indexing), and error handling. When you create an indexer, you specify the data source name, target index name, optional field mappings, a schedule interval (such as "every 5 hours"), and how to handle errors. Indexers support two execution modes: on-demand (run once) and scheduled (recurring).
Change tracking is critical for incremental indexing. Azure SQL Database and Cosmos DB support change detection through the built-in change tracking features (SQL's integrated change tracking or Cosmos DB's change feed). Blob Storage uses last-modified timestamps to detect new or updated blobs. When change tracking is not available, the indexer falls back to full re-indexing, which can be expensive for large datasets. For Azure SQL, you must enable change tracking on both the database and the table before the indexer can use it.
// Creating an indexer via REST API
POST https://{search-service}.search.windows.net/indexers?api-version=2024-07-01
Content-Type: application/json
api-key: {admin-key}
{
"name": "products-indexer",
"dataSourceName": "azure-sql-products",
"targetIndexName": "product-index",
"schedule": {
"interval": "PT2H"
},
"fieldMappings": [
{ "sourceFieldName": "ProductID", "targetFieldName": "id" },
{ "sourceFieldName": "ProductName", "targetFieldName": "name" },
{ "sourceFieldName": "Description", "targetFieldName": "description" }
],
"parameters": {
"batchSize": 100,
"maxFailedItems": 10,
"maxFailedItemsPerBatch": 5
}
}Azure AI Search supports two query parser modes: simple (default) and full Lucene. The simple syntax supports basic query operators — AND, OR, NOT, + (required), - (excluded), and quoted phrases for exact matching. A simple query like "wireless mouse" +Bluetooth -wired searches for documents containing the phrase "wireless mouse," that must include "Bluetooth," and must not include "wired."
The full Lucene syntax adds advanced operators — ~ for fuzzy search (e.g., bluetooth~ matches "bluetooth," "bluetoot," "bluetch"), ^ for field boosting (e.g., title:laptop^3 boosts matches in the title field by 3x), * and ? for wildcard search (e.g., comput* matches "computer," "computing," "computation"), ~n for proximity search (e.g., "laptop battery"~5 matches documents where the words are within 5 terms of each other), and regular expressions enclosed in /. To use full Lucene syntax, set the queryType parameter to full.
The AI-102 exam tests when to use each query type. Use simple syntax for standard keyword searches and metadata filtering — it is faster and sufficient for most e-commerce and content search scenarios. Use full Lucene when you need fuzzy matching for typo-tolerant search, proximity search for phrase matching with word gaps, regex for pattern matching, or field boosting for custom relevance ranking. Remember that full Lucene queries are more resource-intensive and slower, so only use them when the query complexity is justified by the search experience requirements.
Filters are expressed using the OData $filter parameter with standard comparison operators (eq, ne, gt, lt, ge, le), logical operators (and, or, not), and collection operators (in, any, all). A filter narrows the search space before scoring, which makes it very efficient. For example, $filter=category eq 'Electronics' and price le 1000 restricts results to electronics under $1,000. Sorting is done with $orderby — $orderby=rating desc, price asc sorts by rating descending then price ascending. Facets provide aggregated counts for drill-down navigation. A faceted query returns the count of documents in each category — facet=category,count:100 returns up to 100 category values with their document counts. Wildcard queries use * for multi-character and ? for single-character matching, and require the full Lucene query parser.
# Python - full query with filtering, faceting, and sorting
from azure.search.documents import SearchClient
from azure.core.credentials import AzureKeyCredential
client = SearchClient(
endpoint="https://mysearch.search.windows.net",
index_name="product-index",
credential=AzureKeyCredential("YOUR_QUERY_KEY")
)
results = client.search(
search_text="laptop",
filter="category eq 'Electronics' and price le 2000",
order_by=["rating desc", "price asc"],
facets=["category", "brand"],
select=["name", "price", "rating", "category"],
top=10,
query_type="simple"
)
print(f"Total results: {results.get_count()}")
print("Facets:")
for facet_name, facet_values in results.get_facets().items():
for value in facet_values:
print(f" {value['value']}: {value['count']}")
for doc in results:
print(f"{doc['name']} - ${doc['price']} - Rating: {doc['rating']}")Scoring profiles allow you to customize the relevance ranking of search results. By default, Azure AI Search computes a score for each matching document based on term frequency-inverse document frequency (TF-IDF) and the Okapi BM25 algorithm. A scoring profile lets you override this by adding weighted field boosts, magnitude-based functions (boost by recency or numeric value), freshness functions (boost newer documents), tag boosting (boost documents matching specific tags), and distance functions (boost by geographic proximity).
For example, a product search might use a scoring profile that boosts the title field by 5x, boosts recently updated products, and boosts products in the "Featured" category by 2x. Scoring profiles are defined in the index schema and referenced by name in queries. If no scoring profile is specified, the default BM25 scoring is used. You can also override scoring parameters at query time using the scoringParameter option to pass tag values or parameter values to the scoring function.
{
"name": "product-scoring",
"text": {
"weights": {
"name": 5,
"description": 2,
"tags": 3
}
},
"functions": [
{
"type": "freshness",
"fieldName": "lastUpdated",
"boost": 2,
"interpolation": "linear",
"freshness": {
"boostingDuration": "P30D"
}
},
{
"type": "magnitude",
"fieldName": "rating",
"boost": 1.5,
"interpolation": "constant",
"magnitude": {
"boostingRangeStart": 3,
"boostingRangeEnd": 5
}
}
]
}In this exercise you will provision an Azure AI Search service, create an index, index sample product data, and run queries with filtering and faceting.
SearchClient.upload_documents() method.Several index design strategies appear frequently on the exam. Use searchable attributes only on fields that genuinely need full-text search — each searchable field adds to the index size and query latency. Make fields filterable instead of searchable when you only need exact matches. Use facets for categorical drill-down rather than boolean filters because facets provide user-facing navigation counts. When performance is critical, limit the number of searchable fields and use the $select parameter to return only the fields the client needs. Use the query key (read-only key) for client-side queries and reserve the admin key for index management operations. For large datasets, prefer push-based indexing (direct document upload) over pull-based indexing (indexers) when you need real-time freshness, because indexers have a minimum scheduling interval of 5 minutes and each run incurs overhead.
Azure AI Search provides a managed search service with configurable indexes, automated indexers for popular Azure data sources, and flexible query syntax supporting simple and full Lucene modes. Index schema design determines what queries are possible — field attributes control searchability, filterability, sortability, and faceting. Indexers automate data import and support incremental refresh through change tracking. Scoring profiles fine-tune relevance ranking beyond the default BM25 algorithm. Queries support filtering (OData), sorting, faceted navigation, and advanced Lucene features like fuzzy, proximity, and wildcard search. On the AI-102 exam, focus on index design best practices, choosing between simple and full query syntax, and understanding when filters are more efficient than facets. The next chapter extends these concepts with AI enrichment, where cognitive skills transform and enrich content during the indexing process.