← Back to Tutorials Chapter 13

Text-to-Speech and SSML

Azure Text-to-Speech Overview

Azure Text-to-Speech (TTS) converts written text into natural-sounding audio. It is the output counterpart of Speech-to-Text, and together they form the core of Azure's speech AI capabilities. The service supports over 400 neural voices across 140+ languages and locales, each with distinct speaking styles, emotions, and tonal qualities. TTS is used in voice assistants, accessibility tools, audiobook generation, navigation systems, language learning applications, and any scenario where content needs to be consumed audibly rather than visually.

The service is accessed through the Speech SDK's SpeechSynthesizer class or through the REST API for simpler scenarios. The synthesizer accepts either plain text or SSML (Speech Synthesis Markup Language) as input. SSML gives you fine-grained control over every aspect of the synthesized speech — which voice to use, how fast to speak, where to pause, which words to emphasize, and how to pronounce specific terms. For the AI-102 exam, SSML knowledge is tested heavily, and you should be familiar with the most common SSML elements and their attributes.

Neural Voices: Natural-Sounding Synthetic Speech

Microsoft's neural TTS voices use deep neural networks to model the human voice production system. Unlike the older standard voices (which concatenated pre-recorded phonemes and sounded robotic), neural voices produce natural prosody, intonation, and emotional variation. Each neural voice is trained on hours of studio-quality recordings from a specific speaker and can express different speaking styles such as cheerful, sad, angry, excited, friendly, and professional.

Neural voices are categorized as prebuilt neural voices (available without customization) and custom neural voices (trained on your own recordings). Prebuilt voices include region-specific variants like "en-US-JennyNeural" (American English, female), "en-GB-RyanNeural" (British English, male), "ja-JP-NanamiNeural" (Japanese, female), and many others. The voice name follows a consistent naming convention: locale-SpeakerName-Neural. The "Neural" suffix distinguishes them from the deprecated standard voices. For the AI-102 exam, know that all new TTS development should use neural voices, and standard voices are available only for backward compatibility.

Speech Synthesis Markup Language (SSML)

SSML is an XML-based markup language that controls how text is spoken by a TTS engine. It wraps your text in a <speak> root element and uses child elements to control prosody, pronunciation, voice selection, and other speech attributes. The AI-102 exam expects you to read and write SSML that produces specific speech behaviors. The following sections cover the most important SSML elements.

Element	Purpose	Key Attributes
<voice>	Select the speaking voice	name (voice ID)
<lang>	Set the language for a portion of text	xml:lang (locale code)
<prosody>	Control pitch, rate, and volume	pitch, rate, volume
<break>	Insert a pause	time (duration), strength
<emphasis>	Emphasize a word or phrase	level (strong, moderate, reduced)
<phoneme>	Specify exact pronunciation	alphabet, ph (phoneme string)
<say-as>	Interpret text as a specific type	interpret-as (date, time, number, etc.)
<audio>	Insert a prerecorded audio clip	src (URL to audio file)
<mstts:express-as>	Set speaking style and emotion	style, styledegree, role

<break> — Controlling Pauses

The <break> element inserts a pause of a specified duration. Use the time attribute to set an exact duration in milliseconds (e.g., time="500ms") or seconds (e.g., time="2s"). The strength attribute provides named pause levels: x-weak, weak, medium (default), strong, and x-strong. Use breaks to separate sections of content, create dramatic pauses, or improve the natural rhythm of the speech.

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
    xml:lang="en-US">
  Welcome to our tutorial.
  <break time="1s"/>
  Today we will learn about text-to-speech.
  <break strength="strong"/>
  Let us begin with the basics.
</speak>

<emphasis> — Adding Stress to Words

The <emphasis> element increases the stress on a word or phrase, making it stand out from the surrounding speech. The level attribute controls the intensity: strong for maximum emphasis, moderate for normal emphasis, and reduced for de-emphasis. Use emphasis to highlight key terms, contrast two ideas, or convey urgency on specific words.

<phoneme> — Fixing Mispronunciations

The <phoneme> element is your primary tool for correcting how the TTS engine pronounces specific words. It uses phonetic alphabets to specify the exact pronunciation. The most common alphabet is the International Phonetic Alphabet (IPA), specified as alphabet="ipa". The ph attribute contains the phonetic transcription. For example, to make the TTS engine pronounce "Azure" correctly in a technical context, or to pronounce a rare name, you provide the phoneme string.

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
    xml:lang="en-US">
  The word <phoneme alphabet="ipa" ph="ˈreɪz.ər">razor</phoneme>
  sounds different from
  <phoneme alphabet="ipa" ph="ˈrɑː.zɚ">razor</phoneme>.
  The name <phoneme alphabet="ipa" ph="ɑːˈtʃiː">Arché</phoneme>
  is pronounced with a long E sound.
</speak>

<prosody> — Adjusting Rate, Pitch, and Volume

The <prosody> element is the most versatile SSML control. The rate attribute controls speaking speed as a percentage of the default (e.g., rate="80%" for slower, rate="150%" for faster). You can also use named values: x-slow, slow, medium, fast, x-fast. The pitch attribute controls the baseline pitch as a relative semitone value (e.g., pitch="+2st" for higher, pitch="-1st" for lower). The volume attribute controls loudness as a relative level (e.g., volume="+20%") or named value (silent, x-soft, soft, medium, loud, x-loud).

<say-as> — Interpreting Text Types

The <say-as> element tells the TTS engine how to interpret a span of text. This is critical for getting dates, times, numbers, and abbreviations spoken correctly. Common interpret-as values include: date (with format attribute like dmy, mdy, ymd), time (with format hms12 or hms24), cardinal (123 → "one hundred twenty three"), ordinal (1st → "first"), telephone (digits read individually), digits (123 → "one two three"), fraction (1/2 → "one half"), and spell-out (spells each letter).

<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
    xml:lang="en-US">
  The meeting is on
  <say-as interpret-as="date" format="mdy">3/15/2025</say-as>
  at <say-as interpret-as="time" format="hms12">14:30</say-as>.
  My order number is
  <say-as interpret-as="digits">9876</say-as>.
  Call <say-as interpret-as="telephone">800-555-0199</say-as>.
  Please read <say-as interpret-as="spell-out">SSML</say-as>
  as individual letters.
</speak>

<mstts:express-as> — Style and Emotion

The <mstts:express-as> element (Microsoft-specific extension) sets the speaking style for neural voices. The style attribute selects from styles like cheerful, sad, angry, excited, friendly, hopeful, shouting, whispering, and more. The styledegree attribute (0.01 to 2.0) controls the intensity of the style. The role attribute is available for some voices and lets you specify a character role (e.g., "YoungAdultFemale," "OlderAdultMale"). Not all styles are available for all voices — check the documentation for your specific voice's style list.

Custom Voice Models: Creating a Unique Voice from Recordings

Custom Neural Voice (CNV) lets you create a unique synthetic voice that sounds like a specific person. You provide recording studio-quality audio of a speaker reading a script (the training script), along with the matching transcriptions. Microsoft trains a neural voice model on this data, producing a voice that can say any text in the training speaker's voice. Custom voices are categorized as CNV Pro (trained on your recordings) and CNV Lite (trained on fewer recordings with a faster turnaround).

Custom voice training has significant requirements and restrictions. The minimum training data is 300-2000 recorded utterances (about 30 minutes to 2 hours of audio). The audio must be recorded in a professional studio with high-quality equipment, and the speaker must sign a consent form that Microsoft verifies. The trained voice is deployed to a custom endpoint and carries additional costs compared to prebuilt neural voices. Custom voices are subject to Microsoft's Responsible AI standards and undergo a content review before deployment. For the AI-102 exam, know the basic workflow but not the fine details of recording setup.

Audio Output: Streaming vs. File Output

The SpeechSynthesizer can output audio in two ways. Streaming output sends audio data in chunks through an event handler as the synthesis progresses. The callback receives AudioDataStream objects that can be played directly through speakers or processed further. File output writes the complete synthesized audio to a file on disk after synthesis completes. The output format is configurable — you can choose from various audio formats including Riff16Khz16BitMonoPcm, Riff24Khz16BitMonoPcm, and others. For web applications, you typically stream the audio to the browser; for offline content generation (like audiobooks), you save to files.

Code Sample: TTS from Text in Python

import os
import azure.cognitiveservices.speech as speechsdk

speech_key = os.environ["SPEECH_KEY"]
service_region = os.environ["SPEECH_REGION"]

speech_config = speechsdk.SpeechConfig(
    subscription=speech_key,
    region=service_region
)
speech_config.speech_synthesis_voice_name = "en-US-JennyNeural"

# Synthesize to speakers
synthesizer = speechsdk.SpeechSynthesizer(speech_config=speech_config)
text = "Hello and welcome to Azure Text-to-Speech. "
text += "I hope you are enjoying this tutorial."

result = synthesizer.speak_text_async(text).get()

if result.reason == speechsdk.ResultReason.SynthesizingAudioCompleted:
    print("Speech synthesized to speakers.")
elif result.reason == speechsdk.ResultReason.Canceled:
    cancellation = result.cancellation_details
    print(f"Synthesis canceled: {cancellation.reason}")
    print(f"Error details: {cancellation.error_details}")

# Synthesize to a WAV file
audio_config = speechsdk.audio.AudioOutputConfig(
    filename="output.wav"
)
synthesizer = speechsdk.SpeechSynthesizer(
    speech_config=speech_config,
    audio_config=audio_config
)
result = synthesizer.speak_text_async(text).get()

# Synthesize with SSML
ssml = """<speak version="1.0" xmlns="http://www.w3.org/2001/10/synthesis"
    xmlns:mstts="http://www.w3.org/2001/mstts"
    xml:lang="en-US">
  <voice name="en-US-JennyNeural">
    <prosody rate="10%">
      <emphasis level="strong">Listen carefully</emphasis>
      <break time="500ms"/>
      to this <say-as interpret-as="digits">AI</say-as>
      demonstration.
    </prosody>
  </voice>
</speak>"""

result = synthesizer.speak_ssml_async(ssml).get()

// C# - TTS from text and SSML
using Microsoft.CognitiveServices.Speech;
using Microsoft.CognitiveServices.Speech.Audio;

var speechKey = Environment.GetEnvironmentVariable("SPEECH_KEY");
var serviceRegion = Environment.GetEnvironmentVariable("SPEECH_REGION");

var speechConfig = SpeechConfig.FromSubscription(speechKey, serviceRegion);
speechConfig.SpeechSynthesisVoiceName = "en-US-JennyNeural";

using var synthesizer = new SpeechSynthesizer(speechConfig);

var text = "Hello and welcome to Azure Text-to-Speech.";
var result = await synthesizer.SpeakTextAsync(text);

if (result.Reason == ResultReason.SynthesizingAudioCompleted)
{
    Console.WriteLine("Speech synthesized.");
}

// SSML synthesis
var ssml = @"<speak version='1.0' ...>
  <voice name='en-US-JennyNeural'>
    Welcome to SSML synthesis.
  </voice>
</speak>";

result = await synthesizer.SpeakSsmlAsync(ssml);

Creating Audio Content for Accessibility Scenarios

Text-to-Speech is a critical accessibility tool. Screen readers, reading aids for dyslexic users, and audio versions of written content all rely on TTS. When building accessible applications, consider these practices: provide audio controls (play, pause, stop, speed adjustment) so users can control the speech output to match their needs. Use SSML to structure content with appropriate pauses between sections, making it easier to follow. Offer multiple voice options because users have different preferences for voice gender, accent, and speaking rate. For content that includes code blocks, data tables, or complex formatting, use the <say-as> element to ensure the information is conveyed clearly rather than read as raw text.

Summary

Azure Text-to-Speech converts text into natural-sounding speech using neural voices. Plain text synthesis is the simplest approach, but SSML provides full control over voice selection, speaking rate, pitch, volume, pauses, emphasis, pronunciation, style, and emotion. The key SSML elements are <voice>, <prosody>, <break>, <emphasis>, <phoneme>, <say-as>, and <mstts:express-as>. Custom Neural Voice lets you create a unique voice from personal recordings but requires significant training data and ethical approval. The Speech SDK provides the SpeechSynthesizer class for both streaming and file output. For the AI-102 exam, focus on SSML syntax and the differences between neural and standard voices. The next chapter covers Speech Translation for translating spoken language across multiple languages.