Customisation
SSML
Control the avatar's speech, animations, and moreOverview
SSML (speech synthesis markup language) is an XML-like language that can be used to configure your avatar on the fly, ranging from affecting speech, to controlling your avatar's animation and even changing their outfit.
Our SSML commands usually begin with a trl-
prefix, and can be issued via chat
messages sent by either the user or the avatar. For example you could adjust your LLM's prompt
to send particular SSML messages in response to some natural language input, or you could hook
up a button in your application to dispatch an SSML command using the sendMessageToAvatar()
function in our SDK.
You can weave SSML tags with regular speech to affect the sentence that would be spoken, e.g. Let me think <trl-break duration="1.0" /> Ah yes, I remember now - <trl-config speechSpeed='20' /> you asked me to speak quicker!
When SSML tags are sent we automatically filter them where appropriate - e.g. in our client's chat component, or in setups where we have full control over the LLM=>TTS pipeline (to prevent the avatar trying to 'speak' the tag).
Avatar SSML
This section documents SSML tags that relate to your avatar's configuration.
Index
Below you can find all of the available avatar SSML commands, with compatibility indicators for each avatar type (IBA, 3D, VBA). Click on one of the command links to scroll to the relevant documentation section.
<trl-anim />
IBA partially supported
3D supported
VBA partially supported
<trl-break />
IBA partially supported
3D partially supported
VBA partially supported
<trl-config />
IBA partially supported
3D partially supported
VBA partially supported
<trl-state />
IBA supported
3D supported
VBA supported
<trl-content />
IBA partially supported
3D partially supported
VBA partially supported
<trl-morph />
IBA supported
3D supported
VBA supported
<trl-repeat />
IBA not supported
3D not supported
VBA not supported
<trl-start-repeat />
IBA not supported
3D not supported
VBA not supported
<trl-anim />
IBA partially supported
3D supported
VBA partially supported
Allows you to combine animations to animate the avatar's head, face, mouth, etc.
Auxiliary and Core animations
Animations can be played in two different modes - aux
and core
.
Animations played with the core type influence the entire body, only one core animation can play
at a time. You may find it helpful to think of core animations as a 'base' animation, defining
what the avatar's body is doing overall. Issuing two separate commands with the core animation
type will result in the animations being queued, blending from the end of one to the start of the
other.
Aux animations are additive and don't result in queueing, they are instead layered on top of the animations that are already playing. Core animations usually affect the entire body whereas aux animations are supposed to target a few specific bones to achieve some localised animation. This means you could achieve a setup where a core animation makes an avatar stand idle (lightly move/sway), but an aux animation that targets just the neck/head is layered on top to make them nod their head.
Animation IDs
Expand the collapsed section below to see a full list of Animation IDs.
Example usage
Head movements (small, large, shoulder translations)
<trl-anim type='aux' id='ANIM_NAME' duration='1.8' blendStart='0.8' blendEnd='1.0'/>
Core Body
<trl-anim type='core' id='coreNoddingSmall' duration='4' />
Arm & hand movements
<trl-anim type='aux' id='FingersSqueeze' blendStart='1' blendEnd='2'/>
Head & upper body movements
<trl-anim type='aux' id='noddingSmall' duration='3' blendStart='0.5' blendEnd='0.5'/>
Facial Expressions
<trl-anim type='aux' id='mouthOpen' duration='3.5' blendStart='0.5' blendEnd='0.5'/>
Viseme mouth shapes
<trl-anim type='aux' id='viseme' duration='4.0' blendStart='0.5' blendEnd='0.5' />
Parameter | Description |
---|---|
type | Animation type - 'aux' or 'core' |
id | Animation name |
duration | Animation duration in seconds |
blendStart | Blend-in time in seconds |
blendEnd | Blend-out time in seconds |
<trl-break />
IBA partially supported
3D partially supported
VBA partially supported
Introduces a delay in speech.
Example usage
Break duration example
<trl-break duration="1.0" />
Parameter | Description |
---|---|
duration | Speech duration delay in seconds |
<trl-config />
IBA partially supported
3D partially supported
VBA partially supported
Example Usage
Set the speed at which speech is spoken.
<trl-config speechSpeed='20' />
Toggle subtitles on/off
<trl-config subtitlesEnabled="true" />
Interrupt avatar talking with new avatar speech
<trl-config interruptAvatarTalkingWithNewAvatarSpeech="true" />
Ignore user speech while the avatar is talking
<trl-config ignoreUserSpeechWhileAvatarTalking="true" />Whilst the avatar is saying this sentence, the user should be ignored.<trl-config ignoreUserSpeechWhileAvatarTalking="false" />
Change the TTS voice
<trl-config TTSVoice='Joanna' />
<trl-config TTSService="polly" TTSVoice="MatthewNTTS" />
<trl-config TTSService="azure" TTSVoice="ur-PK-UzmaNeural" AzureLanguage="ur-PK" />
Make the avatar look at a point and animate
<trl-config lookat='5,0,0' enable='true' speed='4' wait='2.5' returnSpeed='3.5' absolute='false' /><trl-anim type='aux' id='translateRotateLeftSmall' duration='2.5' />
Reload the avatar config
<trl-config reloadConfig="true" />
Assign key to app or character
<trl-config type="App" key="example-key" value="example-value" /><trl-config type="Character" key="example-key" value="example-value" />
Enable VNS animations
<trl-config enableVNSAnims="true" />
Enable auto animations
<trl-config enableAutoAnims="true" />
Reset session
<trl-config resetSession="true" />
Fullscreen mode
<trl-config FullscreenMode="true" />
<trl-state />
IBA supported
3D supported
VBA supported
Example Usage
Enable listening state
<trl-state enableListening="true" />
<trl-content />
IBA partially supported
3D partially supported
VBA partially supported
Example Usage
Queue command example
<trl-content queue="true" /> Could you repeat that?
Show Background Content
<trl-content ShowBGScreen="true" />
Background Videos
<trl-content BgScreenUrl="URL" /><trl-content BgScreenUrl="https://upload.wikimedia.org/wikipedia/commons/thumb/c/c1/Pacifica_Pier_CA.jpg/1920px-Pacifica_Pier_CA.jpg" />
Background Images
<trl-content BgScreenUrl="https://upload.wikimedia.org/wikipedia/commons/thumb/c/c1/Pacifica_Pier_CA.jpg/1920px-Pacifica_Pier_CA.jpg" />
Foreground Videos
<trl-content position="ScreenAngledSmallLeft" fade="false" speed="1.0" fadetime="0.2" showMovingScreen="true" /><trl-content screen="https://storage.googleapis.com/gtv-videos-bucket/sample/ForBiggerMeltdowns.mp4" />
Foreground Images
<trl-content position="ScreenAngledSmallLeft" fade="false" speed="1.0" fadetime="0.2" showMovingScreen="true" /><trl-content screen="https://jooinn.com/images/brighton-pier-3.jpg" />
Background Color
<trl-content r="0" g="0" b="0" a="0" />
Transitions
<trl-content position="DefaultCenter" fade="false" speed="1.0" fadetime="0.2" showMovingScreen="true" /><trl-content position="ScreenFlatLargeAvatarSmallLeft" fade="false" speed="1.0" fadetime="0.2" showMovingScreen="true" />
Avatar Location
<trl-content position="ANY" location="0.1,-0.1,-2" rotation="0,20,0" speed="1" />
Foreground YouTube
<trl-content screen="https://www.youtube.com/embed/CYXKi2XF5pM?loop=1&autoplay=1&playlist=CYXKi2XF5pM" />
Lighting
<trl-content lightingId="2" />
Lighting Tint
<trl-content lightingId="1" TintColor="1.0,1.0,1.0,1.0" />
Lighting With Blend
<trl-content lightingId="7" lightblending="true" transitionSpeed="10" />
Lighting With Fade
<trl-content lightingId="1" fade="true" fadeInTime="1" darkTime="2" fadeOutTime="1" />
Skylight Intensity
<trl-content lightingId="16" skylightIntensity="0.2" lightblending="true" blendingSpeed="1" />
Screen Intensity
<trl-content screenIntensity="0.4" />
Change Avatar
<trl-content character="amanda" sex="female" />
Subtitles
<trl-content subtitlesEnabled="true" />
<trl-morph />
IBA supported
3D supported
VBA supported
Expand the collapsed section below to see a full list of morph targets (blend shapes).
Example Usage
Blend shape manipulation
<trl-morph id="tongueOut" min="0" max="1.0" speed="1" wait="2" />
<trl-repeat />
IBA not supported
3D not supported
VBA not supported
Repeat the previously spoken text.
<trl-start-repeat />
IBA not supported
3D not supported
VBA not supported
Restrict to repeat only part of the text.