Developer

SDK

Embedding the Trulience Client SDK inside your application

Introduction

The Trulience SDK provides tools to embed interactive avatars into web applications. It facilitates communication with the Trulience backend, allowing for real-time audio and visual end-user interactions with avatars.

Getting Started

Load the Trulience SDK

Include this in your HTML head or body element:

<script src="https://trulience.com/sdk/trulience.sdk.js"></script>

Creating a Trulience Object

The Trulience SDK provides a Builder class for creating and configuring Trulience objects. This simplifies the process of setting up communication with the Trulience backend.

let trulience = Trulience.Builder()
    .setAvatarId("your_avatar_id")
    .setUserName("Guest")
    .enableAvatar(true)
    .setRetry(true)
    .registerVideoElements(videoElements)
    .build();

Builder Methods

Use the following methods to configure the Trulience object:

Method	Description
`setAvatarId(string)`	Specify the avatar ID you wish to use
`setUserName(string)`	Set the name of the user accessing the avatar
`setUserId(string)`	Set a specific user ID (optional)
`enableAvatar(boolean)`	Enable/disable avatar access
`setRetry(boolean)`	Enable/disable automatic authentication retry
`registerVideoElements(object)`	Register the video element to display the avatar
`setCustomString(string)`	Include custom string in authentication request (for REST API)
`setToken(string)`	Provide JWT token for client validation
`build()`	Apply all settings and create the Trulience object

Video Element Registration Example

Specify the id of the video element where you would like to display the avatar:

let videoElements = { 
    remoteVideo: "remoteVideoElementId" 
};
trulience.registerVideoElements(videoElements);

Simple SDK Example

Below is a basic implementation that demonstrates how to use the Trulience SDK:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Trulience SDK Basic Example</title>
    <style>
      #root {
        width: 100vw;
        height: calc(100vh - 185px);
        background-color: #333;
      }
      #textBox {
        width: calc(100vw - 20px);
        font-size: 24px;
        padding: 4px;
      }
      .toolbar {
        display: flex;
        margin: 10px;
        gap: 10px;
      }
    </style>
  </head>
  <body>
    <script src="https://webrtc.github.io/adapter/adapter-latest.js"></script>
    <script src="https://trulience.com/sdk/trulience.sdk.js"></script>

    <div id="root">
      <video style="width:100%; height:100%; background-color:#333" webkit-playsinline playsinline crossorigin="anonymous" id="myvideo" autoplay></video>
    </div>
    <div id="controls" style="position: absolute; bottom: 0; width: 100vw; height:185px;">
      <div class="toolbar">
        <button id="connect" onclick="startCall()">Call</button>
        <button id="disconnect" onclick="endCall()" disabled>End Call</button>
        <button id="micButton" onclick="toggleMic()" disabled>Mute Mic</button>
        <button id="speakerButton" onclick="toggleSpeaker()" disabled>Mute Speaker</button>
      </div>
      <div class="toolbar">
        <textarea id="textBox" rows="3"></textarea>
        <button id="submitButton" onclick="sendText()" disabled>Submit</button>
      </div>
    </div>

    <script>
      var trulience = null;
      const avatarId = "your_avatar_id"; // Replace with your actual avatar ID
      
      let videoElements = {
        remoteVideo: 'myvideo',
      }

      function startCall() {
        // Create a new trulience object
        trulience = Trulience.Builder()
          .setAvatarId(avatarId)
          .setUserName('Guest')
          .enableAvatar(true)
          .setRetry(false)
          .registerVideoElements(videoElements)
          .build();

        // Register for events
        trulience.on('auth-success', () => {
          console.log("Auth succeeded!");
          trulience.connectGateway();
        });
        
        trulience.on('media-connected', () => {
          trulience.setMicEnabled(true);
          trulience.setSpeakerEnabled(true);
          document.getElementById("connect").disabled = true;
          document.getElementById("disconnect").disabled = false;
          document.getElementById("submitButton").disabled = false;
          document.getElementById("micButton").disabled = false;
          document.getElementById("speakerButton").disabled = false;
        });

        // Authenticate
        trulience.authenticate();
      }

      function endCall() {
        if (trulience) {
          trulience.disconnectGateway();
          document.getElementById("connect").disabled = false;
          document.getElementById("disconnect").disabled = true;
          document.getElementById("submitButton").disabled = true;
          document.getElementById("micButton").disabled = true;
          document.getElementById("speakerButton").disabled = true;
        }
      }

      function toggleMic() {
        trulience.toggleMic();
      }

      function toggleSpeaker() {
        trulience.toggleSpeaker();
      }

      function sendText() {
        var msg = document.getElementById("textBox").value;
        document.getElementById("textBox").value = "";
        trulience.sendMessage(msg);
      }
    </script>
  </body>
</html>

Core Functionality

Trulience Class Methods

The Trulience class provides several methods for interacting with avatars:

Method	Description
`getInstance()`	Retrieve a reference to the Trulience object after creation
`sendMessage(string)`	Send a user chat input. This will be sent to the provider you have set up (e.g. an LLM) as is.
`sendMessageToAvatar(string)`	Dispatch a message that isn’t processed by the provider, but instead gets processed by just our client. This can be used to send SSML messages without generating an LLM response.
`toggleMic()`	Toggle microphone between muted and unmuted states
`setMicEnabled(boolean)`	Explicitly set microphone state (true=unmuted, false=muted)
`toggleSpeaker()`	Toggle speaker between muted and unmuted states
`setSpeakerEnabled(boolean)`	Explicitly set speaker state (true=unmuted, false=muted)
`isMicEnabled()`	Check microphone status (returns boolean)
`isSpeakerEnabled()`	Check speaker status (returns boolean)
`isPermissionGranted(string)`	Check if SDK has permission for a resource (e.g., “mic”)
`authenticate()`	Authenticate client with backend (required before using avatar)
`connectGateway()`	Initiate connection with backend to use avatar
`disconnectGateway([reason])`	Terminate connection with backend
`setMediaStream(mediaStream)`	Set the media stream used to drive lip sync. This is useful in situations where you already have an existing TTS media stream that you would prefer to use.
`setLocalMicrophoneDevice(device)`	Allows you to select which microphone device is used

Events System

The Trulience SDK emits various events that can be captured and handled in your application. These events provide insights into authentication, media status, avatar state, and more.

Event Listening

Listen to SDK events:

// Add event listener
Trulience.getInstance().on('auth-success', authSuccessHandler);

// Remove event listener
Trulience.getInstance().off('auth-success', authSuccessHandler);

Available Events

The following events are emitted by the SDK:

Event Name	Description	Parameters
`auth-success`	Authentication success	JSON object with server details
`auth-fail`	Authentication failure	JSON object with error details
`mic-update`	Microphone state change	Boolean (true=unmuted, false=muted)
`cam-update`	Camera state change	Boolean (true=unmuted, false=muted)
`speaker-update`	Speaker state change	Boolean (true=unmuted, false=muted)
`avatar-status-update`	Avatar state change	AvatarStatus value
`media-connecting`	Media connection starting	None
`media-connected`	Media connection established	None
`media-warn`	Usage limit warning	JSON object with warning details
`media-waiting`	All media servers busy	JSON object with server details
`media-busy`	Specific media server busy	JSON object with server details
`media-disconnecting`	Media disconnection starting	CallEndReason value
`media-disconnected`	Media connection dropped	None
`mic-access`	Microphone permission issues	Permission details object
`speech-recognition-start`	Speech recognition started	None
`speech-recognition-error`	Speech recognition error	Error details
`speech-recognition-end`	Speech recognition stopped	None
`speech-recognition-interim-transcript`	Interim speech transcript	Transcripted message
`speech-recognition-final-transcript`	Final speech transcript	Transcripted message
`websocket-connect`	WebSocket connection success	JSON object with server details
`websocket-message`	WebSocket message received	JSON object with message details
`websocket-error`	WebSocket error	Error details
`websocket-close`	WebSocket connection closed	Close details
`load-progress`	Avatar scene loading progress	JSON object with progress details

Constants

AvatarStatus

The AvatarStatus constant represents the current state of the avatar:

Status	Value	Description
`IDLE`	0	Avatar is inactive, not engaged in conversation
`TALKING`	1	Avatar is speaking
`LISTENING`	2	Avatar is processing user input
`UNLOADED`	3	Avatar is not loaded/initialized
`LOADED`	4	Avatar is loaded and ready for interaction
`THINKING`	5	Avatar is processing during complex interactions

Access this constant with: Trulience.AvatarStatus.IDLE, Trulience.AvatarStatus.TALKING etc.

CallEndReason

The CallEndReason constant indicates why a call has ended:

Reason	Value	Description
`HANGED_UP`	0	User intentionally ended the call
`DISCONNECTED`	1	Connection lost due to network/technical issues
`FAILED`	2	Call failed to establish
`UNAUTHORIZED`	3	Authentication/authorization issues

Access this constant with: Trulience.CallEndReason.HANGED_UP, Trulience.CallEndReason.FAILED etc.

Platform-Specific Notes

iOS Considerations

In iOS web browsers (like Safari), there is unique behaviour to be aware of:

Microphone permissions are reset on page reloads
Users must grant microphone permissions again after page reloads
AudioContext is reset when permissions change
Audio output (speaker) is muted by default after permissions change
Users must manually unmute speakers to restore audio

This behaviour is due to iOS security restrictions and should be accounted for in your implementations.