Developer

iOS

Adding a Trulience avatar to your iOS application

Introduction

This guide assumes a high level of familiarity with XCode and Swift/SwiftUI and guides you through creating your own iOS application that displays an avatar. You can skip ahead to the example app section to see all of the code in one place.

Begin by creating an iOS project with XCode. We recommend choosing SwiftUI and using WKWebView (rather than UIKit / UIWebView).

Creating a WebView

Lets set up our webview:

// Webview.swift
import SwiftUI
import WebKit

struct WebView: UIViewRepresentable {
    lazy private var coordinator: Coordinator = Coordinator(self)
    private var webView: WKWebView

    init() {
        self.webView = WebView.createWebView(url: URL(string: "https://trulience.com/avatar/<your-avatar-id>")!)

        // register any SDK events you want to listen for
        webView.configuration.userContentController.add(coordinator, name: "trlChat")
        // ...
    }

    static func createWebView(url: URL) -> WKWebView {
        let configuration = WKWebViewConfiguration()
               
        // 
        configuration.allowsInlineMediaPlayback = true
        configuration.suppressesIncrementalRendering = true
        configuration.defaultWebpagePreferences.allowsContentJavaScript = true
        
        let pref = WKWebpagePreferences.init()
        pref.allowsContentJavaScript = true
        pref.preferredContentMode = .mobile
        configuration.defaultWebpagePreferences = pref
        let webView = WKWebView(frame: .zero, configuration: configuration)
        webView.allowsBackForwardNavigationGestures = true
        webView.load(URLRequest(url: url))
        webView.isOpaque = false // prevent an 'unstyled flash' of white
        
        // optional: AVAudioSession settings
        do {
            try AVAudioSession.sharedInstance().setCategory(.playAndRecord, options: [.defaultToSpeaker, .allowBluetooth])
            try AVAudioSession.sharedInstance().setActive(true, options: .notifyOthersOnDeactivation)
        } catch {
            print("Failed to set audio session category.")
        }
        
        // optional: script to prevent zooming on the webview page
        let disableZoomScriptSrc: String = "var meta = document.createElement('meta');" +
            "meta.name = 'viewport';" +
            "meta.content = 'width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no';" +
            "var head = document.getElementsByTagName('head')[0];" +
            "head.appendChild(meta);"
        let disableZoomScript: WKUserScript = WKUserScript(source: disableZoomScriptSrc, injectionTime: .atDocumentEnd, forMainFrameOnly: true)
        webView.configuration.userContentController.addUserScript(disableZoomScript)

        
        return webView
    }

    func makeUIView(context: Context) -> WKWebView {
        webView.scrollView.contentInsetAdjustmentBehavior = .never
        webView.uiDelegate = context.coordinator
        webView.navigationDelegate = context.coordinator
        return webView
    }

    func makeCoordinator() -> Coordinator {
        Coordinator(self)
    }
}

You’ll also want to create the Coordinator class mentioned throughout the code above:

// Webview.swift
class Coordinator: NSObject, WKScriptMessageHandler, WKUIDelegate, WKNavigationDelegate {
    // ...
}

We’ll use this class later to implement native<=>web communication.

You should now be able to use your WebView in your main ContentView.

// ContentView.swift
struct ContentView: View {
    var body: some View {
        WebView()
        .edgesIgnoringSafeArea([.bottom])
        .background(Color.black) 
    }
}

Microphone Permissions

Ensure your Info.plist contains the Privacy - Microphone Usage Description key with some string value that describes the reason for the permission request, e.g. “Grant microphone access to speak with the avatar.”.

Once the user has granted microphone permissions to your app once, you can prevent the need for repetitive permission popups by implementing this method on your coordinator class:

// Webview.swift
class Coordinator: NSObject, WKScriptMessageHandler, WKUIDelegate, WKNavigationDelegate {
    // ...
    func webView(_ webView: WKWebView, decideMediaCapturePermissionsFor origin: WKSecurityOrigin,
                 initiatedBy frame: WKFrameInfo, type: WKMediaCaptureType) async -> WKPermissionDecision {
        return .grant
    }
// ...

Debugging

You can use Safari on macOS as a remote debugger to inspect the webview’s console. To enable this, you have to set the isInspectable flag on your webview.

if #available(iOS 16.4, *) {
    webView.isInspectable = true
}

Additionally, it’s possible to intercept console.log, console.debug and console.error calls happening inside of the webview through script injection to print them in the XCode console. Begin by defining a WebviewLogger class that reacts to a custom consoleLog message:

class WebViewLogger: NSObject, WKScriptMessageHandler {
    var logs: [String] = []
    
    func log(_ message: String, prefix: String? = "") {
        let today = Date.now
        let formatter3 = DateFormatter()
        formatter3.dateFormat = "HH:mm:ss"
        let timestamp: String = formatter3.string(from: today)
        if let prefix = prefix {
            let loggedMessage: String = timestamp + " :: " + prefix + " :: " + message
            logs.append(loggedMessage)
            print(loggedMessage)
        } else {
            let loggedMessage: String = timestamp + " :: " + message
            logs.append(loggedMessage)
            print(loggedMessage)
        }
    }
    
    func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage) {
        if message.name == "consoleLog", let messageBody = message.body as? String {
            log(messageBody, prefix: "[WEB]")
        }
    }
}

Then instantiate your logger class in your webview class:

let logger = WebViewLogger()

Finally, inject a script into the webview to replace functions like console.log with a custom function that dispatches our consoleLog message:

let configuration = WKWebViewConfiguration()

let source = """
var originalConsoleLog = console.log;
console.log = function() {
    var args = Array.from(arguments).map(function(arg) {
        if (typeof arg === 'undefined') {
            return 'undefined';
        } else if (arg === null) {
            return 'null';
        } else if (typeof arg === 'object') {
            try {
                return JSON.stringify(arg);
            } catch (error) {
                return '[Object]';
            }
        } else {
            return arg.toString();
        }
    });
    window.webkit.messageHandlers.consoleLog.postMessage(args.join(' '));
    originalConsoleLog.apply(console, arguments);
};

window.webkit.messageHandlers.consoleLog.postMessage(`ConsoleError: ${args.join(' ')}`);
originalError.apply(console, arguments);
};
"""
let userScript = WKUserScript(source: source, injectionTime: .atDocumentEnd, forMainFrameOnly: false)

configuration.userContentController.addUserScript(userScript)
configuration.userContentController.add(logger, name: "consoleLog")

// create a WKWebView that uses your configuration
let webView = WKWebView(frame: .zero, configuration: configuration)

Communicating between Native and Web

Your application is now split into two halves, the native Swift code that gets executed on your iOS device, and the javascript web code executed inside of the webview. You can send events across the boundary to communicate between the two.

Swift to Web

Define a callJavaScriptFunction that accepts a function name and optional parameters.

struct WebView: UIViewRepresentable {
    // ...
    func callJavaScriptFunction(functionName: String, parameter: Any? = nil) {
        var script: String
        
        if let param = parameter {
            if let jsonData = try? JSONSerialization.data(withJSONObject: param, options: .fragmentsAllowed),
               let jsonString = String(data: jsonData, encoding: .utf8) {
                script = "window?.NativeBridge?.(functionName)((jsonString));"
            } else {
                print("Error serializing JSON")
                return
            }
        } else {
            script = "window?.NativeBridge?.(functionName)();"
        }
        webView.evaluateJavaScript(script) { (result, error) in
            if let error = error {
                print("Error calling JavaScript function: (error)")
            }
            print("Result (result ?? "None")")
        }
    }
}

Create a function that your native Swift code can invoke:

func switchFlipped(state: Bool) {
    let arg: [String: Any] = [
        "state": state,
    ]
    callJavaScriptFunction(functionName: "switchFlipped", parameter: arg)
}

Finally create a NativeBridge class in your web app to handle the switchFlipped event:

class NativeBridge {
    static INSTANCE = new NativeBridge();

    constructor() {
        return NativeBridge.INSTANCE;
    }

    static getInstance() {
        return NativeBridge.INSTANCE;
    }

    switchFlipped({ state }) {
        // handle switch flip
    }
}

window.NativeBridge = NativeBridge.getInstance();

Web to Swift

Define a utility function for serialising parameters and passing them to a given function. You can think of this as the web counterpart to the callJavaScriptFunction function we defined in Swift earlier.

export const IOSNativeHandler = window.webkit?.messageHandlers;

const callNativeIOSFunction = (func, message) => {
  console.log(`Calling IOSNativeHandler.[${func}].`);
  if (!IOSNativeHandler?.[func]) {
    console.log(`IOSNativeHandler.[${func}] does not exist.`);
    return false;
  }

  try {
    // Check if the message is an object and stringify it
    const stringifiedMessage =
      typeof message === "object" ? JSON.stringify(message) : message;

    // Call the corresponding function with the message parameter
    IOSNativeHandler[func].postMessage(stringifiedMessage);
    console.log(`IOSNativeHandler.[${func}] posted.`);
  } catch (err) {
    console.error("Error while calling iOS native function", err);
    return false;
  }
  return true;
};

You can use it in your code like this:

callNativeIOSFunction("switchFlipped", {state: true})

Finally you can react to the function call in your Swift application with some code like this:

class Coordinator: NSObject, WKScriptMessageHandler, WKUIDelegate, WKNavigationDelegate {
    // ...
    func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage) {
        if message.name == "switchFlipped" {
            if let messageString = message.body as? String {
                // Convert the JSON string to Data
                if let data = messageString.data(using: .utf8) {
                    do {
                        // Deserialize the JSON data into a dictionary
                        if let messageBody = try JSONSerialization.jsonObject(with: data, options: []) as? [String: Any] {
                            // Extract variables
                            if let state = messageBody["state"] as? Bool {
                                // handle switch flip
                            }
                        }
                    } catch {
                        print("Failed to deserialize JSON: (error)")
                    }
                }
            }
        }
    }
}

Check out the example app for a more in-depth look at communicating between native and web. The NativeBridge class in src/utils/NativeBridge.js showcases an event-driven approach on the javascript side. We also demonstrate how callNativeIOSFunction and callNativeAndroidFunction can be unified by a single callNativeAppFunction function that resolves between the two.

Example App

Take a look at this repository to see a full example of how a custom React application can be bundled into an iOS app and loaded locally using WKWebView. This example includes webview logging, permission handling, native<=>web communication and more.

This app loads a native bundle instead of a hosted url, so there are some slight deviations from the guide above. To load a local javascript bundle we use webView.loadFileURL instead of webView.load.

Additionally, we set extra flags when creating the webview to enable certain Web APIs that would otherwise be disabled:

webView.configuration.preferences.setValue(true, forKey: "allowFileAccessFromFileURLs")
webView.configuration.setValue(true, forKey: "allowUniversalAccessFromFileURLs")