Integrating Video Calling in Chat with WebRTC and PubNub

What is WebRTC?

WebRTC is a free and open source project that enables web browsers and mobile devices to provide simple realtime communication. This means that app features like peer-to-peer video conferencing can easily be integrated into a web page. A browser-based video chat can be engineered rapidly with HTML and JavaScript, no back-end code required.

WebRTC allows users to stream peer-to-peer audio and video in modern web browsers. This screenshot is from a WebRTC video call between 2 iOS devices using the Safari web browser.

Making a user’s device a WebRTC client is as simple as initializing a new RTCPeerConnection(); object in front-end JavaScript. Nowadays, WebRTC support comes out of the box with web browsers like Chrome, FireFox, Edge, Safari, and Opera on desktop, as well as native iOS and Android web browsers.

WebRTC Streaming Architecture

Video chat is established on two or more client devices using the WebRTC protocol. The connection can be made using one of two modes. The first mode is peer-to-peer, meaning audio and video packets are streamed directly from client to client with UDP. This works as long as both machines have an IP address that is accessible by the public internet.

Relying on peer-to-peer connections for browser video chat is not wise in production apps. It is common for the Interactive Connectivity Establishment or ICE framework to fail to establish a connection between two users when one or both are behind advanced LAN security.

To mitigate this, you can set your RTC Configuration to first attempt peer-to-peer, and then fall back to relayed connection if peer-to-peer fails.

If publicly accessible IP addresses are not an option, like on enterprise WiFi networks, a WebRTC connection must be established over TCP using a TURN server. The ICE framework will decide if this is necessary as users are trying to connect. A TURN server acts as a relay for video and audio data. TURN instances require bandwidth and machine time – so it’s not free like peer-to-peer streaming.

A developer like yourself can make a TURN server using open source solutions and a general web hosting service. You can also use a TURN provider, like Xirsys. Remember that using a 3rd party TURN provider means that all audio and video data flows through their systems when in transit.

Don’t Build a WebRTC Signaling Server – Use PubNub

WebRTC leaves out a very important component from video calling. A client must use a signaling service to communicate messages with their peer or peers. These messages are for events like:

• I, User A, would like to call you, User B
• User A is currently trying to call you, User B
• I, User B, accept your call User A
• I, User B, reject your call User A
• I, User B, would like to end our call User A
• I, User A, would like to end our call User B
• Text instant messaging like in Slack, Google Hangouts, Skype, Facebook Messenger, etc.
• Session Audio/Video codec and user connectivity data.

These messages are part of the Signaling Transaction Flow which is outlined in the Mozilla Developer Network documentation for WebRTC. This diagram illustrates all of the operations that must take place for an audio or video WebRTC call.

The WebRTC signaling server is an abstract concept. Many services can become this “signaling server” like WebSockets, Socket.IO or PubNub. If you’re tasked with creating a solution for this, you will end up asking: Should we build or should we buy?

PubNub allows a developer like yourself to fully, and cheaply, implement event-driven solutions like a WebRTC signaling service. An Open Source WebRTC library that uses PubNub is available on GitHub. However, the following PubNub solution is even more rapid than building with the WebRTC SDK.

Community Supported Package for WebRTC Calling

PubNub is like a global CDN for realtime data. PubNub’s customers use its IaaS for building realtime chat applications, live GPS location tracking maps, IoT device signaling, and so much more. Developers can focus on building their connected shared experiences and leave the infrastructure to PubNub. There are PubNub SDKs for every programming language and device. Client SDKs enable reliable pub/sub connections, data delivery, and controlling the network; all possible in a few lines of code.

Want to include realtime video and audio conferencing with WebRTC in your web app? This community supported WebRTC JS package, which wraps the PubNub JS SDK, will speed up your development process. You can find the WebRTC JS Package on npm.

Open Source WebRTC Video Chat Example

The open source community has created the WebRTC package for simple 1-to-1 calling with PubNub. You can now provide your users with a peer-to-peer or relayed WebRTC video chat experience. You can use your own STUN/TURN credentials with the same configuration object shown in the MDN RTCConfiguration Documentation (note that Safari only likes an array of URLs).

 

 

The plugin uses PubNub’s Pub/Sub messaging for the WebRTC signaling service. All of the handshakes required by the WebRTC signaling transaction flow are under the covers of the plugin, so you can focus on your app’s higher level code.

The “online user” list in the UI is updated in realtime using PubNub Presence. In order to get this working in your app, you need to enable Presence in the PubNub Admin Dashboard. Your demo key set that was created when you made an account has Presence disabled by default. You need to turn it on for the API key set.

The plugin is available on NPM:

npm install pubnub-js-webrtc

 

---

 

Try the example WebRTC app in the JS WebRTC Package GitHub Repository.

The example app source code is in the example folder.

 

---

You can now build your own fully functioning WebRTC chat app with PubNub!

WebRTC App Tutorial

In this tutorial, we will use plain old JavaScript, HTML, and CSS. If you want to use a modern front-end framework (like Vue, React, or Angular) to build your chat app, check out the PubNub tutorials page or the PubNub Chat Resource Center.

You can use the HTML and CSS in my project example. Copy those files into your project folder.

Doing this makes a very generic chat app user interface. The example app has only 1 global chat, and no private 1:1 chats, although they are easy to implement. Make sure that you also copy the png images from the example to your project.

Open index.html with your favorite text editor. Replace the script tags beneath the body tag of your HTML file with these 2 CDN scripts. Leave the 3rd script tag that refers to app.js. We will write that file together.

<script type="text/javascript" src="https://cdn.pubnub.com/sdk/javascript/pubnub.4.21.7.js"></script>
<script src="https://cdn.jsdelivr.net/npm/pubnub-js-webrtc@latest/dist/pubnub-js-webrtc.js"></script>

The next step is to create your own app.js file in the same directory as your index.html file. The reason we need to make a new app.js is because the script in my example uses Xirsys. My private account is wired to my PubNub Functions server. You will need to make your own back-end server and account if you wish to use a TURN provider like Xirsys. My next blog post will contain a tutorial for building WebRTC apps with TURN.

The app.js script we will write together will use only free peer-to-peer WebRTC connections. If you try to do a video call with 2 devices on the same LAN, your app will work. It is not certain that a video call connection can be made with clients on separate networks (due to NAT security).

app.js

First we will make references to all of the DOM elements from the index.html file. Once we can refer to them in our JavaScript code, we can manipulate them programmatically.

const chatInterface = document.getElementById('chat-interface');
const myVideoSample = document.getElementById('my-video-sample');
const myVideo = document.getElementById('my-video');
const remoteVideo = document.getElementById('remote-video');
const videoModal = document.getElementById('video-modal');
const closeVideoButton = document.getElementById('close-video');
const brokenMyVideo = document.getElementById('broken-my-video');
const brokenSampleVideo = document.getElementById('broken-sample-video');
const usernameModal = document.getElementById('username-input-modal');
const usernameInput = document.getElementById('username-input');
const joinButton = document.getElementById('join-button');
const callConfirmModal = document.getElementById('call-confirm-modal');
const callConfirmUsername = document.getElementById('call-confirm-username');
const yesCallButton = document.getElementById('yes-call');
const noCallButton = document.getElementById('no-call');
const incomingCallModal = document.getElementById('incoming-call-modal');
const callFromSpan = document.getElementById('call-from');
const acceptCallButton = document.getElementById('accept-call');
const rejectCallButton = document.getElementById('reject-call');
const onlineList = document.getElementById('online-list');
const chat = document.getElementById('chat');
const log = document.getElementById('log');
const messageInput = document.getElementById('message-input');
const submit = document.getElementById('submit');

Next, we will add some variables that hold a CSS class name, global app information, and WebRTC configuration info.

const hide = 'hide';
// PubNub Channel for sending/receiving global chat messages
//     also used for user presence with PubNub Presence
const globalChannel = 'global-channel';
let webRtcPhone;
let pubnub;
// An RTCConfiguration dictionary from the browser WebRTC API
// Add STUN and TURN server information here for WebRTC calling
const rtcConfig = {};
let username; // User's name in the app
let myAudioVideoStream; // Local audio and video stream
let noVideoTimeout; // Used to check if a video connection succeeded
const noVideoTimeoutMS = 5000; // Error alert if the video fails to connect

Now we will get into some of the imperative client code for the WebRTC package functionality.

// Init the audio and video stream on this client
getLocalStream().then((localMediaStream) => {
    myAudioVideoStream = localMediaStream;
    myVideoSample.srcObject = myAudioVideoStream;
    myVideo.srcObject = myAudioVideoStream;
}).catch(() => {
    myVideo.classList.add(hide);
    myVideoSample.classList.add(hide);
    brokenMyVideo.classList.remove(hide);
    brokenSampleVideo.classList.remove(hide);
});
// Prompt the user for a username input
getLocalUserName().then((myUsername) => {
    username = myUsername;
    usernameModal.classList.add(hide);
    initWebRtcApp();
});
// Send a chat message when Enter key is pressed
messageInput.addEventListener('keydown', (event) => {
    if (event.keyCode === 13 && !event.shiftKey) {
        event.preventDefault();
        sendMessage();
        return;
    }
});
// Send a chat message when the submit button is clicked
submit.addEventListener('click', sendMessage);
const closeVideoEventHandler = (event) => {
    videoModal.classList.add(hide);
    chatInterface.classList.remove(hide);
    clearTimeout(noVideoTimeout);
    webRtcPhone.disconnect(); // disconnects the current phone call
}
// Register a disconnect event handler when the close video button is clicked
closeVideoButton.addEventListener('click', closeVideoEventHandler);

The new code that we just added:

• Asks the browser if it can access the computer’s webcam and microphone, and stores the stream object to a global variable.
• Prompts the user for an in-app “user name” before we initialize the WebRTC portion of the app.
• Registers event handlers for chat messaging, like when a user clicks the submit button or presses the enter key.
• Makes another event handler for when the user closes the video chat.

Next we are going to add the initialization code for the WebRTC portion of the web application.

const initWebRtcApp = () => {
    // WebRTC phone object event for when the remote peer's video becomes available.
    const onPeerStream = (webRTCTrackEvent) => {
        console.log('Peer audio/video stream now available');
        const peerStream = webRTCTrackEvent.streams[0];
        window.peerStream = peerStream;
        remoteVideo.srcObject = peerStream;
    };
    // WebRTC phone object event for when a remote peer attempts to call you.
    const onIncomingCall = (fromUuid, callResponseCallback) => {
        let username = document.getElementById(fromUuid).children[1].innerText;
        incomingCall(username).then((acceptedCall) => {
            if (acceptedCall) {
                // End an already open call before opening a new one
                webRtcPhone.disconnect();
                videoModal.classList.remove(hide);
                chatInterface.classList.add(hide);
                noVideoTimeout = setTimeout(noVideo, noVideoTimeoutMS);
            }
            callResponseCallback({ acceptedCall });
        });
    };
    // WebRTC phone object event for when the remote peer responds to your call request.
    const onCallResponse = (acceptedCall) => {
        console.log('Call response: ', acceptedCall ? 'accepted' : 'rejected');
        if (acceptedCall) {
            videoModal.classList.remove(hide);
            chatInterface.classList.add(hide);
            noVideoTimeout = setTimeout(noVideo, noVideoTimeoutMS);
        }
    };
    // WebRTC phone object event for when a call disconnects or timeouts.
    const onDisconnect = () => {
        console.log('Call disconnected');
        videoModal.classList.add(hide);
        chatInterface.classList.remove(hide);
        clearTimeout(noVideoTimeout);
    };
    // Lists the online users in the UI and registers a call method to the click event
    //     When a user clicks a peer's name in the online list, the app calls that user.
    const addToOnlineUserList = (occupant) => {
        const userId = occupant.uuid;
        const name = occupant.state ? occupant.state.name : null;
        if (!name) return;
        const userListDomElement = createUserListItem(userId, name);
        const alreadyInList = document.getElementById(userId);
        const isMe = pubnub.getUUID() === userId;
        if (alreadyInList) {
            removeFromOnlineUserList(occupant.uuid);
        } 
        if (isMe) {
            return;
        }
        onlineList.appendChild(userListDomElement);
        userListDomElement.addEventListener('click', (event) => {
            const userToCall = userId;
            confirmCall(name).then((yesDoCall) => {
                if (yesDoCall) {
                    webRtcPhone.callUser(userToCall, {
                        myStream: myAudioVideoStream
                    });
                }
            });
        });
    }
    const removeFromOnlineUserList = (uuid) => {
        const div = document.getElementById(uuid);
        if (div) div.remove();
    };
    pubnub = new PubNub({
        publishKey : '_YOUR_PUBNUB_PUBLISH_API_KEY_HERE_',
        subscribeKey : '_YOUR_PUBNUB_SUBSCRIBE_API_KEY_HERE_'
    });
    // This PubNub listener powers the text chat and online user list population.
    pubnub.addListener({
        message: function(event) {
            // Render a global chat message in the UI
            if (event.channel === globalChannel) {
                renderMessage(event);
            }
        },
        status: function(statusEvent) {
            if (statusEvent.category === "PNConnectedCategory") {
                pubnub.setState({
                    state: {
                        name: username
                    },
                    channels: [globalChannel],
                    uuid: pubnub.getUUID()
                });
                pubnub.hereNow({
                    channels: [globalChannel],
                    includeUUIDs: true,
                    includeState: true
                },
                (status, response) => {
                    response.channels[globalChannel].occupants
                        .forEach(addToOnlineUserList);
                });
            }
        },
        presence: (status, response) => {
            if (status.error) {
                console.error(status.error);
            } else if (status.channel === globalChannel) {
                if (status.action === "join") {
                    addToOnlineUserList(status, response);
                } else if (status.action === "state-change") {
                    addToOnlineUserList(status, response);
                } else if (status.action === "leave") {
                    removeFromOnlineUserList(status.uuid);
                } else if (status.action === "timeout") {
                    removeFromOnlineUserList(response.uuid);
                }
            }
        }
    });
    pubnub.subscribe({
        channels: [globalChannel],
        withPresence: true
    });
    window.ismyuuid = pubnub.getUUID();
    // Disconnect PubNub before a user navigates away from the page
    window.onbeforeunload = (event) => {
        pubnub.unsubscribe({
            channels: [globalChannel]
        });
    };
    // WebRTC phone object configuration.
    let config = {
        rtcConfig,
        ignoreNonTurn: false,
        myStream: myAudioVideoStream,
        onPeerStream,   // is required
        onIncomingCall, // is required
        onCallResponse, // is required
        onDisconnect,   // is required
        pubnub          // is required
    };
    webRtcPhone = new WebRtcPhone(config);
};

The code that we just added to app.js executes after the user enters their “user name” and it:

• Declares all of the plugin event handlers for WebRTC call events
• Adds and removes user online list elements as users come on and offline in the app
• Registers an event handler to make a new video call to a user whenever their name is clicked in the user list UI
• Registers an event handler to render new chat messages whenever one is sent to the global chat, in real-time
• Sets up PubNub to send and listen for realtime messages with the Pub/Sub messaging pattern.
• Initializes the WebRTC package and passes the configuration object to the instance

Before we continue, it is important to note that we need to insert our free PubNub API keys into this function. We can get some forever-free keys using the signup form below. These keys are free up to 1 Million transactions per month, which is great for hobbyists or professional proof-of-concept apps.

You can insert your client Pub/Sub API keys into the app.js file in the PubNub initialization object, like you can see in the earlier code snippet.

pubnub = new PubNub({
    publishKey : '_YOUR_PUBNUB_PUBLISH_API_KEY_HERE_',
    subscribeKey : '_YOUR_PUBNUB_SUBSCRIBE_API_KEY_HERE_'
});

We need to enable the PubNub Presence feature in the PubNub Admin Dashboard. When you create a PubNub key set, the Presence feature is disabled on the key by default. We can enable it for the key by going to the PubNub Admin Dashboard and clicking the toggle switch.

The example app uses presence to show which users are online in the app. We are using the PubNub user UUID to keep unique references to every user in the app. When we do a WebRTC video call operation, we are using the UUID so both users can display the corresponding user name in their UI.

Next, we will need some utility methods to perform UI specific functionality. These are not specific to all WebRTC apps, they are only for running this specific UI that I designed. Add this code to the bottom of the app.js file.

// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
// UI Render Functions
// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
function renderMessage(message) {
    const messageDomNode = createMessageHTML(message);
    log.append(messageDomNode);
    // Sort messages in chat log based on their timetoken (value of DOM id)
    sortNodeChildren(log, 'id');
    chat.scrollTop = chat.scrollHeight;
}
function incomingCall(name) {
    return new Promise((resolve) => {
        acceptCallButton.onclick = function() {
            incomingCallModal.classList.add(hide);
            resolve(true);
        }
        rejectCallButton.onclick = function() {
            incomingCallModal.classList.add(hide);
            resolve(false);
        }
        callFromSpan.innerHTML = name;
        incomingCallModal.classList.remove(hide);
    });
}
function confirmCall(name) {
    return new Promise((resolve) => {
        yesCallButton.onclick = function() {
            callConfirmModal.classList.add(hide);
            resolve(true);
        }
        noCallButton.onclick = function() {
            callConfirmModal.classList.add(hide);
            resolve(false);
        }
        callConfirmUsername.innerHTML = name;
        callConfirmModal.classList.remove(hide);
    });
}
function getLocalUserName() {
    return new Promise((resolve) => {
        usernameInput.focus();
        usernameInput.value = '';
        usernameInput.addEventListener('keyup', (event) => {
            const nameLength = usernameInput.value.length;
            if (nameLength > 0) {
                joinButton.classList.remove('disabled');
            } else {
                joinButton.classList.add('disabled');
            }
            if (event.keyCode === 13 && nameLength > 0) {
                resolve(usernameInput.value);
            }
        });
        joinButton.addEventListener('click', (event) => {
            const nameLength = usernameInput.value.length;
            if (nameLength > 0) {
                resolve(usernameInput.value);
            }
        });
    });
}
function getLocalStream() {
    return new Promise((resolve, reject) => {
        navigator.mediaDevices
        .getUserMedia({
            audio: true,
            video: true
        })
        .then((avStream) => {
            resolve(avStream);
        })
        .catch((err) => {
            alert('Cannot access local camera or microphone.');
            console.error(err);
            reject();
        });
    });
}
function createUserListItem(userId, name) {
    const div = document.createElement('div');
    div.id = userId;
    const img = document.createElement('img');
    img.src = './phone.png';
    const span = document.createElement('span');
    span.innerHTML = name;
    div.appendChild(img);
    div.appendChild(span);
    return div;
}
function createMessageHTML(messageEvent) {
    const text = messageEvent.message.text;
    const jsTime = parseInt(messageEvent.timetoken.substring(0,13));
    const dateString = new Date(jsTime).toLocaleString();
    const senderUuid = messageEvent.publisher;
    const senderName = senderUuid === pubnub.getUUID()
        ? username
        : document.getElementById(senderUuid).children[1].innerText;
    const div = document.createElement('div');
    const b = document.createElement('b');
    div.id = messageEvent.timetoken;
    b.innerHTML = `${senderName} (${dateString}): `;
    div.appendChild(b);
    div.innerHTML += text;
    return div;
}
// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
// Utility Functions
// =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
function sendMessage() {
    const messageToSend = messageInput.value.replace(/?
|/g, '');
    const trimmed = messageToSend.replace(/(\s)/g, '');
    if (trimmed.length > 0) {
        pubnub.publish({
            channel: globalChannel,
            message: {
                text: messageToSend
            }
        });
    }
    messageInput.value = '';
}
// Sorts sibling HTML elements based on an attribute value
function sortNodeChildren(parent, attribute) {
    const length = parent.children.length;
    for (let i = 0; i < length-1; i++) {
        if (parent.children[i+1][attribute] < parent.children[i][attribute]) {
            parent.children[i+1].parentNode
                .insertBefore(parent.children[i+1], parent.children[i]);
            i = -1;
        }
    }
}
function noVideo() {
    const message = 'No peer connection made.
' +
        'Try adding a TURN server to the WebRTC configuration.';
    if (remoteVideo.paused) {
        alert(message);
        closeVideoEventHandler();
    }
}

style.css

We need CSS styles in our app in for a pretty and pleasing user interface. The index.html file already has a reference to the style.css file, so add it to the same folder. The style.css file for this WebRTC app is available in the GitHub Repository.

Done! Now you can deploy your static front-end web files on a web hosting platform like WordPress or GitHub pages. Your WebRTC chat app will be available for use by anyone in the world. The code is mobile compatible, meaning the latest web browsers on iOS and Android will be able to run the app for face to face video!

Frequently Asked Questions (FAQ) about the WebRTC Package

Is the package officially a part of PubNub?

No. It is an open source project that is community supported. If you have questions or need help, reach out to devrel@pubnub.com. If you want to report a bug, do so on the GitHub Issues page.

Does PubNub stream audio or video data?

No. PubNub pairs very well with WebRTC as a signaling service. This means that PubNub signals events from client to client using Pub/Sub messaging. These events include:

• I, User A, would like to call you, User B
• User A is currently trying to call you, User B
• I, User B, accept your call User A
• I, User B, reject your call User A
• I, User B, would like to end our call User A
• I, User A, would like to end our call User B
• Text instant messaging like in Slack, Google Hangouts, Skype, Facebook Messenger, etc.

Can I make a group call with more than 2 participants?

Group calling is possible to develop with WebRTC and PubNub, however, the current PubNub JS WebRTC package can connect only 2 users in a private call. The community may develop this feature in the future but there are no plans for development to date.

I found a bug in the plugin. Where do I report it?

The PubNub JS WebRTC package is an open source, community supported project. This means that the best place to report bugs is on the GitHub Issues page for the code repository. The community will tackle the bug fix at will, so there is no guarantee that a fix will be made. If you wish to provide a code fix, fork the GitHub repository to your GitHub account, push fixes, and make a pull request (process documented on GitHub).

For more examples, check out the PubNub Tutorials page. If you like this plugin, need some help, or want to build something similar, reach out to devrel@pubnub.com. We want to hear your feedback!