This blog article is explaining how to setup web push notifications on a bot framework web chat control. For this, we will use service workers and VAPID. All the code shown here is available in this Github repository containing the full version of it. You can also try this live sample or watch the video below.

If you have any question about this blog article, feel free to contact me on twitter: @meulta

Progressive web apps

The web is in a perpetual evolution. The advantage that apps have over it on mobile platforms is slowly fading away as more and more features are available from the browser. The web community and web browsers team like Microsoft Edge, Google Chrome and Mozilla Firefox are working on enabling the next wave of native-like web experiences, where web content can have the essential capabilities and user experience of native desktop or mobile apps. These web apps can start up instantly, can run in the background and have additional APIs available for developers. We call them: Progressive Web Apps (PWAs).

PWAs are safe, connectivity independent, installable and responsive websites. There is a great chance that you already used one, even without knowing it. Did you already received notifications from Facebook even if the website is not opened in your browser? It is because Facebook is starting to use some of the underlining technology in PWAs. The heart of these new technologies are service workers.

Service workers are slowly being implemented in every browser to help web developers create web apps that are connectivity independent. You can see them as a proxy that goes between the local webpage and your server. It is used to handle caching, push notifications, and even temporary disconnection with background sync.

Bot framework, web chat and the need of push notifications

The Microsoft Bot Framework is a platform that helps developers create a bot which works across multiple channels. You use the Node.js or C# botbuilder SDK to create the bot backend and you can almost automatically make is available on Skype, Slack, Facebook, and a lot more. The combinaison of the web chat and Direct Line is one of these channels. Its name is pretty explicit: it is a web chat control that you can embed in any web page. It is very useful when, for example, you want to embed a support bot chat window directly in your website. Web clients have always been inferior to native clients in one way: background.  Your native client can be granted privileges to run in the background and engage users even when the app isn’t running.  We want to provide that same functionality for the web client

As you might be using at least one instant messaging app on a day to day basis, you should be familiar with the fact that if someone talks to you when the app is closed or reduced you will receive a notification. You can click or tap on it to view the new message and the previous conversation. Having this is important to make the chat UI usable in the long term. It is the same if you are talking with a bot. Usually it will give you an answer pretty quickly as there is no concept of a bot being “away from keyboard” or “not connected”. This said, you might get a message from a bot which:

  • Took some time to compute or get so did not arrive instantly
  • Is a proactive message from the bot at a random time
  • Is a human which took over the conversation and “replaced” the bot
  • Your network doesn’t allow for immediate communication

Push notifications help the user not missing an important message from you. By enabling them to the web client we will fix the gap between native and web clients.

Push notifications

If you never handled push notifications in an app it is important to understand how it works.

When you receive a push notification from an app or a website, even if it looks like they sent it to you directly: it is not technically true. They used a push notification service. This push notification service is linked to the platform you are using. Android uses a push server and Windows another one. It is the same for push notification on a browser: Chrome has its own server, Firefox too, etc.

When you want to be able to send push to a client, you globally have to follow these steps:

  • Step 1: Setup an account on every platform you want to be able to send push to. The push service creates a push sender key that you will need later.
  • Step 2: Subscribe a client to push: this is done and handled by the platform. You do not call directly the push server but you ask the platform to do it. For it to work and for the push server to be able to identify yourself, you have to provide the key it gave you.
  • Step 3: The client get information related to the new subscription: an endpoint on the push server to call to send a new notification and keys for authentication
  • Step 4: Usually, your client code then sends that to your server code to store this subscription information
  • Step 5+: each time you want to send a notification to this client, you use the endpoint and keys that you got from it. The push server will then send that notification to the system which registered (a browser, a mobile, a desktop, etc.). Finally, the system displays it to the user who can see it and click on it.

This is identical for native apps and web apps. However, web notifications are a newer and more progressive spec.  They allow for developers to set up push without all the overhead of accounts on each platform. Your web app will still use the push server associated with each platform but you will generate your own keys. This is done by using VAPID (Voluntary Application Server Identification).

For example, if you setup classic push notifications on chrome, you will have to give a GCM_SENDER_ID you got from a Firebase account. Using VAPID, chrome will still be using Firebase to register and get push events but you will give it your own key, which will also work with other browsers.

In this scenario, your server code is responsible for creating private and public keys. This is a one-time creation process. In your web client code, you use the public key to register to the push server associated with the current browser. The push service understand that you are using VAPID and you get an endpoint and an auth token. This basically only replace step 3 and you can do everything else the same way.  VAPIDs represent the modern approach to push.

To get a deeper understanding of how VAPIDs work, you can check out any of these resources:

When you setup push on a website, you have to do it through a service worker. A service worker is a piece of code which is running side by side with your website client code. The 2 main differences are:

  • It can run even if the website is not open in a tab
  • It is dedicated to network related work such as… push!

In a push scenario, the service worker registers specifically to a “push” event. Its code will be running in the background to get and display what is pushed, and your client code will be responsible for the rest:

  • registering the service worker JavaScript file using serviceWorker.register(…)
  • registering to the push service and getting back the endpoint, key and secret for this push subscription
  • sending the endpoint, key and secret to your server code so it can send a push notification later

Note: when you register for push in your client code, the browser will automatically ask for the user’s permission to enable push notifications.

Ok, enough talking, let’s implement that!

Disclaimer 1: The code I am going to talk about here is from a fully working sample which is available here: https://github.com/meulta/webchat-pushnotifications You can go and have a look at the whole implementation. I will only talk about interesting pieces here.

Disclaimer 2: If you do not know anything about the Bot Framework I highly recommend reading the documentation: https://docs.botframework.com/en-us/

Disclaimer 3: concepts I talk about here will work for any website even if you are not using bot framework 

Disclaimer 4: sorry about all these disclaimers! 😉

Adding push notification to an existing bot

You can try a live version of this sample here: https://webchatpush.azurewebsites.net/web/index.html

The bot we are using in this sample is a really simple one. If you say anything to it, it will start sending one message every 5 seconds. You can stop it by saying “stop”.

You can have a look at the code doing this (https://github.com/meulta/webchat-pushnotifications/blob/master/bot.js#L84-L103) but please do not take it as a reference to send messages proactively to a user from a bot. I tried to keep it as simple as possible as this is not the important part here. You can read more about how to send a message proactively to a user here: https://docs.botframework.com/en-us/azure-bot-service/templates/proactive/

This server code is responsible for creating VAPID keys and send push notification to clients.

  • Creating Vapid keys

If you create your server code in Node.js then you can use a very cool module created by Mozilla which will do almost all the work for you: web-push : https://github.com/web-push-libs/web-push

You will have to first generated Vapid keys using the webPush.generateVapidKeys() function. You only have to do this once or when you want to reset your keys. In our sample, we generate them and store them in a local JSON file. You might want to store this somewhere more secure.

const vapidKeyFilePath = "./vapidKey.json";
var vapidKeys = {};
if (fs.existsSync(vapidKeyFilePath)) {
//if the vapid file exists, then we try to parse its content
//to retrieve the public and private key
//more tests might be necessary here
try {
vapidKeys = JSON.parse(fs.readFileSync(vapidKeyFilePath));
}
catch (e) {
console.error("There is an error with the vapid key file. Log: " + e.message);
process.exit(-1);
}
}
else {
//if the file did not exists, we use the web-push module to create keys
//and store them in the file for future use
//you should copy the public key in the index.js file
vapidKeys = webPush.generateVAPIDKeys();
fs.writeFileSync(vapidKeyFilePath, JSON.stringify(vapidKeys));
console.log("No vapid key file found. One was generated. Here is the public key: " + vapidKeys.publicKey);
}
view raw bot.js hosted with ❤ by GitHub

You then have to call the setVapidDetails() function to configure the web push module to send push notifications using the vapid private key. This will ensure the push server to be sure it comes from you.

webPush.setVapidDetails(
'mailto:example@yourdomain.org',
vapidKeys.publicKey,
vapidKeys.privateKey);
view raw bot.js hosted with ❤ by GitHub
  • Handling event to register push

A bot can receive messages from the user but it can also receive events from the client code. This is very handful to send data to your bot backend code without the user knowing it. In the web chat control it is called the backchannel.

We are going to use this backchannel for the client code to have a way of sending every user push subscription information. We just listen to incoming activities of type event and check that the message name is pushsubscriptionadded (which is one I totally imagine myself, you can pass whatever name you want).

Each time the bot receive a new push subscription, we store it in a local variable associating it to the user internal id in the bot. Note that it might be best to store it in the bot user data.

bot.on("event", function (message) {
if (message.name === "pushsubscriptionadded") {
pushPerUser[message.user.id] = message.value;
}
});
view raw bot.js hosted with ❤ by GitHub
  • Catching messages going out and sending push notifications

In our current scenario, we want to send a push notification to the user each time there is a message sent by the bot. This can easily be done with an event called outgoing. You subscribe to this event then check to see if there is a push notification associated with the user the outgoing message is sent to. If we do, then we use the webPush.sendNotification() function from the web-push module. It will use the VAPID private key and the information from the push subscription to ask the appropriate web push server to send a notification to the browser. It knows which server to talk to thanks to the endpoint property we got from the client in the pushsubscriptionadded event call.

bot.on("outgoing", function (message) {
if (pushPerUser && pushPerUser[message.address.user.id]) {
var pushsub = pushPerUser[message.address.user.id];
webPush.sendNotification({
endpoint: pushsub.endpoint,
TTL: "1",
keys: {
p256dh: pushsub.key,
auth: pushsub.authSecret
}
}, message.text);
}
});
view raw bot.js hosted with ❤ by GitHub

Setting up push in the client

This is the most interesting part. The first role of the client code is to register the service worker in the browser. To do this, we use the navigator.serviceWorker.register() function by giving it the service worker file name. This function return a Promise so you can chain a .then() function to execute some code once the service worker is registered. If the service worker is already registered, it will return the current one.

In our case, we take this opportunity to try to get the existing push notification manager subscription using registration.pushManager.getSubscription() (where registration is the service worker instance). If it does not exist, we will just have to create and return a new one create using registration.pushManager.subscribe() giving it an applicationServerKey. This applicationServerKey is the public key your server generated.

The subscription object we get from this is containing everything the server will need to send a notification to the client: the endpoint, the key and a secret.

In the current sample, all this is done in the setupPush function which takes a callback as a parameter and calls it back with the subscription information.

var setupPush = function (done) {
//first step is registering the service worker file
navigator.serviceWorker.register('service-worker.js')
.then(function (registration) {
//once the sw is registered, we try to get an existing push subscription
return registration.pushManager.getSubscription()
.then(function (subscription) {
//if the subscription exists, then we pass is to the next chained .then function using return
if (subscription) {
return subscription;
}
//if the subscription does not exists, we wrap the VAPID public key and create a new one
//we pass this new once to the next chaind .then function using return
const convertedVapidKey = urlBase64ToUint8Array(VAPID_PUBLICKEY);
return registration.pushManager.subscribe({
userVisibleOnly: true,
applicationServerKey: convertedVapidKey
});
});
})
.then(function (subscription) {
//wrapping the key and secret
const rawKey = subscription.getKey ? subscription.getKey('p256dh') : '';
const key = rawKey ? btoa(String.fromCharCode.apply(null, new Uint8Array(rawKey))) : '';
const rawAuthSecret = subscription.getKey ? subscription.getKey('auth') : '';
const authSecret = rawAuthSecret ? btoa(String.fromCharCode.apply(null, new Uint8Array(rawAuthSecret))) : '';
const endpoint = subscription.endpoint;
//we call back the code that asked to register push notification with the subscription information
done({
endpoint: subscription.endpoint,
key: key,
authSecret: authSecret
});
});
}
view raw index.js hosted with ❤ by GitHub

This setupPush function is called right after we setup the Web Chat control. In the callback function we give to it, we use the Direct Line SDK to send an event of type pushsubscriptionadded to the bot, through the back channel.

setupPush((subscriptionInfo) => {
//once push notifications are setup, we get the subscription info back in this callback
//we use the backchannel to send this info back to the bot using an 'event' activity
botConnection
.postActivity({
type: "event",
name: "pushsubscriptionadded",
value: subscriptionInfo,
from: { id: botConnection.conversationId } //you could define your own userId here
})
.subscribe(id => {
//we store the conversation id which we get back from postActivity(...) in the LocalStorage
//we will need this in case of conversation resuming
localStorage.setItem("pushsample.botConnection.conversationId", botConnection.conversationId);
});
});
});
view raw index.js hosted with ❤ by GitHub

As you can see in the code above, we also store the conversationid in the browser localStorage so it will persist. We need this to be able to resume the conversation when the user clicks on a notification after the tab was closed. We handle this by adding a get parameter to the webpage url: ?isBack=y. To resume a conversation using Direct Line, you just have to give back the conversationid as we do here:

if (getParameterByName("isback") === 'y') {
//if we are resuming an existing conversation, we get back the conversationid from LocalStorage
botConnection = new DirectLine.DirectLine({
secret: DIRECTLINE_SECRET,
conversationId: localStorage.getItem("pushsample.botConnection.conversationId"),
webSocket: false
});
view raw index.js hosted with ❤ by GitHub

Listening to push notification in the background with the service worker

Last but not least, we need to write the code that will sit in the browser and handle push notification event even if the website is not opened.

  • Registering to push

The first piece of code is the one handling the push events.  To do this we use the self.addEventListener() function. It takes an event name (here “push”) and a callback. Each time a new push notification is received, this callback is going to be called. Here, we just call registration.showNotification() which displays it using a nice image and some text. The payload variable is built using the event data (which is the notification text we send from the server).

self.addEventListener('push', function (event) {
//creating the notification message (we should never be in the "no message" case)
var payload = event.data ? event.data.text() : 'No message...';
//we show a notification to the user with the text message
//and an icon which is hosted as a resource on the website
event.waitUntil(
self.registration.showNotification('Chat bot!', {
body: payload,
icon: '/web/img/thinking_morphi.png'
})
);
});
view raw service-worker.js hosted with ❤ by GitHub
  • Handling click on notifications

By default, clicking on a browser notification does nothing. You can add a custom behavior using the ‘notificationclick’ event in the service worker code. Its code is pretty straightforward as we list all the clients (a tab being also seen as a client), we look if one is displaying our web page. If yes and the focus is on another one, we switch to it. If yes and the focus is on it, we do nothing. And finally, if no, we reopen the page adding the ?isBack=y parameter.

self.addEventListener('notificationclick', function (event) {
// Android doesn't close the notification when you click on it
// See: http://crbug.com/463146
event.notification.close();
// This looks to see if the current is already open and
// focuses if it is
event.waitUntil(
//searching for all clients / tab opened in the browser
clients.matchAll({
type: "window"
})
.then(function (clientList) {
//going through the list of clients/tab and trying to find our website
for (var i = 0; i < clientList.length; i++) {
var client = clientList[i];
//if we find it, we put focus back on the tab
if ((client.url.toLowerCase() == baseurl + '/web/index.html' || client.url.toLowerCase() == baseurl + '/web/index.html?isback=y') && 'focus' in client)
return client.focus();
}
if (clients.openWindow) {
//if we did not find it, then we re-open it with the isback=y parameter
//to ensure that we resume the conversation using the conversationid
return clients.openWindow('/web/index.html?isback=y');
}
})
);
});
view raw service-worker.js hosted with ❤ by GitHub

What’s next?

Using web push notifications in a web chat control is obvious. There are a lot of other cases in which it can be really helpful. It can help you notify someone about a trending news, an update on your website or a new friend connection.

Understanding how web notifications are working and adding them to one of your projects is a great first step. Implementing more PWAs’ features can be simpler than you think. At Microsoft, we have recently introduced PWA Builder, which simplifies and automates building a manifest so it’s as easy as providing resources and a description for your app. It will also help you in the process of adding service workers features to your app, such as cache management. In a future version, it will certainly also help you create the service worker code needed to handle push notifications.

In a very near future, service workers will be available in every modern browser: take this opportunity and be part of the Progressive Web Apps world!

If you have any question about this blog article, feel free to contact me on twitter: @meulta