Working with the Key-Value Store (Advanced functions Techniques)

This is the second post of our 4-part series on Advanced PubNub Functions programming techniques. We’ve also talked about Encryption, triggering 3rd party web services, and Event Handler Types.

What is Key-Value Store?

PubNub Functions is incredibly powerful in that it allows you to call a web service in realtime as your data is flowing through the network. But what about storing data? That’s where the Key-Value Store comes into play.

KV Store is short for key-value store, also known as key-value database. The Functions KV Store is designed around low-latency, so it doesn’t have any locks. It’s eventually consistent, meaning you can store something, and that storage will eventually propagate to the network.

Click here for documentation on KV Store. We’ve also got a great overview in our Intro to Functions Training (skip to 20:37 for the KV Store section).

Key-Value Store Overview

In this article, we’ll showcase the different capabilities of KV store, and dive into a simple example of how to handle data stream message events to implement a messaging “Stats Bot” using custom serverless event handling in a realtime Angular 2 web application.

The “Stats Bot” is a demonstration of the Functions KV Store, our highly scalable data store that integrates directly with Functions handlers. Our “Stats Bot” handler processes message events and persist message statistics to the Functions KV Store. Although the example is small (using fixed commands and limited data), it could easily be extended to call remote web services for persistence, natural language detection, sentiment analysis, bridging to external message systems and more.

Here’s how our sample app looks. The “Stats Bot” feature displays recent message statistics so that the web client UI code can stay simple and just display the data.

Functions KV Store

How do we implement a “Stats Bot” in PubNub? First, let’s take a moment to review the relevant concepts and features.

PubNub makes implementing realtime systems much easier by integrating enterprise-grade security, presence, storage and other advanced features into a huge (and growing!) number of client toolkits. The SDKs we’ve personally used with these features include JavaScript, Java, Android, iOS, Ruby and Python. If you’re using another language, check out the PubNub SDK page – chances are, it’s already supported!

The core of PubNub is Data Streams – realtime message channels that allow an unbounded numbers of devices to publish and subscribe to messages. The Functions feature allows developers to attach custom code to those message events in one of two ways, either as a “before publish or fire” event which allows modification of message attributes, or as an “after publish or fire” event which takes place after the message has been published to the message channel. (There is also an “after presence” event for implementing Presence-related functionality).

In this case, we create a BLOCK that intercepts the “before publish or fire” events in a message channel to implement the “Stats Bot” functionality. The big question is, how do we make those stats persistent in the context of data streams? This is a prime use case for the Functions KV Store, which offers scalable data storage capabilities for realtime applications. Here’s how our BLOCK works:

• If the text of the message is the special command “/my_stats”, then we retrieve the “latest_msg” and “count” values for the user from the KV store (meaning the last message sent and total message count respectively).
• Otherwise, we update the “latest_msg” and “count” values for the user in the KV store accordingly.

The KV store is scoped to the PubNub subscribe key we’re using, so it’s easy to create functionality that spans all the data stream channels in our application. Given the available operations (get, set, and get/increment counters), the KV store could be used for caching results of remote APIs, latest activity, or statistics such as voting, presence information, or IP blacklisting and rate limiting.

Obtaining Your PubNub Developer Keys

The first things you’ll need before you can create a realtime application with PubNub are publish and subscribe keys. Just in case you haven’t already, you can create an account, get your keys and be ready to use the PubNub network in less than 60 seconds using the handy signup form.

Once you do that, the publish and subscribe keys look like UUIDs and start with “pub-c-” and “sub-c-” prefixes respectively. Keep those handy – you’ll need to plug them in when initializing the PubNub object in your HTML5 app below.

Setting up the BLOCK

With PubNub Functions, it’s really easy to create code to run in the network. Here’s how to make it happen:

Go to the application instance on the PubNub Admin Dashboard.

Create a new block.

Paste in the block code from the next section.

Start the block, and test it using the “publish message” button and payload on the left-hand side of the screen.

That’s all it takes to create your serverless code running in the cloud!

Diving into the Code – the BLOCK

export default (request) => {
    const db = require('kvstore');

var by = request.message.by || request.params.uuid;
var last_key = by + ":last_msg";
var count_key = by + ":count";

if (request.message && (request.message.text == '/my_stats')) {
    //
    // special "my_stats" command handling
    //
    return db.get(last_key).then((last_msg) => {
        return db.getCounter(count_key).then((count) => {
            request.message.last_msg = last_msg || "<not found>";
            request.message.count = count || 0;
            return request.ok();
        });
    });
}

    //
    // normal case, store the message details
    //
    request.message.at = new Date().toISOString();
    return db.set(last_key, request.message).then((last_result) => {
        return db.incrCounter(count_key).then((count_result) => {
            return request.ok();
        });
    });
}

<!DOCTYPE html>
<html>
  <head>
    <title>Angular 2</title>
    <script src="https://unpkg.com/core-js@2.4.1/client/shim.min.js"></script>
    <script src="https://unpkg.com/zone.js@0.7.2/dist/zone.js"></script>
    <script src="https://unpkg.com/reflect-metadata@0.1.9/Reflect.js"></script>
    <script src="https://unpkg.com/rxjs@5.0.1/bundles/Rx.js"></script>
    <script src="https://unpkg.com/@angular/core/bundles/core.umd.js"></script>
    <script src="https://unpkg.com/@angular/common/bundles/common.umd.js"></script>
    <script src="https://unpkg.com/@angular/compiler/bundles/compiler.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser/bundles/platform-browser.umd.js"></script>
    <script src="https://unpkg.com/@angular/forms/bundles/forms.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js"></script>
    <script src="https://unpkg.com/pubnub@4.3.3/dist/web/pubnub.js"></script>
    <script src="https://unpkg.com/pubnub-angular2@1.0.0-beta.8/dist/pubnub-angular2.js"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" />
  </head>

<body>
  <main-component>
    Loading...
  </main-component>

<div class="container">
    <pre>
    NOTE: make sure to update the PubNub keys below with your keys,
    and ensure that the BLOCK settings are configured properly!
    </pre>
    <h3>MyApp Functions KV Store Integration</h3>
    <br />
    Message:
    <br />
    <input type="text" [(ngModel)]="toSend" placeholder="message" />
    <input type="button" (click)="publish()" value="Send!" />
    <br />
    <small>(type '/my_stats' to see your message stats)</small>
    <hr/>
    <br/>
    <br/>
    <ul>
      <li *ngFor="let item of messages.slice()">
        <div>{{JSON.stringify(item.message)}}</div>
      </li>
    </ul>
</div>

<script>
var app = window.app = {};
app.main_component = ng.core.Component({
    selector: 'main-component',
    template: `...see previous...`

}).Class({
    constructor: [PubNubAngular, function(pubnubService){
        var self = this;
        self.pubnubService = pubnubService;
        self.channelName = 'kv-channel';
        self.JSON = JSON;
        self.messages = [];
        self.toSend = "";

pubnubService.init({
    publishKey: 'YOUR_PUB_KEY',
    subscribeKey: 'YOUR_SUB_KEY',
    ssl:true
});

    pubnubService.subscribe({channels: [self.channelName], triggerEvents: true});
    self.messages = pubnubService.getMessage(this.channelName,function(){
      // no handler necessary, dynamic collection of msg objects
    });
}],

    publish: function(){
        this.pubnubService.publish({ channel: this.channelName, message: {text:this.toSend} });
        this.toSend = "";
    }
});

app.main_module = ng.core.NgModule({
    imports: [ng.platformBrowser.BrowserModule, ng.forms.FormsModule],
    declarations: [app.main_component],
    providers: [PubNubAngular],
    bootstrap: [app.main_component]
}).Class({
    constructor: function(){}
});

document.addEventListener('DOMContentLoaded', function(){
    ng.platformBrowserDynamic.platformBrowserDynamic().bootstrapModule(app.main_module);
});

});
</script>
</body>
</html>