Triggering Webhooks with XHR (Advanced Functions Techniques)

Tutorial Overview

The “Search Bot” is an excellent demonstration of the Functions XHR module, an efficient HTTP request library that integrates directly with Functions handlers. In this tutorial, we dive into an example of intercepting data stream message events to implement a “Search Bot” using custom serverless event handling in a realtime Angular2 web application

We’ll extend our Angular2 messaging app with a “Search Bot” handler to process message events and retrieve search results via a Web Service API. Although the example is small (using fixed commands and limited data), it could easily be extended to call additional web services for persistence, natural language detection, sentiment analysis, bridging to external message systems and more.

We’ll create a PubNub BLOCK of JavaScript that runs entirely in the network and implements the “Search Bot” feature to obtain relevant search results so that the web client UI code can stay simple and just display the data.

PubNub makes implementing realtime systems much easier by integrating enterprise-grade security, presence, storage and other advanced features into a huge (and growing!) number of client toolkits. Check out the PubNub SDK page to see them all.

The core of PubNub is Data Streams – realtime message channels that allow an unbounded numbers of devices to publish and subscribe to messages. The Functions feature allows developers to attach custom code to those message events in one of two ways, either as a “before publish or fire” event which allows modification of message attributes, or as an “after publish or fire” event which takes place after the message has been published to the message channel. (There is also an “after presence” event for implementing Presence-related functionality). Check out our Event Handler Types post for more on those.

Using the XHR module, we create a BLOCK that intercepts the “before publish or fire” events in a message channel to implement the “Search Bot” functionality. The big question is, how do we obtain search results in the context of data streams? This is a prime use case for the Functions XHR module, which offers scalable HTTP request functionality for realtime applications. Here’s how our BLOCK works:

• We examine each message for a “text” attribute.
• Based on the text value, we perform a search GET request from a third-party API.
• We decorate the message with a “search_results” attribute including the result JSON.

The XHR module is extremely configurable, so it’s easy to create functionality to access just about any API you can think of. Here are a few of the considerations where XHR works best:

• The API speaks HTTP (ideally HTTPS for security).
• The API is publicly accessible (otherwise Functions can’t access it).
• The API accepts/produces JSON/form-encoded/plain text (you can do XML, it’s just slightly harder to integrate in Functions JavaScript).
• The API responds in a reasonable time, say 1-5s (the max XHR timeout is 10s).
• The API uses a reasonably easy auth scheme (such as API key, Basic Auth, etc.).

If your API follows those rough parameters, there are a ton of places where XHR can be configured to integrate:

• XHR supports HTTP methods GET, POST, PUT, OPTIONS, DELETE, PATCH.
• XHR supports getting/setting Request and Response headers.
• Codecs are available for query string, base64, and basic auth encoding.
• XHR supports configurable timeouts up to 10s for APIs that need a bit more time.

You’ll want to grab the 35 lines of BLOCK JavaScript and save them to a file, say, pubnub_xhr_block.js. It’s available as a Gist on GitHub for your convenience.

First up, we create a function to handle incoming messages. We include a dependency on the ‘xhr’ module for HTTP/HTTPS API access, ‘codec/query_string’ for query string encoding. Other common codecs to require are ‘codec/auth’ for HTTP Basic Auth encoding, and ‘codec/base64’ for base64 encoding.

export default (request) => {
    const xhr    = require('xhr');
    const query  = require('codec/query_string'); // for HTTP URL encoding
    const auth   = require('codec/auth');         // for HTTP Basic Auth
    const base64 = require('codec/base64');       // for other custom schemes

Next, we configure the API URL, query string options, and HTTP options.

const apiUrl = "https://api.duckduckgo.com/?";
var queryString = query.stringify({
    q : request.message.text,
    format : 'json'
});
var httpOptions = {
    method  : 'GET', // allowed GET/POST/PUT/OPTIONS/DELETE/PATCH
    timeout : 5000,  // max 1000 (10s)
    headers : {
       'Accept' : "application/json"
       // 'Content-Type' : ...
       // 'Authorization' : ...
       // 'If-Modified-Since' : ...
    },
    body    : null,  // form body, JSON string, XML ...
};

    if (request.message && request.message.text) {
        return xhr.fetch(apiUrl + queryString, httpOptions).then((result) => {
            request.message.search_results = JSON.parse(result.body);
            return request.ok();
        });
    }
    return request.ok();
}

<!DOCTYPE html>
<html>
  <head>
    <title>Angular 2</title>
    <script src="https://unpkg.com/core-js@2.4.1/client/shim.min.js"></script>
    <script src="https://unpkg.com/zone.js@0.7.2/dist/zone.js"></script>
    <script src="https://unpkg.com/reflect-metadata@0.1.9/Reflect.js"></script>
    <script src="https://unpkg.com/rxjs@5.0.1/bundles/Rx.js"></script>
    <script src="https://unpkg.com/@angular/core/bundles/core.umd.js"></script>
    <script src="https://unpkg.com/@angular/common/bundles/common.umd.js"></script>
    <script src="https://unpkg.com/@angular/compiler/bundles/compiler.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser/bundles/platform-browser.umd.js"></script>
    <script src="https://unpkg.com/@angular/forms/bundles/forms.umd.js"></script>
    <script src="https://unpkg.com/@angular/platform-browser-dynamic/bundles/platform-browser-dynamic.umd.js"></script>
    <script src="https://unpkg.com/pubnub@4.3.3/dist/web/pubnub.js"></script>
    <script src="https://unpkg.com/pubnub-angular2@1.0.0-beta.8/dist/pubnub-angular2.js"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" />
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" />
  </head>

<body>
  <main-component>
    Loading...
  </main-component>

<div class="container">
    <pre>
    NOTE: make sure to update the PubNub keys below with your keys,
    and ensure that the BLOCK settings are configured properly!
    </pre>
    <h3>MyApp Functions XHR Integration</h3>
    <br />
    Query:
    <br />
    <input type="text" [(ngModel)]="toSend" placeholder="query" />
    <input type="button" (click)="publish()" value="Send!" />
    <hr/>
    <br/>
    <br/>
    <ul>
      <li *ngFor="let item of messages.slice().reverse()">
        <div><b>{{item.message.text}}</b></div>
        <ul>
          <li *ngFor="let result of item.message.search_results.RelatedTopics">
            <a [href]="result.FirstURL">{{result.Text}}</a>
          </li>
        </ul>
      </li>
    </ul>
</div>

<script>
var app = window.app = {};
app.main_component = ng.core.Component({
    selector: 'main-component',
    template: `...see previous...`

}).Class({
    constructor: [PubNubAngular, function(pubnubService){
        var self = this;
        self.pubnubService = pubnubService;
        self.channelName = 'xhr-channel';
        self.messages = [];
        self.toSend = "";

pubnubService.init({
    publishKey: 'YOUR_PUB_KEY',
    subscribeKey: 'YOUR_SUB_KEY',
    ssl:true
});

    pubnubService.subscribe({channels: [self.channelName], triggerEvents: true});
    self.messages = pubnubService.getMessage(this.channelName,function(){
      // no handler necessary, dynamic collection of msg objects
    });
}],

    publish: function(){
        this.pubnubService.publish({ channel: this.channelName, message: {text:this.toSend} });
        this.toSend = "";
    }
});

app.main_module = ng.core.NgModule({
    imports: [ng.platformBrowser.BrowserModule, ng.forms.FormsModule],
    declarations: [app.main_component],
    providers: [PubNubAngular],
    bootstrap: [app.main_component]
}).Class({
    constructor: function(){}
});

document.addEventListener('DOMContentLoaded', function(){
    ng.platformBrowserDynamic.platformBrowserDynamic().bootstrapModule(app.main_module);
});

});
</script>
</body>
</html>