Realtime Stocks with Go Lang - Why It Matters to You

At PubNub we have had a Real Time stocks demo for quite some time, deployed at http://rtstock.co. While it functions, the implementation is not very good. On the backend it uses PHP to generate random stocks data, publish this data to PubNub’s network, populate a channel group with stock names, and define permissions. Together with an Apache server, PHP is used to share PubNub keys with the client a JSON service. It uses many different files to run several different processes and is not easy to start, stop and configure all of the scripts to work properly.

The root cause of these problems is the PHP language. PHP simply wasn’t designed for this type of work. There are far more appropriate programming languages and environments for tasks like creating background-workers, as used in our Real-Time Stocks back-end. One such language is Go, developed at Google.

One of the advantages of Go is that your code is compiled into a binary file and consequently has much better performance than non-compiled languages. Also you don’t need to install any environments or runtimes like in Java. Just compile to a binary on your development machine and you can run it on production using a single executable (configs, html and other assets are not considered). All code dependencies are compiled into the final binary.

In this blog we will rewrite the PHP backend in Go

Configuration

Using Go you can load your configuration from JSON files. Files of this type can be easily converted into a struct. This is a config file with PubNub keys and other options:

And this is the Config struct representation of this config in the Go source code:

The json:"xxx" statement on the right side of each element definition is a special syntax to point the JSON decoder to a specific field in the JSON file. The same is done for the stocks.json config, where are located all required data about each stock. This is the struct for the stocks.

Loops

After the stocks data is loaded from JSON file, we need to run a cycle for each:

Notice that since the done channel is not passed anywhere it will never be resolved in our example, because it’s assumed that it should run indefinitely until you explicitly kill this process.

The RunCycle() method instantiates PubNub instance for the current stock and runs an infinite cycle of updateValuesAndPublish() calls. RunCycle() runs using goroutine, so all stock cycles are parallel. This method generates new and almost random values for stocks then publishes them to associated channel. After publish is done it sleeps a random amount of time. Minimum and maximum (MinTrade and MaxTrade fields respectively) timeouts for each stock are configured inside the stocks.json file:

For all types that can be serialized using `json.Marshal`, the serialization process is performed automatically. You do not have to do it yourself.

Handling response

Each request method, either subscribe-like (long-polling) or non-subscribe call accepts a success and error []byte channels. When a response is received the corresponding channel will be populated with it’s data, otherwise a timeout will occur. This is an examle of publish request handling:

The messaging.GetNonSubscribeTimeout() has a default value of 20 (seconds) and is not strict. You can use any uint16 value you want. To handle a subscribe request, just wrap your select with a for{} block to make it run infinitely:

Serving HTTP

Serving HTTP endpoint is is done with only two functions. The first one is a specific request handler. It exposes the publish and subscribe keys stored in configs to JavaScript client:

The second function runs an HTTP server on specific port and maps two endpoints to their handler functions. At the root endpoint / a public folder is served as statically files. The /get_configs endpoint mapped to be handled by GetConfigHandler function from the example above.

NOTICE: To run HTTP server on 80 port or others lower than 1024 you need root permissions.

Deployment

Go apps can be deployed on multiple PaaS services like Amazon Elastic Beanstalk, Google App Engine, Heroku, Windows Azure, etc. Check out their manuals how to do that. Here, in this post I’ll cover how to setup a Go service on a brand new Ubuntu 15.10 installation.

Since v15.04 Ubuntu uses systemd to manage startup processess. Create a config to start our Go app during system start:

Name this file pubnub-stocks.service and place it in the /lib/systemd/system/ folder where the rest of systemd services are placed. Now perform the following steps:

1. Clone app to /var/pubnub-stocks folder. It contains both configs and static files to serve.
2. Inside /var/pubnub-stocks/public folder run bower install command to install Bower dependencies.
3. If required, change PubNub keys and other parameters inside config/config.json.
4. Copy compiled binary file to /usr/local/bin/pubnub-stocks and turn on an execution flag on it.

After performing above steps you can start, stop and get status of your app whenever you logon to the server by simply typing:

To check out the logs use journalctl command:

Conclusion

That’s it. You can see a live running version of this project at http://rtstock.co/. The source to this project is in the GitHub repo. Go is a far more appropriate language for this type of program than PHP because it can properly handle multiple processes and be a webserver in it’s own right to handle the client credentials. It always pays to use the right tool for the right job.