D3js Tutorial: Building Realtime Data Visualization Graphs

Unfortunately, once a user has loaded a web graph, the data is already stale and the user would normally need to refresh the entire page to get the latest information. However, not having the most current, updated information can be extremely detrimental to a decision making process. Thus, the need for realtime charting!

This D3js tutorial blog post will show how you can fix this problem and use the PubNub Data Stream Network to enhance D3js with realtime graphing without reloading the page or polling with AJAX requests for changes.

Want to see it in action? Check out our live, working bitcoin graph demo here.

PubNub Bitcoin Connection Live JSON Feed

First you'll have to connect to a live stream of data being published to a PubNub channel. This starts by opening up a connection to the PubNub network. Here we are making a connection to the PubNub Bitcoin Exchange GitHub JavaScript Streaming SDK to get Real-Time Bitcoin price updates. If you are looking for a simplified JavaScript SDK for a Bitcoin Exchange socket data stream, you can use this: PubNub Bitcoin Exchange JavaScript SDK.

Next we are able to subscribe to the incoming feed updates on the channel and get ticker updates on the current Bitcoin price:

This will give us a large JSON payload, but we can select the two values we are interested in which is the average value and the timestamp of recorded symbol change:

Historical JSON Bitcoin Data Feed

Loading historical JSON dumps works quite well if we want to know the value of Bitcoin when it was originally recorded. Unfortunately, this does not give us much context about the data unless we leave our page open for a long time. A typical stock dashboard will also include data from the previous 24 or 48 hours.

To accomplish this, we can use the PubNub Storage and Playback service to get data previously published in the channel. When we pull the history, however, we get a long stream of tens of thousands of events in the past 24 hours. This can take long to parse through and cause long wait times before the user sees any data at all. Instead, what we want is shown below; where we take one piece of data from every time increment and put it into one data set:

This can be accomplished by calling the History API and asking for one piece of data at every time interval. We start at a given date in the past, in this example 24 hours. We then call pubnub.history with a count of 1 at the date. When we get data back, store it in a list and call it again with the start date plus our timestamp increment; in this case one hour. To load snapshots of one hour increments in JavaScript we’d do something like this:

The HistoryLoader is wrapped into an object that can be generalized for different use cases. It is utilized in this example like so:

Now that we have our dataset loaded, let’s build a visual graph on a standard Cartesian coordinate system using D3js as our rendering engine.

Building the Bitcoin D3js Graph

This part is going to assume you have some prior knowledge of D3js. Since D3js is a massive library that can apply to thousands of applications we will not go into too much detail about the inner workings of it. We recommend you read some of the D3js Wiki & Documentation before you move forward to familiarize yourself with how D3js works.

D3js Area Chart

Building the area chart is a straightforward process. The Bitcoin Exchange source data arrives as string-based JSON values, but that’s only for this feed type and you’ll see lots of variation from other feed sources. But in this case you can be confident that the values will be string format vs. integer based. We should first convert our data into more readable values so we can sort them easily:

Now we can set up an area graph where the x position is the ticker timestamp and the y position is the ticker value. After some preliminary setup of scales and axes which is not shown we build the graph. This will give us the large area graph on the top of the graph area:

D3js Brushing Window

The brushing window allows our users to dig deeper into the data and see it closer up. Luckily, D3js has this functionality built into the framework which makes it easy to implement. The first step is setting up a brush area that listens for the users clicking and dragging across the area:

This will call our brushed method where we can then change the large graph to represent the brushed area:

This function will take the area that was brushed and reset the domain of the large graph. It will then repaint the large graph in the new domain on the screen.

Adding Tooltip Window in D3js

For some reason, this amazingly well done graph might still not be good enough for your users. They might crave more information about the actual data points themselves. This is where tooltips come in.

The tooltip is the bubble that pops up upon hovering over a circle on the graph. It shows the value at that exact data point gathered. To do this, we add an element that triggers a mouseover event, such as a circle, and display a tooltip when the event is triggered. I based this on an in-depth tutorial, which can be seen here: D3js Adding tooltips to a D3js graph.

Now this is quite lengthy but it does everything in one shot. The first step is taking the data and appending circles at the correct x and y coordinates. Next we add the mouseover listener. This will set the value of the tooltip and show it at the correct x and y coordinate when it happens.

Bitcoin Graph with Real-Time Updates

Now comes the best part: Real-Time! Our graph currently shows the data when you load the page. To get the most up to date data, the client normally would have to reload the page. Since PubNub is constantly publishing this data out along the PubNub API, we can just listen for the latest changes instead of refreshing the page!

To listen to the latest data we need to subscribe to a PubNub channel. This will call a callback method with the latest data precisely when it is published. We can then use this data to update our graph, update the current price, and flip the coin.

You will also see a presence callback in the subscribe code. This will be called when a user enters or leaves the room, allowing us to see how many subscribers there currently are to a given channel.

Now we can update our visual graph. One of the well designed parts of D3js is that adding elements and updating data can be called separately. What we can do with this is just update specific parts of the graph based on the latest data. In the case of the area graph, we can give it the new data, recalculate the area, and tell the graph to redraw:

All of the variables have been defined when we initially loaded the graph when the page was created. In this call we are first figuring out the new domain of the data. We then pass this along to our two area graphs to recalculate the data points and tell it to redraw.

Conclusion on D3js Tutorial for Graphing Realtime Data

To find the full source code for this project take a look at the GitHub D3js Bitcoin Graphing Repository for the demo. You can also find a standalone version of the demo on the Live Bitcoin Dashboard Demo. The power of Real-Time graphing can help your users be more informed when looking at information. Using D3js also makes it easy to add Real-Time functionality to any graph. Every graph that is seen on the D3js tutorials page can easily be made realtime with PubNub using minimal code changes.

Get Started Sign up for a free account and power data visualizations