Visualizing Realtime Warriors Twitter Data using EON.js

Bay Area sports teams have killed it over the past seven years, with championship appearances by the Giants in 2010, 2012, and 2014, the 49ers in 2013, the Warriors in 2015 and 2016, and the Sharks in 2016, creating an influx of notable sports data. As a lifelong Warriors fan (and data geek), I’m really interested as to how this data translates to social media, specifically Twitter. In this demo, I connect to the Twitter firehose using PubNub, capturing all Warrior-related data (i.e. players), and then display it using the open-source framework, Project EON, and its chart builder. What better way to see what sports fans are thinking?

Project Overview

The data visualization of this demo has two main components.

PubNub Twitter Stream

• The Twitter Stream provides a good sample of tweets, reflecting the thoughts and emotions of users. This visualization uses the Twitter Stream to pull tweets that contain keywords about key Warriors players; it also looks at tweets with hashtags of the Warriors and their opponents (in this case, the Cleveland Cavaliers), to deduce which team’s fans are tweeting more.

Project EON

• Project EON is an open-source framework for the visualization that makes a chart with the data from the Twitter Stream in this tutorial. There are many different ways to visualize the data: bar, gauge, spline, and pie (which is what is used in this tutorial), and numerous ways to customize the visualization. Project EON simplifies that process, and, with the EON chart builder, you may not even have to write any code!

Start with HTML

To use EON, the head must contain the following two lines.

<script src="https://pubnub.github.io/eon/v/eon/0.0.10/eon.js" type="text/javascript"></script>

The body of the web page is comprised mainly of a few divs, and the div ids are important in displaying the chart, as well as any text or images.

The chart will attach to the chart div, and, in this tutorial, images are displayed when the mouse hovers over an item in the legend and also based on which team’s hashtags are being tweeted more. The chart only reflects tweets about Warriors players, but h4 id = "bestFans", placed below the legend of the char, is where the number of Warriors and Cavaliers hashtags are compared.

The PubNub Twitter Stream

Now, we are going to use our public Twitter Stream to extract Warriors-related data.

Connecting the Twitter Stream

Connect to the Twitter stream to subscribe to the pubnub-twitter channel with this specific key:

 var channel = 'pubnub-twitter';
 var pubnubTweet = PUBNUB.init({
   subscribe_key: 'sub-c-78806dd4-42a6-11e4-aed8-02ee2ddab7fe',
   ssl: true
 });

Subscribing the Twitter Data

You should make a function to subscribe to the Twitter channel:

   pubnubTweet.subscribe({
     channel: channel,
     callback: processData
   }); //subscribe

This getStreamData() function is called after retrieving some previous data from the Stream. The next section explains how.

Twitter’s History API

Before subscribing all new tweets, let’s retrieve some old data. How many previous tweets you want to go back to is noted by count under history.

 
pubnubTweet.history({
     channel: channel,
     count: 75,
     callback: function(messages) {
       pubnubTweet.each(messages[0], processData); 
     } //callback
}); //history

This gets the tweets from the Twitter Stream, and then the following processData() function parses them for the keywords you are looking for.

Extracting Tweets

Now, create arrays of words that you want to search for (these can also include hashtags.)

As we’re looking for Warriors tweets in this tutorial, the arrays look something like this:

var curryWords = [
   'stephen curry', 'chef curry', '#stephcurry', '#chefcurry', 'curry'
 ]
 var splashWords = [
   'splash brothers', 'splash bros', 'splash'
 ]
 var dubsTeamWords = [ //hardcoded warriorsvthunder
   '#warriors', '#warriorsnation', '#dubnation', '#gsw', '#gswarriors', 'warriors', 'golden state'
 ]
 var cavsTeamWords = [
   '#clevelandcavs', '#cavaliers', '#cavaliersnation', '#cavsnation', 'cavs', 'cavaliers', 'cleveland cavaliers'
 ]

And they should pick up tweets like this:

 
 
 
It is really up to you which words you want to search. You may get tweets that contain a word in the array but that are unrelated to the overall subject: for example, including the word ‘curry’ could pick up a tweet that is about cooking with the spice, not the player. This does slightly skew the data, but many tweets about the athlete Stephen Curry also do contain just that lone word, so it does, for the most part, end up evening out.

After making those arrays, create counts for words in each one.

  
 var numCurryWords = 0;
 var numSplashWords = 0;
 var numDubsTeamWords = 0;
 var numCavsTeamWords = 0;
 var totalNumTweets = 0;

These counts will be incremented each time a tweet containing a word from that array is found. The function that does this is the callback function when you subscribe to the Twitter channel:

 
function processData(data) {
   if (!data) return;
   if (curryWords.some(function(v) { return data.text.toLowerCase().indexOf(v) !== -1; })) {
     numCurryWords += 1;
     totalNumTweets += 1;
     publish2();
     console.log(data);
   } else if (splashWords.some(function(v) { return data.text.toLowerCase().indexOf(v) !== -1; })) {
     numSplashWords += 1;
     totalNumTweets += 1;
     publish2();
     console.log(data);
   } else if (dubsTeamWords.some(function(v) { return data.text.toLowerCase().indexOf(v) !== -1; })) {
     numDubsTeamWords += 1;
     publish2();
     console.log(data);
   } else if (cavsTeamWords.some(function(v) { return data.text.toLowerCase().indexOf(v) !== -1; })) {
     numCavsTeamWords += 1;
     publish2();
     console.log(data);
   } //else if
   bestFans();
} //processData

Wow, that’s a lot of code! In the first part, shown again below, you check if there is any data at all. If there are no tweets (meaning no data), then you return undefined. If there is data, you check that any of the words you’re looking for in any of the arrays are found in tweets from the Twitter Stream. If that’s true, the respective counts are incremented, and the tweets are printed to the console.

 
 if (curryWords.some(function(v) { return data.text.toLowerCase().indexOf(v) !== -1; })) {
     numCurryWords += 1;
     totalNumTweets += 1;
     publish2();
     console.log(data);
   }

You should use this format for each array of words you want to check tweets contain words from, so in this demo, there is one for the array containing Stephen Curry-related words, one for the array containing Splash Brothers-related words, and then one for each array for the Warriors and Cavaliers team hashtags. The full demo includes arrays for other players like Klay Thompson, Draymond Green, and more, but this tutorial is keeping it shorter and simpler.

The final piece of processData() calls the function that checks which team’s hashtags are being used more.

function bestFans() {
   var fanElement = document.getElementById('scrollDiv');
   var fanStr = document.createTextNode('Warriors fans are Tweeting more');
   var notFanStr = document.createTextNode('Cleveland fans are Tweeting more');
   var tieFanStr = document.createTextNode('Both teams\' fans are tweeting equally!');
   if (numDubsTeamWords == numCavsTeamWords) {
     fanElement.innerHTML = tieFanStr;
     fanElement.appendChild(tieFanStr);
   } //if
   else if (numDubsTeamWords > numCavsTeamWords) {
     fanElement.innerHTML = fanStr;
     document.getElementById('scrollDivPic').innerHTML = "<img src="https://s-media-cache-ak0.pinimg.com/favicons/a0c53fc35b36e1c89126baa32eaf57901cd46b7f236147a976ae8328.png?a711a114beca60aae192cca4f173535e" alt="" style="border:0px" />";
     fanElement.appendChild(fanStr);
   } else { // # Cavs hashtags > # Warriors hashtags
     fanElement.innerHTML = notFanStr;
     document.getElementById('scrollDivPic').innerHTML = "<img src="https://d1si3tbndbzwz9.cloudfront.net/basketball/team/7/logo.png" alt="" style="border:0px" />";
     fanElement.appendChild(notFanStr);
   } //else
   if (fanElement.childNodes.length >= 1) {
     fanElement.removeChild(fanElement.firstChild);
   } //if
 } //bestFans()

Now that you finally have the data, you need to display it with an EON chart!

Visualize the Tweets with EON

First, login to your PubNub dashboard to generate your own new pair of Publish and Subscribe keys. Initialize them like so:

var pubnubEon = PUBNUB.init({
  subscribe_key: 'your-sub-key', 
  publish_key: 'your-pub-key'
});

You need publish to a new channel (separate from the Twitter one), and set the x and y values for the chart under message.eon.

var nbaChannel = 'NBAChannel';
 function publish2() {
   pubnubEon.publish({
     channel: nbaChannel,
     message: {
       eon: {
         "Splash Bros": numSplashWords / totalNumTweets,
         "Curry": numCurryWords / totalNumTweets,
       } //eon
     }, //msg
     callback: function(m) {
         console.log(m)
       } //callback
   }); //pubnubEonPublish
 } //publish

This function will be called each time you want data to update.

Embed a donut chart (or whichever type of chart you like) in the HTML web page, and customize the chart with colors, labels, interactivity, and more here:

 
eon.chart({
   channel: nbaChannel,
   pubnub: pubnubEon,
   debug: true,
   generate: {
     bindto: '#chart',
     data: {
       labels: true,
       type: 'donut',
       colors: {
         'Splash Bros': 'blue',
         'Curry': 'red'
       } //colors
     }, //to be continued down below

For it to be displayed in the right spot, the chart must be bound to its divusing the bindto CSS selector. Under generate.data, the type of chart is declared, colors, labels, and sizes are assigned, and more. For more ideas and further aid, the C3.js documentation is helpful as well.

Want to make functions to add interactivity? Do that here, right below colors, under Eon.chart.generate. When the mouse hovers over an item in the legend, an image is shown, and when the mouse leaves, it goes back to how it was before.

 
     legend: {
       show: true,
       item: {
         onmouseover: function(id) {
           if (id == "Curry") {
             document.getElementById('hoverImg').innerHTML = "<img src="https://stats.nba.com/media/players/230x185/201939.png" alt="" style="border:0px" />";
             document.getElementById("hoverImg").style.transitionDuration = "10s";
           } else if (id == "Splash Bros") {
             document.getElementById('hoverImg').innerHTML = "

";
             document.getElementById("hoverImg").style.transitionDuration = "10s";
           }
         },
         onmouseout: function(id) {
             if (id == "Curry") {
               document.getElementById('hoverImg').innerHTML = " ";
               document.getElementById("hoverImg").style.transitionDuration = "10s";
             } else if (id == "Splash Bros") {
               document.getElementById('hoverImg').innerHTML = " ";
               document.getElementById("hoverImg").style.transitionDuration = "10s";
             } //last else if
           } //onmouseout
       } //item
     }, //legend
     tooltip: {
       show: false //gets rid of hover legend
     }
   } //generate
 }); //eon.chart

Possible Next Steps to Take this Further

• Other teams or sports: this could be redone for the Giants, the Sharks, for the Olympics, etc.
• Election: Check political hashtags, candidates, and see what is being said about them.
• Machine Learning: I tried to predict who was winning or who had just scored, but it was, for the most part, inaccurate. There were so many conflicting contributing factors, but would be interesting to try to predict who would win, or to make a better algorithm to guess that.
• Web scraping: ESPN made their API private, so getting their live scores is much more complicated now.

For complete code, including CSS, check out the GitHub repo, and click here to see the live Warriors visualization demo!