Tools to scrape Twitter without any coding

On request I relist some tools that can help anyone to extract data from Twitter without any knowledge of coding. Did I miss a tool? Let me know in the comments and I will add them.

TAGS6.0 - Any OS - LINK

One of my favorite tools to scrape tweets from an account or multiple keyword search. It is a Google Sheets add-on that can update hourly so you don’t have your computer running during the sessions. I have one sheet already scraping for over a year for one keyword related to one of the companies I sometimes work.

My Twitter Scraper - Windows - LINK

A free and open source Twitter Scraper built by Software Engineering student Jason Dixon with the capability to scrape Twitter realtime. It returns username, tweet, time and location in a CSV file. Make sure you have Java 1.8 installed and get your Twitter API OAuth tokens at https://apps.twitter.com by creating a new Twitter application.

NodeXL - Windows - LINK

While a lot of people seem to have trouble with NodeXL to scrape tweets I had some success with it some years ago. NodeXL is an extension for Excel 2007 or higher (I recommend to just use Excel 2007). We probably need to wait for a fix, but sure one to keep in the loop.  

Nayoun & Gephi - Windows - LINK

Not easy to install, but you can use Nayoun & Gephi (as I describe in my other post) to live visualize different aspects of Twitter. You can export the data to a csv file in Gephi.

IFTT - Any OS - LINK

Not for huge volumes of tweets, but you can use some recipes to send certain tweets (like your own or hashtags) to a Google Sheets. Not that almost all recipes are limited on the amount of tweets they can extract at a time.

Naoyun and Gephi

One way to scrape and visualize Twitter Interactions is using Naoyun (http://matthieu-totet.fr/Koumin/tools/naoyun/), a small piece of software created by Matthieu Totet, and Gephi with the Streaming Graph plugin. A similar tool has, for example, been used to show the interactions during the Egyptian Revolution (https://www.youtube.com/watch?v=2guKJfvq4uI) over time. I personally use it to scrape data and visualize live events using some keywords or usernames. For instance the video below that shows one hour of interactions searching for #TDF (Tour de France) last year. It is a great way to identify conversations and key players on a certain subject.

To install Naoyun you have to download it at http://matthieu-totet.fr/Koumin/tools/naoyun/ and install Gephi that can be downloaded at: http://gephi.github.io/ The video below shows how to connect both programs and start live streaming Twitter interactions. The data collected consists of username, media, hashtag, link and status. Exporting is possible at the Data Labory in Gephi.

If you are having problems with running the Naoyun batch file follow this link to activate java in cmd:http://www.youtube.com/watch?v=X05JPr... (is about minecraft but applies)

Fnatic "Cheating" at Dreamhack Winter 2014 - Twitter Visualisation

A little while ago I was attending Dreamhack Winter 2014 as a producer for the League of Legends BYOC tournament, but that wasn’t the most interesting thing that happened on the event. During the quarter final of the CS:GO tournament Fnatic used pixelwalking (when you use an invisible pixel that should not be part of the game) to win the third map.

The other team also used a “cheat” to win some rounds, but eventually Dreamhack decided a rematch was the best option. The community was outraged about the boost Fnatic and expressed their feelings on different social media sites. Using Naoyun and Gephi I scraped the interactions on Twitter for five hours to show the magnitude of the conversation on Twitter. Only using the keywords fnatic, ldlc and overpass gave me over 55.000 tweets (including retweets) sent by over 10.000 Twitter accounts. Making the story trending on Twitter for several hours. Every dot (node) in the picture is a Twitter account and every line (edge) shows that there was an interaction between those accounts (mention or retweet).

Original Image

Interesting to see that Fnatic and Dreamhack is mentioned more compared to several LDLC accounts. Even with the knowledge that LDLC also was using boosts to win some rounds.