DIY: Internet Speed Monitoring and Visualization Pipeline (native)

Did you ever wonder if you get what you are promised by your Internet provider? I did, so I constructed a pipeline to continuously log and visualize up/down speed. To use it you might need to install speedtest-cli, R, pandoc and latex.

There are 2 ways to get this service running, the first one is to manually setup the environment and the second one is to use a vagrant box I prepared. In this post I will describe the manual setup which can be done on Linux or OSX.

Report Results

Using the method described below will check the speed every hour and create a report every day. Both schedules are easily configurable to your needs.

The report consists of 2 parts, which are the graph and the data-table.
The graph shows the up and download values in MB/s over the time. Missing values are shown as gaps in the graph.

The data-table shows the individual data points so it’s easily possible to look up individual values.
data table

Native installation (e.g. if you use an ARM processor)

Install speedtest-cli:

Clone the repo from here.

git clone https://github.com/sivel/speedtest-cli.git

OR install it with pip (as described here):

sudo apt-get install python-pip
sudo pip install speedtest-cli

Install R:

Download for Linux with your favorite package manager like this:

sudo apt-get install r-base

Download for OSX with homebrew like this:

brew tap homebrew/science
brew install Caskroom/cask/xquartz
brew install r

Download for any OS from cran.

Install R libraries:

The current workflow requires multiple R packages:

  • rmarkdown
  • scales
  • ggplot2

Each package can be installed inside R using

install.packages("packageName")

Install Latex:

For Linux I would recommend to install the complete texlive-full because latex is awesome.

sudo apt-get install texlive-full

OSX users might try MacTex.

Install pandoc:

See installation instructions here or use your favorite package manager e.g. sudo apt-get install pandoc

Log the speed data

Save this script which prints your internet speed as check_internet_speed.sh and make it executable using chmod +x check_internet_speed.sh.

Add the following lines to your cron-jobs using crontab -e (don’t forget to put a newline at the end of the file)

*/5 * * * * /PathToTheScript/check_internet_speed.sh >> /PathToMyLogLocation/LogfileName.csv

Create the report

Save this script as create_internet_speed_report.sh and this script as monitor_internet_speed.Rmd and make them executable using chmod +x create_internet_speed_report.sh and chmod +x monitor_internet_speed.Rmd.

Add the following line to your cron-jobs using crontab -e (don’t forget to put a newline at the end of the file)

3 * * * * /PathToTheScript/create_internet_speed_report.sh -i /PathToTheScript/monitor_internet_speed.Rmd -o /PathToMyResultReport/monitor_internet_speed.pdf -d /PathToMyLogLocation/speedtest_values.csv

If you have troubles with the cron job consider these cron gotchas and crontab.guru.
As cron uses a different PATH than your user shell does, I needed to manually set my PATH in the crontab like this PATH=/usr/bin:/usr/local/bin:/bin:/PathToTheScript.

Result

Now you have two schedules running. One checking the Internet speed and writing to a CSV and the second one using the CSV to create a PDF report of your Internet speed. Together they create a fully automatic speed checking and reporting pipeline creating results as shown at the top of the post.

One of my next post will describe how to use the same pipeline without the need to configure the whole setup yourself, relying on a Vagrant image instead.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: