Page History

Log into http://cacti.geant.net/cacti/

Quick Reference

For those who definitely know what they're doing and just need a reminder of the commands:

• ssh -l dante.<surname> prod-cacti01-fra-de.geant.net

• chmod 777 to the /opt/cacti/rra/<name_of_rrd>.rrd                      <-- Richard has permissions to do this step, Ops does not!
• cd /tmp

• rrdtool dump /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd > /tmp/mx1_lon_uk_traffic_in_20683.rrd.xml

• vi /tmp/mx1_lon_uk_traffic_in_20683.rrd.xml
• sudo -u cactiuser mv /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd.bak
• sudo -u cactiuser rrdtool restore /tmp/mx1_lon_uk_traffic_in_20683.rrd.xml  /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd
• sudo -u cactiuser scp -i /home/cactiuser/.ssh/id_dsa /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd cactiuser@prod-cacti02-vie-at.geant.net:/opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd

Detailed Instructions

Locate the graph with the Spike:

...

In this case the rrd file name is /opt/cacti-0.8.7e/rra/mx1_lon_uk_traffic_in_20683.rrd. In the remainder of this description we will use this rrd file. In your case you will need to change the name appropriately. You will also need to remove the Cacti version number from the path, as the real path is /opt/cacti/rra.

Click on the Magnifying Glass icon to the right of the graph and select the time period you wish to zoom into.

...

Continue 'zooming' until you can determine the date and time range of the Spikes. In the above case one spike occurred on 19/01/2016 and the other on 26/01/2016 

Using the Epoch & Unix Timestamp Conversion Tool found at http://www.epochconverter.com/ convert the date/times into their UTC equivalent as Cacti stored its date and time in UTC format.

In this case the UTC ranges are 1453161601 - 1453247999 and  1453766401 - 1453852799. We can also see that the data values on the graph are in the range of 3x10e12 (3.0Tbits) and 3.5x10e12 (3.5Tbits). The RRD Files stores the information in Bytes so we have to divide by 8 to get the magnitude of bytes. (The are 8 bits in a byte). In this case we are looking for data values in the RRD file around 3.75x10e11 and 4.375x10e11

Log into prod-cacti01-fra-de.geant.net.    Username will be of the format dante.<surname> and the password will be the same as in Crowd and used forLog into prod-cacti01-fra-de.geant.net  issues.geant.net.

Change to the /tmp directory and export the data to an XML file format.

Check that the chmod of the rrd file is set to 7777 prior to attempting to remove a spike.

rrdtool dump /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd > /tmp/mx1_lon_uk_traffic_in_20683.rrd.xml

Edit the  /tmp/mx1_lon_uk_traffic_in_20683.rrd.xml and find the offending data. As we know the data is in the order of 10e11, we can search for the following string e+11 . We can see the timestamp is in the right range.

NOTE: The values in the RRD file are in bytes per second. You will need to multiple these values by 8 to obtain the bits per second. The graphs on the website and CSV files that can be downloaded from the website all show the traffic levels in bits per second.

Inbound traffic:

<!-- 2016-01-18 00:00:00 UTC / 1453075200 --> <row><v> 3.5642652369e+07 </v><v> 1.1136736365e+06 </v></row>
<!-- 2016-01-19 00:00:00 UTC / 1453161600 --> <row><v> 9.0115656678e+07 </v><v> 3.1514951986e+06 </v></row>
<!-- 2016-01-20 00:00:00 UTC / 1453248000 --> <row><v> 3.8543163654e+11 </v><v> 9.2684560857e+09 </v></row>
<!-- 2016-01-21 00:00:00 UTC / 1453334400 --> <row><v> 8.0043154538e+07 </v><v> 2.7647348898e+07 </v></row>
<!-- 2016-01-22 00:00:00 UTC / 1453420800 --> <row><v> 1.0178635224e+08 </v><v> 2.1954640988e+07 </v></row>
<!-- 2016-01-23 00:00:00 UTC / 1453507200 --> <row><v> 6.5086071137e+07 </v><v> 2.7845010247e+07 </v></row>

...

<!-- 2016-01-19 00:00:00 UTC / 1453161600 --> <row><v> 9.0115656678e+07 </v><v> 3.1514951986e+06 </v></row>
<!-- 2016-01-20 00:00:00 UTC / 1453248000 --> <row><v> 3.8543163654e+07 </v><v> 9.2684560857e+09 </v></row>
<!-- 2016-01-21 00:00:00 UTC / 1453334400 --> <row><v> 8.0043154538e+07 </v><v> 2.7647348898e+07 </v></row>

Take great care using global search and replace, do a find first to see how many data items are in this order. In the case of 10e9, there are a few so you will have to be more specific in the search and replace e.g find 9.2684560857e+09 and replace with 9.2684560857e+06

...

Make the necessary changes in the text editor, then save your changes. (You may find it easier to WinSCP this file to your local machine for editing in a different text editor if you are not famiiar with UNIX text editors such as vi).

Create a backup of the file you wish to change. Please note we are using mv and not cp as because we will be unable to restore and a rrd file if it exists already onthe on the file system.

sudo -u cactiuser mv /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd.bak

...

sudo -u cactiuser scp -i /home/cactiuser/.ssh/id_dsa /opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd cactiuser@prod-cacti02-vie-at.geant.net:/opt/cacti/rra/mx1_lon_uk_traffic_in_20683.rrd 

The graph should now look a lot better: 

Image Added