To complete this step you need to clone https://gitlab.geant.net/msr-automation/cacti-utilisation  on your local machine. This application will generate the script files which are being referred to in the description below.

Also, You will need to create a folder named "fix-scripts" somewhere on your local machine. In my case I had it under "C:\Users\Omar\projects\fix-scripts". This is the folder where all scripts will be loaded to.

Tip: "application.properties" file needs to be created in "/msr-cacti-utilisation/src/main/resources" inside that file it should have this line "files.outputdirectory=C:\\Users\\Omar\\projects\\fix-scripts\\ " This is the path for "fix-scripts" folder. 

Steps to follow:

  1. Open Eclipse/IntelliJ and browse to "msr-cacti-utilisation" - As mentioned above, this is in Git https://gitlab.geant.net/msr-automation/cacti-utilisation 
  2. Make sure to change the month and year in the "Main" class "cacti-utilisation\src\main\java\org\geant\msr\utilisation\Main.java" Lines 36 and 37
  3. Run the maven project (cacti utilisation) from Eclipse/IntelliJ. This will generate the script files and will save them in ""fix-scripts" folder.
  4. Copy the produced files (C:\Users\Omar\projects\fix-scripts) to your home directory on "prod-cacti01-fra-de.geant.net". You can use WinSCP client from Windows machine.
  5. Log into prod-cacti01-fra-de.geant.net as yourself 
  6. Copy the scripts from your home directory to "/home/cactiuser/scripts" on "prod-cacti01-fra-de.geant.net". To copy multiple files:  sudo cp fix_* /home/cactiuser/scripts/  

change directory to /home/cactiuser/scripts

** Are you able to sudo as cactiuser?  "sudo su cactiuser"

You will need to convert files from Dos to Unix - https://stackoverflow.com/questions/9374991/how-to-convert-files-from-dos-to-unix

  • Run "sudo dos2unix ./fix*" inside  "/home/cactiuser/scripts"
  • Then, run "sudo ./fix_monthly_utilisation.sh -y 2017 -m 7"  ** Pay attention to the year and the month value in the command.

Some circuits will be missing from the MSR GUI, and some circuits will show zeros, to fix that I have added those circuits in bash scripts which you can run, as below:

On prod-cacti01 "/home/cactiuser/scripts"

Results will be shown on this page:  http://test-msr.geant.net:8888/msr/ip_util_ip.jsp

The scripts below are what you've already generated from cacti-utilisation application and copied them from fix-scripts folder.

==================================================================================================================================

Run the following command:

sudo ./fix_monthly_utilisation.sh -y 2017 -m 7

** Pay attention to the year and the month value in the above command.

This scripts calls the following scripts:
./fix_ip_access.sh -y $year -m $month >> /tmp/cacti_util_ip_access_$year_$month.log
./fix_ip_access_ias.sh -y $year -m $month >> /tmp/cacti_util_ip_access_ias_$year_$month.log
./fix_ip_access_ipv6.sh -y $year -m $month >> /tmp/cacti_util_ip_access_ipv6_$year_$month.log
./fix_interconnect.sh -y $year -m $month >> /tmp/cacti_util_interconnect_$year_$month.log
./interconnect_port_summ.sh -y $year -m $month >> /tmp/cacti_util_interconnect_port_summ_$year_$month.log
./fix_interconnect_ipv6.sh -y $year -m $month >> /tmp/cacti_util_interconnect_ipv6_$year_$month.log
./fix_trunks.sh -y $year -m $month >> /tmp/cacti_util_trunks_$year_$month.log
./fix_dws.sh -y $year -m $month >> /tmp/cacti_util_dws_$year_$month.log
./fix_plus.sh -y $year -m $month >> /tmp/cacti_util_plus_$year_$month.log
./fix_lhcone.sh -y $year -m $month >> /tmp/cacti_util_lhcone_$year_$month.log
./fix_eap.sh -y $year -m $month >> /tmp/cacti_util_eap_$year_$month.log

===================================================================================================================================

NOTE: for this page http://test-msr.geant.net:8888/msr/ip_util_ip.jsp the "Usage Peaks" percentage values are taken from opsdb directly not from the opsdb version sitting on MSR. So, in case of weird ( >100%) values in this page, you'll need to re-import opsdb script "opsdb_import_new.sh" which is shown in page 1.1 in this documentation.

** You can see that log files are produced in the /tmp direcory of prod-cacti-fra-de.geant.net. These are useful when diagnosing zero values displayed in the MSR utilisation pages.

if you need to add peers to http://test-msr.geant.net:8888/msr/int_util_ipv6.jsp you will need to look at prod-cacti01 ... /home/cactiuser/scripts and add a line to one of the scripts mentioned above. For instance, to add "NL-SINET" to IPv6:

add "java -cp ".:mysql-connector-java-5.1.16-bin.jar" RRDDump "mx1_ams_nl_traffic_out_46351.rrd" "100000000000" "$year" "$month" "" "" "46519" "ipv6"" ...Also, run the command outside the file for immediate change on the jsp page. In order to get the rrd file name go to https://cacti.geant.org/cacti/graph_view.php?action=tree&tree_id=14&leaf_id=57821 choose "GEANT" and then "R&E Peering" from the menu on the left.  Look for the IPv6 graph and click on the spanner next to it. To get the speed, go to http://opsdb.dante.net/circuits.phtml and search for the circuit

All these scripts rely on the following Java batch program called RRDDump

RRDDump takes 8 parameters: 

#1 RRD file name that will be read

#2 The capacity of the circuit in Bytes

#3 The year you require

#4 The month you require

#5 The from date and time in the format dd-mm-yyyy 00:00:00

#6 The to date and time in the format dd-mm-yyyy 23:59:59

#7 The absid of the corresponding circuit with OpsDB.

#8 The service type of the circuit

** Check users can run this Java program

E.g.

echo "*** ACONET AP *** `date`"
java -cp ".:mysql-connector-java-5.1.16-bin.jar" RRDDump "rt1_vie_at_traffic_in_1404.rrd" "20000000000" "$year" "$month" "" "" "31467" "CCT"


** Improvement - Have a link table that links rrd file names to the name of the circuit, similar to the SPLUNK_LINK table. Then you can use the name to link to the opsdb_circuit table to find the absid of the operational circuit with that name.

Two tables are updated by RRDDump, reports.report_utilization and reports.report_dante_service_utilization. Which table is updated is dependant on the value of the service type parameter.

Sometimes within the log files, you will discover the Spike Detected message. When this occurs, you need to double check the subscription value as the circuit may have been upgraded and alter the time period range of the query.

E.g.

# ACONET AP Spike Correction

java -cp '.:mysql-connector-java-5.1.16-bin.jar' RRDDump "rt1_vie_at_traffic_in_1404.rrd" "20000000000" '2017' '7' '01-07-2017 00:00:00' '15-07-2017 23:59:59' '31467' 'CCT'


*** Add section on reviewing data and checking for zeros/ graph exists in Cacti

*** Add bit about commissioned circuits - Old absids


  • No labels

2 Comments

  1. Emma Apted

    "#2 The capacity of the circuit in Bytes" - This looks more like the capacity in bits per second.

  2. Emma Apted

    It seems that neither reports.report_dante_service_utilization nor reports.report_utilization on the Data Warehouse server have been updated since 2016. We should copy the data from Cacti server to here. Also: Access denied for user 'cactiro'@'%' to database 'reports' on the Cacti server.