Background

Every Four Hours (which is configurable) or when a user presses the Refresh Map button a message is created on the Dashboard Enterprise Service Bus (ESB) in the prodrepopulate.map.queue

The OpsDB Copier process is a runnable JAR file that takes a MySQL dump of the OpsDB data and stores it in /home/local/GEANT/dante.sanigar/dumps/opsdb.sql this file is tarred up into  /home/local/GEANT/dante.sanigar/dumps.tar.gz

The file is rsynced across to the three instances of the Correlator machines:  test-dboard01-corr.geant.net, uat-dboard01-corr.geant.net and prod-dboard01-corr.geant.net to /tmp/received/dumps.tar.gz 

The process writes to a log file /home/local/GEANT/dante.sanigar/copier.log. Changes to the configuration can be made by editing the  /home/local/GEANT/dante.sanigar/config/application.properties 

When the OpsDB Copier finishes, it sends a message to the ESB prodreload.map.queue.

When a message appears in the prodreload.map.queue, the ESB Consumer on the Correlator machine picks up the message and starts to load the data it has obtained from the Operations Database (OpsDB) into the Dashboard Network Map. The file /tmp/received/dumps.tar.gz is untarred to /tmp/received/opsdb.sql and then processed

The Consumer on the Correlator machine is a Tomcat7 webapp called /var/lib/apache-tomcat7/webapps/populate-web

 Apart from the Tomcat Logs located on /var/log/apache-tomcat7, three log files are created in the /var/log/snmptraps/new-dashboard/Populator/ directory: These are allLog.txt, errorLog.txt and warnLog.txt

The configuration files are stored in /var/lib/apache-tomcat7/webapps/populate-web/WEB-INF/classes. 

Building Code

To obtain the source code from Git using GitBash, start GitBash and navigate to your eclipse workspace directory, then run the following:

git clone git@git:dashboard-opsdb-copier

The application uses two xml configuration files and one properties file messaging.xml, common.xml and esb.properties.

The application also depends on three classes:

net.dante.dashboard.data.importAndExport.DatabaseDumper - Found in dashboard-common

net.dante.dashboard.error.ProcessErrorExtractor- Found in dashboard-common

net.dante.service.command.CommandRunner - Found in dashboard-common

PermissionChanger - Found Somewhere - changes permission of dumps.tar.gz to 777

Process Logic

The application has a consumer that listens for message on the networkMapRefreshQueue, when a message is received, it carries out:

databaseDumper.takeDump(); - Produces /home/local/GEANT/dante.sanigar/dumps/opsdb.sql
tarExecutor.execute(); - Produces /home/local/GEANT/dante.sanigar/dumps.tar.gz
permissionChanger.execute(); - changes permission of dumps.tar.gz to 777
transferToAllEnvironmentsAndNotify(); - rsyncs the files across to the Prod, Uat and then Test then sends an ESB message to repopulateMapQueue