Service URL - http://opsdb.dante.net/
- Usual configuration - Points to primary instance.
- Primary Instance - http://prod-opsdb01.geant.net/,
- Secondary Instance - http://prod-opsdb02.geant.net/
OpsDB runs on two servers, named appropriately axxxVMs on each environment (Prod, UAT and Test)
- xxxx-opsdb01.geant.net and xxxxx-opsdb02.geant.net where xxxxx = prod, uat, or test
First Steps
If for any reason the system becomes unavailable:
- Check if the primary instance available by going to: http://prod-opsdb01.geant.net/
- Something wrong with DNS entry (troubleshooting steps)?
- If Primary instance of OPSDB is not available then check if the secondary instance of the OPSDB available by going at http://prod-opsdb02.geant.net/
- If yes,
...
- switch the DNS entry for OPSDB from the ‘Primary(01)’ instance
...
- to the ‘Secondary(02)’ instance. This will allow the general user to continue working on OpsDB whist we continue with our investigations as to why it initially went down.
If we find that both instances have become unavailable, then contact with IT / SWD is of the upmost urgency as further investigation, steps, and decisions will have to be taken across departments (i.e. IT / SWD / OC) as to the best way forward to resolve these issues
Change the Domain Name System (DNS) entry for OpsDB (i.e. Move from one instance to another)
Currently the URL OpsDB.dante.net resolves to our primary instance of our OpsDB Server (which we call ‘Primary’, or ‘01’) – Prod01.geant.net.
If for any reason the primary instance of OpsDB becomes unavailable we need to change the OPDSB.dante.net URL to resolve to our secondary instance of our OpsDB server (which we call ‘Secondary’. or ‘02’) - Prod02.geant.net.
The action usually taken by devOps would be as follows, which you would need to do:
...
OpsDB
- Change the CNAME opsdb.dante.net in Infoblox, to point
...
- to prod-
...
...
Once this has been done the system should then be available to the users once again whilst more detailed investigation takes place into why the Primary instance has become unavailable.
...
If the machine is running follow steps below:
...
Check Apache.
- Has apache failed? Is it running?
...
Currently MySQL data backups are stored in the /opt/vackupsbackups/mysql folder within each VM.
...