Mutiny Cold Standby
Edition 1.0 - 10th February 2022
Introduction
Mutiny Cold Standby is a configuration involving two Mutiny appliances, in which The Primary Appliance is responsible for monitoring the network and its partner, the Secondary, simply stores regular copies of the full system backups (Snapshots), made daily by the Primary, so that it is able to take over as the monitoring system should the Primary appliance fail. This is achieved by manually restoring the most recent backup on the Secondary system once the Primary is dead.
Please note the following caveats:
• Two Mutiny appliances are required – a Primary and a Secondary. A popular choice is a physical Primary and a VM Secondary, although all other combinations are valid.
• The two Mutiny appliances must be connected to the same VLAN.
• In the normal state of operation, each appliance must have a unique IP address on the VLAN and the two systems must be able to communicate freely between themselves using network protocols including ICMP, http/https, rsync and SNMP (that this is normally the case for devices on the same VLAN).
• If the Primary fails it must be completely shut down or disconnected from the VLAN, as when the Secondary takes over the monitoring role, previously held by the Primary, as the Secondary becomes a complete clone of the Primary system and this includes the IP addresses and all other network settings.
• Two new licences are required, one for the Primary and the other for the Secondary. These will replace your existing Mutiny licence.
• If there is a failure of the Primary system, you should only lose up to 24 hours of monitoring data and configuration.
Setting up a New Cold-Standby Pair
In the following instructions set, it is assumed that you already have a working, licensed Mutiny system that is monitoring the network. This will become the Primary system.
Logon as a “Superadmin-Level” user and click through to “Backup/Restore” from the “[Admin] tab.
Set “Auto-save” to “Daily”, as below:
Next, you will need to set up the Secondary System. If you already have an existing appliance that you wish to re-use, this will need a “Factory Reset”. Alternatively, you could purchase a new physical appliance or set up a virtual system from the OVF or ISO files on the “Downloads” Page of the Mutiny Website (https://www.mutiny.com/downloads/)
To perform a Factory Reset you will need to click the button marked “[Reset to Factory Defaults]” in “System Operations” page accessed via the “[System]” tab:
If you choose to build or purchase a new appliance, it will already be in the correct configuration, so you will only need to set the IP address, network mask, default gateway and other network settings, as required. These properties are set via the appliance’s Maintenance (:81/:444) interface user the “Network” tab as below:
Upgrading to the Same Versions
Both Mutiny systems need to be on the same version of both the Mutiny software and the Platform OS. Connect to the downloads page of the Mutiny website (https://www.mutiny.com/downloads/) and download the latest versions of both the “CentOSmutinyBundle” and the “centosPlatformBundle” to your desktop (or other convenient location).
Log on to the Primary system’s Maintenance (:81/:444) Interface and click the “Upgrade” tab. Choose the upgrade files in turn, in the normal manner, starting with the centosPlatformBundle and then click the “[Upload]”.
The Platform bundle may request that you perform a system reboot (usually because the Kernel has been upgraded), so follow this instruction only if directed and wait for the system to reboot before logging on again. If it doesn’t suggest a reboot, you should move directly to upgrading the to the main Mutiny bundle. Choose the CentOSmutinyBundle from your downloaded files and click the “[Upload]” button as above.
Once the Primary has been upgraded and is polling normally again, you should repat the upgrade procedure, with both bundles, on the Secondary system.
Once both the Primary and the Secondary systems are upgraded and running, you will need to send the two Mutiny serial numbers to the Mutiny Support Team using our online form. The Mutiny Serial number can be found in the “Licence Settings” section of the “System Settings” page from the “[Admin]” tab.
You can send us both the serial numbers using the standard “Licence-Request Form” (https://www.mutiny.com/mutiny-support/licence-request/)
Enter both the serial numbers in the box marked “Serial number(s)”. For example:
Mutiny Licensing will then process your request against our customer database records and send you two new licences by email attachment. Once you have received these, each will need to be applied to correct appliance.
Applying for Mutiny Cold-Standby Licences
Locate the licence containing the string “ColdStandby=1”. This will go on the Primary system, that is your currently active Mutiny appliance. It is applied in the “Licence Settings” section of the “System Settings” page from the “[Admin]” tab. Delete the entire contents of the “Licence Key” box before pasting in the full contents of the new licence and click the “[Update]” button. Once the screen has refreshed, you should see a new input box appear at the top of the screen marked “Cold Standby Partner address”. Type the IP address of the other Mutiny appliance that you want to be the Cold-Standby Secondary system, into this box and again click “[Update]”.
Now locate the licence containing the string “ColdStandbySecondary=1”. This will go on the Secondary system. As above, it is applied in the “Licence Settings” section of the “System Settings” page from the “[Admin]” tab. Delete the entire contents of the “Licence Key” box before pasting in the full contents of the new licence and click the “[Update]” button. Once the screen has refreshed, you should see a new input box appear at the top of the screen marked “Cold Standby Partner address”. Type the IP address of the of the other Mutiny appliance, that is already running as the Cold-Standby Primary system, into this box and again click “[Update]”.
Both the appliances should now be operating as a Cold-Standby Pair and the backups on the Primary system will be synchronised onto the Secondary system every hour or so.
Monitoring the Cold-Standby Pair
We have developed an Agent that checks on the condition of our High-Availability configurations, including the Cold Standby and this should be added to the “mutiny” node (IP address 127.0.0.1) on the Primary system.
The “mutiny” node is most-easily located from the Node Manager page, sitting at the top of the network tree.
Add the “Mutiny High-Availability” Monitor” in the Agents section:
After 10 to 15 minutes, the two systems should synchronise:
Mutiny Software Bundle Upgrades
When Mutiny systems are configured for high availability, you only need to upgrade the Primary system as the Secondary will be automatically upgraded when the Primary has completed. Log on to the Primary systems Maintenance (:81/:444) Interface and choose the “Upgrade” tab. Choose the upgrade file in the normal manner and the click “[Upload]” button. Once the bundle has been uploaded and the upgrade started, you will see the following message at the top of the screen:
Adding New or Upgraded Licences
Once you are subscribed to a Mutiny Cold-Standby Licence, your new or upgraded licences will always be supplied in pairs – one for the Primary and one for the Secondary. Please follow the instructions in the previous section of this document - “Applying Mutiny Cold-Standby Licences”.
Normal Operation
During normal operation, there is no need to log on to the Secondary system as it will simply sit on your network collecting the latest automatic daily backups and manual snapshots from the Primary system. However, there are some tasks that will require you to connect to the front end of the Secondary system and log on. These are as follows:
• To add a new or updated Mutiny licence.
• To create a new local user or change the password of an existing user.
• To reassure yourself that the backups are being copied across.
• To turn the Secondary into the new Primary Mutiny system following failure.
When you connect to the Secondary frontend Webpage, you will see something similar to that below. To perform any of the above, you will need to login:
The Secondary system does NOT synchronise usernames/passwords with the Primary system, so these will remain as how you originally set them up, or default. The default access is username:
“superadmin@mutiny.com” and password: “password”.
Note that the Secondary system will not be polling any node – even itself, so if you see a message in the top right-hand corner similar to that below, don’t worry – this is normal:
All the operations that you are likely to need to perform can be found from the “[Admin]” tab as with any standard Mutiny system – see the Mutiny User Manual or Website for details.
Restoring your Mutiny System in the Event of Primary Failure
Should your Primary Mutiny system fail completely, make sure that is shut down or disconnected from the network. Connect to the Secondary system through the normal front end and login as a “Superadmin-Level” user.
Choose “Backup/Restore” from the “[Admin]” table and you will see a list of available Primary backups that you can restore from – normally you would choose the most recent. Click the “Restore” icon and confirm and after a few minutes, you will be disconnected as the Secondary system will be converted to a near-clone of the defunct Primary.
Please then contact the Mutiny Support Team for advice on the best way to deal with your failed Primary system and options for restoring normal service.