Upgrading Stratus Cloud Availability Services

This topic describes how to upgrade Stratus Cloud Availability Services on your KVM compute nodes. You may need to upgrade Availability Services to install updates specific to the KVM-FT hypervisor or to install new features in conjunction with a new Stratus Cloud Workload Services release.

Upgrade Availability Services on one compute node at a time and in one FT pair group at a time. For example, shut down each KVM-FT instance running on the first compute node and then install the new software release. Start each of the KVM-FT instances on the first node, one instance at a time, and verify that they return to the Running/Paired state. When the first compute node and all of its instances are up and running on the upgraded software, repeat the process on the second node of the FT pair group. In each case, the paired KVM-FT instances on the running compute node keep your applications running during the upgrade.

After upgrading Availability Services, you may also need to upgrade the quorum server service (QSS) on your quorum servers. For more information, see Upgrading Quorum Servers for Stratus Cloud Availability Services.

To upgrade Availability Services on KVM compute nodes:

1. In the main menu of Stratus Cloud Workload Services, click Hypervisors.
2. On the Hypervisors page, locate the KVM-FT hypervisors (compute nodes). Decide which hypervisor to upgrade first and record its name. Upgrade only one hypervisor at a time, in the same FT pair group.

3. Click the hypervisor that you want to upgrade and click to view its details page. On the details page:

   • Under Deployed Instances, record the names of the KVM-FT instances that are running on the hypervisor that you selected for upgrade.
   • Under KVM-AX Version, note the current version number of Availability Services.
   Note: The version number is displayed only if KVM-FT instances are deployed and running. Version information is also available on each KVM-FT compute node in the /etc/nova/kvmax.conf file.)

   When you are finished collecting information about the KVM-FT hypervisor, close the details page.

4. In the main menu, click Deployed Applications.
5. On the Deployed Applications page, next to Group by, click Availability and then click Mission Critical to expand the category. The Mission Critical category typically contains most or all of your KVM-FT instances, but you may need to check other categories as well.

6. Click each application to expand it, and locate the KVM-FT instances that are running on the hypervisor that you want to upgrade. For each application associated with these instances, ensure that one of the instances is in the Running state and that the second instance is in the Paired state, which indicates that the pair is running normally in fault-tolerant mode.

   Caution: If any of the instances have advisories or are in a state other than Paired/Running, correct any problems before upgrading the hypervisor.

7. If all KVM-FT instances are paired, carefully locate and shut down only the instances on the hypervisor that you want to upgrade. Click the instance, click , and click to stop the instance.

   As long as the application is currently fault-tolerant, and you stop only the instance on the hypervisor you want to upgrade, it does not matter whether that particular instance is in the Running or Paired state. The other instance keeps the application running.

8. Monitor the instances as they go from the Stopped to Shutoff state. Ensure that all of the instances are in the Shutoff state before you continue. You can monitor the transitions on the Deployed Applications page and also on the Advisories tab of the Dashboard page.
9. When all of the instances on the hypervisor that you want to upgrade are in the Shutoff state, log on to the console of the compute node as the root user (or be prepared to use sudo to run commands as root).

10. If you have not already done so, install the yum-utils package for access to additional yum commands:

    # yum install yum-utils

11. Verify that there are no unfinished yum transactions by entering the following yum command:

    # yum-complete-transaction --cleanup-only
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: mirror.sanctuaryhost.com
    * epel: mirror.cogentco.com
    * extras: centos.mirror.nac.net
    * updates: mirror.wiredtree.com
    No unfinished transactions left.

    If there are unfinished transactions, resolve them before continuing with the upgrade.

12. Download the new Availability Services installation script from the Stratus Cloud Solution Downloads and Support page at http://www.stratus.com/services-support/downloads to a local management PC.
13. Transfer the installation script to the /opt/Release directory. For example, use a secure copy (SCP) utility to copy the file from the local management PC to the compute node.

14. On the compute node, switch to the /opt/Release directory:

    # cd /opt/Release

15. Run the following command to make the installation script an executable file, where script is the name of the script:

    # chmod a+x script.sh

16. Run the installation script in the following format, where the axlinkethdev1 and axlinkethdev2 arguments represent the Ethernet devices to which the KVM-FT A-link cables are connected and detectedLink represents the network (adapter, virtual LAN, or channel bond) that handles data traffic for the instances:

    # ./kvm-ax-n.n.n.n.n install axlinkethdev1 axlinketdev2 detectedLink

    The detectedLink network is monitored for link up/down state transitions. A link down state indicates that KVM-FT instances may have lost network connectivity for this hypervisor. The KVM-FT software uses the state transitions to determine possible actions.

17. The installation script begins to upgrade the Availability Services software. When prompted, restart the KVM-FT node to apply the changes.

18. After restarting the node, log on and run the kvmax-healthcheck script to confirm that the upgrade was successful. Verify the newly installed version numbers and examine each section of the output for any failures. If you need assistance correcting any problems, contact your service representative.

    Run the script as follows:

    # /opt/stratus/scripts/kvmax-healthcheck.sh
    - -  - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - -
    KVM-AX Installer Verification Output
    KVM-AX Version : n.n.n.n.n
    FTCore Version : n.n.n.n-n.n
    - -  - - - - - - - - - - - - - - - - - - - - - - - - - - -- - - - - -
    .
    .
    .
    

19. In the main menu of Workload Services, click Deployed Applications.
20. On the Deployed Applications page, next to Group by, click Availability and then click Mission Critical to expand the category.
21. Locate the KVM-FT instances that are on the hypervisor that you just upgraded.

22. Start only one KVM-FT instance and monitor its progress on the Deployed Applications page and on the Dashboard page as it transitions through various states. It is normal to see critical (red) and warning (orange) states on the Dashboard page as startup and sync proceeds.

23. When the Dashboard status reaches the normal (green) state and advisories are resolved, go back to the Deployed Applications page. For the KVM-FT instance that you started, ensure that the paired instances in the application have returned to the Running and Paired states.
24. Start the additional KVM-FT instances on the hypervisor that you upgraded one at a time in the same manner.
25. After starting the KVM-FT instances, verify that the new Availability Services version is recognized by Workload Services. On the Hypervisors page, click the upgraded hypervisor, click to view the details page, and verify the version number under KVM-AX Version. (The version is displayed only if KVM-FT instances are deployed and running.)
26. Repeat steps 1-25 for the second hypervisor in the FT pair group.
27. Repeat steps 1-26 for additional FT pair groups.
28. If needed, upgrade the quorum service on your quorum servers.See Upgrading Quorum Servers for Stratus Cloud Availability Services.