Using load balancing for highly available applications

Application architecture

The application includes the following Compute Engine components:

• VPC network: a virtual network within Google Cloud that can provide global connectivity using its own routes and firewall rules.
• Firewall rule: a Google Cloud firewall lets you allow or deny traffic to your instances.
• Instance template: a template used to create each VM instance in the managed instance group.
• Regional managed instance group: a group of VM instances running the same application across multiple zones.
• Global static external IP address: a static IP address that is accessible on external networks and can be attached to a global resource.
• Global load balancer: a load balancer that allows backend instances to be distributed across multiple regions. Use a global load balancer when your users need access to the same applications and content, and you want to provide access using a single anycast IP address.
• Health check: a policy used by the load balancer to evaluate the responsiveness of the application on each VM instance.

Launching the web application

This tutorial uses a web application that is stored on GitHub. If you would like learn more about how the application was implemented, see the GoogleCloudPlatform/python-docs-samples repository on GitHub.

Launch the web application on every VM in an instance group by including a startup script in an instance template. Additionally, run the instance group in a dedicated VPC network to keep this tutorial's firewall rules from interfering with any existing resources running in your project.

Create a VPC network

Using a VPC network protects existing resources in your project from being affected by the resources that you will create for this tutorial. A VPC network is also required to restrict incoming traffic so that it must go through the load balancer.

Create a VPC network to encapsulate the firewall rules for the demo web application:

1. In the Google Cloud console, go to the VPC networks page.

   Go to VPC networks

2. Click Create VPC Network.

3. Under Name, enter web-app-vpc.

4. Set Subnet creation mode to Custom.

5. Create a new subnet as follows:

   1. In the Subnets section, set the Name field, enter web-app-vpc-subnet.
   2. In the Region drop-down, select us-central1.
   3. Make sure that the IP stack type option is set to IPv4.
   4. In the Primary IPv4 range section, enter the IPv4 range 10.2.0.0/24.

6. At the bottom of the page, click Create.

Wait until the VPC network is created before continuing.

Create a firewall rule

After the VPC network is created, set up a firewall rule to allow HTTP traffic to the VPC network:

1. In the Google Cloud console, go to the Firewalls page.

   Go to Firewalls

2. Click Create firewall rule.

3. In the Name field, enter allow-web-app-http.

4. Set Network to web-app-vpc.

5. Make sure that the following options are set as given:

   • Direction of traffic option is set to Ingress.
   • Action on match option is set to Allow.

6. In the Targets drop-down, select All instances in the network.

7. Set Source filter to IPv4 ranges.

8. In the Source IP ranges field, enter 130.211.0.0/22, 35.191.0.0/16 to allow for load balancer health checks.

9. Under Protocols and ports, do the following:

   1. Select Specified protocols and ports.
   2. Select TCP.
   3. In the Ports field, enter 80 to allow access for HTTP traffic.

10. Click Create.

Create an instance template

Create a template that you will use to create a group of VM instances. Each instance created from the template launches a demo web application by using a startup script.

1. In the Google Cloud console, go to the Instance templates page.

   Go to Instance templates

2. Click Create instance template.

3. Under Name, enter load-balancing-web-app-template.

4. Under Machine configuration, set the Machine type to e2-medium.

5. Click the Advanced options section to expand.

6. Click the Networking section and do the following:

   1. In the Network interfaces section, delete any existing network interfaces by clicking the delete icon next to them.
   2. Click Add a network interface, and then select the web-app-vpc network. This forces each instance created with this template to run on the previously created network.
   3. In the Subnetwork drop-down, select web-app-vpc-subnet.
   4. Click Done.

7. Click the Management section and do the following:

   1. In the Automation section, enter the following startup script:

      apt-get update
      apt-get -y install git python3-pip python3-venv
      git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
      python3 -m venv venv
      ./venv/bin/pip3 install -Ur ./python-docs-samples/compute/managed-instances/demo/requirements.txt
      ./venv/bin/pip3 install gunicorn
      ./venv/bin/gunicorn --bind 0.0.0.0:80 app:app --daemon --chdir ./python-docs-samples/compute/managed-instances/demo
      

      The script gets, installs, and launches the web application when a VM instance starts up.

8. Leave the default values for the other options.

9. Click Create.

Wait until the template is created before continuing.

Create a regional managed instance group

To run the web application, use the instance template to create a regional managed instance group:

1. In the Google Cloud console, go to the Instance groups page.

   Go to Instance groups

2. Click Create instance group.

3. For Name, enter load-balancing-web-app-group.

4. For Instance template, select load-balancing-web-app-template.

5. Set Number of instances to 6. If this field is disabled, turn off autoscaling first.

   To turn off autoscaling, go to the Autoscaling section. In the Autoscaling mode drop-down, select Off: do not autoscale.

6. For Location, select Multiple zones.

7. For Region, select us-central1.

8. For Zones, select the following zones from the drop-down list:

   • us-central1-b
   • us-central1-c
   • us-central1-f

9. Leave the default values for the other options.

10. Click Create. This redirects you back to the Instance groups page.

    You might need to wait a few minutes until all of the instances in the group are running.

Configuring the load balancer

To use a load balancer to direct traffic to your web application, you must reserve an external IP address to receive all incoming traffic. Then, create a load balancer that accepts traffic from that IP address and redirects that traffic to the instance group.

Reserve a static IP address

Use a global static external IP address to provide the load balancer with a single point of entry for receiving all user traffic. Compute Engine preserves static IP addresses even if you change or delete any affiliated Google Cloud resources. This allows the web application to always have the same entry point, even if other parts of the web application might change.

1. In the Google Cloud console, go to the IP addresses page.

   Go to IP addresses

2. Click Reserve external static IP address.

3. In the Name field, enter web-app-ipv4.

4. Set IP version to IPv4.

5. Set Type to Global.

6. Click Reserve.

Create a load balancer

This section explains the steps required to create a global load balancer that directs HTTP traffic.

This load balancer uses a frontend to receive incoming traffic and a backend to distribute this traffic to healthy instances. Because the load balancer is made of multiple components, this task is divided into five parts:

• Select the load balancer type
• Name the load balancer
• Configure the frontend
• Configure the backend
• Review and finalize

Complete all the parts to create the load balancer.

Select the load balancer type

1. In the Google Cloud console, go to the Load balancing page.

   Go to Load balancing

2. Click Create load balancer.
3. For Type of load balancer, select Application Load Balancer (HTTP/HTTPS) and click Next.
4. For Public facing or internal, select Public facing (external) and click Next.
5. For Global or single region deployment, select Best for global workloads and click Next.
6. For Load balancer generation, select Global external Application Load Balancer and click Next.
7. Click Configure.

Name the load balancer

1. In the left panel, for Load balancer name, enter web-app-load-balancer.

Configure the frontend

1. On the Frontend configuration page, under Name, enter web-app-ipv4-frontend.
2. Set the Protocol to HTTP.
3. Set the IP version to IPv4.
4. Set the IP address to web-app-ipv4.
5. Set the Port to 80.
6. Click Done to create the frontend.

Configure the backend

1. In the left panel, click Backend configuration.
2. Click Backend services & backend buckets drop-down to open a menu, and then click Create a backend service.
3. In the new window, for the Name of the backend service, enter web-app-backend.
4. In the Backends section, do the following:
   1. Set Instance group to load-balancing-web-app-group.
   2. Set Port numbers to 80. This allows HTTP traffic between the load balancer and the instance group.
   3. Under Balancing mode, select Utilization.
   4. Click Done.

5. Create the health check for the backend of the load balancer as follows:

   1. Click the Health check drop-down, and then click Create a health check. A new window opens.
   2. In the new window under Name, enter web-app-load-balancer-check.
   3. Set the Protocol to HTTP.
   4. Under Port, enter 80.
   5. For this tutorial, set the Request path to /health, which is a path that the demo web application is set up to respond to.

   6. Set the following Health criteria:

      1. Set Check interval to 3 seconds. This defines the amount of time from the start of one probe to the start of the next one.
      2. Set Timeout to 3 seconds. This defines the amount of time that Google Cloud waits for a response to a probe. Its value must be less than or equal to the check interval.
      3. Set Healthy Threshold to 2 consecutive successes. This defines the number of sequential probes that must succeed in order for the instance to be considered healthy.
      4. Set Unhealthy Threshold to 2consecutive failures. This defines the number of sequential probes that must fail in order for the instance to be considered unhealthy.

   7. Click Create to create the health check.

6. Leave the default values for the other options.

7. Click Create to create the backend service.

Review and finalize

Verify your load balancing settings before creating the load balancer:

1. In the left panel of the Create global external Application Load Balancer page, click Review and finalize.

2. On the Review and finalize page, verify that Frontend uses an IP address with a Protocol of HTTP.

3. On the same page, verify the following Backend settings:

   • The Backend service is web-app-backend.
   • The Endpoint protocol is HTTP.
   • The Health check is web-app-load-balancer-check.
   • The Instance group is load-balancing-web-app-group.

4. Click Create to finish creating the load balancer.

You might need to wait a few minutes for the load balancer to finish being created.

Test the load Balancer

Verify that you can connect to the web application by using the load balancer as follows:

1. In the Google Cloud console, go to the Load balancing page.

   Go to Load balancing

2. In the Name column, click web-app-load-balancer to expand the load balancer you just created.

3. To connect to the web-app using the external static IP addresses, do the following:

   1. In the Frontend section, copy the IP address shown in the IP:Port column.

   2. Open a new browser tab and paste the IP address into the address bar. This should display the demo web application:

      

   Notice that, whenever you refresh the page, the load balancer connects to different instances in different zones. This happens because you are not connecting to an instance directly; you are connecting to the load balancer, which selects the instance you are redirected to.

   When you are done, close the browser tab for the demo web application.

Simulating a zonal outage

You can observe the functionality of the load balancer by simulating the widespread unavailability of a zonal outage. This simulation works by forcing all of the instances located in a specified zone to report an unhealthy status on the /health request path. When these instances report an unhealthy status, they fail the load balancing health check, prompting the load balancer to stop directing traffic to these instances.

1. Monitor which zones the load balancer is directing traffic to.

   1. In the Google Cloud console, go to Cloud Shell.

      Open Cloud Shell

      Cloud Shell opens in a pane of the Google Cloud console. It can take a few seconds for the session to initialize.

   2. Save the static external IP address of your load balancer as follows:

      1. Get the external IP address from the frontend forwarding rule of the load balancer by entering the following command in your terminal:

         gcloud compute forwarding-rules describe web-app-ipv4-frontend --global
         

         The output looks as follows. Copy the EXTERNAl_IP_ADDRESS from the output.

         IPAddress: EXTERNAl_IP_ADDRESS
         ...
         

      2. Create a local bash variable:

         export LOAD_BALANCER_IP=EXTERNAl_IP_ADDRESS
         

         Replace EXTERNAl_IP_ADDRESS with the external IP address that you copied.

   3. To monitor which zones the load balancer is directing traffic to, run the following bash script:

      while true
      do
          BODY=$(curl -s "$LOAD_BALANCER_IP")
          NAME=$(echo -n "$BODY" | grep "load-balancing-web-app-group" | perl -pe 's/.+?load-balancing-web-app-group-(.+?)<.+/\1/')
          ZONE=$(echo -n "$BODY" | grep "us-" | perl -pe 's/.+?(us-.+?)<.+/\1/')
          echo $ZONE
          sleep 2 # Wait for 2 seconds
      done
      

      This script continuously attempts to connect to the web application by using the IP address for the frontend of the load balancer, and outputs which zone the web application is running from for each connection.

      The resulting output should include zones us-central1-b, us-central1-c, and us-central1-f:

      us-central1-f
      us-central1-b
      us-central1-c
      us-central1-f
      us-central1-f
      us-central1-c
      us-central1-f
      us-central1-c
      us-central1-c
      

      Keep this terminal open.

2. While your monitor is running, begin simulating the zonal outage.

   1. In Cloud Shell, open a second terminal session by clicking the add Add button.

   2. Create a local bash variable for the project ID:

      export PROJECT_ID=PROJECT_ID
      

      where PROJECT_ID is the project ID for your current project, which is displayed on each new line in the Cloud Shell:

      user@cloudshell:~ (PROJECT_ID)$
      

   3. Create a local bash variable for the zone that you want to disable. To simulate a failure of zone us-central1-f, use the following command:

      export DISABLE_ZONE=us-central1-f
      

      Then, run the following bash script. This script causes the demo web application instances in the disabled zone to output unhealthy responses to the load balancer health check. Unhealthy responses prompt the load balancer to direct traffic away from these instances.

      export MACHINES=$(gcloud --project=$PROJECT_ID compute instances list --filter="zone:($DISABLE_ZONE)" --format="csv(name,networkInterfaces[0].accessConfigs[0].natIP)" | grep "load-balancing-web-app-group")
      for i in $MACHINES;
      do
        NAME=$(echo "$i" | cut -f1 -d,)
        IP=$(echo "$i" | cut -f2 -d,)
        echo "Simulating zonal failure for zone $DISABLE_ZONE, instance $NAME"
        curl -q -s "https://$IP/makeUnhealthy" >/dev/null --retry 2
      done
      

      After a short delay, the load balancer stops directing traffic to the unhealthy zones, so the output from the first terminal window stops listing zone us-central1-f:

      us-central1-c
      us-central1-c
      us-central1-c
      us-central1-b
      us-central1-b
      us-central1-c
      us-central1-b
      us-central1-c
      us-central1-c
      

      This indicates that the load balancer is directing traffic only to the healthy, responsive instances.

      Keep both terminals open.

   4. In the second terminal, create a local bash variable for the zone that you want to restore. To restore traffic to zone us-central1-f, use the following command:

      export ENABLE_ZONE=us-central1-f
      

      Then, run the following bash script. This script causes the demo web application instances in the enabled zone to output healthy responses to the load balancer health check. Healthy responses prompt the load balancer to begin distributing traffic back toward these instances.

      export MACHINES=$(gcloud --project=$PROJECT_ID compute instances list --filter="zone:($ENABLE_ZONE)" --format="csv(name,networkInterfaces[0].accessConfigs[0].natIP)" | grep "load-balancing-web-app-group")
      for i in $MACHINES;
      do
        NAME=$(echo "$i" | cut -f1 -d,)
        IP=$(echo "$i" | cut -f2 -d,)
        echo "Simulating zonal restoration for zone $ENABLE_ZONE, instance $NAME"
        curl -q -s "https://$IP/makeHealthy" >/dev/null --retry 2
      done
      

      After a few minutes, the output from the first terminal window gradually lists zone us-central1-fagain:

      us-central1-b
      us-central1-b
      us-central1-c
      us-central1-f
      us-central1-c
      us-central1-c
      us-central1-b
      us-central1-c
      us-central1-f
      

      This indicates that the load balancer is directing incoming traffic to all zones again.

      Close both terminals when you have finished.