Developer Setup: Multiconnect on Minikube

This guide will walk you through how to set up the WhatsApp Business API client locally on Minikube.

This document covers:

Prerequisites

  1. Set up a Kubernetes cluster locally using minikube.
  2. Install the kubectl command line tool.
  3. It's recommended you set up a test account locally for fast development and testing new releases.
  4. This example runs the WhatsApp Business API in High Availability mode. We highly recommended you read through the Availability and Scaling guide to learn more about High Availability and Multiconnect.

Setting up a High Availability cluster

  1. Create a ~/biz directory to store the setup scripts:
    mkdir ~/biz; cd ~/biz;
    
  2. Clone all the configuration files from the installation/kubernetes directory of the WhatsApp-Business-API-Setup-Scripts Github repo to ~/biz.
  3. Create a MySQL database with a persistent volume by running the following command:
    kubectl apply -f mysql.yaml
    
    Expected output:
    persistentvolume/mysql-volume created
    persistentvolumeclaim/mysql-volume-claim created
    service/mysql-service created
    deployment.extensions/mysql-deployment created
    
  4. Verify MySQL is now running with by running the following command:
    kubectl get pods
    
    Example output:
    NAME                               READY   STATUS    RESTARTS   AGE
    mysql-deployment-5d4f898-xtkpj     1/1     Running   0          53m
    
  5. Create a whatsapp-config secret from db.env by running the following command:
    kubectl create secret generic whatsapp-config --from-env-file=db.env
    
    Expected output:
    secret/whatsapp-config created
    
    You can check the secret was created by running the following command:
    kubectl get secrets
    
    Expected output:
    NAME                TYPE        DATA        AGE
    whatsapp-config     Opaque      5           52s
    
    Check the details of the secret by running the following command:
    kubectl describe secrets/whatsapp-config
    
    Expected output:
    Name:         whatsapp-config
    Namespace:    default
    Labels:       <none>
    Annotations:  <none>
    
    Type:  Opaque
    
    Data
    ====
    wa-db-password:  8 bytes
    wa-db-port:      4 bytes
    wa-db-username:  4 bytes
    wa-db-engine:    5 bytes
    wa-db-hostname:  13 bytes
    
  6. To send or receive media messages, the WhatsApp Business API client requires a persistent volume shared between the Webapp, Master and Coreapp deployments. Create a PersistentVolume and PersistentVolumeClaim for media messages by running the following command:
    kubectl apply -f volume.yaml
    
    This command will set up a local media volume mounted at /usr/local/wamedia inside the Kubernetes cluster with 1GB size and request 1GB physical size from it through a PersistentVolumeClaim for development and testing purposes. You can check the status of media volume and media volume claim by running the following command:
    kubectl get pv media-volume
    
    kubectl get pvc media-volume-claim
    
  7. Replace the $VERSION environment variable with the latest WhatsApp Business API client version (e.g., 2.23.4) in the containers section of the webapp.yaml, master.yaml, and coreapp.yaml files.
    webapp.yaml
    containers:
    - name: whatsapp-web
      image: docker.whatsapp.biz/web:v2.23.4
    master.yaml
    containers:
    - name: whatsapp-master
      image: docker.whatsapp.biz/coreapp:v2.23.4
    coreapp.yaml
    containers:
    - name: whatsapp-coreapp
      image: docker.whatsapp.biz/coreapp:v2.23.4
  8. Deploy the Webapp, Master, and Coreapp by running the following commands:
    To deploy the Webapp:
    kubectl apply -f webapp.yaml
    
    Expected output:
    horizontalpodautoscaler.autoscaling/whatsapp-web-autoscaler created
    service/whatsapp-web-service created
    deployment.apps/whatsapp-web-deployment created
    
    To deploy the Master:
    kubectl apply -f master.yaml
    
    Expected output:
    horizontalpodautoscaler.autoscaling/whatsapp-master-autoscaler created
    service/whatsapp-master-service created
    deployment.apps/whatsapp-master-deployment created
    
    To deploy the Coreapp:
    kubectl apply -f coreapp.yaml
    
    Expected output:
    horizontalpodautoscaler.autoscaling/whatsapp-coreapp-autoscaler created
    service/whatsapp-coreapp-service created
    deployment.apps/whatsapp-coreapp-deployment created
    
    You can check if the Master/Coreapp/Webapp pod is running with:
    kubectl get pods
    
    Expected output
    NAME                                           READY   STATUS    RESTARTS   AGE
    mysql-deployment-5d4f898-xtkpj                 1/1     Running   0          53m
    whatsapp-coreapp-deployment-78c4d987b8-4cpz4   1/1     Running   0          2s
    whatsapp-coreapp-deployment-78c4d987b8-l5qjj   1/1     Running   0          2s
    whatsapp-master-deployment-8598d7bf6b-56fvn    1/1     Running   7          15m
    whatsapp-master-deployment-8598d7bf6b-9r6lc    1/1     Running   7          16m
    whatsapp-web-deployment-cd4c5785c-9vn6l        1/1     Running   0          16m
    whatsapp-web-deployment-cd4c5785c-mn7kf        1/1     Running   0          16m
    
    Default configurations will create 2 Master pods, 2 Coreapp pods and 2 Webapp pods. Each pod requests 128MB memory and 0.15 CPUs. Change spec.template.spec.containers.resources in the respective YAML files if you want different resource settings.
  9. You can download and configure our Postman collection to easily interact with the WhatsApp Business API. When you run a Kubernetes cluster using Minikube, you need to check the Kubernetes cluster IP by running:
    minikube ip
    
    Find which port the Webapp runs on using:
    kubectl get services/whatsapp-web-service
    
    Expected output:
    NAME                   TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
    whatsapp-web-service   NodePort   10.101.114.46   <none>        443:32477/TCP   25m
    
    The above result means that port 443 inside the Webapp container is mapped to the port 32477 on the Kubernetes cluster.
    You need to use https://{{your-minikube-cluster-ip}}:{{your-webapp-service-targetport}} as the API root URL when using the Postman collection.
  10. Log in and obtain an authentication token.
  11. Perform a health check to verify all pods are running properly. Example response:
    'health': {
        '172.17.0.11:whatsapp-coreapp-deployment-78c4d987b8-l5qjj': {
            'errors': [
                {
                    'code': 1011,
                    'title': 'Service not ready',
                    'details': 'Wacore is not instantiated. Please check wacore log for details.'
                }
            ]
        },
        '172.17.0.6:whatsapp-master-deployment-8598d7bf6b-56fvn': {
            'errors': [
                {
                    'code': 1011,
                    'title': 'Service not ready',
                    'details': 'Wacore is not instantiated. Please check wacore log for details.'
                }
            ]
        },
        '172.17.0.7:whatsapp-coreapp-deployment-78c4d987b8-4cpz4': {
            'errors': [
                {
                    'code': 1011,
                    'title': 'Service not ready',
                    'details': 'Wacore is not instantiated. Please check wacore log for details.'
                }
            ]
        },
        '172.17.0.8:whatsapp-master-deployment-8598d7bf6b-9r6lc': {
            'gateway_status': 'unregistered',
            'role': 'primary_master'
        }
    },
    'meta': {
        'version': 'v2.23.4',
        'api_status': 'stable'
    }
    
    Because your WhatsApp Business API client is not yet registered, you will see 'gateway_status': 'unregistered' for your primary_master in the health status response.
  12. Follow the Registration documentation to register the WhatsApp Business API client.
  13. Perform another health check after registration to verify one of the Coreapps now has a connected status. Example response:
    'health': {
        '172.17.0.11:whatsapp-coreapp-deployment-78c4d987b8-l5qjj': {
            'gateway_status': 'disconnected',
            'role': 'coreapp'
        },
        '172.17.0.6:whatsapp-master-deployment-8598d7bf6b-56fvn': {
            'gateway_status': 'disconnected',
            'role': 'secondary_master'
        },
        '172.17.0.7:whatsapp-coreapp-deployment-78c4d987b8-4cpz4': {
            'gateway_status': 'connected',
            'role': 'coreapp'
        },
        '172.17.0.8:whatsapp-master-deployment-8598d7bf6b-9r6lc': {
            'gateway_status': 'disconnected',
            'role': 'primary_master'
        }
    },
    'meta': {
        'version': 'v2.23.4',
        'api_status': 'stable'
    }
    
    Note: In High Availability mode, only one Coreapp (whatsapp-coreapp-deployment-78c4d987b8-4cpz4 in this example) will be connected to the Whatsapp server; all other pods including the primary Master will have a gateway_status of disconnected. If whatsapp-coreapp-deployment-78c4d987b8-4cpz4 goes down, whatsapp-coreapp-deployment-78c4d987b8-l5qjj will replace it and connect to the Whatsapp server to maintain High Availability.

You have now set up the WhatsApp Business API in High Availability mode. In this mode, only one Coreapp is able to connect to the WhatsApp server to send messages at any given time. If you want to have multiple Coreapps sending messages at the same time to increase message throughput, follow the steps in the Setting up a highly available Multiconnect cluster section below.

Setting up a highly available Multiconnect cluster

  1. Use the shards endpoint to set up 2 shards; you should see an HTTP response with status 201 Created.
  2. Perform a health check to verify all pods are running properly. Example response:
    'health': {
        '172.17.0.11:whatsapp-coreapp-deployment-78c4d987b8-l5qjj': {
            'gateway_status': 'connected',
            'role': 'coreapp'
        },
        '172.17.0.6:whatsapp-master-deployment-8598d7bf6b-56fvn': {
            'gateway_status': 'disconnected',
            'role': 'secondary_master'
        },
        '172.17.0.7:whatsapp-coreapp-deployment-78c4d987b8-4cpz4': {
            'gateway_status': 'connected',
            'role': 'coreapp'
        },
        '172.17.0.8:whatsapp-master-deployment-8598d7bf6b-9r6lc': {
            'gateway_status': 'connected',
            'role': 'primary_master'
        }
    },
    'meta': {
        'version': 'v2.23.4',
        'api_status': 'stable'
    }
    
    Note: In MultiConnect mode with 2 shards, 2 Coreapps (whatsapp-coreapp-deployment-78c4d987b8-l5qjj and whatsapp-coreapp-deployment-78c4d987b8-4cpz4 in this example) will be connected to the WhatsApp server, and the primary Master (whatsapp-master-deployment-8598d7bf6b-9r6lc in this example) will also connect to the WhatsApp server.

So far in this example, you have 2 Coreapp containers and message loads are split between them. However, if one of the Coreapp containers goes down, half of message sends will fail. In order to maintain High Availability in this new Multiconnect setup, you can start a third Coreapp to tolerate 1 Coreapp failure, which is similar to the diagram shown in the Multiconnect Introduction.

  1. To start the third Coreapp container, change the number of replicas to 3 in the coreapp.yaml file:
    spec:
      replicas: 3
    
  2. Redeploy the Coreapp by running:
    kubectl apply -f coreapp.yaml
    
    Expected output:
    horizontalpodautoscaler.autoscaling/whatsapp-coreapp-autoscaler unchanged
    service/whatsapp-coreapp-service unchanged
    deployment.apps/whatsapp-coreapp-deployment configured
    
  3. Perform a health check to verify all pods are running properly. Example response:
    'health': {
        '172.17.0.10:whatsapp-coreapp-deployment-78c4d987b8-hwqp6': {
            'gateway_status': 'disconnected',
            'role': 'coreapp'
        },
        '172.17.0.11:whatsapp-coreapp-deployment-78c4d987b8-l5qjj': {
            'gateway_status': 'connected',
            'role': 'coreapp'
        },
        '172.17.0.6:whatsapp-master-deployment-8598d7bf6b-56fvn': {
            'gateway_status': 'disconnected',
            'role': 'secondary_master'
        },
        '172.17.0.7:whatsapp-coreapp-deployment-78c4d987b8-4cpz4': {
            'gateway_status': 'connected',
            'role': 'coreapp'
        },
        '172.17.0.8:whatsapp-master-deployment-8598d7bf6b-9r6lc': {
            'gateway_status': 'connected',
            'role': 'primary_master'
        }
    },
    'meta': {
        'version': 'v2.23.4',
        'api_status': 'stable'
    }
    
    The new whatsapp-coreapp-deployment-78c4d987b8-hwqp6 container now acts as a standby container and is not currently connected to the WhatsApp server. If either whatsapp-coreapp-deployment-78c4d987b8-l5qjj or whatsapp-coreapp-deployment-78c4d987b8-4cpz4 stops working, whatsapp-coreapp-deployment-78c4d987b8-hwqp6 will connect to the WhatsApp server to maintain an overall shard count of 2.

Upgrading

It is highly recommended you backup your current application settings before upgrading. See the Backup and Restore documentation for more information.

  1. Replace the $VERSION environment variable with the version you would like to upgrade to (e.g., 2.23.5) in the containers section of the webapp.yaml, master.yaml, and coreapp.yaml files:
    webapp.yaml
    containers:
    - name: whatsapp-web
      image: docker.whatsapp.biz/web:v2.23.5
    master.yaml
    containers:
    - name: whatsapp-master
      image: docker.whatsapp.biz/coreapp:v2.23.5
    coreapp.yaml
    containers:
    - name: whatsapp-coreapp
      image: docker.whatsapp.biz/coreapp:v2.23.5
  2. Upgrade the Webapp, Master, and Coreapp, and monitor rollout status.
    To upgrade the Webapp:
    kubectl apply -f webapp.yaml
    
    Expected output:
    kubectl rollout status deployments/whatsapp-web-deployment
    
    To upgrade the Master:
    kubectl apply -f master.yaml
    
    Expected output:
    kubectl rollout status deployments/whatsapp-master-deployment
    
    To upgrade the Coreapp:
    kubectl apply -f coreapp.yaml
    
    Expected output:
    kubectl rollout status deployments/whatsapp-coreapp-deployment
    

Uninstalling

  1. Delete the deployments:
    kubectl delete -f webapp.yaml
    kubectl delete -f master.yaml
    kubectl delete -f coreapp.yaml
    kubectl delete -f mysql.yaml
    
  2. Delete the volume:
    kubectl delete -f volume.yaml
    
  3. Delete the secret:
    kubectl delete secrets/whatsapp-config
    

Troubleshooting

Check if all pods are running:

kubectl get pods

Example output:

NAME                                           READY   STATUS    RESTARTS   AGE
mysql-deployment-5d4f898-xtkpj                 1/1     Running   0          1h
whatsapp-coreapp-deployment-78c4d987b8-4cpz4   1/1     Running   2          1h
whatsapp-coreapp-deployment-78c4d987b8-hwqp6   1/1     Running   0          24m
whatsapp-coreapp-deployment-78c4d987b8-l5qjj   1/1     Running   2          1h
whatsapp-master-deployment-8598d7bf6b-56fvn    1/1     Running   9          1h
whatsapp-master-deployment-8598d7bf6b-9r6lc    1/1     Running   8          1h
whatsapp-web-deployment-cd4c5785c-f99n4        1/1     Running   0          50m
whatsapp-web-deployment-cd4c5785c-s5phx        1/1     Running   0          50m

If any pod is not ready (i.e., shows 0/1 in the READY column), you can get logs for the particular pod by running the example command:

kubectl logs whatsapp-coreapp-deployment-78c4d987b8-4cpz4 > whatsapp-coreapp-deployment-78c4d987b8-4cpz4.txt

You can find the logs in the whatsapp-coreapp-deployment-78c4d987b8-4cpz4.txt file in the current directory.

To collect logs for a particular deployed service, such as the Webapp, run:

kubectl logs deployments/whatsapp-web-deployment > whatsapp-web-deployment.txt

You can find the logs in the whatsapp-web-deployment.txt file in the current directory.


This software uses code of FFmpeg licensed under the LGPLv2.1 and its source can be downloaded here.