Introducing Distributed Load Testing Using locust on Azure Kubernetes Service ( AKS )

Updated: Now, Azure Load Testing has supported Locust script😄

https://learn.microsoft.com/en-us/azure/load-testing/quickstart-create-run-load-test-with-locust?tabs=portal

Updated: Now, Azure has released Azure Load Testing. https://docs.microsoft.com/en-us/azure/load-testing/

— — —

Today I’m open-sourcing distributed-load-testing-using-locust-on-aks, making an easy to scalable distributed load testing using Azure Kubernetes Service (AKS) and locust.

koudaiii/distributed-load-testing-using-locust-on-aks

If you’ve used a performance tools, then you are probably familiar with the concept of Distributed Load Testing. Distributed Load Testing lets you check your application for High Availability and Scalability, and Resiliency, and Reliability.

However, Azure currently does not offer a managed service for load test, so you need to prepare a performance tool on Azure. If you want to check performance, you can quickly get a distributed load testing by using distributed-load-testing-using-locust-on-aks.

Table of Contents

1. What does distributed-load-testing-using-locust-on-aks solve?
2. What does distributed-load-testing-using-locust-on-aks do?
3. Getting started with distributed-load-testing-using-locust-on-aks
4. How does distributed-load-testing-using-locust-on-aks work?
5. Conclusion

1. What does distributed-load-testing-using-locust-on-aks solve?

Two prominent issues led me to develop distributed-load-testing-using-locust-on-aks:

1 Azure currently does not have a managed service for load testing. So you need to prepare a tool for load testing yourself.

I want to select a simple tool and also can run on my laptop pc.

2 Even if you prepared a performance tool, your performance tool might have a bottleneck. So your application might not be a correct result of performance. Currently, Web Applications often have auto-scale.

You need to solve the bottleneck of this tool. Then, performance tests can confirm where broken in your application. And, your applications can demonstrate degenerate driving methods appropriate. And the application can establish scaling at the right time.

2. What does distributed-load-testing-using-locust-on-aks do?

It precisely solves the two problems listed above:

1. distributed-load-testing-using-locust-on-aks solves a complex operation using locust on Azure Kubernetes Service (AKS):
2. distributed-load-testing-using-locust-on-aks builds a scalable environment for your workload using locust on Azure Kubernetes Service (AKS) :

3. Getting started with distributed-load-testing-using-locust-on-aks

distributed-load-testing-using-locust-on-aks has the bootstrap script. So you can run bootstrap script.

$ script/bootstrap

If you have Azure Kubernetes Service (AKS) environment, you might apply kubernetes/ manifest files to your Azure Kubernetes Service (AKS).

you need run kubectl port-forwardcommand because you can access WebUI.

$ kubectl port-forward svc/locust-master 8089:8089 -n locust
(Command+t)
$ open http://localhost:8089
# Set your target URL

Please check out distributed-load-testing-using-locust-on-aks and consider trying out the project and giving feedback.

I did a load test on a sample application in a repository called myapp, assuming a scenario that I experienced in the past. This scenario is based on a spike that occurred when the application was introduced on TV.

Statistics

Request Transition Graph

I checked myapp can scalable application because of application can scale until request peak.

$ kubectl get hpa -A -w
NAMESPACE   NAME         REFERENCE                  TARGETS    MINPODS   MAXPODS   REPLICAS   AGE
locust      locust-hpa   Deployment/locust-worker   100%/40%   2         100       5          5m15s
myapp       myapp-hpa    Deployment/myapp           29%/40%    1         100       1          2m31s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       1          2m47s
locust      locust-hpa   Deployment/locust-worker   99%/40%    2         100       5          5m32s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       2          3m2s
locust      locust-hpa   Deployment/locust-worker   99%/40%    2         100       10         5m47s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       2          3m18s
myapp       myapp-hpa    Deployment/myapp           73%/40%    1         100       2          3m48s
locust      locust-hpa   Deployment/locust-worker   95%/40%    2         100       10         6m32s
locust      locust-hpa   Deployment/locust-worker   95%/40%    2         100       20         6m47s
locust      locust-hpa   Deployment/locust-worker   95%/40%    2         100       24         7m3s
locust      locust-hpa   Deployment/locust-worker   76%/40%    2         100       24         7m33s
myapp       myapp-hpa    Deployment/myapp           89%/40%    1         100       2          4m49s
locust      locust-hpa   Deployment/locust-worker   76%/40%    2         100       41         7m47s
myapp       myapp-hpa    Deployment/myapp           89%/40%    1         100       4          5m3s
myapp       myapp-hpa    Deployment/myapp           89%/40%    1         100       5          5m19s
locust      locust-hpa   Deployment/locust-worker   59%/40%    2         100       41         8m33s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          5m49s
locust      locust-hpa   Deployment/locust-worker   59%/40%    2         100       47         8m48s
locust      locust-hpa   Deployment/locust-worker   59%/40%    2         100       47         9m34s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          6m50s
locust      locust-hpa   Deployment/locust-worker   59%/40%    2         100       47         10m
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          7m51s
locust      locust-hpa   Deployment/locust-worker   59%/40%    2         100       47         11m
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          8m52s
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          9m53s
locust      locust-hpa   Deployment/locust-worker   60%/40%    2         100       47         13m
myapp       myapp-hpa    Deployment/myapp           59%/40%    1         100       5          10m

As you can see, it scaled from 2 locust workers to a maximum of 47. Also, one of myapp’s responses went down due to the sudden load, but myapp could scale up to 5 workers before the peak.

When the performance test is done, let’s clean up.

$ script/clean

4. How does distributed-load-testing-using-locust-on-aks work?

distributed-load-testing-using-locust-on-aks builds locust on Azure Kubernetes Service (AKS). locust on Azure Kubernetes Service (AKS) has master and some workers. In addition, locust on Azure Kubernetes Service (AKS) has included hpa(horizontal pod autoscaler) setting for the worker. Therefore, distributed-load-testing-using-locust-on-aks realizes distributed load testing depending on the load.

Why use Kubernetes?

To increase the scalability of the load tool, I used hpa of kubernetes because manual scale is very difficult. Nowadays, web applications support scalability as a matter of course. I solved a bottleneck in my load tool this way.

Why use Azure Kubernetes Service (AKS)?

• If you have an account in Azure, you can easily create and delete the Kubernetes infrastructure using az commands.
• If you have a Web Application on Azure, you can performance test in Azure.

5. Conclusion

Using distributed-load-testing-using-locust-on-aks, you can easily use distributed load testing on Azure and get the experience that you’re used to in terms of end-to-end performance.

I hope this article explains why and how I built distributed-load-testing-using-locust-on-aks. Please make sure to check out the GitHub repository, star/fork it, and give feedback through the issue tracker or on Twitter(@koudaiii).