🌶️ 👈🏻 🌥️ Query Counting: Basic Django Performance Testing 👕 🦗 🚨

Hello everyone. We have prepared a translation of another useful material for students of the course "Web-developer in Python" , which started yesterday.

You can often hear about testing methods such as TDD, and how to test the business logic of an application. However, testing application performance is a completely different task. There are many different ways, but the most common approach is to create an environment in which you can conduct a DDoS attack on your application and observe its behavior. This is a very interesting topic, but this is not what I want to talk about today. Today we will look at a simpler test, one that you can do using the default Django unit tests: that is, testing the number of times your application accesses the database.

Testing this is very simple, and this is exactly the aspect that can hurt application performance in the early stages. This aspect is the very first to be tested when something starts to work slowly. The good news is that there is only one thing you need to know to write tests of this kind: the assertNumQueries method , and it is pretty easy to use. Here is an example:

from django.test import TestCase, Client
from django.urls import reverse
from trucks.models import Truck

class TrucksTestCase(TestCase):
    def test_list_trucks_view_performance(self):
        client = Client()

        Truck.objects.create(...)

        with self.assertNumQueries(6):
            response = client.get(reverse("trucks:list_trucks"))

        self.assertEqual(response.context["trucks_list"], 1)

The above code claims that during the view, the "trucks:list_trucks"application will access the database only 6 times. But there is something else, note that before starting, we first create a new object Truck, and after that we say that trucks_listthere is at least one object in the context data of the view . In this kind of tests, this is important because you need a guarantee that you are not testing on an empty data set. It’s important to understand that just instantiating a class is Trucknot enough. You need to check if it has been included in the context. Perhaps you are filtering the list of trucks, so it is likely that your instance Truckwill not be included in the result.

Having done all of the above, we have already made significant progress, but there is another important step that is easy to forget about. If we want our views to scale, we must ensure that performance does not decrease as the number of returned items grows. In the end, we still have a performance problem if we turn to the database not 6 times to get one item, but 106 if we have 100 items. We need a constant number of database calls, which will not depend on the number of returned items. Fortunately, this problem is also solved very simply, we need to add one more (or several) elements to the database and again count the number of hits. This is how the test will look in the final version:

from django.test import TestCase, Client
from django.urls import reverse
from trucks.models import Truck

class TrucksTestCase(TestCase):
    def test_list_trucks_view_performance(self):
        client = Client()

        Truck.objects.create(...)

        with self.assertNumQueries(6):
            response = client.get(reverse("trucks:list_trucks"))

        self.assertEqual(response.context["trucks_list"], 1)

        Truck.objects.create(...)

        with self.assertNumQueries(6):
            response = client.get(reverse("trucks:list_trucks"))

        self.assertEqual(response.context["trucks_list"], 2)

Note that we again check the number of items returned in the context, but on the second run we expect 2 trucks ( Truck). The reason for this behavior is similar to the first case.

Ensuring a constant number of calls to the database when adding new data is more priority than ensuring a small number of calls in general.

The last thing to do is to make sure that your data is as hydrated as possible. This means that you need to create related data that will be used during the processing of your view. If you do not, there is a risk that your application will access the database more often in production than in the test (although it may succeed). In our example, we needed to create TruckDrivera company for ourTruck.

from trucks.models import Truck, TruckDriver
...
        truck = Truck.objects.create(...)
        TruckDriver.objects.create(name="Alex", truck=truck)

If the number of database calls is no longer constant after performing the steps described above, then look for more information about the select_related and prefetch_related methods .

That's all for today, I hope that from this moment on you will begin to check the number of queries of your application to the database at the very beginning of the project. This will not take much time, but will prevent problems that may arise with the increase in the number of users of your application.

By the way, you can still catch the course . See you.

Query Counting: Basic Django Performance Testing

More articles: