Django provides a powerful and flexible admin interface out of the box. It comes in handy while browsing data, doing straightforward daily tasks or performing CRUD operations.
At Sufle, we use Django heavily and Django Admin is part of our daily operations. For one of our projects, it was fast and reliable in the beginning. However, when our tables started hitting millions of rows, we experienced slow search times and timeouts most of the time. While our data continues to grow, we have learned a lot about what works well and what does not with Django Admin. With some simple adjustments, it is possible to continue to use admin even with large datasets. Here are the things we found, which might work for you.
First things first, for foreign keys and calculated fields there is no need to run extra queries per row. Performing multi queries for each object can lead to significantly slower response times.
What you can do here is to override the
get_queryset method in your model admin class. You can use
select_releated on your foreign keys.
class MyModelAdmin(ModelAdmin): def get_queryset(self, request): queryset = super().get_queryset(request) return queryset.select_related('foreign_key_1', 'foreign_key_2')
Or you can do annotations to save queries.
class MyModelAdmin(ModelAdmin): def get_queryset(self, request): queryset = super().get_queryset(request) return queryset.annotate(field_count=Count('field'))
Standart search works fine but after getting into a point of millions of rows, searching becomes a major issue. When there is more than one field in the
search_fields, the SQL query has multiple JOIN statements chained one after the other. Our solution was introducing DjangoQL package here.
DjangoQLSearchMixin replaces the standard Django search with the DjangoQL search. DjangoQL helps us search through only specified fields, related fields included. All you need to do is adding
DjangoQLSearchMixin to your model admin:
from djangoql.admin import DjangoQLSearchMixin class MyModelAdmin(DjangoQLSearchMixin, ModelAdmin): pass
DjangoQL autocompletes model fields, supports logical operators, parenthesis and table joins. You can get rid of multiple JOIN statements with just a few lines of code and focus on what you want to search without limitations rather than hard coding
After reaching a certain size, expensive
COUNT(*) queries become a burden on
change_list and cause timeout errors.
MySQL is our go-to solution and naturally, we use Django-MySQL to extend Django’s built-in MySQL support. The package comes with the handy
count_tries_approx method which returns the approximate count instead of the exact count for the table. In this way, we eliminate a full table scan on InnoDB and achieve a significantly faster result.
class MyModelAdmin(ModelAdmin): def get_queryset(self, request): qs = super().get_queryset(request) return qs.count_tries_approx()
SELECT COUNT(*) AS `__count` FROM `mytable`
SELECT TABLE_ROWS FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'mytable'
Hope these useful tricks help you run Django Admin while you scale and grow.
An AWS Certified Developer Associate, Burak is an experienced software engineer. With his experience in various industries including global technology companies, he follows his passion for going beyond the limits to build excellent products with collaboration and knowledge sharing.
Cookies are small files that are sent to and stored in your computer by the websites you visit. Next time you visit the site, your browser will read the cookie and relay the information back to the website or element that originally set the cookie.
Cookies allow us to recognize you automatically whenever you visit our site so that we can personalize your experience and provide you with better service.