Two Scoops Study Session 16

This time we studied Two Scoops of Django Chapter 24, 'Improving Django Performance', and Chapter 25, 'Asynchronous Task Queues'. Chapter 24 focused on the perspective of performance optimization, while Chapter 25 centered on when to use asynchronous task queues and what to be careful about.

Performance improvement needs to be approached across multiple layers. For Query, you should reduce duplicate queries and eliminate slow queries by leveraging indexes, and for Database, you shouldn't blindly pile up logs or temporary data, and should carefully examine each DB environment's configuration.

Beyond that, you also need to think about where to place your Cache and when cache invalidation should occur, and for static files like HTML/CSS/JavaScript, compression and minification are important. Depending on the situation, you can also make use of upstream caches or a CDN.

However, the core message is that premature optimization can be harmful. For small-scale services or early-stage products, it's more important to build features that actually work before optimizing, and you should avoid the mistake of worrying about performance issues in advance when you have almost no users, which can bring development itself to a halt.

Asynchronous task queues are suitable for tasks where processing the result takes a long time. Conversely, they don't fit tasks where the user needs to see the result immediately or use it right away, so you need to carefully choose based on the nature of the task.

When using async tasks, a few principles matter.

• Tasks should be written to maintain idempotency, since there's always a possibility of retries.
• Important data should not be stored only inside the queue. Given the possibility of failures and retries, it's safer to keep data in a separate storage and use the queue only to pull and execute it.
• The queue is not free. Even though it runs invisibly, it ultimately consumes computing resources, so an inefficient design becomes a bottleneck.

In the deep-dive section, we also introduced additional materials. Through the translated Postgres Performance article, we looked at practical tips such as caches, hit rates, and how to use indexes, and broadened our understanding by directly querying an actual DB.

We also covered several Celery-related articles, including an approach of not putting important data into the queue, a design to prevent duplicate sends in an email-sending task, and Celery - Best Practices. Finally, through an article on AMQP, we organized the terminology and background of how message queues work internally.

Overall, this study session brought together the basic principles of Django performance tuning and the correct criteria for using asynchronous tasks. It was especially impressive that we came away with practical hints for applying Celery or PostgreSQL optimizations to our own personal projects.