My question from StackOverflow:
My technology stack is Redis as a channels backend, Postgresql as a database, Daphne as an ASGI server, Nginx in front of a whole application. Everything is deployed using Docker Swarm, with only Redis and Database outside. I have about 20 virtual hosts, with 20 interface servers, 40 http workers and 20 websocket workers. Load balancing is done using Ingress overlay Docker network. Ouch, and I'm also using django-cacheops for low timeout caching (up to 15 min)
The problem is, sometimes very weird things happen regarding performance. Most of requests are handled in under 400ms, but sometimes request can take up to 2-3s, even during very small load. Profiling workers with Django Debug Toolbar or middleware-based profilers shows nothing (timing 50-100ms or so)
My question: is there any good method of profiling a whole request path with django-channels? I would like how much time each phase takes, i.e when request was processed by Daphne, when worker started processing, when it finished, when interface server sent response to the client. Currently, I have no idea how to solve this.
EDIT:
Some more information:
Website isn't that big, it tops at 1k users online, with some real-time features like notifications, chat, and real-time updates of few views. It looks like sometimes website is "blocked", like communication between redis / interface servers / workers were incredibly slooow.
Maybe you have some experience about server / worker ratios, fine tuning channel and redis settings?
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/18cd97f8-5505-470b-8c22-3d97b1586d83%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
No comments:
Post a Comment