Re: Connection timeouts on high load

On Thu, Apr 30, 2015 at 10:00 AM, sephii <sylvaintersideral@gmail.com> wrote:
> Hello,
>
> I have an application made with Django 1.7 and the Django Rest
> Framework and I'm in the phase of load testing it. My setup is made of
> 3 servers:
>
> - Nginx + gunicorn
> - Gunicorn
> - Postgresql + Memcached
>
> Nginx is configured as a loadbalancer so I can add more gunicorn
> instances if needed. In the current setup it already balances the load
> between its own gunicorn instance and the other server with the
> gunicorn instance. The project code is on the loadbalancer and it's
> shared on the gunicorn server via an NFS share.

I would not do that, but it is unlikely to cause many issues. NFS has
many gotchas.

>
> I'm using Locustio to simulate about 4000 users, that make a request
> every 30-60 seconds, with about 200 new users per second. The servers
> are handling the load until I get to about 2000 users, and then nginx
> starts returning 502 and 504 errors with the following in the logs:

I think you are being wildly optimistic about the performance of a
single app server. If the app server is not overloaded, add more
workers, if it is, add more app servers. Add some sort of monitoring
(I like munin) to all the servers to measure load - cpu, interrupts,
memory, disk io, memcached stats, postgre stats, nginx stats - and see
where your bottleneck is.

On the plus side, if you have 4000 users making requests every 30
seconds, you'll be able to afford many app servers ;)

Cheers

Tom

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAFHbX1Lzw2j2XRKCU3RNFqV7uZZW3%3Dp%3Da8cj%3DNo88gKiYj_XJg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.