Monday, August 30, 2010

Re: potential issue re in memory django file uploading.

On Aug 30, 1:54 pm, dave b <db.pub.m...@gmail.com> wrote:
> On 30 August 2010 11:04, Russell Keith-Magee <russ...@keith-magee.com> wrote:
>
>
>
>
>
> > On Sun, Aug 29, 2010 at 8:26 PM, dave b <db.pub.m...@gmail.com> wrote:
> >  1) An actual problem where you can clearly describe the circumstances
> > or sequence of events that would allow an attack to occur, and
> >  2) Something that is actually Django's problem -- by which I mean,
> > something that is actually Django's responsibility to solve, rather
> > than something that is a webserver configuration issue.
>
> > At this point, it's not clear to me that either of these two things
> > are true. Based on your messages and the feedback from Graham and
> > Steve, it sounds like you're describing an attack that *could* exist,
> > but only if you've got a misconfigured (or badly implemented) web
> > server.
>
> > If you believe that I'm wrong, and there *is* an actual problem, you
> > need to convince us. This doesn't mean posting large wads of Django's
> > source code and proposed patches over multiple messages. It means
> > describing in clear, concise language exactly what conditions need to
> > exist for a problem to occur.
>
> > Yours,
> > Russ Magee %-)
>
> Morning. Will do so below here.
> Just do remember, there is more than one way to run a httpd, some of
> us run our own custom stuff ;) and not everyone is using a setup like
> you have.
>
> -------------------
>
> Feature: Attacker crashes your django installation via file uploading
> As attacker
> I want to crash your django installation
> To take your site down or reduce its availability, so I can steal the
> underpants and then profit!!!!
>
> Background:
> Given I am an attacker
> And you have uploads enabled with the default settings (memory and
> temporary file).
> And you are running on a platform with /tmp
>
> Feature: I upload a 1gb file and have this go into system memory
> Given I have a 1gb file
> When I uploaded it to the website
> Then I should see that your system now has used an additional 1gb of /tmp
> And available system memory is now reduced

How is any available system memory reduced. Normally '/tmp' is disk,
not memory and since data is streamed when being written to disk, the
only memory impact on the application would be what is required to
hold one block of the data in memory while it is being read/written.
The whole file would not be getting stored in memory because when >
2.5MB, it wouldn't be using in memory file uploader, but the disk
based one.

> So basically I was saying there are two problems.
> One is if the httpd isn't behaving properly (this is probably not
> entirely true) with respect to the content length field and abusing
> memory limitation.

Huh. Don't know what you are suggesting here. Web servers do not load
all request content into memory at one time, they stream data, usually
through the web application actually pulling it through a chunk at a
time when the application wants it. Thus is matters little what
content length is to the web server.

> The second issue is that there is no *default* set limit on temporary
> file uploads, so any file larger than 2.5mb can find its way to /tmp
> and there is no limit on the size of these files in django core.
> That is there is no set limit on the size of a temporary file upload.

I don't actually use Django so not 100% sure, but yes there possibly
isn't an equivalent of LimitRequestBody definable within Django unless
can be done with middleware. But then it is as I pointed out the wrong
place to be doing it. The better place is in the web server. This is
because the web server can reject it immediately. If you do it in the
application, then for any hosting mechanism that has to proxy data
through to the application, eg., mod_wsgi daemon mode, fastcgi, scgi,
uWSGI etc, then the web server will have already started proxying
through the data to the application process, thus wasting cycles and
resources, by the time the application decides it doesn't want to
handle it. You are thus always better off rejecting large requests as
early as possible in the pipeline.

So, yes it may make sense for Django to have a fail safe and allow you
to specify a maximum on upload size if it doesn't already, but that is
only of use where you haven't set up your production web server
properly to protect you from abuses, something you should be doing.

> The second problem is going to exist within the bounds of the set
> limits of the webserver and the various mods that are used with
> django.
>
> In an extreme and very unlikely case, the httpd may ungzip the data
> from the attacker and modify the content length (when it knows what it
> should be - the connection is terminated ) with django getting a large
> amount of data to store from a much smaller user body request.

No it will not. Did you not read my other post?

The mod_deflate filter does not modify the content length as it cant.
This is because it has to stream the decompressed data and by the time
it knows the actual content length, the original content length header
has long ago been passed through to the application and there is no
opportunity to update it. As such. any application will only see the
length as being what the original content length said it was and would
truncate the data at that. As I said before, it is technically not
possible to use mod_deflate on request content with a WSGI
application.

Anyway, I would have to agree with Russell, you are simply not making
yourself clear enough and to added to that seem to keep echoing
statements that have been refuted.

For the third time I ask you whether you have actually gone and tested
your hypothesis and can provide a working test case that demonstrates
the problem.

FWIW, there are much simpler ways to bring a site down than this. I
suggest you go research Slowloris.

http://en.wikipedia.org/wiki/Slowloris

Graham

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment