Monday, November 24, 2014

Re: How to deal with CSRF middleware from a crawler

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBCAAGBQJUc5nrAAoJEC0ft5FqUuEh53AP+wegYReDOTj6bWSMT/bH5Fgp
7eMJpS5VwAQPnrk8Pk8tn39eyiLDE/i0cBSHVXv0jyDEENp3LBM/L2tXVEUpDtRB
lPzOA3MaGplLayjhY6FhP9UU9/VD1HK8A5E7CvUzsAi7TpjoEynH8MWBtrRpBVD9
DdBEzosdqM+VPw6oWWbWR/c7nUMi1E3Fr7qsdsc745d49qNo3EoUUWh+ilpMaIaK
lxKXWNFNoEPZ8BMHv7vFp1uKxNOh7FJZ2jJ/zRwZyKDfe8ee6p7gWmTjKnC8EoKg
flSViFHbACLRRfpAFmBce3sSxOkt7wIs/GAHv63V1VjGuwK9RLeHg2LRbzdvjdFc
PXiBy7mF4h/8SYMsehfYiZhDiog+mCgmfrAxpREJkVfi3semL9/x1WU8L37QlFU7
lsd405q2fWPJM2Ow69ywGmLjApzQoNR2IwtYOWEwVldgMGIg0yauyeMeDHS23sY1
1w81A09D3sWVw8+30CRwDAedq7hw6cowhWBLgh9ly9thY+YZGHUbeS1fwK2FHfkC
miIi9S7mmXKTqBfEmLgN66i/4e/DlhCnhY8BprtzNp7jca1NZkZYHZgm5La3FiMB
B0nOm7qVsQ6DugEeyfJFry3npj+Elq7FQElutvbrGJw5aJDzeAFUoR+BUK0tWOCW
t8ibVpa0vMDryf1wMuRm
=lRFE
-----END PGP SIGNATURE-----
Hi Torsten,

On 11/24/2014 01:00 PM, Torsten Bronger wrote:
> We use crawlers, which in our case a Python scripts that read data
> from disk and send a lot of HTTP POST requests to the Django
> deployment. The POST requests hit the same URLs/Views that are also
> used by the browser to edit something with a web <form>.
>
> If I activate the CSRF middleware, does this mean that our crawlers
> have to make GET requests before every POST in order to get the CSRF
> token? This would slow down them significantly ... Can one exclude
> certain usernames from the CSRF checks? Or do you see another way
> of keeping the number of HTTP requests small in the crawlers?

Unless you've modified the CSRF implementation locally, all it does is
check that the CSRF token provided in a cookie matches the one provided
in the POST data. This is effective because browser same-origin policy
prevents malicious JS from reading the cookie value that the user's
browser will send, or from controlling the sent cookie value.

But this means that the CSRF protection is simple to bypass in a case
like yours: you can just set the CSRF cookie and the POST var to the
same value in all your crawler's requests. It doesn't matter what that
value is.

Carl

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/547399E3.30009%40oddbird.net.
For more options, visit https://groups.google.com/d/optout.

No comments:

Post a Comment