Friday, March 30, 2012

Questions about UnicodeDecodeError

I have a situation where I am reading data that I have no control over and
inserting it into a db. The data is http headers. I am storing them in
postgres db in a text field and the db encoding is SQL_ASCII. Since the
data can be anything even non compliant http headers with anything for its
values I don't want to modify the data before I store it in the db.
However this is causing issues with certain values causing the
UnicodeDecodeError. For example I have a specific case where the user
agent is set to 'KC\xd4\xda\xcf\xdf\xc9\xfd\xbc\xb6'. I have been trying
to look for a way to deal with these cases gracefully in the models
__unicode__ method but nothing I have tried has worked.

Thanks,
Ali Mesdaq
Security Researcher
Cell: +1 (619) 952-8488 | Fax: +1 (408) 321-9818
Email: ali.mesdaq@fireeye.com

Next Generation Threat Protection
http://www.FireEye.com <http://www.fireeye.com/>

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

No comments:

Post a Comment