Thursday, September 30, 2010

Re: UnicodeEncodeError

On Wed, Sep 29, 2010 at 5:32 PM, werefr0g <> wrote:
Sorry, sorry.... I proposed Jean to send me the actual file to check its encoding. I asked for his OS too in order to see what editor are available and how it allows to "transcode" the text.

Not the contents of the file is irrelevant to this problem: it is the file's name that is causing the problem. The last bit of the traceback is:

 File "/usr/languages/python/2.6/lib/python2.6/", line 18, in exists
   st = os.stat(path)

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe7' in position 53: ordinal not in range(128)

"path" here includes the file name, no file data is involved. Django is passing a unicode string "path" to the os.stat() function. On many operating systems, Python must actually pass a bytestring, not unicode, to the underlying OS routine that implements "stat".  Therefore Python must convert the unicode string to a bytestring using some encoding. The encoding it uses is whatever is returned by os.getfilesystemencoding: As noted in that documentation, on Unix the encoding will be:

... the user's preference according to the result of nl_langinfo(CODESET), or None if the nl_langinfo(CODESET) failed.

That's a pretty obscure description but it boils down to the encoding associated with the currently set locale. (And on some systems successfully setting a locale with an encoding like utf-8 requires installing some "extra" language packs.) So the key to fixing this problem is to ensure the locale of the running server is successfully set to one with an encoding such as utf-8, which supports (can encode) the full range of unicode values. Unfortunately details on setting locales differs from machine to machine so it is hard to give specific instructions here.


You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to
To unsubscribe from this group, send email to
For more options, visit this group at

No comments:

Post a Comment