Re: Django 1.2.3, Oracle with Character Set WE8MSWIN1252 - 'utf8' codec can't decode bytes

Hi Ian,

Thanks for the response.

With cx_Oracle(version 5.0.3), the retrieval of that field value works fine as in my original email.

It's only when i directly use the Django models way of accessing that it fails.

Below two examples will make it more clear.

This is using Django models and it fails

==============================

>>> TerminologyMap.objects.filter(id=316)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 68, in __repr__

    data = list(self[:REPR_OUTPUT_SIZE + 1])

  File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 83, in __len__

    self._result_cache.extend(list(self._iter))

  File "/usr/lib/python2.5/site-packages/django/db/models/query.py", line 269, in iterator

    for row in compiler.results_iter():

  File "/usr/lib/python2.5/site-packages/django/db/models/sql/compiler.py", line 672, in results_iter

    for rows in self.execute_sql(MULTI):

  File "/usr/lib/python2.5/site-packages/django/db/models/sql/compiler.py", line 741, in <lambda>

    result = iter((lambda: cursor.fetchmany(GET_ITERATOR_CHUNK_SIZE)),

  File "/usr/lib/python2.5/site-packages/django/db/backends/oracle/base.py", line 552, in fetchmany

    for r in self.cursor.fetchmany(size)])

  File "/usr/lib/python2.5/site-packages/django/db/backends/oracle/base.py", line 625, in _rowfactory

    value = to_unicode(value)

  File "/usr/lib/python2.5/site-packages/django/db/backends/oracle/base.py", line 636, in to_unicode

    return force_unicode(s)

  File "/usr/lib/python2.5/site-packages/django/utils/encoding.py", line 88, in force_unicode

    raise DjangoUnicodeDecodeError(s, *e.args)

django.utils.encoding.DjangoUnicodeDecodeError: 'utf8' codec can't decode bytes in position 22-24: invalid

 data. You passed in 'Registro guardado con \xe9xito' (<type 'str'>)

This is using cx_Oracle and it works fine

===============================

>>> cx_Oracle.version

'5.0.3'

>>> cursor.execute("select to_term from terminology_map where id=316")

>>> cursor.fetchone()[0]

'Registro guardado con \xe9xito'

Regards,

Anurag

On Wed, Dec 1, 2010 at 10:51 PM, Ian <ian.g.kelly@gmail.com> wrote:

On Nov 30, 8:31 pm, Anurag Chourasia <anurag.choura...@gmail.com>
wrote:

> On Oracle 10.2 with Character-Set set to WE8MSWIN1252,
>
> When using Django, I try to select a Oracle row which contains a field with
> value as 'Páginas', i encounter the following error "'utf8' codec can't
> decode bytes "

The NLS_LANG setting used by Django should guarantee that the data
comes back as UTF-8 regardless of the database character set.

What version of cx_Oracle are you using?
Is the column type VARCHAR2 or NVARCHAR2?
What do you get if you try the following, substituting the appropriate
values?

$ export NLS_LANG=.UTF8
$ python
>>> import cx_Oracle
>>> conn = cx_Oracle.connect('username/password@dsn')
>>> cursor = conn.cursor()
>>> cursor.execute("SELECT PROBLEM_COLUMN FROM TERMINOLOGY_MAP WHERE ID = 206")
>>> print repr(cursor.fetchone()[0])

Thanks,
Ian

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.