dealing with BLOB field and encoding of the text.

Hi All,

I have hit a road block in trying to read BLOB data {of some encoding
other than UTF-8 or any other unicode} in Django for display.

The SQLite database that I am dealing with has a table, the columns of
it which stores the command, output are of type BLOB and the data
being stored is in SHIFT_JIS encoding. this database also contain an
other table, which contains the encoding information.

From the error I can make out that at the model level, conversion of
data to utf-8 is failing. I do not know if adding _unicode_() method
explicitly will solve this problem.

Please advice on the approach that I should take to overcome this
problem. with out compromising on changing the field type of the
database to something suitable other than blob. I do not have control
on changing the schema of the database table :(

The model code that I am using is:

class BlobField(models.Field):
def db_type(self):
return 'blob'

class VarStr(models.Field):
def db_type(self):
return 'varchar'

class Result(models.Model):
tcid = VarStr('Testcase ID')
cmd = BlobField('Command')
ec = VarStr('Status Code')
op = BlobField('Output', null=True)
mtext = BlobField('Match', null=True)
txetm = BlobField('Do Not Match', null=True)
mdiff = BlobField('Output Diff.', null=True)
gdiff = BlobField('GD Diff.', null=True)
result = VarStr()

class Meta:
db_table = 'results'

I am getting the following error:

OperationalError at /report/
Could not decode to UTF-8 column 'cmd' with text 'mkdir
folder_lazy_dog_�ӂ����݈�_�I����'
Request Method: POST
Request URL: http://127.0.0.1:8000/report/
Django Version: 1.3 pre-alpha SVN-14347
Exception Type: OperationalError
Exception Value:

Could not decode to UTF-8 column 'cmd' with text 'mkdir
folder_lazy_dog_�ӂ����݈�_�I����'

Exception Location: /usr/lib/python2.6/dist-packages/django/db/models/
sql/compiler.py in <lambda>, line 744
Python Executable: /usr/bin/python
Python Version: 2.6.6
Python Path: ['/LA/src/lighthouse', '/usr/local/lib/python2.6/dist-
packages/Pygments-1.3.1-py2.6.egg', '/usr/lib/python2.6', '/usr/lib/
python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/usr/lib/
python2.6/lib-old', '/usr/lib/python2.6/lib-dynload', '/usr/local/lib/
python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/usr/
lib/python2.6/dist-packages/PIL', '/usr/lib/python2.6/dist-packages/
gst-0.10', '/usr/lib/pymodules/python2.6', '/usr/lib/python2.6/dist-
packages/gtk-2.0', '/usr/lib/pymodules/python2.6/gtk-2.0']
Server time: Tue, 26 Oct 2010 00:39:31 +0530

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to django-users+unsubscribe@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-users?hl=en.