středa 22. srpna 2012

Psycopg2 UnicodeEncodeError when using "psycopg2.extensions.adapt"

Just a little trick. If you are using Psycopg2´s adapt method to adapt your classes for usage in SQL statements (like this), you may run into problems with unicode strings.

Let´s say you have object which you would like to map to some composite PG type. Any string attribute which will contain unicode chars may give you exception like this (when called adapt and getquoted on it):

       .... some traceback .... 

       return adapted.getquoted()
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 40-44: ordinal not in range(256)

It gave me few moments, but I found solution in this discussion. Because the encoding is passed through by connection object, it is necessary to call prepare methods on all adapted items (by adapted, I mean this: adapted = psycopg2.extensions.adapt(value)). 

When your class for adaptation is passed to psycopg´s execute, psycopg first try to call prepare if available. So in your class, you get real connection object by prepare method. Then, after calling adapt on attribute values, you can call prepare on them too. 

Little example. 


   1 from psycopg2.extensions import adapt, ISQLQuote
   2 
   3 class FooBarComposite:
   4         
   5         def __init__(self, foo, bar):
   6                 self.foo = foo
   7                 self.bar = bar
   8                 
   9         def __conform__(self, protocol):
  10                 if protocol is ISQLQuote:
  11                         return self
  12         
  13         def prepare(self, conn):
  14                 self._conn = conn
  15         
  16         def getquoted(self):
  17                 
  18                 adapted_foo = adapt(self.foo)
  19                 if hasattr(adapted_foo, 'prepare'):
  20                         adapted_foo.prepare(self._conn)
  21                 
  22                 adapted_bar = adapt(self.bar)
  23                 if hasattr(adapted_bar, 'prepare'):
  24                         adapted_bar.prepare(self._conn)
  25                 
  26                 result = adapt((adapted_foo, adapted_bar))
  27                 if hasattr(result, 'prepare'):
  28                         result.prepare(self._conn)
  29                 
  30                 return result.getquoted()


Now, class FooBarComposite can be used directly as an parameter of SQL statements and unicode attributes will work properly.