#86: Building bsddb.so for Python

I want to build the _bsddo.so module for my Python install, and link it to an installation of BerkeleyDB 4.8 on my OS X 10.6 machine. OS X 10.6 comes with Python 2.6 and a stock bsddb module, but does not ship the actual binary _bsddb.so, which makes the entire module useless. As soon as you attempt to use bsddb, you will get:

ImportError: No module named _bsddb
I don't want to re-compile the entire Python package or install a second Python package for one measly shared object file.

Building _bsddb.so for OS X 10.6 system Python

Grab a copy of the Python source code corresponding to the version you want. The stock 10.6 operating system ships with Python 2.6.1, so you'll want to grab the tarball from the reference site below.

Unpack it and go into the Modules/ subdirectory within the unpacked source directory. There you'll find _bsddb.c.

Compile _bsddb.c into a shared object via:
gcc -O2 -arch x86_64 -arch i386 -shared -o _bsddb.so _bsddb.c -I/usr/local/BerkeleyDB/include -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -L/usr/local/lib -L/usr/local/BerkeleyDB/lib -ldb -framework Python

Swap the -I/usr/local/BerkeleyDB/include and -L/usr/local/BerkeleyDB/lib paths for your own. If you're on PPC, obviously use -arch ppc instead of i386 or x86_64.

During compilation, you may get:
_bsddb.c:4525: error: ‘DB_XIDDATASIZE’ undeclared (first use in this function) _bsddb.c:4525: error: (Each undeclared identifier is reported only once _bsddb.c:4525: error: for each function it appears in.) _bsddb.c: In function ‘DBTxn_prepare’: _bsddb.c:5950: error: ‘DB_XIDDATASIZE’ undeclared (first use in this function) _bsddb.c: In function ‘init_bsddb’: _bsddb.c:7099: error: ‘DB_XA_CREATE’ undeclared (first use in this function)

This will only happen if you were using BerkeleyDB 4.8 with Python 2.6.1. The reason is that bdb 4.8 has renamed DB_XIDDATASIZE to DB_GID_SIZE, and removed DB_XA_CREATE entirely.

To fix this, do a global search/replace in _bsddb.c for DB_XIDDATASIZE and replace it with DB_GID_SIZE. Also, remove all lines mentioning DB_XA_CREATE (such as _bsddb.c:7099). This is safe unless you were going to use XA Resource Manager support in BerkeleyDB, which I was not.

To install, you can place your freshly minted _bsddb.so file in /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/. Alternatively, if you are concerned about system updates overwriting your object file, you can place it in /Library/Python/2.6/site-packages/lib-dynload, AND make sure to add a .pth file to the site-packages directory pointing to the new lib-dynload directory.

I knew my little exploration with readline.so was going to pay off.

References used:

Think you've got a better solution? Help 92049143cabb7ba896d7c06e19906303_small yliu out by posting your solution

Python "bindings" for Oracle Berkeley DB

http://www.jcea.es/programacion/pybsddb.htm - found by 92049143cabb7ba896d7c06e19906303_small yliu on February 22, 2010, 12:33 AM UTC

solving the DB_XIDDATASIZE and DB_XA_CREATE problems

Python 2.6.1 Release

http://www.python.org/download/releases/2.6.1/ - found by 92049143cabb7ba896d7c06e19906303_small yliu on February 22, 2010, 12:33 AM UTC