[Python-Dev] 2.3.1 is (almost) a go (original) (raw)

Skip Montanaro skip at pobox.com
Wed Sep 24 12:41:28 EDT 2003 

>> Many are deserting what at times appears to be a sinking bsddb ship
>> in favor of pickles...  For long-running SpamBayes apps like
>> pop3proxy and the Outlook plugin, pickles make sense if shared access
>> to the scoring database is not needed.

Anthony> :-(

Anthony> I haven't been following the spambayes lists too closely. Are
Anthony> there concrete problems with bsddb that are cropping up, or
Anthony> just a general wariness of it?

It's a combination of things. First and foremost, using the pickle storage is much faster assuming your database is not too huge and you have a long-running spambayes app. Second, people still use pre-2.3 Windows versions of Python to run their spambayes apps. By default, these people get dumbdbm by default. That's a disaster waiting to happen because of storage size and speed concerns. There were also bugs in dumbdbm (which have since been fixed).

Anthony> If there _is_ a problem with bsddb, it needs to be
Anthony> addressed. Too many things depend on it.

I think it's more that bsddb is an innocent bystander in a drive-by shooting, though some folks have been seeing DB_RUNRECOVERY (sp?) errors. The anydbm-style access to Berkeley databases doesn't provide the necessary locking for multiple process (or multiple thread?) access.

I agree if there are problems they should be fixed. One step in the right direction would probably be to create a default DBEnv for use by the bsddb.*open() compatibility calls. Also, the docs should mention threading issues for people using the more featureful API so they don't unwittingly shoot themselves in the foot. (Maybe DBEnv=None should not be allowed in open calls if threads are enabled?)

Skip

More information about the Python-Dev mailing list