I have measured the time taken by various operations for 3 pure-Python database modules (buzhug, KirbyBase and Gadfly) and compared them with SQLite

The tests are those described on the SQLite comparisons pages, which compares performance of SQLite to that of MySQL and PostGreSQL

The tests

Here are the tests :

insert : create the base and insert n elements (n= 1000 or 25000) in it

The database has 3 fields : a (integer, from 1 to n),b (random integer between 1 and 100000),c (a string, value = 'fifty nine' if b=59)
For gadfly and SQLite two options are possible : with an index on field a, or without index

The values of a,b,c are stored in a list recs

SQL statements

cursor.execute("CREATE TABLE t1(a INTEGER, b INTEGER, c VARCHAR(100))")
if make_index:
    cursor.execute("CREATE INDEX i3 ON t1(a)")
for a,b,c in recs:
    cursor.execute("INSERT INTO t1 VALUES(%s,%s,'%s')" %(a,b,c))
conn.commit()

buzhug code

db=Base('t1').create(('a',int),('b',int),('c',str))
for rec in recs:
    db.insert(*rec)
db.commit()

select1 : 100 selections to count the number of records and the average of field b for values of b between 10*n and 10*n + 1000 for n = 1 to 100

SQL statements

for i in range(100):
    sql = 'SELECT count(*), avg(b) FROM t1 WHERE b>=%s AND b<%s' %(100*i,1000+100*i)
    cursor.execute(sql)
    nb,avg = cursor.fetchall()[0]

buzhug code

for i in range(100):
    recs = db.select(['b'],b=[100*i,999+100*i])
    nb = len(recs)
    if nb:
        avg = sum([r.b for r in recs])/nb

select2 : 100 selections to count the number of records and the average of field b for values of c with the string 'one', 'two', ...,'ninety nine' inside. It uses the keyword LIKE for SQL database (I couldn't do the test for Gadfly which doesn't support LIKE) ; for Kirbybase and buzhug I use regular expressions

The strings for each number between 0 and 99 are stored in the list num_strings

SQL statements
for num_string in num_strings: sql = "SELECT count(*), avg(b) FROM t1 WHERE c LIKE '%%%s%%'" %num_string cursor.execute(sql) nb,avg = cursor.fetchall()[0]

buzhug code
for num_string in num_strings: pattern = re.compile(".*"+num_string+".*") recs = db.select(['b'],'p.match(c)',p=pattern) nb = len(recs) if nb: avg = sum([r.b for r in recs])/nb

delete1 : delete all the records where the field c contains the string 'fifty'. There again I couldn't do the test for gadfly

SQL statements	`sql="DELETE FROM t1 WHERE c LIKE '%fifty%';" cursor.execute(sql) conn.commit()`
buzhug code	`db.delete(db.select(['__id__'],'p.match(c)',p=re.compile('.fifty.')))`

delete2 : delete all the records for which the field a is >10 and < 20000

SQL statements	`sql="DELETE FROM t1 WHERE a>10 AND a<20000;" cursor.execute(sql) conn.commit()`
buzhug code	`db.delete(db.select(['__id__'],'x < a < y',x=10,y=20000))`

update1 : 1000 updates, multiply b by 2 for records where 10*n <= a < 10*(n+1) for n = 0 to 999

SQL statements

for i in range(100):
	sql="UPDATE t1 SET b=b*2 WHERE a>=%s AND a<%s;\n" \
		%(10*i,10*(i+1))
	cursor.execute(sql)
conn.commit()

buzhug code

for i in range(100):
    for r in db.select(a=[10*i,10*i+9]):
        db.update(r,b = r.b*2)

update2 : 1000 updates to set c to a random value where a = 1 to 1000

New values of field c are stored in a list new_c

SQL statements
for i in range(0,1000): sql="UPDATE t1 SET c='%s' WHERE a=%s" %(new_c[i],i) cursor.execute(sql) conn.commit()

buzhug code
recs = db.select_for_update(['a','c'],a=[1,999]) for r in recs: db.update(r,c = new_c[r.a])

The tests have been made on a Windows XP machine, with Python 2.4.1

Versions : buzhug 0.3, KirbyBase 1.9, gadfly 1.0.0, PySQLite 2.2.2 installed with the Windows binary for Python 2.4

Results

Here are the results

1000 records

           buzhug    kirbybase        gadfly           sqlite
                      index     no index    index   no index  index
create     0.31       0.12        0.59      1.87    0.33      0.41
select1    0.20       4.02        3.96      4.13    0.07      0.18
select2    0.80       4.08        -         -       0.17      0.16
delete1    0.04       0.06        -         -       0.26      0.28
delete2    0.16       0.10        0.07      0.07    0.22      0.17
update1    0.48       2.77        5.22      5.48    0.34      0.21
update2    0.74      25.19       35.39      0.83    0.68      0.41

25000 records

             buzhug   kirbybase   gadfly (1)          sqlite
                                no index         no index   index
create       6.54       2.49       19.50         1.62       2.49
select1      3.68      97.19       74.71         2.29       2.30
select2     18.99      96.42        -            3.64       3.69
delete1      0.77       1.18        -            1.04       1.20
delete2      2.61       1.79        1.02         0.44       0.62
update1      1.30      11.33      110.78         1.72       0.46
update2      0.77      22.51      975.67         9.35       0.46

(1) not tested with index, creation time is +INF

Conclusions

buzhug is usually much faster than KirbyBase and Gadfly for selection of records. On the 25,000 item base, the average is 10 times faster than KirbyBase and 275 times faster than Gadlfy. KirbyBase is faster for insertion of new records (because all data are stored in a single file) and for the delete2 test. Gadfly is only faster for the delete2 test

SQLite is faster than buzhug on nearly all operations. On the 25,000 item base, the average on all operations is 2,7 times faster (with or without index). buzhug is only faster on delete1