[Python-Dev] xmlrpclib speed boost (original) (raw)

Skip Montanaro skip@pobox.com (Skip Montanaro)
Mon, 1 Oct 2001 13:56:58 -0500 

>> This change may be deemed not to be the correct fix as far backwards
>> compatibility is concerned (it uses the "from m import x as y"
>> feature which was new with 2.0 I think).  If someone alters this fix,
>> please don't put the import back into the functions that call
>> cgi.escape.

Thomas> Why not ? Moving the import to the top level just causes the
Thomas> slowdown to occur at a different moment. If this is really the
Thomas> problem, the slowdown should occur only the first time you use a
Thomas> particular function (unless you explicitly un-import cgi somehow
Thomas> ?)

Trust me, this really is the problem. Instead of getting imported once, it gets imported once for every string and once for every key in every dictionary. I timed it before and after. Compare Marshaller.dump_string before

def dump_string(self, value):
    from cgi import escape
    self.write("<value><string>%s</string></value>\n" % escape(value))

and after

def dump_string(self, value):
    self.write("<value><string>%s</string></value>\n" % _escape(value))

I don't care how little work it is to import a module a second time, it is probably going to be on the same order of magnitude as that write call. In Marshaller.dump_struct the import was inside the for loop. I originally pulled it up out of the for loop but left it inside the method. That got me about a 25% boost in my simple test. I was ready to check in that one change and thought, "aw hell, might as well see what happens if I pull the import all the way out to the top level". Dump performance went all the way back up to where 0.9.8 is.

Thomas> Importing cgi only in the functions that actually use it, in
Thomas> order to avoid the slowdown unless it's really necessary, sure
Thomas> seems like a sensible solution to me :-)

Except dumping structs (dictionaries) and strings happens a lot. Even if all we ever did was dump ints, floats and lists, the total cost of my change would be one import.

Skip