[Python-Dev] FW: [Python-Help] Python threads with suncc (Forte 6.1) compiler (original) (raw)

Tim Peters tim.one@home.com
Sat, 4 Aug 2001 23:17:50 -0400

[Guido, on reworking select()]

Yes, I think we do need this. PyObjectAsFileDescriptor() calls the fileno() method which could be a Python call.

Ya, but for me to believe that matters, I have to believe people worried about high-performance select are passing objects with Python-implemented fileno methods to begin with. That's quite a stretch, isn't it? They have to pay for fileno() calls regardless, the question is only whether they pay it once or twice per object.

Maybe we could use a more sophisticated construct where we use a short array allocated on the stack for lists of fewer than 100 (say) files but use malloc for larger ones?

Another idea: screw it . We have to allocate three lists anyway to hold the results. Allocate them at the start instead of the end, and then they can do double-duty, holding the cached fileno() results across the select() call.

Fleshing it out a little,

def list2set(inputlist, fdset): "Return (list of file descriptors, biggest fd seen); set fdset slots." n = len(inputlist) result = [None] * n if _MSC_VER: biggest = 0 else: biggest = -1 for i in range(n): obj = inputlist[i] fd = PyObject_AsFileDescriptor(obj) if fd == -1: return NULL, -1 result[i] = obj, fd FD_SET(fd, fdset) if not _MSC_VER: biggest = max(biggest, fd) return result, biggest

iolist is the result of an earlier list2set() call. It gets shrunk

to hold just the objects corresponding to the fds in fdset.

def set2list(iolist, fdset): lastindex = 0 for obj, fd in iolist: if FD_ISSET(fd, fdset): iolist[lastindex] = obj lastindex += 1 if lastindex < len(iolist): del iolist[lastindex:]

So it still does the perversely optimized caching, but no stack space required, no free() calls, only the 3 malloc() calls that are needed anyway, and up to 3 realloc() calls to shrink the output lists. It does create a bunch of 2-tuples, but they're on a free list and come cheap.