[Python-Dev] New branch for r23c2 work (original) (raw)

Skip Montanaro skip@pobox.com
Mon, 21 Jul 2003 10:34:40 -0500 

Anthony> Could someone have a quick look at 774751 and decide if it's
Anthony> worth spending time on before rc2? (synopsis: sometime between
Anthony> 2.1 and 2.2, binding to a socket on MacOS X started taking a
Anthony> long long long time.)

I took the proverbial quick look. It appears the call to getsockaddrarg is taking all the time. I added some calls to gettimeofday surrounding the bits of functionality in sock_bind:

    struct timeval t;
    long usec;

    gettimeofday(&t, NULL); usec = t.tv_usec;
    if (!getsockaddrarg(s, addro, &addr, &addrlen))
            return NULL;
    gettimeofday(&t, NULL);
    fprintf(stderr, "getsockaddrarg: %d\n", (t.tv_usec-usec));
    gettimeofday(&t, NULL); usec = t.tv_usec;
    Py_BEGIN_ALLOW_THREADS
    res = bind(s->sock_fd, addr, addrlen);
    Py_END_ALLOW_THREADS
    gettimeofday(&t, NULL);
    fprintf(stderr, "bind+threads: %d\n", (t.tv_usec-usec));

and get this sort of output running on my Powerbook:

getsockaddrarg: 139824
bind+threads: 30
getsockaddrarg: 139884
bind+threads: 17
getsockaddrarg: 138592
bind+threads: 17
getsockaddrarg: 139504
bind+threads: 19

Instrumenting Python 2.2.3 sock_bind on a Linux machine generates this sort of output:

getsockaddrarg: 13
bind+threads: 7
getsockaddrarg: 14
bind+threads: 8
getsockaddrarg: 14
bind+threads: 8
getsockaddrarg: 15
bind+threads: 7
getsockaddrarg: 14
bind+threads: 7

Moving the time checks into getsockaddrarg generates this output on my Powerbook:

tuple check: 4
parse tuple: 26
setipaddr: 141153
htons: 3

Looking at setipaddr I see that Just added some lock acquisition code in May:

1.265        (jvr      09-May-03):          Py_BEGIN_ALLOW_THREADS
1.265        (jvr      09-May-03):          ACQUIRE_GETADDRINFO_LOCK
1.153        (loewis   21-Jul-01):          error = getaddrinfo(NULL, "0", &hints, &res);
1.265        (jvr      09-May-03):          Py_END_ALLOW_THREADS
1.265        (jvr      09-May-03):          /* We assume that those thread-unsafe getaddrinfo() versions
1.265        (jvr      09-May-03):             *are* safe regarding their return value, ie. that a
1.265        (jvr      09-May-03):             subsequent call to getaddrinfo() does not destroy the
1.265        (jvr      09-May-03):             outcome of the first call. */
1.265        (jvr      09-May-03):          RELEASE_GETADDRINFO_LOCK

revision 1.265
date: 2003/05/09 07:53:18;  author: jvr;  state: Exp;  lines: +45 -10
[ 731644] & [ 604210 ] Release the GIL around getaddrinfo(), yet protect
access with lock on those platforms that getaddrinfo() isn't (known to be)
thread-safe. Thanks to MvL for mentoring this patch.

Without digging deeper, I'm going to suggest that's the culprit. I'll leave it to others to figure out what can be done.

Skip