Issue 604210: release GIL around getaddrinfo() (original) (raw)
Created on 2002-09-03 21:48 by jhylton, last changed 2022-04-10 16:05 by admin. This issue is now closed.
Messages (7)
Author: Jeremy Hylton (jhylton) 
Date: 2002-09-03 21:48
If getaddrinfo() is thread-safe, then we should release the interpreter lock before calling it. There is every reason to believe that this operation will be slow.
Author: Martin v. Löwis (loewis) * 
Date: 2002-09-04 06:25
Logged In: YES user_id=21627
How do you know getaddrinfo is thread-safe? In particular, when using the getaddrinfo emulation, it is not.
Author: Jeremy Hylton (jhylton)
Date: 2002-09-04 19:20
Logged In: YES user_id=31392
The getaddrinfo() on Linux says it thread-safe. It's a name-fangled call. Is it part of some standard? It would certainly be possible to make the fallback implementation thread-safe. Or, if we don't believe it is thread-safe, we could add a separate lock to protect it. The socket module of 2.1 vintage had a separate lock for this purpose. And the 2.1 vintage Python performs much better when a multi-threaded app does DNS lookups.
Author: Neal Norwitz (nnorwitz) *
Date: 2002-09-04 20:18
Logged In: YES user_id=33168
I looked on OpenBSD (I think, may have been NetBSD) and FreeBSD. The man pages were the same, both old (circa 1995), but they said their implementation was not thread-safe. I did a man on Dec UNIX and it said it conformed to POSIX.1g Draft 6.6, but didn't indicate if it was thread safe. Solaris 8 doesn't say anything in the man page.
Here's a NetBSD 1.6 man page dated 2002/05/14 and it's still not thread-safe: http://216.239.37.100/search?q=cache:4bTvQQqcwq4C:www.tac.eu.org/cgi-bin/man-cgi%3Fgetaddrinfo%2B3+getaddrinfo+thread+safe&hl=en&ie=UTF-8
This: http://216.239.33.100/search?q=cache:q5egqJ5_mv4C:mail.gnu.org/pipermail/guile-devel/2001-October/004039.html+getaddrinfo+thread+safe&hl=en&ie=UTF-8 says "getaddrinfo and getnameinfo are the recommended APIs. They avoid hidden static data; they're supposed to be thread-safe; they handle multiple address families."
It seems safest to do what Jeremy proposes and add a lock. There can always be a define if getaddrinfo is thread safe or not. Maybe we can even determine with autoconf.
Note: I get much better performance with this patch under linux.
Author: Martin v. Löwis (loewis) *
Date: 2002-09-05 08:38
Logged In: YES user_id=21627
RFC 2553 specifies that getaddrinfo, getipnodebyname, and all the other new netdb functions must be thread-safe - hence the BSD man pages document it as a bug that they are not.
So adding an additional lock, in general, might be overkill. I'd suggest a strategy that assumes, in general, that getaddrinfo is thread-safe. If it isn't (by hard-coded knowledge), I'd propose just to continue using the interpreter lock - those systems deserve to lose.
As for the fallback implementation - feel free to do anything about it that you can find the time for. Recording it as thread-unsafe might be fine, adding a lock is another option, using the thread-safe API inside the fallback (where available) is a third option.
I would not care too much, except that this is used on Windows. Notice that it won't be used on Windows if Python is compiled with VC.NET, since that provides a "native" getaddrinfo (and Python knows that it does). So if Python 2.3 is going to be compiled with VC.NET, I would not worry about the fallback implementation at all.
Author: Just van Rossum (jvr) *
Date: 2003-05-03 18:07
Logged In: YES user_id=92689
FWIW, I opened a bug regarding this very same issue (I didn't know if this item): python.org/sf/731644. That item now also contains a new patch that takes care of locking.
Author: Just van Rossum (jvr) *
Date: 2003-05-09 07:57
Logged In: YES user_id=92689
The last patch from python.org/sf/731644 was accepted by MvL, and consequently checked in as socketmodule.c rev. 1.265.