Issue 7735: socket.create_connection() creates IPv6 DNS requests even when built with --disable-ipv6 (original) (raw)

Issue7735

Created on 2010-01-18 22:51 by Evan.Teran, last changed 2022-04-11 14:56 by admin. This issue is now closed.

Messages (16)
msg98036 - (view)	Author: Evan Teran (Evan.Teran)	Date: 2010-01-18 22:51
I have encountered an issue where python will do a AAAA request even when built without IPv6. This becomes an issue because on some configurations this seems to cause a 5 second delay on DNS lookups (that is a separate issue of course). Basically here is what I am seeing: #!/usr/bin/python import urllib2 print urllib2.urlopen('http://python.org/').read(100) results in the following: 0.000000 10.102.0.79 -> 8.8.8.8 DNS Standard query A python.org 0.000023 10.102.0.79 -> 8.8.8.8 DNS Standard query AAAA python.org 0.005369 8.8.8.8 -> 10.102.0.79 DNS Standard query response A 82.94.164.162 5.004494 10.102.0.79 -> 8.8.8.8 DNS Standard query A python.org 5.010540 8.8.8.8 -> 10.102.0.79 DNS Standard query response A 82.94.164.162 5.010599 10.102.0.79 -> 8.8.8.8 DNS Standard query AAAA python.org 5.015832 8.8.8.8 -> 10.102.0.79 DNS Standard query response AAAA 2001:888:2000:d::a2 looking at socket.py in create_connection() (line 500 on my python 2.6.4 stdlib) the code is like this: for res in getaddrinfo(host, port, 0, SOCK_STREAM): af, socktype, proto, canonname, sa = res sock = None try: sock = socket(af, socktype, proto) if timeout is not _GLOBAL_DEFAULT_TIMEOUT: sock.settimeout(timeout) sock.connect(sa) return sock except error, msg: if sock is not None: sock.close() The 3rd argument is the socket type, which is set to 0 which apparently means unspecified. It seems to me that if python is built without IPv6 support it should instead pass AF_INET since even if it does get an IPv6 response it can't possibly use it.
msg98037 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2010-01-18 23:13
Can you propose a patch? I personally consider this not worth the effort. Unless there is a DNS problem, this cases shouldn't really be noticable, as the socket module will immediately report that IPv6 is not supported (without even consulting the operating system). So if there is a problem, it must be with DNS, in which case I would rather recommend to fix the DNS setup.
msg172679 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-11 19:15
they are noticeable! If the sock.connect fails with the IPv4 address, it will then try to connect to an IPv6 address (which fails with an TypeError then). The original error is hidden!
msg172701 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-10-11 22:12
I would suggest closing as won't fix. The way I understand it, --disable-ipv6 simply allows building on systems without IPv6 support. It's not meant to disable IPv6 requests on an IPv6-compliant system. If your IPv6 connectivity doesn't work properly, you should disable it in your system configuration, not try to recompile Python!
msg172703 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-11 22:41
The fact is I have disabled IPv6 with python's --disable-ipv6 switch. If you think that this switch shouldn't be supported anymore, either remove it or document it as obsolete. Telling me to disable IPv6 in my system configuration is just arrogant. I may not be able to do so and I might have had good reasons to disable IPv6 support in python. It might even be the case that IPv6 support is working on my machine. So, that's a bad argument. And the fact that I may be able to disable IPv6, won't change the value of socket.has_ipv6 magically for me.
msg172705 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-10-11 22:44
> Telling me to disable IPv6 in my system configuration is just > arrogant. No, it's perfectly reasonable. On the other hand, recompiling Python without IPv6 to workaround a system-level problem with IPv6 is totally silly. Do you recompile all other programs which might attempt IPv6 communications?
msg172706 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-11 22:54
Why do you think I'm disabling IPv6 because I have a "system level problem"? I am not recompiling python to workaround system level problems. And I don't recompile any other program. The problem is in python, it can be fixed with a one line patch. but you prefer to insult me instead of admitting that there is a problem.
msg172707 - (view)	Author: Antoine Pitrou (pitrou) *	Date: 2012-10-11 22:55
> Why do you think I'm disabling IPv6 because I have a "system level > problem"? So please explain the problem you're having.
msg172708 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-11 23:00
The OPs description is pretty clear. There's no good reason to ask for IPv6 addresses if IPv6 is disabled. The create_connection will try them all, and if connections to IPv4 addresses fail, you'll end up with a TypeError.
msg172719 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-12 07:12
It's not quite true what I wrote. Actually you get a "error: getsockaddrarg: bad family" error. But regardless of the error, there just is no need to ask for IPv6 addresses!
msg172726 - (view)	Author: Martin v. Löwis (loewis) *	Date: 2012-10-12 07:44
The switch --disable-ipv6 is supported and works as intended. It is not the intention of the switch to disable AAAA lookups. Instead, it disables support for IPv6 sockets. Requesting that the switch disables any code that somehow deals with IPv6 is unreasonable - it would also affect the ipaddress module and urllib.parse. So closing this as won't fix.
msg172729 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-12 08:02
The switch disables support for IPv6 sockets, and since IPv6 support is disabled, there is no need to try to lookup IPv6 addresses in create_connection. They just cannot be used afterwards. I didn't request that the switch disables any code that somehow deals with IPv6. I'm just talking about that one function! You refuse to acknowledge the problem based on broken assumptions and bogus arguments.
msg172731 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-10-12 08:12
btw AAAA lookups do not work if python is configured with --disable-ipv6, see http://bugs.python.org/issue16208
msg178634 - (view)	Author: STINNER Victor (vstinner) *	Date: 2012-12-31 01:10
I spoke with schmir on IRC about this issue because he considers the request as valid, and he doesn't understand why the issue was closed with "wont fix". -- schmir use case: - a network where IPv6 doesn't work (for an unknown reason) - a Linux system with IPv6 support and a network interface with a local link address (fe80:...), maybe also with a IPv6 public address (it doesn't really matter for the use case) - Python configured with --disable-ipv6 "to ensure that Python will not send any IPv6 request" (according to schmir) (./configure --disable-ipv6) - the DNS client is configured to return IPv4 addresses and then IPv6 addresses - call socket.create_connection((hostname, 80)) - the computer cannot connect to this host in IPv4 nor IPv6 Current behaviour: - Python calls getaddrinfo() with ai_family=0 (AF_UNSPEC) - the DNS client (of the glibc) asks for A (IPv4) and AAAA (IPv6) DNS records - Python tries to connect to the IPv4 address and fails (connection error for example) - Python tries to connect to the IPv6 address and fails with a TypeError because AF_INET6 is not supported - the final exception is misleading: TypeError instead of a connection error schmir expects that --disable-ipv6 would "really" disable IPv6 *everywhere* in Python, which is wrong. Python may still get IPv6 adddresses from getaddrinfo() if the system does somehow support IPv6. schmir proposes the change create_connection() to really disable IPv6 there: force AF_INET if socket.has_ipv6 is False. https://github.com/SiteSupport/gevent/commit/9b1bccffc11455112076189f35023291cf97a2a2 I agree that something is wrong here, but I also disagree with schmir. In my opinion, --disable-ipv6 is the wrong answer to the initial problem (ensure that the computer will not use IPv6 because IPv6 doesn't work somewhere in the network). The correct solution is to disable IPv6 system wide. See for example https://wiki.archlinux.org/index.php/IPv6#Disable_IPv6 to disable IPv6 system wide on Linux. It's simpler and safer to disable IPv6 system wide instead of fixing all applications. Other applications will have the same issue (ask AAAA DNS records) in the same use case. "wont fix" is the correct status for this issue: we agree that there is a bug, but it will not be fixed, because --disable-ipv6 is the wrong solution. -- Note: ai_flags=AI_ADDRCONFIG would not change anything because the network interface has an IPv6 local link address. See http://sourceware.org/bugzilla/show_bug.cgi?id=12377 for a discussion about this in the glibc. Note: schmir uses Gentoo and disabled the ipv6 use flag of its python package (USE=-ipv6).
msg178643 - (view)	Author: Daniel Shahaf (danielsh)	Date: 2012-12-31 04:51
Victor, thanks for the summary. I view things a little differently, so I'll offer my POV: 1. The OPs (Evan and Ralf) build with --disable-ipv6. 2. --disable-ipv6 disables support for IPv6 sockets. (as per Martin, and AFAIK as per the common meaning of that flag in other configure scripts.) It does not disable AAAA DNS queries in stdlib DNS client modules. It does not disable IPv6 support in the 'ipaddress' module. It does not signify that IPv6 is/isn't supported by libc. It is orthogonal to whether any interface has IPv6 addresses configured. 3. socket.create_connection() requests IPv6 addresses when Python was built with --disable-ipv6. 4. The OPs claim that (3) is a bug, reasoning that, as create_connection() should not try to connect(2) to IPv6 addresses (per (2)), it's pointless, wasteful, or wrong for it to request them in the first place. 5. Ralf provided a one-line patch for (4): https://github.com/SiteSupport/gevent/commit/9b1bccffc11455112076189f35023291cf97a2a2 In other words, the issue revolves around configuring Python not to create IPv6 sockets on IPv6-capable systems. HTH
msg178691 - (view)	Author: Ralf Schmitt (schmir)	Date: 2012-12-31 15:24
Daniel is pretty much spot on, thanks for that! Regarding the use case: I disabled IPv6 system wide when building packages via gentoo's USE flag. I didn't do anything in order to configure IPv6 or remove it. My local network interface having a local link address is a result of that. I've been told multiple times to fix my setup. And I said multiple times that the setup is not at fault here. > schmir expects that --disable-ipv6 would "really" disable IPv6 > *everywhere* in Python, which is wrong. Python may still get IPv6 > adddresses from getaddrinfo() if the system does somehow support > IPv6. I did not say that. In fact I wrote in : """I didn't request that the switch disables any code that somehow deals with IPv6. I'm just talking about that one function!""" > Python may still get IPv6 adddresses from getaddrinfo() if the > system does somehow support IPv6. That would be nice. But that's currently not the case. see http://bugs.python.org/issue16208 haypo, you also keep talking of an initial problem, which you assume must be there somewhere in my network - which I try to workaround with --disable-ipv6. There is no problem on my side that I'm trying to fix. It's just that I have disabled IPv6 via gentoo's USE flags, since I don't use it. I've also been telling this multiple times, I don't know why I'm being completely ignored here. > "wont fix" is the correct status for this issue: we agree that there > is a bug, but it will not be fixed, because --disable-ipv6 is the > wrong solution. again. it can't be a solution since there is no problem unless this option is being used and then there's a problem in python.