[Python-Dev] int/long FutureWarning (original) (raw)

Martin v. L�wis martin@v.loewis.de
30 Nov 2002 01:03:51 +0100

"Mark Hammond" <mhammond@skippinet.com.au> writes:

True. But can you come up with a reasonable example? A few of us have this problem, and almost everyone has agreed that hex constants are generally used in operations where the numeric value is not important, but the bit pattern is.

The problem is that you won't know whether an integer was hex. If I do

f = open("/tmp/large","w") f.seek(4278190080)

(which is 0xff000000l) then, under your proposed change, this might seek to a negative offset on some systems, instead of the positive offset.

Likewise

[None] * (1l << 32)

now gives an OverflowError. With your proposed change, it gives

[]

(actually, all calls to mul will change their behaviour).

I haven't seen a real example of where your scenario actually exists - not to mention that any such code would have to be new (as that constant does not work with "i" today) so therefore the author should use the new correct type code!

See, this is a central mistake: Under your proposed change, existing code will change. Whether or not the constant was original hexadecimal, or whether it was a constant at all, is not known at the time ParseTuple makes its decision.

I'm not sure if you are playing devil's advocate, or your contempt for our problem is real?

At the moment, I just want everybody to understand what the implications of proposed changes are. It seems we are still in disagreement about facts.

I don't know whether dropping the OverflowError test would cause real problems, since existing code likely does not trigger these errors (or else it would not work correctly now).

However, I do believe that extension authors will complain if their extension crashes if Python has failed to check a range, and converted a positive value to a negative one. Some underlying C library might react badly when it gets large negative numbers.

No - no one is trying to say any of that at all. The problem is simply breaking existing code. That code was not broken in the first place. We understand the rationale - just we also have lots of existing code.

The question is what the proposed correction should be. I'm in favour of using stricter semantics when the semantics changes (i.e. give more exceptions than before); authors of applications that can accept to drop some of the built-in tests then need to make explicit changes to drop them.

You seem to favour the reverse policy: drop the stronger checks by default, and give authors who want them an option to add them back. The problem with this strategy is that authors might not know that they have to change anything. With my strategy, they will find out automatically.

Well, I wouldn't be happy, but it wouldn't kill me, but would have to be a semi-blind replace to keep me sane (Maybe I would just write a script to convert all my .py files with hex constants to decimal - the bitwise ops will still get me, but locating them may be OK.)

That's what I did with h2py.py. It now generates decimal constants to silence the warning, and a unary - to get the negative value, since I cannot guesss whether the constant really was meant to be negative or not.

Of course, someone will have to update PyXPCOM in the Mozilla tree, and any thing else using hex constants in that way but not actively maintained may struggle and become unavailable for a little while.

That's why Python 2.3 adds the warning. There is no change to semantics, and over a year of time to make modification (when you update to Python 2.4).

Plenty of extension authors not on python-dev (including plenty of home-grown extensions taking win32con constants, for example), and obviously I can't speak for them. And as Jack's latest mail points out, supporting multiple versions of Python becomes almost impossible. The cost is real, and not able to be borne only by people on this list.

If you can't afford to break code now, accept the warning. If you don't like the warning, silence it with a global filter.

I estimate that Python users confronted with the warning will go through the same process as readers of this list:

When users truly understand the issues, and can make an educated decision what to do, the warning has served its purpose.

Regards, Martin