[Python-Dev] Adding bytes.frombuffer() constructor to PEP 467 (was: [Python-ideas] Adding bytes.frombuffer() constructor (original) (raw)

Chris Barker chris.barker at noaa.gov
Fri Oct 21 17:57:38 EDT 2016

On Thu, Oct 20, 2016 at 11:48 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>>>>> len(getbuiltinmethods()) >> 230 > > So what? No one looks in all the methods of builtins at once.

Yes, Python implementation developers do, which is why it's a useful part of defining the overall "size" of Python and how that is growing over time.

sure -- but of course, the trick is that adding *one" new method is never a big deal by itself.

I'm confused though -- IIUC, you are proposing adding a iobuffers module to the std lib -- how is that not growing the "size" of Python?

I'm still confused about the "io" in "iobuffers" -- I've used buffers a lot -- for passing data around between various C libs -- numpy, image processing, etc... I never really thought of it as IO though. which is why a simple frombuffer() seems to make a lot of sense to me, without any other stuff. (to be honest, I reach for Cyton these days for that sort of thing though)

and we

make it easier for educators to decide whether or not they should be

introducing their students to the new capabilities.

advanced domain specific use cases (see

http://learning-python.com/books/python-changes-2014-plus.html for one generalist author's perspective on the vast gulf that can arise between "What professional programmers want" and "What's relevant to new programmers")

thanks for the link -- I'll need to read the whole thing through -- though from a glance, I have a slightly different perspective, as an educator as well:

Python 3, in general, is harder to learn and less suited to scripting, while potentially more suited to building larer systems.

I came to this conclusion last year when I converted my introductory class to py3.

Some of it is the redundancy and whatnot talked about in that link -- yes, those are issue or me. But more of it is real, maybe important change. Interestingly, the biggest issue with the transition: Unicode, is one thing that has made life much easier for newbies :-)

But the big ones are things like:

The more to be iterable focused rather than sequence focused -- iterables really are harder to wrap one's head around when you are first learning. And I was surprised at how often I had to wrap list() around stuff when converting my examples and exercise solutions.

I've decided to teach the format() method for string formatting -- but it is harder to wrap your head around as a newbie.

Even the extra parens in print() makes it a bit harder to script() well.

Use with: -- now I have to explain context managers before they can even read a file.. (or gloss over it and jsut say " copy this code to open a file"

Anyway, I've been meaning to write a Blog post about this, that would be better formed, but you get the idea.

In short, I really appreciate the issues here -- though I really don't see how adding one method to a fairily obscure builtin really applies -- this is nothing like having three(!) ways to format strings.

Which is more comprehensible and discoverable, dict.setdefault(), or

collections.defaultdict()?

Well, setdefault is Definitively more discoverable! not sure what your point is.

As it happens, the homework for my intro class this week can greatly benefit from setdefault() (or defaultdict() ) -- and in the last few years, far fewer newbies have discovered defaultdict() for their solutions. Empirical evidence for discoverability.

As for comprehensible -- I give a slight nod to .setdefault() - my solution to the HW uses that. I can't say I have a strong argument as to why -- but having (what looks like) a whole new class for this one extra feature seems a bit odd, and makes one look carefully to see what else might be different about it...

Micro-optimisations like dict.setdefault() typically don't make sense in isolation - they only make sense in the context of a particular pattern of thought. Now, one approach to such patterns is to say "We just need to do a better job of teaching people to recognise and use the pattern!". This approach tends not to work very well - you're often better off extracting the entire pattern out to a higher level construct, giving that construct a name, and teaching that, and letting people worry about how it works internally later.

hmm -- maybe -- but to me, that example isn't really a pattern of thought (to me) -- I actually remember my history of learning about setdefault(). I found myself writing a bunch of code something like:

if key not in a_dict: a_dict[key] = something a_dict['key'].somethign_or_other()

Once I had written that code a few times, I thought: "There has got to be a cleaner way to do this", looked at the dict methods and eventually found setdefault() (took an embarrassingly long time). I did think -- "this has got to be a common enough pattern to be somehow supported" but I will say that it never, ever dawned on me to think: "this is got to be a common enough pattern that someone would have made a special kind of dictionary for it" -- I thought of it as a feature you'd want with a dict -- into a different kind of dict.

So in the end, if one were to ask ME whether Python should have a .setdefault() method on dict, or an DefaultDict object -- I'd say the method.

bytes and bytarray are already low-level objects -- adding low-level > functionality to them makes perfect sense.

They're not really that low level. They're relatively low level (especially for Python),

sure -- but that is the context here.

> And no, this is not just for asycio at all -- it's potentially useful > for any byte manipulation.

Anyway, I think frombuffer() would be a very nice thing to have, and where it goes is secondary.

Yes, which is why I think the end goal should be a public iobuffers module in the standard library.

I guess I'd need to see a proposal for what would go in that module to have any opinion on whether it would be a good idea. Is anyone actually working on such a proposal?

-CHB

--

Christopher Barker, Ph.D. Oceanographer

Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception

Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20161021/e2ff7772/attachment.html>

More information about the Python-Dev mailing list