[Python-Dev] LinkedHashSet/LinkedHashMap equivalents (original) (raw)

Delaney, Timothy C (Timothy) tdelaney at avaya.com
Thu Mar 10 06:00:20 CET 2005

Steven Bethard wrote:

def filterdups(iterable): seen = set() for item in iterable: if item not in seen: seen.add(item) yield item

Adding this to, say, itertools would cover all my use cases. And as long as you don't have too many duplicates, filterdups as above should keep memory consumption down better.

Thinking about this further - memory usage would be almost identical. By the time you completed the iterable, you would have built up exactly the same set internally - although probably not as memory efficient since it would be being built piecemeal. OTOH, an ordered set has a bit of extra memory for maintaining the order, so it's going to be pretty close.

The only thing this gains you (and it's significant) is the ability to work on any iterable lazily.

Tim Delaney

More information about the Python-Dev mailing list