[Python-ideas] [Python-Dev] hello, new dict addition for new eve ? (original) (raw)

James Hutchison jamesghutchison at gmail.com
Mon Jan 2 19:09:13 CET 2012

Here are some "utility" functions I had to write in my current project:

add entries from d2 to d1

supports only dict and numeric types for now

recursive, applies function to all subdicts

def addDictEntries(d1, d2)

take the max of entries from d2 to d1

supports only dict and numeric types for now

recursive, applies function to all subdicts

def maxDictEntries(d1, d2):

take the min of entries from d2 to d1

supports only dict and numeric types for now

recursive, applies function to all subdicts

def minDictEntries(d1, d2)

merges entries from d2 to d1 like update does

if an entry already exists and the values differ then throws an exception

recursive, applies function to all subdicts

def mergeDictEntriesSafe(d1, d2)

traverses through the dict and converts all defaultdicts to dicts

this is done so that missing elements will throw an exception rather

than creating a default entry

also, defaultdicts aren't picklable so this is required in order to

serialize a dict def unDefaultDict(d)

same as unDefaultDict but isn't recursive. Only applies to top level keys

def unDefaultDictShallow(d)

The usage for all this varies but I found that combining / adding dicts together becomes very prevalent when you split datasets up for multihreading/multiprocessing purposes and need to recombine it later.

so I propose that a "combine" function is added to the dict class and all dict subclasses. The function would have the following arguments:

def combine(self, d2, conflictFunc=None):

If combineFunc is a function, then that function is called and the result is used. val = conflictFunc(k1,k2); If combineFunc is a string, then self's key value calls conflictFunc and the new value is the return value

def combine(self, d2, combineFunc=None):
    if combineFunc == None:
        for k,i in d2.items():
            if(not k in self):
                self[k] = copy.deepcopy(i);
            else:
                if(hasattr(i,'keys')):
                    self[k].combine(i);
                else:
                    if self[k] != i:
                        raise someUsefulException();
        return;
    if (type(combineFunc) == str):
        for k,i in d2.items():
            if(not k in self):
                self[k] = copy.deepcopy(i);
            else:
                if(hasattr(i,'keys')):
                    self[k].combine(i,combineFunc);
                else:
                    self[k] = getattr(self[k],combineFunc)(i);
    else:
        for k,i in d2.items():
            if(not k in self):
                self[k] = copy.deepcopy(i);
            else:
                if(hasattr(i,'keys')):
                    self[k].combine(i,combineFunc);
                else:
                    self[k] = combineFunc(self[k],i);

examples:

d.combine(d2); # update with d2 but throw an exception if any keys have different values d.combine(d2, myCustomFunc); # if conflict, then result is = myCustomFunc(v1,v2) which would be defined elsewhere d.combine(d2, 'add'); # if conflict, then result is = v1 + v2

print(d1) {'a': 'b', 'c': {'g': 5}} print(d2) {'a': 'ecause', 'c': {'g': 10}} d1.combine(d2,'add'); print(d1) {'a': 'because', 'c': {'g': 15}}

I think doing so would be more powerful and less ambiguous than trying to give dict a '+'

A second function I suggest adding is asDict(). This would return a dictionary with all sub dictionaries-types converted to the dict type. I.E. all defaultdicts and counters become just plain dicts. This would be the same as my unDefaultDict function. It could take an optional argument to do the conversion only to the shallow keys

def asDict(self, shallow=False)

More information about the Python-ideas mailing list