[FIXED] Any better way to flatten and rebuild nested dict

Issue

One solution posted by myself, but I'm not satisfied with it, which is too complicated.

Sometimes, we need to flatten one dict, and sometimes in reverse, for example: a nested dict looks like this:

{'a': 'A', 'b': {'c': 'BC', 'd': [4, 44, 444]}}

and a flatten dict:

{'a': 'A', 'b__c': 'BC', 'b__d': [4, 44, 444]}

as we can see, in this case, we use double underscore to represent the nested level.

I wrote two functions to do these, Please check the answer I posted, but I still prefer any robost 3rd party lib or any other idea to make these two functions eaiser?

And besides, I also wanna know what is the right way to flatten value, which type is a list filled with dict objects? Let's put it this way, what should a flatten dict should be, if we replace the number 4, 44, 444 with dicts intead.

Solution

I wrote two functions to convert flatten and nested dict, but I prefer to find some robost lib to handle these cases.

def flatten(nested_dict, result=None, prefix=''):
    from collections import OrderedDict
    if result is None:
        result = dict()
    for k, v in nested_dict.items():
        new_k = '__'.join((prefix, k)) if prefix else k
        if not (isinstance(v, dict) or isinstance(v, OrderedDict)):
            result.update({new_k: v})
        else:
            flatten(v, result, new_k)
    return result


def rebuild(flatten_dict, result=None):
    from collections import defaultdict
    import json

    def tree():
        return defaultdict(tree)

    def rec(keys_iter, value):
        _r = tree()
        try:
            _k = next(keys_iter)
            _r[_k] = rec(keys_iter, value)
            return _r
        except StopIteration:
            return value

    if result is None:
        result = dict()

    for k, v in flatten_dict.items():
        keys_nested_iter = iter(k.split('__'))
        cur_level_dict = result
        while True:
            try:
                k = next(keys_nested_iter)
                if k in cur_level_dict:
                    cur_level_dict = cur_level_dict[k]
                else:
                    cur_level_dict[k] = json.loads(json.dumps(rec(keys_nested_iter, v)))
            except StopIteration:
                break

    return result


if __name__ == "__main__":
    my_dict = {'a': 'A',
               'b':
                   {
                       'd': [4, 44, 444],
                       'c': 'BC'
                   }
               }

    my_flatten_dict = flatten(my_dict)
    assert my_flatten_dict == {'a': 'A', 'b__c': 'BC', 'b__d': [4, 44, 444]}

    my_nested_dict = rebuild(my_flatten_dict)
    assert my_nested_dict == {'a': 'A', 'b': {'c': 'BC', 'd': [4, 44, 444]}}

Answered By - Menglong Li