[FIXED] Dictionaries whose values are other dictionaries

Issue

I have a code whose purpose was to replace the "$oid", "$id" keys by keys without the $ sign. The function that does this takes a dictionary and returns a new dictionary. My file contains several "lines" and this is why I needed to do the parsing (below the code I will post a data sample):

import json 

def key_replacer(dictionary):
    new_dictionary = {}
    for k, v in dictionary.items():
        if k in ('$oid', '$date'):
            return v
        elif isinstance(v, dict):
            v = key_replacer(v)
        elif isinstance(v, list):
            tmp = []
            for itm in v:
                tmp.append(key_replacer(itm))
            v = tmp
        new_dictionary[k] = v
    return new_dictionary

def parse_ndjson(data):
    return [json.loads(l) for l in data.splitlines()]

with open('C:\\Windows\\files\\test.json', 'r', encoding="utf8") as handle:
    data = handle.read()
    dicts = parse_ndjson(data)

for d in dicts:
    new_d = key_replacer(d)
    json_string=json.dumps(new_d)
    print(json_string) #because I want a .json format at the end. 

For the following data sample, it works (I initially thought the error was because of the true or the number values):

{"_id":{"$oid":"5d1dc6ba0d27b66b64575bc5"},"quiz":[],"name":"MyName","startDate":{"$date":"2019-07-08T10:00:00Z"},"service":{"$oid":"5d160dc22549bc51860b2736"},"__v":0}
{"_id":{"$oid":"5d5be8a3221257336792a5ad"},"quiz":[],"name":"YourName","repeatable":true,"alertText":"3","startDate":{"$date":"2019-08-19T10:00:00Z"},"service":{"$oid":"5cf8f93af70b3e01bb970fe6"},"__v":0}

It returns:

{"_id": "5d1dc6ba0d27b66b64575bc5", "quiz": [], "name": "MyName", "startDate": "2019-07-08T10:00:00Z", "service": "5d160dc22549bc51860b2736", "__v": 0}
{"_id": "5d5be8a3221257336792a5ad", "quiz": [], "name": "YourName", "repeatable": true, "alertText": "3", "startDate": "2019-08-19T10:00:00Z", "service": "5cf8f93af70b3e01bb970fe6", "__v": 0}

Below there is a long example line where the code stopped working and it gave me AttributeError: 'int' object has no attribute 'items'.

{"_id":{"$oid":"5ae9595a97c91e0d18d50d82"},"name":"sample","settings":[{"title":"sampletitle","types":["easy","hard"]},{"title":"sampletitle2","types":["easy","hard"]}],"video":"https://www.youtube.com/watch?v=vvvvvvv","testTypes":[{"$oid":"5cad910"},{"$oid":"5cad9"}],"storage":"https://s3.eu-central-2.amazonaws.com/smthsmthsmth/","tempSettings":{"edit":true,"max":2,"min":1},"completed":true,"url":"https://something.com/pic.png","StartedAt":{"$date":"2021-04-01T06:31:41.786Z"}}

AttributeError                            Traceback (most recent call last)
<ipython-input-12-78e23a72f1de> in <module>
     24 
     25 for d in dicts:
---> 26     new_d = key_replacer(d)
     27     json_string=json.dumps(new_d)
     28     print(json_string)

<ipython-input-12-78e23a72f1de> in key_replacer(dictionary)
     11             tmp = []
     12             for itm in v:
---> 13                 tmp.append(key_replacer(itm))
     14             v = tmp
     15         new_dictionary[k] = v

<ipython-input-12-78e23a72f1de> in key_replacer(dictionary)
     11             tmp = []
     12             for itm in v:
---> 13                 tmp.append(key_replacer(itm))
     14             v = tmp
     15         new_dictionary[k] = v

<ipython-input-12-78e23a72f1de> in key_replacer(dictionary)
      3 def key_replacer(dictionary):
      4     new_dictionary = {}
----> 5     for k, v in dictionary.items():
      6         if k in ('$oid', '$date'):
      7             return v

AttributeError: 'str' object has no attribute 'items'

When using print right below for itm in v: I get a dictionary and a string instead of a dictionary:

{'title': 'sampletitle', 'types': ['easy', 'hard']} easy

Why does this happen and what can I do with my code?

Solution

The problem that is causing the code to fail:

The problem lies in the line for itm in v: of key_replacer function where you are you are iterating list items, and passing those items it to the key_replacer() function recursively. The key_replacer function is expecting to have a dictionary as a parameter but the list items that you are passing can be anything right? Just in case of the below instance:

{'title': 'sampletitle', 'types': ['easy', 'hard']}

your list corresponding to key 'types' contains strings and is not a dictionary instance.

Solution: Adding this in the top of your key_replacer() function

 isinstance(dictionary, dict)

Below is the key_replacer code that you can use, hopefully, it will work for you:

def key_replacer(inp):
    if not isinstance(inp, dict) and not isinstance(inp, list):
        return inp
    if isinstance(inp, dict):
        new_dictionary = {}
        for k, v in inp.items():
            if k in ('$oid', '$date'):
                return v
            elif isinstance(v, dict):
                v = key_replacer(v)
            elif isinstance(v, list):
                tmp = []
                for itm in v:
                    tmp.append(key_replacer(itm))
                v = tmp
            new_dictionary[k] = v
        return new_dictionary
    tmp = []
    for itm in inp:
        tmp.append(key_replacer(itm))
    return tmp

Answered By - Alok Raj