[FIXED] Namedtuple are not rendering properly in Flask

Issue

I am using Flask/Python application and facing issue in rendering namedtuple type data in jinja template.

I'm using versions

• Python 3.6
• Flask 1.1.2
• Jinja2 3.0.1

Below is my working sample code:

from collections import namedtuple
from flask import Flask, render_template, render_template_string

# ....
app = Flask(__name__)

Category = namedtuple(
'Category',
['name', 'priority', 'menus', 'has_next_level']
)

MenuItem = namedtuple(
    'MenuItem',
    ['title', 'url', 'target', 'id', 'parent_id', 'object_type']
)

# --- Sample input value render_template()
menu_content = {
    50: {
        'AGENTS': Category(
            name='AGENTS', 
            priority=1, 
            menus=[
                MenuItem(
                    title='Agent2', 
                    url='/monitoring/dashboard/6/agent/2', 
                    target=None, 
                    id=2, 
                    parent_id=None, 
                    object_type='agent'
                    ), 
                MenuItem(
                    title='Postgres Enterprise Manager Host', 
                    url='/monitoring/dashboard/6/agent/1', 
                    target=None, 
                    id=1, 
                    parent_id=None, 
                    object_type='agent'
                )], 
            has_next_level=True
        )
    }
}

output = None
with app.app_context():
    output = render_template_string(
        '{{ menu_content|tojson|safe }}',
        menu_content=menu_content
    )

print(output)

Output, I am getting:

{"50": {"AGENTS": ["AGENTS", 1, [["Agent2", "/monitoring/dashboard/6/agent/2", null, 2, null, "agent"], ["Postgres Enterprise Manager Host", "/monitoring/dashboard/6/agent/1", null, 1, null, "agent"]], true]}

Expected output:

{"50": {"AGENTS": {"has_next_level": true, "menus": [{"id": 2, "object_type": "agent", "parent_id": null, "target": null, "title": "Agent2", "url": "/monitoring/dashboard/6/agent/2"}, {"id": 1, "object_type": "agent", "parent_id": null, "target": null, "title": "Postgres Enterprise Manager Host", "url": "/monitoring/dashboard/6/agent/1"}]}

Am I missing anything here?

Solution

Thanks for adding details to your question, so I could reproduce your issue.

1. Solution to your problem
2. Explanation

Solution:

from collections import namedtuple
from flask import Flask, render_template, render_template_string
import json
from collections import OrderedDict

# Function that will be used to convert the namedtuple to a dict
def namedtuple_asdict(obj):
    if hasattr(obj, "_asdict"): # detect namedtuple
        return OrderedDict(zip(obj._fields, (namedtuple_asdict(item) for item in obj)))
    elif isinstance(obj, str): # iterables - strings
        return obj
    elif hasattr(obj, "keys"): # iterables - mapping
        return OrderedDict(zip(obj.keys(), (namedtuple_asdict(item) for item in obj.values())))
    elif hasattr(obj, "__iter__"): # iterables - sequence
        return type(obj)((namedtuple_asdict(item) for item in obj))
    else: # non-iterable cannot contain namedtuples
        return obj

# ....
app = Flask(__name__)

Category = namedtuple(
'Category',
['name', 'priority', 'menus', 'has_next_level']
)

MenuItem = namedtuple(
    'MenuItem',
    ['title', 'url', 'target', 'id', 'parent_id', 'object_type']
)

# --- Sample input value render_template()
menu_content = {
    50: {
        'AGENTS': Category(
            name='AGENTS', 
            priority=1, 
            menus=[
                MenuItem(
                    title='Agent2', 
                    url='/monitoring/dashboard/6/agent/2', 
                    target=None, 
                    id=2, 
                    parent_id=None, 
                    object_type='agent'
                    ), 
                MenuItem(
                    title='Postgres Enterprise Manager Host', 
                    url='/monitoring/dashboard/6/agent/1', 
                    target=None, 
                    id=1, 
                    parent_id=None, 
                    object_type='agent'
                )], 
            has_next_level=True
        )
    }
}

# Convert the dict of dict of namedtuple etc. to JSON string
menu_content_JSONified = json.dumps(namedtuple_asdict(menu_content))

output = None
with app.app_context():
    output = render_template_string(
        '{{ menu_content|safe }}',
        menu_content=menu_content_JSONified
    )

print(output)

Output:

{"50": 
    {"AGENTS": 
        {"name": "AGENTS", 
        "priority": 1, 
        "menus": [
            {"title": "Agent2", "url": "/monitoring/dashboard/6/agent/2", "target": null, "id": 2, "parent_id": null, "object_type": "agent"}, 
            {"title": "Postgres Enterprise Manager Host", "url": "/monitoring/dashboard/6/agent/1", "target": null, "id": 1, "parent_id": null, "object_type": "agent"}], 
        "has_next_level": true
        }
    }
}

By calling json.dumps you will get a JSON string. If you only use namedtuple_asdict you will get an OrderedDict.

Explanation:

The main problem is that the object you want to JSONify is not supported by default by the json.JSONEncoder (cf. the conversion table here: https://docs.python.org/3.6/library/json.html#py-to-json-table).

First, the object menu_content is a dict of dict of namedtuple.

Secondly, this namedtuple contains a str, an int, a list and a bool.

Thirdly, the list contains 2 namedtuples.

So we have to find a way to tell how to correctly convert this type of structure to JSON.

We could extend json.JSONEncoder (like the ComplexEncoder example from the docs (https://docs.python.org/3.6/library/json.html), or create a function that could deal with that.

Here, we use the function that will detect the type of each object and convert it in order to get the correct.

The credit of this function goes to @MisterMiyagi who posted it here: Serializing a nested namedtuple into JSON with Python >= 2.7. I just made a little modification for Python 3.

You noticed that your namedtuples were converted to list: that's because the conversion table used by the json.JSONEncoder (see link above) tells to convert Python's lists and tuples to JSON array.

And namedtuple is a sublcass of tuple, so your namedtuples are converted to JSON arrays.

Examples:

menu_item = MenuItem(
                    title='Agent2', 
                    url='url2', 
                    target=None, 
                    id=2, 
                    parent_id=None, 
                    object_type='agent'
                    )

menu_item_json_dumps = json.dumps(menu_item)
print(menu_item_json_dumps)
# OUTPUT: ["Agent2", "url2", null, 2, null, "agent"]

menu_item_as_dict_json_dumps = json.dumps(menu_item._asdict())
print(menu_item_as_dict_json_dumps)
# OUTPUT: {"title": "Agent2", "url": "url2", "target": null, "id": 2, "parent_id": null, "object_type": "agent"}

You can find some more information about that in @martineau answer here: Why doesn't JSONEncoder work for namedtuples?

Calling the function above will call _asdict() when needed. You could adapt it in order for examlpe to skip some None values or whatever.

Answered By - Rivers