[FIXED] Inconsistent json serializer in django.core.serializers (unescaped double quotes)

Issue

If I take a simple ListView, and wish to return JSON:

from django.core import serializers
from django.http import JsonResponse
class MyListView(ListView):

def get_queryset(self):
    return Thing.objects.all()

def get(self, request, *args, **kwargs):
    data = serializers.serialize("json", self.get_queryset())
    return JsonResponse(data, status=200, safe=False)

This will give me "valid" json. However, the json will be delivered like this:

It's not only an eyesore, if you then try to print that (eg via console.log), those """ on lines 2, 8, 11 will become \". (This has been noted in other questions). To parse that json in the browser, you need to run it through JSON.parse twice (or transpile the escaped chars another way).

On the other hand, if I do the below code (and this is my present hack for getting around the problem):

data = list(map(lambda thing: {"pk": thing.pk, <etc>}, self.get_queryset()))
return JsonResponse(data, status=200, safe=False)

This gives me valid json without the additional " chars (excuse the cat pic):

My question: clearly, the browser is getting different inputs. Yes, they're both valid json (they pass validators) - but the first won't clear JSON.parse(). Is there any way to force the core serializer to conform to the second json output format? At present I'm having to write a lot of boilerplate serializer code that's making my classes look messy.

Solution

The inbuilt django serializer outputs a String. The django JsonpResponse takes key+value data and stringifies it with the right HTTP headers added. That's fine if you pass a dictionary or list, but if you pass a String that aready has " chars in it (like the output of the core serializer does), JsonResponse blithely escapes the existing " because you can't have " in an unescaped string of course. Yay teamwork.

Obviously, with this deeper understanding of what's going on, it's a bit stupid to expect the serializer to output anything other than a string - that's what serializers do, right?

So then I might think, can I get the JsonResponse encoder to ignore the " somehow in the string? JsonResponse in 1.9 can provide encoder arguments.

Yes, but none of the arguments include magically_skip_double_quotes=true, and even if it did it wouldn't be able to parse {"quotation": "\"Fine weather!\" said Bob."} properly.

And at that point, you kind of realise that you actually want to just:

from django.forms import model_to_dict
data = [model_to_dict(thing) for thing in things]

And that's probably the easiest way of going about it.

Answered By - Escher