[FIXED] Serializing a function in python

Issue

I am trying to serialize a code and send it as a json...

def f(x): return x*x
def fi(x): return int(x[0])

code_string = marshal.dumps(fi.func_code)

jsn = {"code":code_string)
json.dumps(jsn) # doesnt work if code_string is from fi

So... the above code block works if my function is f(x)

But fails for fi(x)

Original exception was:

Traceback (most recent call last):
  File "/home/mohitdee/Documents/python_scala/rdd.py", line 41, in <module>
    send_data(json.dumps(jsn))
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
    return _iterencode(o, 0)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x83 in position 32: invalid start byte
[48001 refs]

How do i resolve this in python

Solution

Marshall is a binary protocol, i.e. a bunch of bytes with very custom interpretation. It's not text, it doesn't conform to in any particular text encoding. It is, for the most part, just a sequence of bits. If you absolutely need to embed those in a text protocol like JSON, you need to escape the bytes that don't make valid characters in the relevant encoding (to be safe, assume a subset of ASCII). The canonical solution is base64:

import base64

code_string = marshal.dumps(fi.func_code)
code_base64 = base64.b64encode(code_string)

jsn = {"code": code_base64}

Answered By - user395760