[FIXED] how to replace non ascii char in python

Issue

I need to replace non ASCII char like ¾ in Python but I get

SyntaxError: Non-ASCII character '\xc2' in file test.py but no encoding declared; see http://www.python.org/peps/pep-0263.html for details`

After following the directions on the webpage, I am getting

UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 449: ordinal not in range(128)

Here's my code:

data = data.replace(u"½", u"1/2")
data = re.sub(u"¾", u"3/4", data, flags=re.DOTALL)

What do I need to change in my code?

---

my file is:

#!/usr/bin/python

with codecs.open("file.txt", "r", "utf8") as myfile:
    data = myfile.read()

data = data.replace(u"½", u"1/2")

file.txt is:

hello world ½

Solution

You're reading into the local variable data as bytes but then treating data it like it's already a unicode object.

Change this:

with open(file_name, "r") as myfile:
    data = myfile.read()

To this:

import io

with io.open(file_name, encoding="utf8") as myfile:
    data = myfile.read()

Answered By - wim