[FIXED] How can I change the encoding in Spyder to UTF - 8?

Issue

I want to change the encoding in Spyder to UTF - 8.

Could someone explain me how I reach this, please? Thanks for helping me!

EDIT

def deleteDoubleValues (derivativeAngle):
    
    position = []
    cleanValues = [] #leeres Array erstellen 
    cleanValues.append(derivativeAngle[0]) 
    for i in range (len(derivativeAngle)-1):  
        
        if format(derivativeAngle[i],'.4f') != format(derivativeAngle[i+1],'.4f'):
            #i=i+1
            position.append(i)
            cleanValues.append(derivativeAngle[i+1]) 
    if format(derivativeAngle[i],'.4f') != format(derivativeAngle[i+1],'.4f'):
        position.append(i+1)
        
    return cleanValues, position

def deletePositionAccel(position,strideData):
    xAcceleration = strideData["accel x"].to_numpy()
    yAcceleration = strideData["accel y"].to_numpy()
  
    xValues = []
    yValues = []
    
    for i in range(len(position)):
        xValues.append(xAcceleration(position[i]))
        yValues.append(yAcceleration(position[i]))
  
    return xValues, yValues

When I subsequently run def deletePositionAccel() in the main, I always get the following error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 1382: invalid continuation byte

I don't know why, because the CSV file I work with is in UTF - 8.

derivativeAngle=[ 9.88   -2.12   29.88   -2.12    9.88   16.88    9.88    4.88    9.88
   -2.12    9.88   16.88   10.88    9.88   10.88    9.88    4.88    3.88
   -2.12    9.88    3.88   10.88   10.88    9.88    9.88   10.88   10.88
   15.88   16.88   16.88   22.88   34.88   41.88   53.88   60.88   -2.12
   72.88   84.88   97.88  110.88  128.88  141.88  159.88  172.88  191.88
  203.88  222.88  241.88  266.88  272.88  297.88  303.88  322.88  303.88
  279.88  240.88  166.88   97.88   22.88  -46.12  -64.12  -90.12 -139.12
 -134.12 -164.12 -190.12   -2.12 -202.12 -226.12 -221.12 -227.12 -234.12
 -214.12 -214.12 -215.12 -215.12 -208.12 -196.12 -189.12 -183.12 -184.12
 -189.12 -183.12 -177.12 -165.12 -152.12 -146.12   -2.12 -152.12 -170.12
 -171.12 -177.12 -171.12 -177.12 -170.12 -159.12 -133.12 -108.12  -77.12
  -52.12  -27.12   -8.12   21.88   47.88   -2.12   73.88   84.88   91.88
  109.88  122.88  103.88  110.88  110.88  109.88  109.88  110.88   91.88
   78.88   66.88   53.88   47.88   34.88   29.88   -2.12   22.88   22.88
   15.88   16.88   10.88    3.88    9.88    4.88   -2.12   16.88   -2.12
    3.88  -15.12   -8.12  -15.12   -8.12   -8.12   -2.12   -8.12   -8.12
   -9.12   -8.12   -8.12   -2.12   -9.12]

This is how my csv - file looks like:

    xAcceleration=[  0.11   0.14   0.22   0.23   0.23   0.2    0.16   0.17   0.12   0.13
       0.13   0.1    0.12   0.13   0.12   0.12   0.12   0.12   0.03   0.03
       0.03   0.05   0.     0.04   0.03   0.     0.03  -0.03  -0.04  -0.07
      -0.15  -0.18  -0.14  -0.28  -0.31  -0.34  -0.52  -0.52  -0.64  -0.45
      -0.72  -0.77  -0.9   -1.12  -0.93  -0.75  -0.3   -0.54  -0.87   0.33
       1.46  -2.53  -6.01  -7.76  -5.53 -15.57 -17.11 -16.93 -18.82 -15.84
     -15.49 -15.   -14.66 -13.86 -13.09 -12.26 -11.5  -10.17 -10.17  -9.26
      -8.3   -7.19  -5.31  -4.49  -3.97  -3.12  -2.23  -1.3   -0.38   0.51
       0.75   0.99   1.63   3.33   4.19   4.86   5.23   5.6    6.16   6.95
       6.95   7.87   9.95  11.15  12.17  13.14  14.69  15.34  15.72  15.64
      15.54  15.45  15.23  14.64  14.    14.    13.02  11.84   5.01  -4.19
     -16.76  -8.5    1.82  -0.42  -1.1    0.03   0.53   0.27  -0.32  -0.46
      -0.46  -0.36  -0.09   0.09  -1.08  -0.74  -0.74  -0.43  -0.84  -0.79
      -0.84  -0.89  -0.83  -0.82  -0.85  -0.85  -0.91  -0.97  -0.94  -0.88
      -0.79  -0.73  -0.77  -0.77  -0.75  -0.76  -0.71  -0.7   -0.64  -0.66]

.

yAcceleration=[[-1.41 -1.32 -1.21 -1.13 -1.13 -1.03 -0.96 -0.86 -0.85 -0.82 -0.82 -0.74
     -0.55 -0.52 -0.46 -0.42 -0.33 -0.31 -0.21 -0.14 -0.14 -0.11  0.    0.17
      0.31  0.49  0.6   0.91  1.07  1.29  1.54  2.    2.04  1.97  2.04  2.
      2.45  2.45  2.85  3.    3.02  3.11  3.99  3.85  4.11  3.5   2.75  4.45
      5.62  5.43  1.09  8.58  9.26  7.84  4.69  4.17  6.89  8.85  7.86  6.61
      5.81  5.48  5.3   4.63  4.47  4.55  4.98  6.68  6.68  7.54  8.15  8.44
      8.6   8.57  8.54  8.41  8.2   7.76  7.45  7.58  7.44  6.98  6.62  6.53
      6.33  6.04  5.58  4.8   4.37  3.85  3.85  3.04  1.62  0.97  0.63  0.32
     -0.38 -0.76 -1.25 -2.07 -2.39 -2.47 -2.37 -1.34 -0.61 -0.61 -0.33 -0.69
      2.73  2.48  4.25  2.13  0.12  0.93  1.76  1.22  1.62  1.54  1.19  1.15
      1.53  1.07  1.15  1.47  0.67  0.66  0.66  0.63  0.13  0.11  0.01 -0.06
      0.    0.03  0.11  0.11  0.2   0.25  0.34  0.31  0.32  0.27  0.31  0.31
      0.4   0.42  0.39  0.35  0.25  0.2 ]

Stack Trace:

Traceback (most recent call last):

  File "C:\Users\nadin\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_code
    self.showtraceback(running_compiled_code=True)

  File "C:\Users\nadin\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2043, in showtraceback
    value, tb, tb_offset=tb_offset)

  File "C:\Users\nadin\Anaconda3\lib\site-packages\IPython\core\ultratb.py", line 1385, in structured_traceback
    self, etype, value, tb, tb_offset, number_of_lines_of_context)

  File "C:\Users\nadin\Anaconda3\lib\site-packages\IPython\core\ultratb.py", line 1297, in structured_traceback
    elist = self._extract_tb(tb)

  File "C:\Users\nadin\Anaconda3\lib\site-packages\IPython\core\ultratb.py", line 1278, in _extract_tb
    return traceback.extract_tb(tb)

  File "C:\Users\nadin\Anaconda3\lib\traceback.py", line 72, in extract_tb
    return StackSummary.extract(walk_tb(tb), limit=limit)

  File "C:\Users\nadin\Anaconda3\lib\traceback.py", line 363, in extract
    f.line

  File "C:\Users\nadin\Anaconda3\lib\traceback.py", line 285, in line
    self._line = linecache.getline(self.filename, self.lineno).strip()

  File "C:\Users\nadin\Anaconda3\lib\linecache.py", line 16, in getline
    lines = getlines(filename, module_globals)

  File "C:\Users\nadin\Anaconda3\lib\linecache.py", line 47, in getlines
    return updatecache(filename, module_globals)

  File "C:\Users\nadin\Anaconda3\lib\linecache.py", line 137, in updatecache
    lines = fp.readlines()

  File "C:\Users\nadin\Anaconda3\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 1382: invalid continuation byte

Note: strideData stands for the imported CSV file.

Solution

Spyder is telling you that your source file is ISO 8859-1, a.k.a. Latin-1. That is a guess on Spyder's part, but probably an accurate one, based on a name or affiliation or copyright notice in a comment. That is not the problem. It is true that the Python interpreter expects source to be UTF-8 by default. But it will only complain if it encounters a wrong encoding in, say, a string literal. And when it does, it won't run. Instead it will give the error SyntaxError: Non-UTF-8 code starting with ... but no encoding declared. Your error is happening when your code runs and reads a file. Changing the encoding of the Python source will not fix that.

Your traceback is pretty explicit. The input file being complained of isn't UTF-8. Not valid UTF-8 anyway. I know you believe that it is, but (at the risk of appearing to restate the obvious) the message says "in the process of decoding a file that is expected to be UTF-8, a byte sequence was encountered at position 1382 that is not a valid UTF-8 byte sequence. The offending byte is 0xDF." The encoding library is an operational definition of UTF-8, so it is always correct on this topic.

It may be that the input file with the unexpected encoding is not the file you think it is.

It is a reasonable guess that the input file is actually, like your source, also Latin-1. In Latin-1 the byte 0xDF is ß and my hunch is the one before it is Ä; but there are too many possibilities to guess accurately.

Answered By - BoarGules