[FIXED] Python3 - 'utf-16-le' codec can't encode character '\udce2' in position 12: surrogates not allowed

Issue

I recently changed servers that my python script was running on and I now get the this error:

'utf-16-le' codec can't encode character '\udce2' in position 12: surrogates not allowed

Script was running fine on previous server. The script takes commandline arguments and uses mutagen for mp3 tag processing. Here's part of the script itself:

    if args.title:
        audio.add(TIT2(encoding=3, text=u"" + args.title))
    if args.artist:
        audio.add(TPE1(encoding=3, text=u"" + args.artist))
    if args.album:
        audio.add(TALB(encoding=3, text=u"" + args.album))
    if args.year:
        audio.add(TYER(encoding=3, text=u"" + args.year))
    if args.track:
        audio.add(TRCK(encoding=3, text=u"" + args.track))
    if args.genre:
        audio.add(TCON(encoding=3, text=u"" + args.genre))
    if args.comment:
        audio.add(COMM(encoding=3, lang='eng', desc=u'', text=u"" + args.comment))
    if args.TXXX:
        # We used append when created TXXX argument, so we have an array of all TXXX tags
        # Tag is given in the form of --TXXX "SCRIPTURETEXT:Job 1:1" so we will iterate through them
        # and split on first colon. This should give tag's custom name (desc) in first element of array and
        # tag contents (text) in the second element
        for x in args.TXXX:
            tmptag = x.split(':', maxsplit=1)
            audio.add(TXXX(encoding=3, desc=u"" + tmptag[0], text=u"" + tmptag[1]))
    if args.picture:
        # Embed cover-art in ID3 metadata
        img_path = "/var/www/apps/audiodb/_resources/images/speakers/" + args.picture
        if path.isfile(img_path): # check if file exists
            audio.add(APIC(encoding=3, mime='image/jpg', type=3,
                          desc=u'Cover', data=open(img_path, 'rb').read()))

    audio.save(filename,v2_version=3)
    print ("MP3 tags successfully written.")

Solution

I discovered the answer to the problem. My python script was called via PHP using the exec command. When the python script parsed the commandline arguments, one of the fields contained the character – which caused the UTF error message. So, in my php script, I added these lines before I called the exec command.

$locale = 'en_US.utf-8';
setlocale(LC_ALL, $locale);
putenv('LC_ALL='.$locale);

Thanks, to the Unicode to PHP exec post for helping figure out my issue.

Answered By - user5919866