问题现象

升级过程进行备份时候报 UnicodeEncodeError:'utf-8'codec can't encode characters in position 44-47:surrogates not allowed

[ INFO ]-[x2openEuler-upgrade]-[Backup]:get backup dir and files in sut.conf
Traceback(most recent call last):
File "/usr/bin/x2openEuler-upgrade",line 9,in
mainO
File "sut/workflow/actor.py",line 329,in actor.main
File "sut/workflow/actorpy",line 161,in actorrun_backup
File "./sut/backup/_init_py",line 9,in process
File "sut/backup/actorpy",line 446,in actor.for_backup_process
File "sut/backup/actor.py",line 359,in actor.BackupFiles.do_files_backup
File "sut/backup/actorpy",line 139,in actor.BackupFiles.get_all_backup_files
File "sut/backup/actorpy",line 128,in actor.BackupFiles.get_other_files
File "sut/backup/actor.py",line 130,in actor.BackupFiles.get_other_files
File "sut/backup/actorpy",line 77,in actor.BackupFiles.write_backup_list_to_file
UnicodeEncodeError:'utf-8'codec can't encode characters in position 44-47:surrogates not allowed
ERROR ]-[centos2openEuler]:x2openEuler-upgrade backup failed,please check
2
##THE END##

分析

备份目录下有文件的文件名不能被utf-8进行转码

解决

把下面的脚本保存成一个check_utf8.py文件，执行python3 check_utf8.py, 在升级过程，有时候是没有python3,可以借助x2openEuler工具中封装的python3,比如执行/usr/local/x2openEuler-python/bin/python3 check_utf8.py

import os
import json
dirname_list = ["/boot","/root","/run","/usr","/etc"]
garbled_dict = {"result": []}
result_path = "garbled_result.txt"

def covert_to_utf8(filename):
    try:
        encode_filename = filename.encode("utf-8")
    except UnicodeEncodeError as e:
        garbled_dict.get("result").append(filename)
        print(f"错误的文件名：{str(filename).encode('utf-8','replace').decode('utf-8')}")
        return None, str(e)
    return encode_filename,None

def check_utf8_file(dirname):
    for maindir, subdir, file_name_lists in os.walk(dirname):
        try:
            covert_to_utf8(maindir)
        except Exception as e:
            print(f"err is {e}")
            print("*****{0}/{1}******".format(maindir,subdir))
            continue

        for filename in file_name_lists:
            apath = os.path.join(maindir, filename)
            try:
                utf8_filename, error = covert_to_utf8(apath)
            except Exception as e:
                print(e)
                print("{0}/{1}".format(maindir,subdir))
                continue

for dirname in dirname_list:
    check_utf8_file(dirname)

print(f"garbled_dict is {garbled_dict}")

try:
    data_str = json.dumps(garbled_dict,default=lambda obj: __dict__,indent=2)
except (TypeError,ValueError) as err:
    print(f"Dumps json file {result_path} failed, please check your raw data")
    exit(0)
try:
    with os.fdopen(os.open(result_path, os.O_WRONLY |os.O_CREAT| os.O_TRUNC, 0o660), 'w') as jsonfile:
                   jsonfile.write(data_str)
except OSError as err:
     print("Generate json file {0} failed, please check your raw data!".format(result_path))

脚本执行之后，发现/usr下面有个图标是乱的，把它移到其他目录或删除即可