pickle常见Error解决

1. pickle OverflowError: cannot serialize a bytes object larger than 4 GiB

进行pickle.dump时出现上述错误，可以加上“protocol=4”参数。依据：https://docs.python.org/3/library/pickle.html#data-stream-format

2. pickle EOFError: Ran out of input

进行pickle.load时出现上述错误，一种可能是文件没有被正常打开和关闭，如执行下面代码，在“query_ids = pkl.load(handle_id)”处会出现此错误：

........
score_file = open("clipv3_query.pkl", "wb")
pkl.dump(query_reps, score_file, protocol=4)

with open("clipv3_query.pkl", "rb") as handle_id:
    try:
        query_ids = pkl.load(handle_id)
        print(query_ids.shape)
        print(query_ids[:5])
    except Exception as e:
        print(e)

改成如下代码后，运行正常：

with open("clipv3_query.pkl", "wb") as score_file:
    pkl.dump(query_reps, score_file, protocol=4)

with open("clipv3_query.pkl", "rb") as handle_id:
    try:
        query_ids = pkl.load(handle_id)
        print(query_ids.shape)
        print(query_ids[:5])
    except Exception as e:
        print(e)