解决使用copy.deepcopy()拷贝Tensor或model时报错只支持用户显式创建的Tensor问题

原创

已于 2023-01-29 23:55:45 修改 · 1.1w 阅读

标签

#python #deepcopy #Tensor #requires_grad #grad_fn

于 2023-01-29 23:05:22 首次发布

本文探讨了在PyTorch中使用copy.deepcopy()时遇到的问题，即无法拷贝requires_grad=True的非叶子节点Tensor，并提供了相应的解决办法。

模型训练过程中常需边训练边做validation或在训练完的模型需要做测试，通常的做法当然是先创建model实例然后掉用load_state_dict()装载训练出来的权重到model里再调用model.eval()把模型转为测试模式，这样写对于训练完专门做测试时当然是比较合适的，但是对于边训练边做validation使用这种方式就需要写一堆代码，如果能使用copy.deepcopy()直接深度拷贝训练中的model用来做validation显然是比较简洁的写法，但是由于copy.deepcopy()的限制，写model里代码时如果没注意，调用copy.deepcopy(model)时可能就会遇到这个错误：Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment，详细错误信息如下:

 File "/usr/local/lib/python3.6/site-packages/prc/framework/model/validation.py", line 147, in init_val_model
    val_model = copy.deepcopy(model)
  File "/usr/lib64/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(state, memo)
  File "/usr/lib64/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/usr/lib64/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/lib64/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.6/copy.py", line 306, in _reconstruct
    value = deepcopy(value, memo)
  File "/usr/lib64/python3.6/copy.py", line 180, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "/usr/lib64/python3.6/copy.py", line 280, in _reconstruct
    state = deepcopy(sta