Tokenizer class Qwen2Tokenizer

Traceback (most recent call last):
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/train/sft_train.py", line 218, in <module>
    train()
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/train/sft_train.py", line 195, in train
    run_sft(
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/train/sft_train.py", line 49, in run_sft
    model, tokenizer = load_model_and_tokenizer(
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/llm_base/load_tokenizer.py", line 183, in load_model_and_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
  File "/home/ziva/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 748, in from_pretrained
    raise ValueError(
ValueError: Tokenizer class Qwen2Tokenizer does not exist or is not currently imported.

解决方法：pip install --upgrade transformers==4.40.1

然后报错：W&B offline. Running your script from this directory will only write metadata locally. Use wandb disabled to completely turn off W&B.
Traceback (most recent call last):
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/train/sft_train.py", line 11, in <module>
    from dbgpt_hub.llm_base.load_tokenizer import load_model_and_tokenizer
  File "/home/ziva/DB-GPT-Hub/dbgpt_hub/llm_base/load_tokenizer.py", line 7, in <module>
    from trl import AutoModelForCausalLMWithValueHead
  File "/home/ziva/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/trl/__init__.py", line 5, in <module>
    from .core import set_seed
  File "/home/ziva/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/trl/core.py", line 25, in <module>
    from transformers import top_k_top_p_filtering
ImportError: cannot import name 'top_k_top_p_filtering' from 'transformers' (/home/ziva/miniconda3/envs/dbgpt_hub/lib/python3.10/site-packages/transformers/__init__.py)

升级版本太高，与python冲突，更新python也没用

pip install  transformers==4.37.0  -i https://mirrors.aliyun.com/pypi/simple/
这个版本合适