DeepSeek-R1 Windows微调实施手册

版本：1.1 | 适用系统：Windows 10/11 + WSL2 | 测试硬件：NVIDIA RTX 3090/4090

一、环境准备阶段

1.1 系统要求

1.2 WSL2环境部署

powershellCopy Code
# 以管理员身份运行PowerShell
wsl --install -d Ubuntu-22.04
wsl --set-version Ubuntu-22.04 2
wsl --shutdown

# 安装NVIDIA驱动for WSL2
winget install Nvidia.CUDA --version 12.2

1.3 CUDA环境配置

bashCopy Code
# 在WSL终端执行
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
sudo apt update && sudo apt install -y cuda-toolkit-12-2
echo 'export PATH=/usr/local/cuda-12.2/bin:$PATH' >> ~/.bashrc
source ~/.bashrc

二、依赖安装

2.1 Python环境

bashCopy Code
conda create -n deepseek python=3.10 -y
conda activate deepseek
pip install torch==2.1.0+cu121 torchvision==0.16.0+cu121 --extra-index-url https://download.pytorch.org/whl/cu121

2.2 核心库安装

bashCopy Code
pip install \
  transformers==4.35.2 \
  accelerate==0.24.1 \
  datasets==2.15.0 \
  peft==0.6.0 \
  bitsandbytes==0.41.3.post2 \
  tensorboard==2.14.0

2.3 FlashAttention优化

bashCopy Code
git clone https://github.com/Dao-AILab/flash-attention
cd flash-attention && MAX_JOBS=4 python setup.py install

三、数据准备

3.1 数据集格式转换

将您的JSON数据转换为Hugging Face数据集格式：

pythonCopy Code
from datasets import Dataset, Features, Value

features = Features({
    "instruction": Value("string"),
    "input": Value("string"), 
    "output": Value("string")
})

dataset = Dataset.from_json("your_data.jsonl", features=features)
dataset.save_to_disk("formatted_data")

3.2 数据预处理

pythonCopy Code
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-r1-7b")
tokenizer.pad_token = tokenizer.eos_token

def preprocess(examples):
    texts = [f"Instruction: {ins}\nInput: {inp}\nOutput: {out}" 
            for ins, inp, out in zip(examples["instruction"], examples["input"], examples["output"])]
    
    return tokenizer(
        texts,
        max_length=1024,
        truncation=True,
        padding="max_length",
        return_tensors="pt"
    )

dataset = dataset.map(preprocess, batched=True, batch_size=1000)

四、微调执行

4.1 LoRA配置

pythonCopy Code
from peft import LoraConfig

lora_config = LoraConfig(
    r=32,
    lora_alpha=64,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

4.2 训练参数

pythonCopy Code
from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir="./deepseek-r1-finetuned",
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=2e-5,
    num_train_epochs=3,
    fp16=True,
    logging_steps=50,
    save_steps=500,
    evaluation_strategy="no",
    optim="adamw_torch_fused",
    report_to="tensorboard"
)

4.3 启动训练

pythonCopy Code
from transformers import Trainer, DataCollatorForLanguageModeling

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=dataset,
    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
    peft_config=lora_config
)

trainer.train()

五、验证与推理

5.1 加载微调模型

pythonCopy Code
from peft import PeftModel

model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-r1-7b")
model = PeftModel.from_pretrained(model, "./deepseek-r1-finetuned/checkpoint-1500")

5.2 测试推理

pythonCopy Code
input_text = "Instruction: 生成党建办绩效考核报告..."
inputs = tokenizer(input_text, return_tensors="pt").to("cuda")

outputs = model.generate(
    inputs.input_ids,
    max_length=1024,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

print(tokenizer.decode(outputs, skip_special_tokens=True))

六、性能调优建议

6.1 显存优化策略

6.2 多GPU配置

bashCopy Code
# 启动分布式训练
accelerate launch --num_processes=2 --mixed_precision=fp16 train.py

七、常见问题排查

7.1 CUDA内存不足

pythonCopy Code
# 在TrainingArguments中添加：
gradient_checkpointing=True,
gradient_accumulation_steps=16

7.2 依赖冲突解决

bashCopy Code
# 创建隔离环境
python -m venv deepseek-env
source deepseek-env/bin/activate
pip install --force-reinstall torch==2.1.0+cu121

7.3 训练速度慢

bashCopy Code
# 启用CUDA内核优化
export PYTORCH_CUDA_ALLOC_CONF="backend:cudaMallocAsync"
export NVIDIA_TF32_OVERRIDE=1  # 启用TF32加速

附：完整目录结构

textCopy Code
deepseek-finetune/
├── data/
│   ├── raw/                 # 原始数据
│   └── processed/           # 预处理后的数据集
├── scripts/
│   ├── train.py             # 训练主程序
│   └── inference.py         # 推理脚本
├── outputs/                 # 模型检查点
└── logs/                    # TensorBoard日志

‌注‌：建议在WSL中通过\\wsl.localhost\Ubuntu-22.04\home\<user>\deepseek-finetune访问项目目录

如需原生Windows方案，请参考NVIDIA Windows ML指南