Calling Source Code to Start

This section shows how to use MindSpore Transformers to quickly pull up a LoRA low-parameter fine-tuning task based on the Llama2-7B model. To use other models and tasks via MindSpore Transformers, please read the corresponding model documentation.

Preparing Weights File

MindSpore Transformers provides pre-trained weights and word list files that have been converted for pre-training, fine-tuning and inference. Users can also download the official HuggingFace weights and use them after converting the model weights. For convenience, this file won't go into too much detail about converting the original weights here, but you can refer to the Llama2 documentation and weight conversion for more details. Please download the MindSpore weights, the converted .ckpt file, and the tokenizer.model file for subsequent processing.

Model Name	MindSpore Weights	HuggingFace Weights
Llama2-7B	llama2_7b.ckpt	Llama-2-7b-hf

Word list download link: tokenizer.model

Preparing Dataset

The dataset file alpaca_data.json used in the fine-tuning process can be obtained at Stanford Alpaca.
Install the fastchat tool. The version must be greater than or equal to 0.2.13.
```
  pip install fastchat>=0.2.13
```
Data Preprocessing

The following command needs to be executed in the MindSpore Transformers code root directory, and replaces {path} with the local path below.
1. Execute mindformers/tools/dataset_preprocess/llama/alpaca_converter.py, and use the fastchat tool to add prompt templates to convert the raw dataset into a multi-round conversation format.
```
  python mindformers/tools/dataset_preprocess/llama/alpaca_converter.py \
    --data_path /{path}/alpaca_data.json \
    --output_path /{path}/alpaca-data-conversation.json
```
  Parameter descriptions
  - data_path: Input the path to the downloaded file.
  - output_path: Save path of the output file.
2. Execute mindformers/tools/dataset_preprocess/llama/llama_preprocess.py, and generate MindRecord data and convert data with prompt templates to MindRecord format.
```
  python mindformers/tools/dataset_preprocess/llama/llama_preprocess.py \
    --dataset_type qa \
    --input_glob /{path}/alpaca-data-conversation.json \
    --model_file /{path}/tokenizer.model \
    --seq_length 4096 \
    --output_file /{path}/alpaca-fastchat4096.mindrecord
```
  Parameter descriptions
  - dataset_type: Preprocessed data types. The options include "wiki" and "qa."
    - "wiki" is used to process the Wikitext2 dataset, which is suitable for the pre-training and evaluation stages.
    - "qa" is used to process the Alpaca dataset, converting it into a question-answer format, which is suitable for the fine-tuning stage. For other dataset conversion scripts, please refer to the corresponding model documentation.
  - input_glob: Path to the converted alpaca file.
  - model_file: Path to the model tokenizer.model file.
  - seq_length: Sequence length of the output data.
  - output_file: Save path of the output file.
3. The console outputs the following, proving that the format conversion was successful.
```
  # Console outputs
  Transformed 52002 records.
  Transform finished, output files refer: {path}/alpaca-fastchat4096.mindrecord
```

Initiating Fine-tuning

In the MindSpore Transformers code root directory, execute the following command to launch the fine-tuning task:

bash scripts/msrun_launcher.sh "run_mindformer.py \
 --config configs/llama2/lora_llama2_7b.yaml \
 --train_dataset_dir /{path}/alpaca-fastchat4096.mindrecord \
 --load_checkpoint /{path}/llama2_7b.ckpt \
 --auto_trans_ckpt True \
 --use_parallel True \
 --run_mode finetune" 8

Command Explanation:

scripts/msrun_launcher.sh: Script for launching distributed tasks.
"run_mindformer.py ...": Parameter string for the Python task executed on each card, including:
- run_mindformer.py: One-click startup script.
- --config: Specifies the task configuration file path, e.g., configs/llama2/lora_llama2_7b.yaml.
- --train_dataset_dir: Specifies the dataset path, e.g., /{path}/alpaca-fastchat4096.mindrecord.
- --load_checkpoint: Specifies the checkpoint file path, e.g., /{path}/llama2_7b.ckpt.
- --auto_trans_ckpt True: Enables automatic checkpoint partitioning.
- --use_parallel True: Enables distributed task execution.
- --run_mode finetune: Sets the run mode to fine-tuning.
8: Sets the task to runs on 8 NPUs.

When the following log appears on the console:

Start worker process with rank id:0, log file:output/msrun_log/worker_0.log. Environment variable [RANK_ID=0] is exported.
Start worker process with rank id:1, log file:output/msrun_log/worker_1.log. Environment variable [RANK_ID=1] is exported.
Start worker process with rank id:2, log file:output/msrun_log/worker_2.log. Environment variable [RANK_ID=2] is exported.
Start worker process with rank id:3, log file:output/msrun_log/worker_3.log. Environment variable [RANK_ID=3] is exported.
Start worker process with rank id:4, log file:output/msrun_log/worker_4.log. Environment variable [RANK_ID=4] is exported.
Start worker process with rank id:5, log file:output/msrun_log/worker_5.log. Environment variable [RANK_ID=5] is exported.
Start worker process with rank id:6, log file:output/msrun_log/worker_6.log. Environment variable [RANK_ID=6] is exported.
Start worker process with rank id:7, log file:output/msrun_log/worker_7.log. Environment variable [RANK_ID=7] is exported.

It indicates that the fine-tuning task is started, the progress can be monitored in the output/msrun_log/ directory.

For more details on Llama2, and more startup approaches, please refer specifically to the Llama2 README documentation for more support.