Finetuning or training ?

[trmoro]

My issue may be basic, but I've try finetuning vigogne with custom french instructions, but when i'm using this model, I never get the responses that I wrote.

So, I wondering if I'm using finetuning right or if I have to use a different method to train the model.

[pereconteur]

Hi @trmoro !

I contacted @bofenghuang to see if he could show me how to use finetune.py by giving me a demo with the instruction: "Donne moi le top 10 du ballon d'or 2016" and outputting the top 10. And all this while showing me the changes in the .json file. In short, a mini tuto of how to do it (in video). I'm waiting for his answer, but if you succeeded, could you make a screen recording and send it to me?

I would like to have the steps of the fine-tuning that would help me a lot!

[pereconteur]

I say this because I have a Macbook M1 and I don't have CUDA with NVIDIA.

If you can also explain me the steps to do and the prerequisite I would be extremely grateful!

[bofenghuang]

Hi @trmoro!

Have you checked the loss curve during fine-tuning? This can provide insight into whether the model is learning and improving. Also, how much training data did you use?

Regarding hyperparameters, you might want to try using more aggressive values for parameters related to LoRA. This could potentially improve the model's performance.

[trmoro]

Hi @bofenghuang

For the training, I did 150 epochs. The loss curve is decreasing fastly after epoch 25 (loss = 0.4 at epoch 25, loss =0.09 at epoch 50)
For the moment, I have a small dataset, only 60 entries but I'm adding data to it.

I try with modified LoRA parameters.

Here is the parameters I used before :

python3 finetune.py
--model_name_or_path hf
--train_file "data/dataset.json"
--output_dir "outputs/dataset"
--run_name "dataset_vigogne"
--overwrite_output_dir
--model_max_length_percentile 100
--preprocessing_num_workers 4
--dataloader_num_workers 1
--lora_r 8
--lora_alpha 16
--lora_dropout 0.05
--target_modules "q_proj" "v_proj"
--per_device_train_batch_size 16
--per_device_eval_batch_size 8
--gradient_accumulation_steps 8
--num_train_epochs 150
--logging_steps 25
--save_strategy "steps"
--save_steps 200
--save_total_limit 3
--learning_rate 3e-4
--report_to "wandb"