Here, we will explore how to Fine-tune a base Tiny-Llama model<\/a> on a cleaned Alpaca dataset.<\/p>\n
![\"Fine-tuning](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth.png\")
<\/figure>\n<\/div>\nLearning Objectives<\/h4>\n\n- Understand fine-tuning and different methods of it.<\/li>\n
- Learn about tools and techniques for efficient fine-tuning.<\/li>\n
- Learn about WandB for logging training logs.<\/li>\n
- Fine-tune Tiny-Llama on the Alpaca dataset in Colab.<\/li>\n<\/ul>\n
This article was published as a part of the <\/strong><\/em>Data Science Blogathon.<\/strong><\/em><\/a><\/p>\n\nTable of contents<\/h2>\n<\/div>\nWhat is LLM Fine-Tuning?<\/h2>\n
Fine-tuning is the process of making a pre-trained model learn new knowledge. The pre-trained model is a general-purpose model trained on a large amount of data. However, in most cases, they fail to perform as intended, and fine-tuning is the most effective way to make the model adapt to specific use cases. For example, base LLMs<\/a> do well at text generation on single-turn QA but struggle with multi-turn conversations like chat models. <\/p>\nThe base models need to be trained on transcripts of dialogues to be able to hold multi-turn conversations. Fine-tuning is essential to mold pre-trained models into different avatars. The quality of Fine-tuned models depends on the quality of data and base model capabilities. There are multiple ways to model fine-tuning, like LoRA, QLoRA, etc.<\/p>\n
Let\u2019s briefly go through these concepts.<\/p>\n
LoRA<\/h4>\n
LoRA stands for Low-rank Adaptation, a popular fine-tuning technique in which we select a few trainable parameters instead of updating all the parameters via a low-rank approximation of original weight matrices. The LoRA model can be Fine-tuned faster on less compute-intensive hardware.<\/p>\n
QLoRA<\/h4>\n
QLoRA or Quantized LoRA is a step further than the LoRA. Instead of a full-precision model, it quantizes the model weights to lower floating point precision before applying LoRA. Quantization is the process of downcasting higher bit values to lower values. A 4-bit quantization process involves quantizing the 16-bit weights to 4-bit float values. <\/p>\n
Quantizing the model leads to a substantial reduction in model size with comparable accuracy to the original model. In QLoRA, we take a quantized model and apply LoRA to it. The models can be quantized in multiple ways, such as through llama.cpp, AWQ, bitsandbytes, etc. <\/p>\n
Fine-Tuning with Unsloth<\/h2>\n
Unsloth is an open-source platform for fine-tuning popular Large Language Models faster. It supports popular LLMs, including Llama-2 and Mistral, and their derivatives like Yi, Open-hermes, etc. It implements custom triton kernels and a manual back-prop engine to improve the speed of the model training.<\/p>\n
Here, we will use the Unsloth to Fine-tune a base 4-bit quantized Tiny-Llama model on the Alpaca<\/a> dataset. The model is quantized with bits and bytes, and kernels are optimized with OpenAI\u2019s Triton.<\/p>\n\n![\"Unsloth\"](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth-1.png\")
<\/figure>\n<\/div>\nLogging with WandB<\/h2>\n
In Machine learning, it is crucial to log training and evaluation metrics. This gives us a complete picture of the train run. Weights and Biases<\/a> (WandB) is an open-source library for visualizing and tracking machine learning experiments. It has a dedicated web app for visualizing training metrics in real-time. It also lets us manage production models centrally. We will use WandB only to track our Tiny-Llama fine-tuning run. <\/p>\nTo use WandB, sign up for a free account and create an API key<\/a>.<\/p>\nNow, let\u2019s start fine-tuning our model.<\/strong><\/p>\nHow to Fine-tune Tiny-Llama?<\/h2>\n
Fine-tuning is a compute-heavy task. It requires a machine with 10-15 GB of VRAM, or you can use Colab\u2019s free Tesla T4 GPU runtime.<\/p>\n
Now install Unsloth and WandB<\/p>\n
%%capture\nimport torch\nmajor_version, minor_version = torch.cuda.get_device_capability()\n!pip install wandb\nif major_version >= 8:\n # Use this for new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40)\n !pip install \"unsloth[colab_ampere] @ git+https:\/\/github.com\/unslothai\/unsloth.git\"\nelse:\n # Use this for older GPUs (V100, Tesla T4, RTX 20xx)\n !pip install \"unsloth[colab] @ git+https:\/\/github.com\/unslothai\/unsloth.git\"\npass<\/code><\/pre>\nThe next thing is to load the 4-bit quantized pre-trained model with Unsloth.<\/p>\n
from unsloth import FastLanguageModel\nimport torch\nmax_seq_length = 4096 # Choose any! We auto support RoPE Scaling internally!\ndtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\nload_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n model_name = \"unsloth\/tinyllama-bnb-4bit\", # \"unsloth\/tinyllama\" for 16bit loading\n max_seq_length = max_seq_length,\n dtype = dtype,\n load_in_4bit = load_in_4bit,\n)<\/code><\/pre>\nThis will install the model locally. The 4-bit model size will be around 760 MBs.<\/p>\n
Now apply PEFT<\/a> to the 4-bit Tiny-Llama model.<\/p>\nmodel = FastLanguageModel.get_peft_model(\n model,\n r = 32, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n \"gate_proj\", \"up_proj\", \"down_proj\",],\n lora_alpha = 32,\n lora_dropout = 0, # Currently only supports dropout = 0\n bias = \"none\", # Currently only supports bias = \"none\"\n use_gradient_checkpointing = True, # @@@ IF YOU GET OUT OF MEMORY - set to True @@@\n random_state = 3407,\n use_rslora = False, # We support rank stabilized LoRA\n loftq_config = None, # And LoftQ\n)<\/code><\/pre>\nPrepare Data<\/h3>\n
The next step is to prepare the dataset for fine-tuning. As I mentioned earlier, we will use a cleaned Alpaca dataset<\/a>. This is a cleaned version of the original Alpaca dataset. It follows the instruction-input-response format. Here is an example of Alpaca data<\/p>\n\n![\"Fine-Tuning\"](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth.webp\")
<\/figure>\n<\/div>\nNow, let\u2019s prepare our data.<\/p>\n
@title prepare data\n\n#alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that\n provides further context.\n Write a response that appropriately completes the request.\n\n### Instruction:\n{}\n\n### Input:\n{}\n\n### Response:\n{}\"\"\"\n\nEOS_TOKEN = tokenizer.eos_token\ndef formatting_prompts_func(examples):\n instructions = examples[\"instruction\"]\n inputs = examples[\"input\"]\n outputs = examples[\"output\"]\n texts = []\n for instruction, input, output in zip(instructions, inputs, outputs):\n # Must add EOS_TOKEN, otherwise your generation will go on forever!\n text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n texts.append(text)\n return { \"text\" : texts, }\npass\n\nfrom datasets import load_dataset\ndataset = load_dataset(\"yahma\/alpaca-cleaned\", split = \"train\")\ndataset = dataset.map(formatting_prompts_func, batched = True,)<\/code><\/pre>\nNow, split the data into train and eval data. I have taken small eval data as larger eval data slows down the training.<\/p>\n
dataset_dict = dataset.train_test_split(test_size=0.004)<\/code><\/pre>\nConfigure WandB<\/h3>\n
Now, configure Weights and Biases in your current runtime.<\/p>\n
# @title wandb init\nimport wandb\nwandb.login()<\/code><\/pre>\nProvide API key to log in to WandB when prompted.<\/p>\n
Set up environment variables.<\/p>\n
%env WANDB_WATCH=all\n%env WANDB_SILENT=true<\/code><\/pre>\nTrain Model<\/h3>\n
So far, we have loaded the 4-bit model, created the LoRA configuration, prepared the dataset, and configured WandB. The next step is to train the model on the data. For that, we need to define a trainer from the Trl library. We will use the SFTrainer from Trl. But before that, initialize WandB and define appropriate training arguments.<\/p>\n
import os\n\nfrom trl import SFTTrainer\nfrom transformers import TrainingArguments\nfrom transformers.utils import logging\nimport wandb\n\nlogging.set_verbosity_info()\nproject_name = \"tiny-llama\" \nentity = \"wandb\"\n# os.environ[\"WANDB_LOG_MODEL\"] = \"checkpoint\"\n\nwandb.init(project=project_name, name = \"tiny-llama-unsloth-sft\")<\/code><\/pre>\nTraining Arguments<\/p>\n
args = TrainingArguments(\n per_device_train_batch_size = 2,\n per_device_eval_batch_size=2,\n gradient_accumulation_steps = 4,\n evaluation_strategy=\"steps\",\n warmup_ratio = 0.1,\n num_train_epochs = 1,\n learning_rate = 2e-5,\n fp16 = not torch.cuda.is_bf16_supported(),\n bf16 = torch.cuda.is_bf16_supported(),\n optim = \"adamw_8bit\",\n weight_decay = 0.1,\n lr_scheduler_type = \"linear\",\n seed = 3407,\n output_dir = \"outputs\",\n report_to=\"wandb\", # enable logging to W&B\n # run_name=\"tiny-llama-alpaca-run\", # name of the W&B run (optional)\n logging_steps=1, # how often to log to W&B\n logging_strategy = 'steps',\n save_total_limit=2,\n )<\/code><\/pre>\nThis is important for training. To keep GPU usage low, keep the train, eval batch, and gradient accumulating steps low. The logging_steps is the number of steps before metrics are logged to WandB.<\/p>\n
Now, initialize the SFTTrainer.<\/p>\n
trainer = SFTTrainer(\n model = model,\n tokenizer = tokenizer,\n train_dataset = dataset_dict[\"train\"],\n eval_dataset=dataset_dict[\"test\"],\n dataset_text_field = \"text\",\n max_seq_length = max_seq_length,\n dataset_num_proc = 2,\n packing = True, # Packs short sequences together to save time!\n args = args,\n)<\/code><\/pre>\nNow, start the training.<\/p>\n
trainer_stats = trainer.train()\nwandb.finish()<\/code><\/pre>\nDuring the training run, WandB will track the training and eval metrics. You visit the given dashboard link and see it in real-time.<\/p>\n
This is a screenshot from my run on a Colab notebook.<\/p>\n
\n![\"Fine-tuning\"](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth-1.webp\")
<\/figure>\n<\/div>\nThe training speed will depend on multiple factors, including the training and eval data sizes, train and eval batch size, and the number of epochs. If you encounter GPU usage issues, try reducing batch and gradient accumulation step sizes. The train batch size = batch_size_per_device * gradient_accumulation_steps. And the number of optimization steps = total training data\/batch size. You can play with the parameters and see which works better.<\/p>\n
You can visualize the training and evaluation loss of your training on the WandB dashboard.<\/p>\n
Train Loss<\/p>\n
\n![\"Train](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth-2.webp\")
<\/figure>\n<\/div>\nEval Loss<\/p>\n
\n![\"Eval](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth-3.webp\")
<\/figure>\n<\/div>\nInferencing<\/h3>\n
You can save the LoRA adapters locally or push them to the HuggingFace Repository.<\/p>\n
model.save_pretrained(\"lora_model\") # Local saving\n# model.push_to_hub(\"your_name\/lora_model\", token = \"...\") # Online saving<\/code><\/pre>\nYou can also load the saved model from the disk and use it for inferencing.<\/p>\n
if False:\n from unsloth import FastLanguageModel\n model, tokenizer = FastLanguageModel.from_pretrained(\n model_name = \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n max_seq_length = max_seq_length,\n dtype = dtype,\n load_in_4bit = load_in_4bit,\n )\n\ninputs = tokenizer(\n[\n alpaca_prompt.format(\n \"capital of France?\", # instruction\n \"\", # input\n \"\", # output - leave this blank for a generation!\n )\n]*1, return_tensors = \"pt\").to(\"cuda\")\n\noutputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\ntokenizer.batch_decode(outputs)<\/code><\/pre>\nFor streaming Model responses.<\/p>\n
from transformers import TextStreamer\n\ntext_streamer = TextStreamer(tokenizer)\n_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 64)<\/code><\/pre>\nSo, this was all about fine-tuning a Tiny-Llama model with WandB logging.<\/p>\n
Here is the Colab Notebook<\/a> for the same.<\/p>\nConclusion<\/h2>\n
Small LLMs can be beneficial for deploying on compute-restricted hardware, such as personal computers, mobile phones, and other wearables, etc. Fine-tuning allows these models to perform better on downstream tasks. In this article, we learned how to Fine-tune a base language model on a dataset.<\/p>\n
Key Takeaways<\/h4>\n\n- Fine-tuning is the process of making a pre-trained model adapt to a specific new task.<\/li>\n
- Tiny-Llama is an LLM with only 1.1 billion parameters and is trained on 3 trillion tokens.<\/li>\n
- There are different ways to Fine-tune LLMs, like LoRA and QLoRA.<\/li>\n
- Unsloth is an open-source platform that provides CUDA-optimized LLMs to speed up fine-tuning LLMs.<\/li>\n
- Weights and Biases (WandB) is a tool for tracking and storing ML experiments.<\/li>\n<\/ul>\n
Frequently Asked Questions<\/h2>\n\nQ1. What is LLM fine-tuning?<\/b><\/strong> <\/p>\nA. Fine-tuning, in the context of machine learning, especially deep learning, is a technique where you take a pre-trained model and adapt it to a new, specific task.<\/p>\n<\/p><\/div>\n
Q2. Can I Fine-tune LLMs for free?<\/b><\/strong> <\/p>\nA. It is possible to Fine-tune smaller LLMs for free on Colab over the Tesla T4 GPU with QLoRA.<\/p>\n<\/p><\/div>\n
Q3. What are the benefits of Fine-tuning LLM?<\/b><\/strong> <\/p>\nA. Fine-tuning vastly enhances LLM\u2019s capability to perform downstream tasks, like role play, code generation, etc.<\/p>\n<\/p><\/div>\n
Q4. What is Tiny-Llama?<\/b><\/strong> <\/p>\nA. Tiny-Llama trained on 3 trillion tokens is an LLM with 1.1B parameters. The model adopts the original Llama-2 architecture.<\/p>\n<\/p><\/div>\n
Q5. What is Unsloth used for?<\/b><\/strong> <\/p>\nA. Unsloth is an open-source tool that provides faster and more efficient LLM fine-tuning by optimizing GPU kernels with Triton.<\/p>\n<\/p><\/div><\/div>\n
The media shown in this article is not owned by Analytics Vidhya and is used at the Author\u2019s discretion.<\/strong><\/p>\nRelated<\/em><\/h3>\n<\/p>\n\n\n\n
\n![\"Sunil](\"https:\/\/wordpress-1016567-4521551.cloudwaysapps.com\/wp-content\/uploads\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth.jpg\")
\n<\/a>\n<\/div>\n<\/div>\n<\/div>\n\n- SEO Powered Content & PR Distribution. Get Amplified Today.<\/a><\/li>\n
- PlatoData.Network Vertical Generative Ai. Empower Yourself. Access Here.<\/a><\/li>\n
- PlatoAiStream. Web3 Intelligence. Knowledge Amplified. Access Here.<\/a><\/li>\n
- PlatoESG. Carbon,<\/a> CleanTech,<\/a> Energy,<\/a> Environment,<\/a> Solar,<\/a> Waste Management.<\/a> Access Here.<\/a><\/li>\n
- PlatoHealth. Biotech and Clinical Trials Intelligence. Access Here.<\/a><\/li>\n
- Source:<\/span> https:\/\/www.analyticsvidhya.com\/blog\/2024\/02\/fine-tuning-a-tiny-llama-model-with-unsloth\/<\/a><\/li>\n<\/ul>\n","protected":false},"author":1,"featured_media":3093067,"template":"Default","meta":{"_eb_attr":"","type":"","auto_type":false,"post":"","stream":"","stream_url":"","waveform_data":[],"duration":0,"start":0,"end":0,"bpm":0,"downloadable":false,"download_url":"","purchase_title":"","purchase_url":"","post-count-all":0,"like_count":0,"download_count":0,"editor_note":"","copyright":"","captions":[],"sources":[]},"genre":[42022],"station_tag":[21246,38012,33649,38013,34957,21579,3759,69356,26866,71322,69935,78123,71824,3629,48576,13775,45379,31352,9160,4263,40460,40465,40210,47852,69352,3761,13519,68956,9873,68950,4046,43445,18340,68867,4604,3821,12974,10063,12194,10791,4243,48551,6009,3718,4135,48552,48553,5128,10055,72867,40024,30993,48554,68863,12619,12341,14391,8952,4139,14383,40971,69769,3719,4591,79293,47564,43446,24733,13083,69017,48555,48563,9772,71996,3887,4145,10957,78142,11474,11499,3720,74677,39912,13680,33521,4149,40297,8637,71795,4690,45027,40590,5340,11928,11290,9945,3724,11927,11006,5425,5157,3642,40775,9831,13780,15624,74674,77612,12075,10420,9095,5512,78198,4561,78977,9163,11538,69099,9423,40053,5404,4167,4063,10727,70616,40932,11374,44081,11107,46301,40054,11957,4250,41831,9165,9884,12620,452,12135,9166,12390,69835,11100,9369,29027,69030,69742,10369,40057,69383,39836,40486,12646,48556,5175,5070,4742,13776,11103,3903,12012,10844,40035,7814,40168,40059,19651,3647,5022,13838,4311,4425,48557,3732,3801,11038,3849,3694,4185,12342,3650,16264,48567,69344,40819,22734,9365,10091,19265,3953,11381,70063,11192,40062,40170,70606,68944,75037,3701,3653,48558,13752,4196,3805,4597,74716,4572,3806,9704,10954,4318,4573,4010,10786,69921,33094,5388,48580,29647,79280,458,34918,79297,4965,12523,4279,7142,69397,69548,69376,21292,39916,71567,10223,4089,13777,3737,11796,9377,34144,4353,8690,11767,12471,475,4093,3961,5633,4094,9413,10997,18123,9483,11457,69836,70539,23391,39917,20482,14094,73493,10350,10785,48559,69346,71570,48560,3916,68953,4207,15633,10750,11184,11934,68946,14287,28475,3663,9252,68864,69348,48879,76398,68865,10683,41129,41326,40421,40422,12344,68458,40330,12019,9083,44683,5634,476,3861,7457,8453,9642,8570,11181,4017,9876,39920,12063,40042,9221,4579,10419,9057,14213,23484,3940,4622,73158,72403,7118,4110,10148,70050,11815,9619,11484,12623,3710,3776,69754,12142,13274,29564,3921,13653,14276,75514,4624,3745,4118,69188,4459,77609,3976,4483,12345,12462,12444,3712,14394,13695,4548,40348,13785,4120,4860,9979,3714,21389,31196,4412,30317,9283,39843,3669,4123,39844,70890,40350,10316,76390,9889,4035,4128,75195,44368,68954,48561,13215,39845,48569,69447,28675,48566,69021,13778,68948,9055,3671,19267,9084,4036,3813,22249,69860,10956,8813,7332,25082,6986,4895,14386,40985,7291,25688,14821,36846,40077,70380,12837,3984,37458,10367,68866,24736,12835,39873,48587,75082,68958,23339,13218,68959,9633,4590,31564,32754,48571,68963,9178,4368,69189,36520,11454,27168,48562,4240,40104,68949,13217,14555],"artist":[42023],"mood":[],"activity":[],"_links":{"self":[{"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/station\/3093066"}],"collection":[{"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/station"}],"about":[{"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/types\/station"}],"author":[{"embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/users\/1"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/media\/3093067"}],"wp:attachment":[{"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/media?parent=3093066"}],"wp:term":[{"taxonomy":"genre","embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/genre?post=3093066"},{"taxonomy":"station_tag","embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/station_tag?post=3093066"},{"taxonomy":"artist","embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/artist?post=3093066"},{"taxonomy":"mood","embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/mood?post=3093066"},{"taxonomy":"activity","embeddable":true,"href":"https:\/\/platodata.io\/wp-json\/wp\/v2\/activity?post=3093066"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
This article was published as a part of the <\/strong><\/em>Data Science Blogathon.<\/strong><\/em><\/a><\/p>\n Fine-tuning is the process of making a pre-trained model learn new knowledge. The pre-trained model is a general-purpose model trained on a large amount of data. However, in most cases, they fail to perform as intended, and fine-tuning is the most effective way to make the model adapt to specific use cases. For example, base LLMs<\/a> do well at text generation on single-turn QA but struggle with multi-turn conversations like chat models. <\/p>\n The base models need to be trained on transcripts of dialogues to be able to hold multi-turn conversations. Fine-tuning is essential to mold pre-trained models into different avatars. The quality of Fine-tuned models depends on the quality of data and base model capabilities. There are multiple ways to model fine-tuning, like LoRA, QLoRA, etc.<\/p>\n Let\u2019s briefly go through these concepts.<\/p>\n LoRA stands for Low-rank Adaptation, a popular fine-tuning technique in which we select a few trainable parameters instead of updating all the parameters via a low-rank approximation of original weight matrices. The LoRA model can be Fine-tuned faster on less compute-intensive hardware.<\/p>\n QLoRA or Quantized LoRA is a step further than the LoRA. Instead of a full-precision model, it quantizes the model weights to lower floating point precision before applying LoRA. Quantization is the process of downcasting higher bit values to lower values. A 4-bit quantization process involves quantizing the 16-bit weights to 4-bit float values. <\/p>\n Quantizing the model leads to a substantial reduction in model size with comparable accuracy to the original model. In QLoRA, we take a quantized model and apply LoRA to it. The models can be quantized in multiple ways, such as through llama.cpp, AWQ, bitsandbytes, etc. <\/p>\n Unsloth is an open-source platform for fine-tuning popular Large Language Models faster. It supports popular LLMs, including Llama-2 and Mistral, and their derivatives like Yi, Open-hermes, etc. It implements custom triton kernels and a manual back-prop engine to improve the speed of the model training.<\/p>\n Here, we will use the Unsloth to Fine-tune a base 4-bit quantized Tiny-Llama model on the Alpaca<\/a> dataset. The model is quantized with bits and bytes, and kernels are optimized with OpenAI\u2019s Triton.<\/p>\n In Machine learning, it is crucial to log training and evaluation metrics. This gives us a complete picture of the train run. Weights and Biases<\/a> (WandB) is an open-source library for visualizing and tracking machine learning experiments. It has a dedicated web app for visualizing training metrics in real-time. It also lets us manage production models centrally. We will use WandB only to track our Tiny-Llama fine-tuning run. <\/p>\n To use WandB, sign up for a free account and create an API key<\/a>.<\/p>\n Now, let\u2019s start fine-tuning our model.<\/strong><\/p>\n Fine-tuning is a compute-heavy task. It requires a machine with 10-15 GB of VRAM, or you can use Colab\u2019s free Tesla T4 GPU runtime.<\/p>\n Now install Unsloth and WandB<\/p>\n The next thing is to load the 4-bit quantized pre-trained model with Unsloth.<\/p>\n This will install the model locally. The 4-bit model size will be around 760 MBs.<\/p>\n Now apply PEFT<\/a> to the 4-bit Tiny-Llama model.<\/p>\n The next step is to prepare the dataset for fine-tuning. As I mentioned earlier, we will use a cleaned Alpaca dataset<\/a>. This is a cleaned version of the original Alpaca dataset. It follows the instruction-input-response format. Here is an example of Alpaca data<\/p>\n Now, let\u2019s prepare our data.<\/p>\n Now, split the data into train and eval data. I have taken small eval data as larger eval data slows down the training.<\/p>\n Now, configure Weights and Biases in your current runtime.<\/p>\n Provide API key to log in to WandB when prompted.<\/p>\n Set up environment variables.<\/p>\n So far, we have loaded the 4-bit model, created the LoRA configuration, prepared the dataset, and configured WandB. The next step is to train the model on the data. For that, we need to define a trainer from the Trl library. We will use the SFTrainer from Trl. But before that, initialize WandB and define appropriate training arguments.<\/p>\n Training Arguments<\/p>\n This is important for training. To keep GPU usage low, keep the train, eval batch, and gradient accumulating steps low. The logging_steps is the number of steps before metrics are logged to WandB.<\/p>\n Now, initialize the SFTTrainer.<\/p>\n Now, start the training.<\/p>\n During the training run, WandB will track the training and eval metrics. You visit the given dashboard link and see it in real-time.<\/p>\n This is a screenshot from my run on a Colab notebook.<\/p>\n The training speed will depend on multiple factors, including the training and eval data sizes, train and eval batch size, and the number of epochs. If you encounter GPU usage issues, try reducing batch and gradient accumulation step sizes. The train batch size = batch_size_per_device * gradient_accumulation_steps. And the number of optimization steps = total training data\/batch size. You can play with the parameters and see which works better.<\/p>\n You can visualize the training and evaluation loss of your training on the WandB dashboard.<\/p>\n Train Loss<\/p>\n Eval Loss<\/p>\n You can save the LoRA adapters locally or push them to the HuggingFace Repository.<\/p>\n You can also load the saved model from the disk and use it for inferencing.<\/p>\n For streaming Model responses.<\/p>\n So, this was all about fine-tuning a Tiny-Llama model with WandB logging.<\/p>\n Here is the Colab Notebook<\/a> for the same.<\/p>\n Small LLMs can be beneficial for deploying on compute-restricted hardware, such as personal computers, mobile phones, and other wearables, etc. Fine-tuning allows these models to perform better on downstream tasks. In this article, we learned how to Fine-tune a base language model on a dataset.<\/p>\n A. Fine-tuning, in the context of machine learning, especially deep learning, is a technique where you take a pre-trained model and adapt it to a new, specific task.<\/p>\n<\/p><\/div>\n A. It is possible to Fine-tune smaller LLMs for free on Colab over the Tesla T4 GPU with QLoRA.<\/p>\n<\/p><\/div>\n A. Fine-tuning vastly enhances LLM\u2019s capability to perform downstream tasks, like role play, code generation, etc.<\/p>\n<\/p><\/div>\n A. Tiny-Llama trained on 3 trillion tokens is an LLM with 1.1B parameters. The model adopts the original Llama-2 architecture.<\/p>\n<\/p><\/div>\n A. Unsloth is an open-source tool that provides faster and more efficient LLM fine-tuning by optimizing GPU kernels with Triton.<\/p>\n<\/p><\/div><\/div>\n The media shown in this article is not owned by Analytics Vidhya and is used at the Author\u2019s discretion.<\/strong><\/p>\nTable of contents<\/h2>\n<\/div>\n
What is LLM Fine-Tuning?<\/h2>\n
LoRA<\/h4>\n
QLoRA<\/h4>\n
Fine-Tuning with Unsloth<\/h2>\n
<\/figure>\n<\/div>\nLogging with WandB<\/h2>\n
How to Fine-tune Tiny-Llama?<\/h2>\n
%%capture\nimport torch\nmajor_version, minor_version = torch.cuda.get_device_capability()\n!pip install wandb\nif major_version >= 8:\n # Use this for new GPUs like Ampere, Hopper GPUs (RTX 30xx, RTX 40xx, A100, H100, L40)\n !pip install \"unsloth[colab_ampere] @ git+https:\/\/github.com\/unslothai\/unsloth.git\"\nelse:\n # Use this for older GPUs (V100, Tesla T4, RTX 20xx)\n !pip install \"unsloth[colab] @ git+https:\/\/github.com\/unslothai\/unsloth.git\"\npass<\/code><\/pre>\nfrom unsloth import FastLanguageModel\nimport torch\nmax_seq_length = 4096 # Choose any! We auto support RoPE Scaling internally!\ndtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+\nload_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.\n\nmodel, tokenizer = FastLanguageModel.from_pretrained(\n model_name = \"unsloth\/tinyllama-bnb-4bit\", # \"unsloth\/tinyllama\" for 16bit loading\n max_seq_length = max_seq_length,\n dtype = dtype,\n load_in_4bit = load_in_4bit,\n)<\/code><\/pre>\nmodel = FastLanguageModel.get_peft_model(\n model,\n r = 32, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128\n target_modules = [\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n \"gate_proj\", \"up_proj\", \"down_proj\",],\n lora_alpha = 32,\n lora_dropout = 0, # Currently only supports dropout = 0\n bias = \"none\", # Currently only supports bias = \"none\"\n use_gradient_checkpointing = True, # @@@ IF YOU GET OUT OF MEMORY - set to True @@@\n random_state = 3407,\n use_rslora = False, # We support rank stabilized LoRA\n loftq_config = None, # And LoftQ\n)<\/code><\/pre>\nPrepare Data<\/h3>\n
<\/figure>\n<\/div>\n@title prepare data\n\n#alpaca_prompt = \"\"\"Below is an instruction that describes a task, paired with an input that\n provides further context.\n Write a response that appropriately completes the request.\n\n### Instruction:\n{}\n\n### Input:\n{}\n\n### Response:\n{}\"\"\"\n\nEOS_TOKEN = tokenizer.eos_token\ndef formatting_prompts_func(examples):\n instructions = examples[\"instruction\"]\n inputs = examples[\"input\"]\n outputs = examples[\"output\"]\n texts = []\n for instruction, input, output in zip(instructions, inputs, outputs):\n # Must add EOS_TOKEN, otherwise your generation will go on forever!\n text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN\n texts.append(text)\n return { \"text\" : texts, }\npass\n\nfrom datasets import load_dataset\ndataset = load_dataset(\"yahma\/alpaca-cleaned\", split = \"train\")\ndataset = dataset.map(formatting_prompts_func, batched = True,)<\/code><\/pre>\ndataset_dict = dataset.train_test_split(test_size=0.004)<\/code><\/pre>\nConfigure WandB<\/h3>\n
# @title wandb init\nimport wandb\nwandb.login()<\/code><\/pre>\n%env WANDB_WATCH=all\n%env WANDB_SILENT=true<\/code><\/pre>\nTrain Model<\/h3>\n
import os\n\nfrom trl import SFTTrainer\nfrom transformers import TrainingArguments\nfrom transformers.utils import logging\nimport wandb\n\nlogging.set_verbosity_info()\nproject_name = \"tiny-llama\" \nentity = \"wandb\"\n# os.environ[\"WANDB_LOG_MODEL\"] = \"checkpoint\"\n\nwandb.init(project=project_name, name = \"tiny-llama-unsloth-sft\")<\/code><\/pre>\nargs = TrainingArguments(\n per_device_train_batch_size = 2,\n per_device_eval_batch_size=2,\n gradient_accumulation_steps = 4,\n evaluation_strategy=\"steps\",\n warmup_ratio = 0.1,\n num_train_epochs = 1,\n learning_rate = 2e-5,\n fp16 = not torch.cuda.is_bf16_supported(),\n bf16 = torch.cuda.is_bf16_supported(),\n optim = \"adamw_8bit\",\n weight_decay = 0.1,\n lr_scheduler_type = \"linear\",\n seed = 3407,\n output_dir = \"outputs\",\n report_to=\"wandb\", # enable logging to W&B\n # run_name=\"tiny-llama-alpaca-run\", # name of the W&B run (optional)\n logging_steps=1, # how often to log to W&B\n logging_strategy = 'steps',\n save_total_limit=2,\n )<\/code><\/pre>\ntrainer = SFTTrainer(\n model = model,\n tokenizer = tokenizer,\n train_dataset = dataset_dict[\"train\"],\n eval_dataset=dataset_dict[\"test\"],\n dataset_text_field = \"text\",\n max_seq_length = max_seq_length,\n dataset_num_proc = 2,\n packing = True, # Packs short sequences together to save time!\n args = args,\n)<\/code><\/pre>\ntrainer_stats = trainer.train()\nwandb.finish()<\/code><\/pre>\n<\/figure>\n<\/div>\n<\/figure>\n<\/div>\n<\/figure>\n<\/div>\nInferencing<\/h3>\n
model.save_pretrained(\"lora_model\") # Local saving\n# model.push_to_hub(\"your_name\/lora_model\", token = \"...\") # Online saving<\/code><\/pre>\nif False:\n from unsloth import FastLanguageModel\n model, tokenizer = FastLanguageModel.from_pretrained(\n model_name = \"lora_model\", # YOUR MODEL YOU USED FOR TRAINING\n max_seq_length = max_seq_length,\n dtype = dtype,\n load_in_4bit = load_in_4bit,\n )\n\ninputs = tokenizer(\n[\n alpaca_prompt.format(\n \"capital of France?\", # instruction\n \"\", # input\n \"\", # output - leave this blank for a generation!\n )\n]*1, return_tensors = \"pt\").to(\"cuda\")\n\noutputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)\ntokenizer.batch_decode(outputs)<\/code><\/pre>\nfrom transformers import TextStreamer\n\ntext_streamer = TextStreamer(tokenizer)\n_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 64)<\/code><\/pre>\nConclusion<\/h2>\n
Key Takeaways<\/h4>\n
\n
Frequently Asked Questions<\/h2>\n
Related<\/em><\/h3>\n<\/p>\n
\n
\n<\/a>\n<\/div>\n<\/div>\n<\/div>\n\n