examples/models/cerebras/cerebras_lora.ipynb

{
  "cells": [
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "0ctRYDdUyshU"
      },
      "source": [
        "# [xTuring](https://github.com/stochasticai/xturing) - LLaMA efficient fine-tuning tutorial\n",
        "\n",
        "This tutorial aims to show how easy it is to perform fine-tuning with xTuring. This notebook shows how to fine-tune LLaMA 7B model on GPU which has limited memory, it requires only 9GB VRAM"
      ]
    },
    {
      "attachments": {},
      "cell_type": "markdown",
      "metadata": {
        "id": "DfZriB2ez6kp"
      },
      "source": [
        "## 1. Install the `xTuring` library"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bNFVFKld2Ju4"
      },
      "outputs": [],
      "source": [
        "!pip install xturing --upgrade"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0TwnHA9zz9v6"
      },
      "source": [
        "## 2. Download and unzip the dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KRO0TVPP2iFy"
      },
      "outputs": [],
      "source": [
        "!wget https://d33tr4pxdm6e2j.cloudfront.net/public_content/tutorials/datasets/alpaca_data.zip\n",
        "!unzip alpaca_data.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tCFfBj2n0CBw"
      },
      "source": [
        "## 3. Load the dataset and initialize the model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pwvVo7Sx2vVj"
      },
      "outputs": [],
      "source": [
        "from xturing.datasets.instruction_dataset import InstructionDataset\n",
        "from xturing.models import BaseModel\n",
        "\n",
        "instruction_dataset = InstructionDataset(\"/content/alpaca_data\")\n",
        "# Initializes the model\n",
        "model = BaseModel.create(\"cerebras_lora_int8\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GPIwmGIp1yPz"
      },
      "source": [
        "## 4. Start the finetuning"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "BHa8P_jM7jCx"
      },
      "outputs": [],
      "source": [
        "# Finetuned the model\n",
        "model.finetune(dataset=instruction_dataset)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pp6iDEYxesRe"
      },
      "source": [
        "## 5. Generate an output text with the fine-tuned model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lYfFHtW33j4B"
      },
      "outputs": [],
      "source": [
        "# Once the model has been finetuned, you can start doing inferences\n",
        "output = model.generate(texts=[\"Why LLM models are becoming so important?\"])\n",
        "print(\"Generated output by the model: {}\".format(output))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3CJgRvFVevwK"
      },
      "source": [
        "## Do you have any questions?\n",
        "\n",
        "You can open an issue in our [GitHub repo](https://github.com/stochasticai/xturing) \n"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}