L5_comparing_llama_models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fde90571-0799-48af-b315-247f85c2269b",
   "metadata": {},
   "source": [
    "# Comparing Llama Models"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ce53734-47df-454d-bb29-c135b2a1b338",
   "metadata": {},
   "source": [
    "- Load helper function to prompt Llama models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "469f76b7-acd9-4192-93f2-a37d4ac9d196",
   "metadata": {
    "height": 29
   },
   "outputs": [],
   "source": [
    "from utils import llama, llama_chat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c6cd039f-d768-44a7-b81b-bfab14a33a90",
   "metadata": {},
   "source": [
    "### Task 1: Sentiment Classification\n",
    "- Compare the models on few-shot prompt sentiment classification.\n",
    "- You are asking the model to return a one word response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "44d44286-dd86-48bb-9457-72dca64b5b37",
   "metadata": {
    "height": 182
   },
   "outputs": [],
   "source": [
    "prompt = '''\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: Positive\n",
    "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
    "Sentiment: Negative\n",
    "Message: Can't wait to order pizza for dinner tonight!\n",
    "Sentiment: ?\n",
    "\n",
    "Give a one word response.\n",
    "'''"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38108527-032d-411b-bb3e-31471694758f",
   "metadata": {},
   "source": [
    "- First, use the 7B parameter chat model (`llama-2-7b-chat`) to get the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d58a65f2-a8e2-4cf4-abc4-c65fd64710c9",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Hungry\n"
     ]
    }
   ],
   "source": [
    "response = llama(prompt,\n",
    "                 model=\"togethercomputer/llama-2-7b-chat\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24729f9d-0899-472b-9025-12824b962a52",
   "metadata": {},
   "source": [
    "- Now, use the 70B parameter chat model (`llama-2-70b-chat`) on the same task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3f62ef43-0b6a-4c11-bed9-8f49ec4f0802",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Positive\n"
     ]
    }
   ],
   "source": [
    "response = llama(prompt,\n",
    "                 model=\"togethercomputer/llama-2-70b-chat\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c0c5ded-3bbe-4cac-898c-578655ea47cc",
   "metadata": {},
   "source": [
    "### Task 2: Summarization\n",
    "- Compare the models on summarization task.\n",
    "- This is the same \"email\" as the one you used previously in the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "07d248f0-a610-4696-aacd-12ca94b2fb8c",
   "metadata": {
    "height": 568
   },
   "outputs": [],
   "source": [
    "email = \"\"\"\n",
    "Dear Amit,\n",
    "\n",
    "An increasing variety of large language models (LLMs) are open source, or close to it. The proliferation of models with relatively permissive licenses gives developers more options for building applications.\n",
    "\n",
    "Here are some different ways to build applications based on LLMs, in increasing order of cost/complexity:\n",
    "\n",
    "Prompting. Giving a pretrained LLM instructions lets you build a prototype in minutes or hours without a training set. Earlier this year, I saw a lot of people start experimenting with prompting, and that momentum continues unabated. Several of our short courses teach best practices for this approach.\n",
    "One-shot or few-shot prompting. In addition to a prompt, giving the LLM a handful of examples of how to carry out a task — the input and the desired output — sometimes yields better results.\n",
    "Fine-tuning. An LLM that has been pretrained on a lot of text can be fine-tuned to your task by training it further on a small dataset of your own. The tools for fine-tuning are maturing, making it accessible to more developers.\n",
    "Pretraining. Pretraining your own LLM from scratch takes a lot of resources, so very few teams do it. In addition to general-purpose models pretrained on diverse topics, this approach has led to specialized models like BloombergGPT, which knows about finance, and Med-PaLM 2, which is focused on medicine.\n",
    "For most teams, I recommend starting with prompting, since that allows you to get an application working quickly. If you’re unsatisfied with the quality of the output, ease into the more complex techniques gradually. Start one-shot or few-shot prompting with a handful of examples. If that doesn’t work well enough, perhaps use RAG (retrieval augmented generation) to further improve prompts with key information the LLM needs to generate high-quality outputs. If that still doesn’t deliver the performance you want, then try fine-tuning — but this represents a significantly greater level of complexity and may require hundreds or thousands more examples. To gain an in-depth understanding of these options, I highly recommend the course Generative AI with Large Language Models, created by AWS and DeepLearning.AI.\n",
    "\n",
    "(Fun fact: A member of the DeepLearning.AI team has been trying to fine-tune Llama-2-7B to sound like me. I wonder if my job is at risk? 😜)\n",
    "\n",
    "Additional complexity arises if you want to move to fine-tuning after prompting a proprietary model, such as GPT-4, that’s not available for fine-tuning. Is fine-tuning a much smaller model likely to yield superior results than prompting a larger, more capable model? The answer often depends on your application. If your goal is to change the style of an LLM’s output, then fine-tuning a smaller model can work well. However, if your application has been prompting GPT-4 to perform complex reasoning — in which GPT-4 surpasses current open models — it can be difficult to fine-tune a smaller model to deliver superior results.\n",
    "\n",
    "Beyond choosing a development approach, it’s also necessary to choose a specific model. Smaller models require less processing power and work well for many applications, but larger models tend to have more knowledge about the world and better reasoning ability. I’ll talk about how to make this choice in a future letter.\n",
    "\n",
    "Keep learning!\n",
    "\n",
    "Andrew\n",
    "\"\"\"\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Summarize this email and extract some key points.\n",
    "\n",
    "What did the author say about llama models?\n",
    "```\n",
    "{email}\n",
    "```\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "135f00bb-51d9-492a-920b-5b286ab5f5c4",
   "metadata": {},
   "source": [
    "- First, use the 7B parameter chat model (`llama-2-7b-chat`) to summarize the email."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "99c60da6-9b3e-4f11-a1d5-8e5e273e91ac",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The author discusses the proliferation of large language models (LLMs) and the various ways to build applications using them, ranging from prompting to fine-tuning. Key points include:\n",
      "\n",
      "1. LLMs are becoming more open source, giving developers more options for building applications.\n",
      "2. Different approaches to building applications include prompting, one-shot or few-shot prompting, fine-tuning, and pretraining.\n",
      "3. The author recommends starting with prompting for most teams, as it allows for quick development, and gradually moving to more complex techniques if needed.\n",
      "4. Fine-tuning a smaller model can yield superior results than prompting a larger, more capable model in some cases, depending on the application.\n",
      "5. Choosing a specific model also requires consideration, as smaller models require less processing power but may not have as much knowledge about the world or reasoning ability as larger models.\n",
      "6. The author plans to discuss how to make this choice in a future letter.\n",
      "\n",
      "Regarding llama models, the author mentions that a member of the DeepLearning.AI team has been trying to fine-tune a model called Llama-2-7B to sound like them, but the author does not provide any further information or insights about llama models.\n"
     ]
    }
   ],
   "source": [
    "response_7b = llama(prompt,\n",
    "                model=\"togethercomputer/llama-2-7b-chat\")\n",
    "print(response_7b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e84506c-c6b9-4512-9fbc-1e27140d4a2f",
   "metadata": {},
   "source": [
    "- Now, use the 13B parameter chat model (`llama-2-13b-chat`) to summarize the email."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "b6af5fb7-8538-4831-bf02-a18474b04c61",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure! Here's a summary of the email and some key points:\n",
      "\n",
      "Summary:\n",
      "The author discusses different approaches to building applications using large language models (LLMs), ranging from prompting to fine-tuning, and provides recommendations on when to use each approach. They also discuss the trade-offs between using smaller or larger models and the importance of choosing the right model for the application.\n",
      "\n",
      "Key points:\n",
      "\n",
      "1. Prompting: Giving a pretrained LLM instructions to build a prototype quickly, without a training set.\n",
      "2. One-shot or few-shot prompting: Providing a handful of examples to the LLM for better results.\n",
      "3. Fine-tuning: Training an LLM further on a small dataset for a specific task.\n",
      "4. Pretraining: Training an LLM from scratch, but requires significant resources.\n",
      "5. Choosing a development approach: Depending on the application, the author recommends starting with prompting and gradually moving to more complex techniques if needed.\n",
      "6. Choosing a specific model: Smaller models require less processing power but may not have as much knowledge, while larger models have better reasoning ability but require more resources.\n",
      "7. Fine-tuning a smaller model: May not yield superior results as compared to prompting a larger, more capable model, especially for complex reasoning tasks.\n",
      "8. The author will discuss how to choose the right model for the application in a future letter.\n"
     ]
    }
   ],
   "source": [
    "response_13b = llama(prompt,\n",
    "                model=\"togethercomputer/llama-2-13b-chat\")\n",
    "print(response_13b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08aa0c61-ff78-4372-b4fb-a51be7bf11eb",
   "metadata": {},
   "source": [
    "- Lastly, use the 70B parameter chat model (`llama-2-70b-chat`) to summarize the email."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "8a031c7e-d4d0-404d-867c-27f8a0ab96eb",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The author of the email, Andrew, discusses the various ways to build applications using large language models (LLMs), including prompting, one-shot or few-shot prompting, fine-tuning, and pretraining. He recommends starting with prompting and gradually moving on to more complex techniques if necessary. He also mentions the challenges of fine-tuning a proprietary model like GPT-4 and notes that smaller models may not always deliver superior results.\n",
      "\n",
      "The author also mentions a \"fun fact\" about a member of the DeepLearning.AI team trying to fine-tune Llama-2-7B to sound like him, which suggests that the author may be exploring the possibility of creating a personalized AI model.\n",
      "\n",
      "Some key points from the email include:\n",
      "\n",
      "1. There are several ways to build applications using LLMs, each with increasing cost and complexity.\n",
      "2. Prompting is a quick and easy way to build a prototype, and it's a good starting point for most teams.\n",
      "3. Fine-tuning is a more complex approach that requires a small dataset of your own, but it can be accessible to more developers.\n",
      "4. Pretraining your own LLM from scratch is resource-intensive and is only recommended for specialized models.\n",
      "5. Choosing the right model depends on your application, with smaller models requiring less processing power but larger models having more knowledge and better reasoning ability.\n",
      "6. The author will discuss how to choose a specific model in a future letter.\n"
     ]
    }
   ],
   "source": [
    "response_70b = llama(prompt,\n",
    "                model=\"togethercomputer/llama-2-70b-chat\")\n",
    "print(response_70b)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e6f051d-a7e4-45cc-b162-646862084994",
   "metadata": {},
   "source": [
    "#### Model-Graded Evaluation: Summarization\n",
    "\n",
    "- Interestingly, you can ask a LLM to evaluate the responses of other LLMs.\n",
    "- This is known as **Model-Graded Evaluation**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a77de7b-7339-4ff3-b8fd-63cb448d4ed8",
   "metadata": {},
   "source": [
    "- Create a `prompt` that will evaluate these three responses using 70B parameter chat model (`llama-2-70b-chat`).\n",
    "- In the `prompt`, provide the \"email\", \"name of the models\", and the \"summary\" generated by each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "e10e1d72-10ab-43dd-80f8-74fcf57e28a4",
   "metadata": {
    "height": 500
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Based on the summaries provided, it seems that all three models (llama-2-7b-chat, llama-2-13b-chat, and llama-2-70b-chat) were able to capture the main points of the email. However, there are some differences in the way the information is presented and the level of detail provided.\n",
      "\n",
      "Llama-2-7b-chat's summary is the shortest and most concise, focusing on the key points of the email. It does not provide any additional information or insights beyond what is mentioned in the email.\n",
      "\n",
      "Llama-2-13b-chat's summary is slightly longer and provides more context, including the author's recommendations for choosing a development approach and the trade-offs between using smaller or larger models. It also mentions the \"fun fact\" about the DeepLearning.AI team trying to fine-tune Llama-2-7B.\n",
      "\n",
      "Llama-2-70b-chat's summary is the longest and most detailed, providing a comprehensive overview of the email's content. It includes all the key points mentioned in the other two summaries and adds additional information about the challenges of fine-tuning a proprietary model like GPT-4.\n",
      "\n",
      "Overall, all three models seem to have performed well in summarizing the email, but llama-2-70b-chat's summary is the most comprehensive and detailed.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Given the original text denoted by `email`\n",
    "and the name of several models: `model:<name of model>\n",
    "as well as the summary generated by that model: `summary`\n",
    "\n",
    "Provide an evaluation of each model's summary:\n",
    "- Does it summarize the original text well?\n",
    "- Does it follow the instructions of the prompt?\n",
    "- Are there any other interesting characteristics of the model's output?\n",
    "\n",
    "Then compare the models based on their evaluation \\\n",
    "and recommend the models that perform the best.\n",
    "\n",
    "email: ```{email}`\n",
    "\n",
    "model: llama-2-7b-chat\n",
    "summary: {response_7b}\n",
    "\n",
    "model: llama-2-13b-chat\n",
    "summary: {response_13b}\n",
    "\n",
    "model: llama-2-70b-chat\n",
    "summary: {response_70b}\n",
    "\"\"\"\n",
    "\n",
    "response_eval = llama(prompt,\n",
    "                model=\"togethercomputer/llama-2-70b-chat\")\n",
    "print(response_eval)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "179a5581-bb28-4eba-be19-045be0779ed0",
   "metadata": {},
   "source": [
    "### Task 3: Reasoning ###\n",
    "- Compare the three models' performance on reasoning tasks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "1f98a66a-6d17-4a6d-bae8-881a5d5a1d4d",
   "metadata": {
    "height": 97
   },
   "outputs": [],
   "source": [
    "context = \"\"\"\n",
    "Jeff and Tommy are neighbors\n",
    "\n",
    "Tommy and Eddy are not neighbors\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "ff5198c1-701d-4e07-93a1-38ab7a12e07c",
   "metadata": {
    "height": 63
   },
   "outputs": [],
   "source": [
    "query = \"\"\"\n",
    "Are Jeff and Eddy neighbors?\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "da3a23b9-0dde-4cc2-83cc-b9344b5983d4",
   "metadata": {
    "height": 194
   },
   "outputs": [],
   "source": [
    "prompt = f\"\"\"\n",
    "Given this context: ```{context}```,\n",
    "\n",
    "and the following query:\n",
    "```{query}```\n",
    "\n",
    "Please answer the questions in the query and explain your reasoning.\n",
    "If there is not enough informaton to answer, please say\n",
    "\"I do not have enough information to answer this questions.\"\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "81d63881-485c-4170-9bb0-a935ea79e4de",
   "metadata": {},
   "source": [
    "- First, use the 7B parameter chat model (`llama-2-7b-chat`) for the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "544f3d8b-14e4-497b-9a3e-3bcc9f9f623c",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure, I'd be happy to help! Based on the information provided, we can answer the query as follows:\n",
      "\n",
      "Are Jeff and Eddy neighbors?\n",
      "\n",
      "No, Jeff and Eddy are not neighbors.\n",
      "\n",
      "Reasoning:\n",
      "\n",
      "* Jeff and Tommy are neighbors (given)\n",
      "* Tommy and Eddy are not neighbors (given)\n",
      "\n",
      "So, Jeff and Eddy are not neighbors because they are not in a direct relationship (neighborhood) with each other.\n"
     ]
    }
   ],
   "source": [
    "response_7b_chat = llama(prompt,\n",
    "                        model=\"togethercomputer/llama-2-7b-chat\")\n",
    "print(response_7b_chat)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd66b3dc-a373-4c30-8554-9ef855a076c2",
   "metadata": {},
   "source": [
    "- Now, use the 13B parameter chat model (`llama-2-13b-chat`) for the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "6fa0f0e9-d26e-4736-a053-abe175b4d5f7",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Based on the information provided, I do not have enough information to answer the question \"Are Jeff and Eddy neighbors?\" because there is no information about the relationship between Jeff and Eddy. The only information provided is that Tommy and Eddy are not neighbors, but there is no information about Jeff's relationship with either Tommy or Eddy. Therefore, I cannot determine whether Jeff and Eddy are neighbors or not.\n"
     ]
    }
   ],
   "source": [
    "response_13b_chat = llama(prompt,\n",
    "                        model=\"togethercomputer/llama-2-13b-chat\")\n",
    "print(response_13b_chat)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d7dc11-c961-4307-ba92-d0b88b9d8fc3",
   "metadata": {},
   "source": [
    "- Lastly, use the 70B parameter chat model (`llama-2-70b-chat`) for the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "b3477846-ae26-4411-afa8-747261f0d6f9",
   "metadata": {
    "height": 63
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  No, Jeff and Eddy are not neighbors.\n",
      "\n",
      "The given information states that Jeff and Tommy are neighbors, and Tommy and Eddy are not neighbors. Since Tommy is not a neighbor of Eddy, it means that Eddy is not a neighbor of Tommy. Therefore, Jeff, who is a neighbor of Tommy, cannot be a neighbor of Eddy.\n"
     ]
    }
   ],
   "source": [
    "response_70b_chat = llama(prompt,\n",
    "                        model=\"togethercomputer/llama-2-70b-chat\")\n",
    "print(response_70b_chat)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1017fdb2-3e5d-4f4b-862d-acdb3dc41e5c",
   "metadata": {},
   "source": [
    "#### Model-Graded Evaluation: Reasoning\n",
    "\n",
    "- Again, ask a LLM to compare the three responses.\n",
    "- Create a `prompt` that will evaluate these three responses using 70B parameter chat model (`llama-2-70b-chat`).\n",
    "- In the `prompt`, provide the `context`, `query`,\"name of the models\", and the \"response\" generated by each model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "2c742104-ef6d-43f6-afa0-6f8ae67ace4a",
   "metadata": {
    "height": 449
   },
   "outputs": [],
   "source": [
    "prompt = f\"\"\"\n",
    "Given the context `context:`,\n",
    "Also also given the query (the task): `query:`\n",
    "and given the name of several models: `mode:<name of model>,\n",
    "as well as the response generated by that model: `response:`\n",
    "\n",
    "Provide an evaluation of each model's response:\n",
    "- Does it answer the query accurately?\n",
    "- Does it provide a contradictory response?\n",
    "- Are there any other interesting characteristics of the model's output?\n",
    "\n",
    "Then compare the models based on their evaluation \\\n",
    "and recommend the models that perform the best.\n",
    "\n",
    "context: ```{context}```\n",
    "\n",
    "model: llama-2-7b-chat\n",
    "response: ```{response_7b_chat}```\n",
    "\n",
    "model: llama-2-13b-chat\n",
    "response: ```{response_13b_chat}```\n",
    "\n",
    "model: llama-2-70b-chat\n",
    "response: ``{response_70b_chat}```\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "dc6f1cde-68e2-4e67-96da-a98dd2e6bc45",
   "metadata": {
    "height": 80
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Evaluation of each model's response:\n",
      "\n",
      "1. llama-2-7b-chat:\n",
      "\t* Accuracy: The model's response accurately answers the query by stating that Jeff and Eddy are not neighbors.\n",
      "\t* Contradictory response: No, the response does not provide a contradictory answer.\n",
      "\t* Other characteristics: The model's output provides a clear and concise explanation of the reasoning behind the answer, citing the given information and using logical deduction.\n",
      "2. llama-2-13b-chat:\n",
      "\t* Accuracy: The model's response does not provide an answer to the query, stating that there is not enough information to determine whether Jeff and Eddy are neighbors.\n",
      "\t* Contradictory response: No, the response does not provide a contradictory answer.\n",
      "\t* Other characteristics: The model's output acknowledges the limitations of the given information and does not make an incorrect assumption or provide a misleading answer.\n",
      "3. llama-2-70b-chat:\n",
      "\t* Accuracy: The model's response accurately states that Jeff and Eddy are not neighbors.\n",
      "\t* Contradictory response: No, the response does not provide a contradictory answer.\n",
      "\t* Other characteristics: The model's output provides a brief and straightforward answer without elaborating on the reasoning behind it.\n",
      "\n",
      "Comparison of the models based on their evaluation:\n",
      "\n",
      "1. llama-2-7b-chat: The model provides an accurate answer and a clear explanation of the reasoning behind it. It effectively uses the given information to deduce the answer.\n",
      "2. llama-2-70b-chat: The model also provides an accurate answer, but its output is less detailed and does not offer an explanation.\n",
      "3. llama-2-13b-chat: The model does not provide an answer, stating that there is not enough information. While it acknowledges the limitations of the given information, it does not offer any insight or deduction.\n",
      "\n",
      "Recommendation:\n",
      "Based on their performance in answering the query accurately and providing informative responses, llama-2-7b-chat and llama-2-70b-chat are the top-performing models. Therefore, the recommendation is to use either of these models for tasks that require deducing relationships between entities based on given information.\n"
     ]
    }
   ],
   "source": [
    "response_eval = llama(prompt, \n",
    "                      model=\"togethercomputer/llama-2-70b-chat\")\n",
    "\n",
    "print(response_eval)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a56dd65",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}