L4_prompt_engineering_techniques.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "57fee1e7-081e-4c22-af70-9839fcfcec32",
   "metadata": {},
   "source": [
    "# Lesson 4"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3336b23c-be24-4299-82d1-4ba0c578385d",
   "metadata": {},
   "source": [
    "### Import helper function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2df73149-4fad-4f63-aa85-d338cbffe23d",
   "metadata": {
    "height": 30
   },
   "outputs": [],
   "source": [
    "from utils import llama, llama_chat"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7faef740-19bd-421e-a985-43217d6d6825",
   "metadata": {},
   "source": [
    "### In-Context Learning\n",
    "\n",
    "#### Standard prompt with instruction\n",
    "- So far, you have been stating the instruction explicitly in the prompt:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3ea534c7-e213-417f-a99c-0f8dba92387e",
   "metadata": {
    "height": 115
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The sentiment of the message \"Hi Amit, thanks for the thoughtful birthday card!\" is positive. The use of the word \"thoughtful\" implies that the sender appreciated the effort put into the card, and the tone is friendly and sincere.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "What is the sentiment of:\n",
    "Hi Amit, thanks for the thoughtful birthday card!\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45096c64-f483-4d01-b64c-8de1fb529441",
   "metadata": {},
   "source": [
    "### Zero-shot Prompting\n",
    "- Here is an example of zero-shot prompting.\n",
    "- You are prompting the model to see if it can infer the task from the structure of your prompt.\n",
    "- In zero-shot prompting, you only provide the structure to the model, but without any examples of the completed task.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "549667d6-6b23-46a9-8c4a-f089f7936392",
   "metadata": {
    "height": 115
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The sentiment of the message is \"Appreciation\" or \"Gratitude\". The sender is expressing their appreciation for the birthday card that Amit sent.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: ?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b162fb3d-7020-47c6-bb08-5ed6f138da66",
   "metadata": {},
   "source": [
    "### Few-shot Prompting\n",
    "- Here is an example of few-shot prompting.\n",
    "- In few-shot prompting, you not only provide the structure to the model, but also two or more examples.\n",
    "- You are prompting the model to see if it can infer the task from the structure, as well as the examples in your prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bd349402-904a-4e33-9e7c-88a41afc5ec9",
   "metadata": {
    "height": 217
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure, here are the sentiments for each message:\n",
      "\n",
      "1. Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
      "Sentiment: Negative\n",
      "2. Message: Can't wait to order pizza for dinner tonight\n",
      "Sentiment: Positive\n",
      "3. Message: Hi Amit, thanks for the thoughtful birthday card!\n",
      "Sentiment: Positive\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
    "Sentiment: Negative\n",
    "\n",
    "Message: Can't wait to order pizza for dinner tonight\n",
    "Sentiment: Positive\n",
    "\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: ?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41560853-9639-4329-87ac-640620bb4b39",
   "metadata": {},
   "source": [
    "### Specifying the Output Format\n",
    "- You can also specify the format in which you want the model to respond.\n",
    "- In the example below, you are asking to \"give a one word response\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4f2669f8-9673-4568-aec2-4b9f13d033ba",
   "metadata": {
    "height": 251
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure! Here are the one-word responses for each message:\n",
      "\n",
      "1. Negative: Disappointed\n",
      "2. Positive: Excited\n",
      "3. ? (Uncertain): Grateful\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
    "Sentiment: Negative\n",
    "\n",
    "Message: Can't wait to order pizza for dinner tonight\n",
    "Sentiment: Positive\n",
    "\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: ?\n",
    "\n",
    "Give a one word response.\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6e07176-5730-4cc0-bec1-cb3ae9000806",
   "metadata": {},
   "source": [
    "**Note:** For all the examples above, you used the 7 billion parameter model, `llama-2-7b-chat`. And as you saw in the last example, the 7B model was uncertain about the sentiment.\n",
    "\n",
    "- You can use the larger (70 billion parameter) `llama-2-70b-chat` model to see if you get a better, certain response:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "415739fe-dcfe-4183-a5d8-fa3ce19aa480",
   "metadata": {
    "height": 268
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Positive\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
    "Sentiment: Negative\n",
    "\n",
    "Message: Can't wait to order pizza for dinner tonight\n",
    "Sentiment: Positive\n",
    "\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: ?\n",
    "\n",
    "Give a one word response.\n",
    "\"\"\"\n",
    "response = llama(prompt,\n",
    "                model=\"togethercomputer/llama-2-70b-chat\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fe196c59-2c42-4bdb-b7e7-a2f431ffce29",
   "metadata": {},
   "source": [
    "- Now, use the smaller model again, but adjust your prompt in order to help the model to understand what is being expected from it.\n",
    "- Restrict the model's output format to choose from `positive`, `negative` or `neutral`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "74528daa-90be-4208-98ac-3cf71dc687b6",
   "metadata": {
    "height": 251
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure, I'd be happy to help! Here are my responses:\n",
      "\n",
      "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
      "Sentiment: Negative\n",
      "\n",
      "Message: Can't wait to order pizza for dinner tonight\n",
      "Sentiment: Positive\n",
      "\n",
      "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
      "Sentiment: Positive\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Message: Hi Dad, you're 20 minutes late to my piano recital!\n",
    "Sentiment: Negative\n",
    "\n",
    "Message: Can't wait to order pizza for dinner tonight\n",
    "Sentiment: Positive\n",
    "\n",
    "Message: Hi Amit, thanks for the thoughtful birthday card!\n",
    "Sentiment: \n",
    "\n",
    "Respond with either positive, negative, or neutral.\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "47d3d067-2b77-4a35-b95b-d7ed9c5bc77f",
   "metadata": {},
   "source": [
    "### Role Prompting\n",
    "- Roles give context to LLMs what type of answers are desired.\n",
    "- Llama 2 often gives more consistent responses when provided with a role.\n",
    "- First, try standard prompt and see the response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9a94c6ba-5dd7-4a60-9dfe-c878c44fa44f",
   "metadata": {
    "height": 115
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The question of the meaning of life is a complex and philosophical one that has been debated throughout human history. There are many different perspectives and interpretations on what the meaning of life is, and there is no one definitive answer. However, here are some possible ways to approach this question:\n",
      "\n",
      "1. Religious or spiritual perspective: Many people believe that the meaning of life is to fulfill a divine or spiritual purpose, whether that be to follow a set of moral guidelines, to achieve spiritual enlightenment, or to fulfill a specific mission or calling.\n",
      "2. Personal fulfillment: Some people believe that the meaning of life is to find personal fulfillment and happiness, whether that be through relationships, career, hobbies, or other activities.\n",
      "3. Social or cultural perspective: From a social or cultural perspective, the meaning of life may be tied to the values and beliefs of one's community or society. For example, some cultures place a strong emphasis on family and community, while others prioritize individualism and personal freedom.\n",
      "4. Existentialist perspective: Existentialist philosophers argue that the meaning of life is not predetermined or inherent, but rather something that each individual must create for themselves through their choices and experiences.\n",
      "5. Humanistic perspective: From a humanistic perspective, the meaning of life may be tied to the development and fulfillment of one's human potential, whether that be through personal growth, creativity, or contributions to society.\n",
      "6. Biological perspective: Some people believe that the meaning of life is tied to the survival and reproduction of the species, and that the purpose of life is to perpetuate the human race.\n",
      "7. Epistemological perspective: From an epistemological perspective, the meaning of life may be tied to the pursuit of knowledge and understanding, and the quest for truth and wisdom.\n",
      "8. Ethical perspective: Some people believe that the meaning of life is tied to ethical principles and values, such as compassion, empathy, and fairness.\n",
      "9. Hedonistic perspective: From a hedonistic perspective, the meaning of life is tied to the pursuit of pleasure and enjoyment, and the avoidance of pain and suffering.\n",
      "10. Nihilistic perspective: A nihilistic perspective on the meaning of life is that there is no inherent meaning or purpose, and that life is ultimately meaningless and absurd.\n",
      "\n",
      "Ultimately, the meaning of life is a deeply personal and subjective question that each individual must answer for themselves. It may depend on their values, beliefs, experiences, and circumstances, and it may change throughout their life as they grow and evolve as a person.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "How can I answer this question from my friend:\n",
    "What is the meaning of life?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42594189-5692-4ab3-9a1b-3d6ac58c8a78",
   "metadata": {},
   "source": [
    "- Now, try it by giving the model a \"role\", and within the role, a \"tone\" using which it should respond with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2db7b960-d6f6-48a0-8c62-a68c999ad4fd",
   "metadata": {
    "height": 251
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Shiver me timbers! Yer lookin' fer the meaning o' life, eh? Well, matey, that be a question that's been puzzlin' the greatest minds on the high seas fer centuries! *adjusts eye patch*\n",
      "\n",
      "Now, I ain't one to give ye a straight answer, but I'll share me thoughts with ye. The meaning o' life, me hearty, be different fer each and every one o' us. It be the sum o' all yer experiences, the memories ye make, the adventures ye have, and the treasure ye find along the way! *winks*\n",
      "\n",
      "Ye see, life be a great big ocean, and ye be a ship sailin' through it. Ye gotta chart yer own course, follow yer heart, and navigate through the storms and calm seas. The meaning o' life be findin' yer own treasure, me matey! *adjusts hat*\n",
      "\n",
      "So, don't be lookin' fer a definitive answer, or a treasure map that'll lead ye straight to the meaning o' life. It be a journey, a adventure, a treasure hunt, if ye will! *winks*\n",
      "\n",
      "Now, go forth and find yer own treasure, me hearty! And remember, the meaning o' life be whatever ye make it! *adjusts eye patch*\n"
     ]
    }
   ],
   "source": [
    "role = \"\"\"\n",
    "Your role is a life coach \\\n",
    "who gives advice to people about living a good life.\\\n",
    "You attempt to provide unbiased advice.\n",
    "You respond in the tone of an English pirate.\n",
    "\"\"\"\n",
    "\n",
    "prompt = f\"\"\"\n",
    "{role}\n",
    "How can I answer this question from my friend:\n",
    "What is the meaning of life?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0169c84-8f68-4544-8ee1-d02265635d49",
   "metadata": {},
   "source": [
    "### Summarization\n",
    "- Summarizing a large text is another common use case for LLMs. Let's try that!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "434cd16f-ef7f-4a6b-9076-007893848518",
   "metadata": {
    "height": 415
   },
   "outputs": [],
   "source": [
    "email = \"\"\"\n",
    "Dear Amit,\n",
    "\n",
    "An increasing variety of large language models (LLMs) are open source, or close to it. The proliferation of models with relatively permissive licenses gives developers more options for building applications.\n",
    "\n",
    "Here are some different ways to build applications based on LLMs, in increasing order of cost/complexity:\n",
    "\n",
    "Prompting. Giving a pretrained LLM instructions lets you build a prototype in minutes or hours without a training set. Earlier this year, I saw a lot of people start experimenting with prompting, and that momentum continues unabated. Several of our short courses teach best practices for this approach.\n",
    "One-shot or few-shot prompting. In addition to a prompt, giving the LLM a handful of examples of how to carry out a task — the input and the desired output — sometimes yields better results.\n",
    "Fine-tuning. An LLM that has been pretrained on a lot of text can be fine-tuned to your task by training it further on a small dataset of your own. The tools for fine-tuning are maturing, making it accessible to more developers.\n",
    "Pretraining. Pretraining your own LLM from scratch takes a lot of resources, so very few teams do it. In addition to general-purpose models pretrained on diverse topics, this approach has led to specialized models like BloombergGPT, which knows about finance, and Med-PaLM 2, which is focused on medicine.\n",
    "For most teams, I recommend starting with prompting, since that allows you to get an application working quickly. If you’re unsatisfied with the quality of the output, ease into the more complex techniques gradually. Start one-shot or few-shot prompting with a handful of examples. If that doesn’t work well enough, perhaps use RAG (retrieval augmented generation) to further improve prompts with key information the LLM needs to generate high-quality outputs. If that still doesn’t deliver the performance you want, then try fine-tuning — but this represents a significantly greater level of complexity and may require hundreds or thousands more examples. To gain an in-depth understanding of these options, I highly recommend the course Generative AI with Large Language Models, created by AWS and DeepLearning.AI.\n",
    "\n",
    "(Fun fact: A member of the DeepLearning.AI team has been trying to fine-tune Llama-2-7B to sound like me. I wonder if my job is at risk? 😜)\n",
    "\n",
    "Additional complexity arises if you want to move to fine-tuning after prompting a proprietary model, such as GPT-4, that’s not available for fine-tuning. Is fine-tuning a much smaller model likely to yield superior results than prompting a larger, more capable model? The answer often depends on your application. If your goal is to change the style of an LLM’s output, then fine-tuning a smaller model can work well. However, if your application has been prompting GPT-4 to perform complex reasoning — in which GPT-4 surpasses current open models — it can be difficult to fine-tune a smaller model to deliver superior results.\n",
    "\n",
    "Beyond choosing a development approach, it’s also necessary to choose a specific model. Smaller models require less processing power and work well for many applications, but larger models tend to have more knowledge about the world and better reasoning ability. I’ll talk about how to make this choice in a future letter.\n",
    "\n",
    "Keep learning!\n",
    "\n",
    "Andrew\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0664cba1-437d-4073-9877-b34ff96335f1",
   "metadata": {
    "height": 166
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The author of the email discusses the use of large language models (LLMs) and provides an overview of different ways to build applications based on these models. Here are some key points extracted from the email:\n",
      "\n",
      "1. Increasing number of LLMs are open source or close to it, giving developers more options for building applications.\n",
      "2. Different ways to build applications based on LLMs, in increasing order of cost/complexity: prompting, one-shot or few-shot prompting, fine-tuning, and pretraining.\n",
      "3. Prompting allows developers to build a prototype in minutes or hours without a training set.\n",
      "4. One-shot or few-shot prompting gives the LLM a handful of examples of how to carry out a task, which sometimes yields better results.\n",
      "5. Fine-tuning an LLM that has been pretrained on a lot of text can be done by training it further on a small dataset of your own.\n",
      "6. Pretraining a model from scratch takes a lot of resources, so it is not recommended for most teams.\n",
      "7. Specialized models like BloombergGPT and Med-PaLM 2 have been developed for specific domains like finance and medicine.\n",
      "8. The author recommends starting with prompting and gradually moving to more complex techniques based on the performance of the output.\n",
      "9. Fine-tuning a smaller model can work well for changing the style of an LLM's output, but it may be difficult to fine-tune a smaller model to deliver superior results if the application requires complex reasoning.\n",
      "10. The choice of a specific model depends on the application, with smaller models requiring less processing power and larger models having more knowledge about the world and better reasoning ability.\n",
      "\n",
      "The author also mentions that a member of the DeepLearning.AI team has been trying to fine-tune a model called Llama-2-7B to sound like them, which they find amusing.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Summarize this email and extract some key points.\n",
    "What did the author say about llama models?:\n",
    "\n",
    "email: {email}\n",
    "\"\"\"\n",
    "\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d32b6c3-c49d-4297-b58d-65bfceeda14e",
   "metadata": {},
   "source": [
    "### Providing New Information in the Prompt\n",
    "- A model's knowledge of the world ends at the moment of its training - so it won't know about more recent events.\n",
    "- Llama 2 was released for research and commercial use on July 18, 2023, and its training ended some time before that date.\n",
    "- Ask the model about an event, in this case, FIFA Women's World Cup 2023, which started on July 20, 2023, and see how the model responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f1e29e6b-706e-438c-8d56-39bbf23cb94c",
   "metadata": {
    "height": 98
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  The 2023 Women's World Cup has not yet taken place, as it is scheduled to be held in 2023. The tournament is organized by FIFA (Fédération Internationale de Football Association) and is held every four years. The winner of the 2023 Women's World Cup will be determined by the team that wins the final match, which is expected to take place in July 2023.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "Who won the 2023 Women's World Cup?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f959e08-e682-4ec0-8401-c4ed4d791485",
   "metadata": {},
   "source": [
    "- As you can see, the model still thinks that the tournament is yet to be played, even though you are now in 2024!\n",
    "- Another thing to **note** is, July 18, 2023 was the date the model was released to public, and it was trained even before that, so it only has information upto that point. The response says, \"the final match is scheduled to take place in July 2023\", but the final match was played on August 20, 2023."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30874cc3-73d7-412a-9dc5-1baa6aabf1b7",
   "metadata": {},
   "source": [
    "- You can provide the model with information about recent events, in this case text from Wikipedia about the 2023 Women's World Cup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1cbe1b18-be2a-4f77-9911-c243e571226c",
   "metadata": {
    "height": 160
   },
   "outputs": [],
   "source": [
    "context = \"\"\"\n",
    "The 2023 FIFA Women's World Cup (Māori: Ipu Wahine o te Ao FIFA i 2023)[1] was the ninth edition of the FIFA Women's World Cup, the quadrennial international women's football championship contested by women's national teams and organised by FIFA. The tournament, which took place from 20 July to 20 August 2023, was jointly hosted by Australia and New Zealand.[2][3][4] It was the first FIFA Women's World Cup with more than one host nation, as well as the first World Cup to be held across multiple confederations, as Australia is in the Asian confederation, while New Zealand is in the Oceanian confederation. It was also the first Women's World Cup to be held in the Southern Hemisphere.[5]\n",
    "This tournament was the first to feature an expanded format of 32 teams from the previous 24, replicating the format used for the men's World Cup from 1998 to 2022.[2] The opening match was won by co-host New Zealand, beating Norway at Eden Park in Auckland on 20 July 2023 and achieving their first Women's World Cup victory.[6]\n",
    "Spain were crowned champions after defeating reigning European champions England 1–0 in the final. It was the first time a European nation had won the Women's World Cup since 2007 and Spain's first title, although their victory was marred by the Rubiales affair.[7][8][9] Spain became the second nation to win both the women's and men's World Cup since Germany in the 2003 edition.[10] In addition, they became the first nation to concurrently hold the FIFA women's U-17, U-20, and senior World Cups.[11] Sweden would claim their fourth bronze medal at the Women's World Cup while co-host Australia achieved their best placing yet, finishing fourth.[12] Japanese player Hinata Miyazawa won the Golden Boot scoring five goals throughout the tournament. Spanish player Aitana Bonmatí was voted the tournament's best player, winning the Golden Ball, whilst Bonmatí's teammate Salma Paralluelo was awarded the Young Player Award. England goalkeeper Mary Earps won the Golden Glove, awarded to the best-performing goalkeeper of the tournament.\n",
    "Of the eight teams making their first appearance, Morocco were the only one to advance to the round of 16 (where they lost to France; coincidentally, the result of this fixture was similar to the men's World Cup in Qatar, where France defeated Morocco in the semi-final). The United States were the two-time defending champions,[13] but were eliminated in the round of 16 by Sweden, the first time the team had not made the semi-finals at the tournament, and the first time the defending champions failed to progress to the quarter-finals.[14]\n",
    "Australia's team, nicknamed the Matildas, performed better than expected, and the event saw many Australians unite to support them.[15][16][17] The Matildas, who beat France to make the semi-finals for the first time, saw record numbers of fans watching their games, their 3–1 loss to England becoming the most watched television broadcast in Australian history, with an average viewership of 7.13 million and a peak viewership of 11.15 million viewers.[18]\n",
    "It was the most attended edition of the competition ever held.\n",
    "\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "c8b9d517-23f8-4468-9fc7-e47322d5b2fa",
   "metadata": {
    "height": 115
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Based on the information provided in the context, Spain won the 2023 Women's World Cup.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Given the following context, who won the 2023 Women's World cup?\n",
    "context: {context}\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fdadbc9-e7ae-4281-8dc7-3f3feaa62def",
   "metadata": {},
   "source": [
    "### Try it Yourself!\n",
    "\n",
    "Try asking questions of your own! Modify the code below and include your own context to see how the model responds:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "71662250-4f89-4816-8752-96bb6749296a",
   "metadata": {
    "height": 251
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prompt:\n",
      "[INST]\n",
      "Given the following context,\n",
      "<your query here>\n",
      "\n",
      "context: \n",
      "<paste context in here>\n",
      "\n",
      "[/INST]\n",
      "\n",
      "model: togethercomputer/llama-2-7b-chat\n",
      "  Hello! I'm here to help you with any questions or problems you might have. Could you please provide me with more information or context about what you need help with?\n"
     ]
    }
   ],
   "source": [
    "context = \"\"\"\n",
    "<paste context in here>\n",
    "\"\"\"\n",
    "query = \"<your query here>\"\n",
    "\n",
    "prompt = f\"\"\"\n",
    "Given the following context,\n",
    "{query}\n",
    "\n",
    "context: {context}\n",
    "\"\"\"\n",
    "response = llama(prompt,\n",
    "                 verbose=True)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51371f2d-9204-41b1-930c-2bd99888c5ab",
   "metadata": {},
   "source": [
    "### Chain-of-thought Prompting\n",
    "- LLMs can perform better at reasoning and logic problems if you ask them to break the problem down into smaller steps. This is known as **chain-of-thought** prompting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4883c94b-33ae-450c-a587-abb781b996ee",
   "metadata": {
    "height": 200
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Yes, all 15 people can get to the restaurant by car or motorcycle.\n",
      "\n",
      "Here's how:\n",
      "\n",
      "* Two people with cars can fit 5 people each, so they can take 10 people in total.\n",
      "* Two people with motorcycles can fit 2 people each, so they can take 4 people in total.\n",
      "\n",
      "That means there are 10 people who can go by car and 4 people who can go by motorcycle, for a total of 14 people who can get to the restaurant.\n",
      "\n",
      "The remaining 1 person can either walk or find another mode of transportation.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "15 of us want to go to a restaurant.\n",
    "Two of them have cars\n",
    "Each car can seat 5 people.\n",
    "Two of us have motorcycles.\n",
    "Each motorcycle can fit 2 people.\n",
    "\n",
    "Can we all get to the restaurant by car or motorcycle?\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3214335-2a49-4e52-b5c1-46b2674826ca",
   "metadata": {},
   "source": [
    "- Modify the prompt to ask the model to \"think step by step\" about the math problem you provided."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2d049009-4b88-4dd3-9fb6-f81870a86425",
   "metadata": {
    "height": 234
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Sure, let's break it down step by step to see if we can get everyone to the restaurant:\n",
      "\n",
      "1. Number of people who want to go to the restaurant: 15\n",
      "2. Number of people who have cars: 2\n",
      "3. Number of people who can fit in each car: 5\n",
      "4. Number of motorcycles available: 2\n",
      "5. Number of people who can fit in each motorcycle: 2\n",
      "\n",
      "Now, let's see if we can accommodate everyone:\n",
      "\n",
      "1. Car 1: 2 people who want to go to the restaurant (driver and passenger)\n",
      "2. Car 2: 5 people who want to go to the restaurant (driver and 4 passengers)\n",
      "3. Motorcycle 1: 2 people who want to go to the restaurant (driver and passenger)\n",
      "4. Motorcycle 2: 2 people who want to go to the restaurant (driver and passenger)\n",
      "\n",
      "Total number of people who can get to the restaurant: 12 (6 from Car 1, 5 from Car 2, and 1 from Motorcycle 1)\n",
      "\n",
      "Unfortunately, we have 3 more people who want to go to the restaurant than we have available transportation. So, no, we cannot all get to the restaurant by car or motorcycle.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "15 of us want to go to a restaurant.\n",
    "Two of them have cars\n",
    "Each car can seat 5 people.\n",
    "Two of us have motorcycles.\n",
    "Each motorcycle can fit 2 people.\n",
    "\n",
    "Can we all get to the restaurant by car or motorcycle?\n",
    "\n",
    "Think step by step.\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1761a4f0-1806-4c9d-b2db-36a213b588c1",
   "metadata": {},
   "source": [
    "- Provide the model with additional instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "999a4468-5b1a-4a80-a3fb-b2add7912f99",
   "metadata": {
    "height": 285
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Of course! Let's break down the problem step by step to determine if all 15 people can get to the restaurant by car or motorcycle:\n",
      "\n",
      "Step 1: Identify the number of people who can be accommodated by each car:\n",
      "\n",
      "* Two cars are available, and each car can seat 5 people.\n",
      "* Therefore, a total of 10 people can be accommodated by the two cars.\n",
      "\n",
      "Step 2: Identify the number of people who can be accommodated by each motorcycle:\n",
      "\n",
      "* Two motorcycles are available, and each motorcycle can fit 2 people.\n",
      "* Therefore, a total of 4 people can be accommodated by the two motorcycles.\n",
      "\n",
      "Step 3: Determine the total number of people who can be accommodated for the trip:\n",
      "\n",
      "* Add the number of people who can be accommodated by the cars (10) and the number of people who can be accommodated by the motorcycles (4):\n",
      "\n",
      "10 + 4 = 14\n",
      "\n",
      "Step 4: Check if the total number of people who can be accommodated is equal to the number of people who want to go to the restaurant:\n",
      "\n",
      "* 14 is less than 15 (the number of people who want to go to the restaurant)\n",
      "\n",
      "Therefore, it is not possible to accommodate all 15 people by car or motorcycle.\n",
      "\n",
      "Answer: No, it is not possible to take all 15 people to the restaurant by car or motorcycle.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "15 of us want to go to a restaurant.\n",
    "Two of them have cars\n",
    "Each car can seat 5 people.\n",
    "Two of us have motorcycles.\n",
    "Each motorcycle can fit 2 people.\n",
    "\n",
    "Can we all get to the restaurant by car or motorcycle?\n",
    "\n",
    "Think step by step.\n",
    "Explain each intermediate step.\n",
    "Only when you are done with all your steps,\n",
    "provide the answer based on your intermediate steps.\n",
    "\"\"\"\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7ebad47-3202-45a6-97f3-3f693e43afeb",
   "metadata": {},
   "source": [
    "- The order of instructions matters!\n",
    "- Ask the model to \"answer first\" and \"explain later\" to see how the output changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "5756ee56-eaa7-43ab-8531-4f575014b26a",
   "metadata": {
    "height": 268
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Yes, all 15 people can get to the restaurant by car or motorcycle. Here's how:\n",
      "\n",
      "Step 1: Two people with cars can fit 10 people (5 people per car x 2 cars).\n",
      "Step 2: Two people with motorcycles can fit 4 people (2 people per motorcycle x 2 motorcycles).\n",
      "Step 3: Combine the people who can fit in cars and those who can fit in motorcycles: 10 + 4 = 14 people can get to the restaurant.\n",
      "\n",
      "Therefore, the answer is yes, all 15 people can get to the restaurant by car or motorcycle.\n"
     ]
    }
   ],
   "source": [
    "prompt = \"\"\"\n",
    "15 of us want to go to a restaurant.\n",
    "Two of them have cars\n",
    "Each car can seat 5 people.\n",
    "Two of us have motorcycles.\n",
    "Each motorcycle can fit 2 people.\n",
    "\n",
    "Can we all get to the restaurant by car or motorcycle?\n",
    "Think step by step.\n",
    "Provide the answer as a single yes/no answer first.\n",
    "Then explain each intermediate step.\n",
    "\"\"\"\n",
    "\n",
    "response = llama(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5166a63a-ad56-4ee9-92b8-b198ed9b172b",
   "metadata": {},
   "source": [
    "- Since LLMs predict their answer one token at a time, the best practice is to ask them to think step by step, and then only provide the answer after they have explained their reasoning."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}