00_Prerequisites/bedrock_basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "88a5ab2f-d044-4956-b75b-7408d9c3e323",
   "metadata": {},
   "source": [
    "# Amazon Bedrock boto3 Prerequisites\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*\n",
    "\n",
    "---\n",
    "\n",
    "In this demo notebook, we demonstrate how to use the [`boto3` Python SDK](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html) to work with [Amazon Bedrock](https://aws.amazon.com/bedrock/) Foundation Models.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6aeedd9f-f0a3-4f8e-934d-22f6f7a89de5",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "\n",
    "Run the cells in this section to install the packages needed by the notebooks in this workshop. ⚠️ You will see pip dependency errors, you can safely ignore these errors. ⚠️\n",
    "\n",
    "IGNORE ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "108c611c-7246-45c4-9f1e-76888b5076eb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install --no-build-isolation --force-reinstall \\\n",
    "    \"boto3>=1.28.57\" \\\n",
    "    \"awscli>=1.29.57\" \\\n",
    "    \"botocore>=1.31.57\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27610c0f-7de6-4440-8f76-decf30e3c5ca",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Create the boto3 client\n",
    "\n",
    "Interaction with the Bedrock API is done via the AWS SDK for Python: [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/index.html).\n",
    "\n",
    "#### Use different clients\n",
    "The boto3 provides different clients for Amazon Bedrock to perform different actions. The actions for [`InvokeModel`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModel.html) and [`InvokeModelWithResponseStream`](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) are supported by Amazon Bedrock Runtime where as other operations, such as [ListFoundationModels](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_ListFoundationModels.html), are handled via [Amazon Bedrock client](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_Operations_Amazon_Bedrock.html).\n",
    "\n",
    "\n",
    "#### Use the default credential chain\n",
    "\n",
    "If you are running this notebook from [Amazon Sagemaker Studio](https://aws.amazon.com/sagemaker/studio/) and your Sagemaker Studio [execution role](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html) has permissions to access Bedrock you can just run the cells below as-is. This is also the case if you are running these notebooks from a computer whose default AWS credentials have access to Bedrock.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae2b2a05-78a9-40ca-9b5e-121030f9ede1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import sys\n",
    "\n",
    "import boto3\n",
    "\n",
    "boto3_bedrock = boto3.client('bedrock')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e9174c4-326a-463e-92e1-8c7e47111269",
   "metadata": {},
   "source": [
    "#### Validate the connection\n",
    "\n",
    "We can check the client works by trying out the `list_foundation_models()` method, which will tell us all the models available for us to use "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f67b4466-12ff-4975-9811-7a19c6206604",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "boto3_bedrock.list_foundation_models()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f690043-df45-448f-8fa6-1ea8b06f1087",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## `InvokeModel` body and output\n",
    "\n",
    "The `invoke_model()` method of the Amazon Bedrock runtime client (`InvokeModel` API) will be the primary method we use for most of our Text Generation and Processing tasks - whichever model we're using.\n",
    "\n",
    "Although the method is shared, the format of input and output varies depending on the foundation model used - as described below:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4650fa3-a831-4039-9fd6-749926d35979",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "### Amazon Titan Large\n",
    "\n",
    "#### Input\n",
    "```json\n",
    "{   \n",
    "    \"inputText\": \"<prompt>\",\n",
    "    \"textGenerationConfig\" : { \n",
    "        \"maxTokenCount\": 512,\n",
    "        \"stopSequences\": [],\n",
    "        \"temperature\": 0.1,  \n",
    "        \"topP\": 0.9\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputTextTokenCount\": 613,\n",
    "    \"results\": [{\n",
    "        \"tokenCount\": 219,\n",
    "        \"outputText\": \"<output>\"\n",
    "    }]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7adb6bee-7654-4269-9127-9afa4e823454",
   "metadata": {},
   "source": [
    "### Anthropic Claude\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"prompt\": \"\\n\\nHuman:<prompt>\\n\\nAnswer:\",\n",
    "    \"max_tokens_to_sample\": 300,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_k\": 250,\n",
    "    \"top_p\": 1,\n",
    "    \"stop_sequences\": [\"\\n\\nHuman:\"]\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"completion\": \"<output>\",\n",
    "    \"stop_reason\": \"stop_sequence\"\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb94aa2f-30da-499b-b3f5-02f102dbb1ea",
   "metadata": {},
   "source": [
    "### Stability AI Stable Diffusion XL\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"text_prompts\": [\n",
    "        {\"text\": \"this is where you place your input text\"}\n",
    "    ],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 0,\n",
    "    \"steps\": 50\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{ \n",
    "    \"result\": \"success\", \n",
    "    \"artifacts\": [\n",
    "        {\n",
    "            \"seed\": 123, \n",
    "            \"base64\": \"<image in base64>\",\n",
    "            \"finishReason\": \"SUCCESS\"\n",
    "        },\n",
    "        //...\n",
    "    ]\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80f4adca-cfc4-439b-84b7-e528398684e3",
   "metadata": {
    "tags": []
   },
   "source": [
    "---\n",
    "\n",
    "## Common inference parameter definitions\n",
    "\n",
    "### Randomness and Diversity\n",
    "\n",
    "Foundation models generally support the following parameters to control randomness and diversity in the \n",
    "response.\n",
    "\n",
    "**Temperature** – Large language models use probability to construct the words in a sequence. For any \n",
    "given next word, there is a probability distribution of options for the next word in the sequence. When \n",
    "you set the temperature closer to zero, the model tends to select the higher-probability words. When \n",
    "you set the temperature further away from zero, the model may select a lower-probability word.\n",
    "\n",
    "In technical terms, the temperature modulates the probability density function for the next tokens, \n",
    "implementing the temperature sampling technique. This parameter can deepen or flatten the density \n",
    "function curve. A lower value results in a steeper curve with more deterministic responses, and a higher \n",
    "value results in a flatter curve with more random responses.\n",
    "\n",
    "**Top K** – Temperature defines the probability distribution of potential words, and Top K defines the cut \n",
    "off where the model no longer selects the words. For example, if K=50, the model selects from 50 of the \n",
    "most probable words that could be next in a given sequence. This reduces the probability that an unusual \n",
    "word gets selected next in a sequence.\n",
    "In technical terms, Top K is the number of the highest-probability vocabulary tokens to keep for Top-\n",
    "K-filtering - This limits the distribution of probable tokens, so the model chooses one of the highest-\n",
    "probability tokens.\n",
    "\n",
    "**Top P** – Top P defines a cut off based on the sum of probabilities of the potential choices. If you set Top \n",
    "P below 1.0, the model considers the most probable options and ignores less probable ones. Top P is \n",
    "similar to Top K, but instead of capping the number of choices, it caps choices based on the sum of their \n",
    "probabilities.\n",
    "For the example prompt \"I hear the hoof beats of ,\" you may want the model to provide \"horses,\" \n",
    "\"zebras\" or \"unicorns\" as the next word. If you set the temperature to its maximum, without capping \n",
    "Top K or Top P, you increase the probability of getting unusual results such as \"unicorns.\" If you set the \n",
    "temperature to 0, you increase the probability of \"horses.\" If you set a high temperature and set Top K or \n",
    "Top P to the maximum, you increase the probability of \"horses\" or \"zebras,\" and decrease the probability \n",
    "of \"unicorns.\"\n",
    "\n",
    "### Length\n",
    "\n",
    "The following parameters control the length of the generated response.\n",
    "\n",
    "**Response length** – Configures the minimum and maximum number of tokens to use in the generated \n",
    "response.\n",
    "\n",
    "**Length penalty** – Length penalty optimizes the model to be more concise in its output by penalizing \n",
    "longer responses. Length penalty differs from response length as the response length is a hard cut off for \n",
    "the minimum or maximum response length.\n",
    "\n",
    "In technical terms, the length penalty penalizes the model exponentially for lengthy responses. 0.0 \n",
    "means no penalty. Set a value less than 0.0 for the model to generate longer sequences, or set a value \n",
    "greater than 0.0 for the model to produce shorter sequences.\n",
    "\n",
    "### Repetitions\n",
    "\n",
    "The following parameters help control repetition in the generated response.\n",
    "\n",
    "**Repetition penalty (presence penalty)** – Prevents repetitions of the same words (tokens) in responses. \n",
    "1.0 means no penalty. Greater than 1.0 decreases repetition."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce22c308-ebbf-4ef5-a823-832b7c236e31",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Try out the models\n",
    "\n",
    "With some theory out of the way, let's see the models in action! Run the cells below to see basic, synchronous example invocations for each model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a0a79b9",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import botocore\n",
    "import json \n",
    "\n",
    "bedrock_runtime = boto3.client('bedrock-runtime')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "893872fe-04fa-4f09-9736-6c6173ec1fc2",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Amazon Titan Large"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7df55eed-a3cf-426c-95ea-ec60dade6477",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "174a8eb1-f9a4-4946-bfe2-550d21487f48",
   "metadata": {},
   "source": [
    "Next, we will construct the body with the `prompt_data` above, and add a optional parameters like `topP` and `temperature`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dd2bb671-6b10-4948-9e5e-95d6ced3b86f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "try:\n",
    "\n",
    "    body = json.dumps({\"inputText\": prompt_data, \"textGenerationConfig\" : {\"topP\":0.95, \"temperature\":0.2}})\n",
    "    modelId = \"amazon.titan-tg1-large\"\n",
    "    accept = \"application/json\"\n",
    "    contentType = \"application/json\"\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"results\")[0].get(\"outputText\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d7c0fe6-576a-4380-89aa-726bab5d65ff",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Anthropic Claude"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a725de2-bdea-4d86-b12d-d1d7cdda010b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ba33ac0-fa16-4c4f-b882-e838d0cb5830",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"prompt\": prompt_data, \"max_tokens_to_sample\": 500})\n",
    "modelId = \"anthropic.claude-instant-v1\"  # change this to use a different version from the model provider\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body.get(\"completion\"))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc498bea",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Stability Stable Diffusion XL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "173e51a2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"a landscape with trees\"\n",
    "body = json.dumps({\n",
    "    \"text_prompts\": [{\"text\": prompt_data}],\n",
    "    \"cfg_scale\": 10,\n",
    "    \"seed\": 20,\n",
    "    \"steps\": 50\n",
    "})\n",
    "modelId = \"stability.stable-diffusion-xl-v1\"\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    print(response_body[\"result\"])\n",
    "    print(f'{response_body.get(\"artifacts\")[0].get(\"base64\")[0:80]}...')\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a271fa6-13fd-480a-87a5-3702d29a5c43",
   "metadata": {},
   "source": [
    "**Note:** The output is a [base64 encoded](https://docs.python.org/3/library/base64.html) string of the image data. You can use any image processing library (such as [Pillow](https://pillow.readthedocs.io/en/stable/)) to decode the image as in the example below:\n",
    "\n",
    "```python\n",
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45072848-000a-4c22-8f08-2647e5c2230e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import base64\n",
    "import io\n",
    "from PIL import Image\n",
    "\n",
    "base_64_img_str = response_body.get(\"artifacts\")[0].get(\"base64\")\n",
    "image = Image.open(io.BytesIO(base64.decodebytes(bytes(base_64_img_str, \"utf-8\"))))\n",
    "image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4621a301-53e4-4182-9fce-8ee422813e9d",
   "metadata": {},
   "source": [
    "## Generate streaming output\n",
    "\n",
    "For large language models, it can take noticeable time to generate long output sequences. Rather than waiting for the entire response to be available, latency-sensitive applications may like to **stream** the response to users.\n",
    "\n",
    "Run the code below to see how you can achieve this with Bedrock's `invoke_model_with_response_stream()` method - returning the response body in separate chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c69627e3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "prompt_data = \"\"\"Command: Write me a blog about making strong business decisions as a leader.\n",
    "\n",
    "Blog:\n",
    "\"\"\"\n",
    "\n",
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-tg1-large\"  # (Change this, and the request body, to try different models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    stream = response.get('body')\n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                if 'outputText' in chunk_obj:\n",
    "                    text = chunk_obj.get('outputText', None)\n",
    "                    print(text,end='')\n",
    "                    if not text :\n",
    "                        break\n",
    "                    #text = chunk_obj['outputText']\n",
    "                    clear_output(wait=True)\n",
    "                    output.append(text)\n",
    "                    display_markdown(Markdown(''.join(output)))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adf097eb-1bfa-41f1-8135-cdd4ad6e9983",
   "metadata": {},
   "source": [
    "### Anthropic Claude (messages API)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd800ef5-78f7-43d4-b73e-c37c7609cb40",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you'd like to try your own prompt, edit this parameter!\n",
    "prompt_data = \"\"\"Human: Write me 500 word paragraph about making strong business decisions as a leader.\n",
    "\n",
    "Assistant:\n",
    "\"\"\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ef24e0c-dda8-4f5d-bbcb-db389e67e713",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "messages_API_body = {\n",
    "    \"anthropic_version\": \"bedrock-2023-05-31\", \n",
    "    \"max_tokens\": 512,\n",
    "    \"messages\": [\n",
    "        {\n",
    "            \"role\": \"user\",\n",
    "            \"content\": [\n",
    "                {\n",
    "                    \"type\": \"text\",\n",
    "                    \"text\": prompt_data\n",
    "                }\n",
    "            ]\n",
    "        }\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be7fbbe8-78b8-4071-b80c-3f954185fad8",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "from IPython.display import clear_output, display, display_markdown, Markdown\n",
    "\n",
    "body = json.dumps(messages_API_body)\n",
    "modelId = \"anthropic.claude-v2\"  # (Change this to try different model versions)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    \n",
    "    stream = response.get('body')\n",
    "    \n",
    "    \n",
    "    output = []\n",
    "\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                if 'delta' in chunk_obj:\n",
    "                    delta_obj = chunk_obj.get('delta', None)\n",
    "                    if delta_obj:\n",
    "                        text = delta_obj.get('text', None)\n",
    "                        print(text,end='')\n",
    "                        if not text :\n",
    "                            break\n",
    "                    # output.append(text[0]) if type(text) is list and len(text)>0 else output.append('')\n",
    "                    # display_markdown(Markdown(text))\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ef3451d-b66a-4b11-a1ed-734bf9e7bbec",
   "metadata": {},
   "source": [
    "## Generate embeddings\n",
    "\n",
    "Use text embeddings to convert text into meaningful vector representations. You input a body of text \n",
    "and the output is a (1 x n) vector. You can use embedding vectors for a wide variety of applications. \n",
    "Bedrock currently offers Titan Embeddings for text embedding that supports text similarity (finding the \n",
    "semantic similarity between bodies of text) and text retrieval (such as search).\n",
    "\n",
    "At the time of writing you can use `amazon.titan-embed-text-v1` as embedding model via the API. The input text size is 8192 tokens and the output vector length is 1536.\n",
    "\n",
    "To use a text embeddings model, use the InvokeModel API operation or the Python SDK.\n",
    "Use InvokeModel to retrieve the vector representation of the input text from the specified model.\n",
    "\n",
    "\n",
    "\n",
    "#### Input\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"inputText\": \"<text>\"\n",
    "}\n",
    "```\n",
    "\n",
    "#### Output\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"embedding\": []\n",
    "}\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9645dbd8",
   "metadata": {},
   "source": [
    "Let's see how to generate embeddings of some text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1085cc56",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "prompt_data = \"Amazon Bedrock supports foundation models from industry-leading providers such as \\\n",
    "AI21 Labs, Anthropic, Stability AI, and Amazon. Choose the model that is best suited to achieving \\\n",
    "your unique goals.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5c54b424",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "body = json.dumps({\"inputText\": prompt_data})\n",
    "modelId = \"amazon.titan-embed-text-v1\"  # (Change this to try different embedding models)\n",
    "accept = \"application/json\"\n",
    "contentType = \"application/json\"\n",
    "\n",
    "try:\n",
    "\n",
    "    response = bedrock_runtime.invoke_model(\n",
    "        body=body, modelId=modelId, accept=accept, contentType=contentType\n",
    "    )\n",
    "    response_body = json.loads(response.get(\"body\").read())\n",
    "\n",
    "    embedding = response_body.get(\"embedding\")\n",
    "    print(f\"The embedding vector has {len(embedding)} values\\n{embedding[0:3]+['...']+embedding[-3:]}\")\n",
    "\n",
    "except botocore.exceptions.ClientError as error:\n",
    "\n",
    "    if error.response['Error']['Code'] == 'AccessDeniedException':\n",
    "           print(f\"\\x1b[41m{error.response['Error']['Message']}\\\n",
    "                \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "                 \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "                 \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")\n",
    "\n",
    "    else:\n",
    "        raise error\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a48a0e8-147d-4525-a6b2-68a09af1b2c4",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "In this notebook we showed some basic examples of invoking Amazon Bedrock models using the AWS Python SDK. You're now ready to explore the other labs to dive deeper on different use-cases and patterns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8bb76df-4e99-4ebe-a954-53992ad317dc",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-east-1:081325390199:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}