advanced_functionality/tensorflow_iris_byom/tensorflow_BYOM_iris.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorFlow BYOM: Train locally and deploy on SageMaker.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "This notebook's CI test result for us-west-2 is as follows. CI test results in other regions can be found at the end of the notebook. \n",
    "\n",
    "![This us-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-2/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "1. [Introduction](#Introduction)\n",
    "2. [Prerequisites and Preprocessing](#Prequisites-and-Preprocessing)\n",
    "    1. [Permissions and environment variables](#Permissions-and-environment-variables)\n",
    "    2. [Model definitions](#Model-definitions)\n",
    "    3. [Data Setup](#Data-setup)\n",
    "3. [Training the network locally](#Training)\n",
    "4. [Set up hosting for the model](#Set-up-hosting-for-the-model)\n",
    "    1. [Export from TensorFlow](#Export-the-model-from-tensorflow)\n",
    "    2. [Import model into SageMaker](#Import-model-into-SageMaker)\n",
    "    3. [Create endpoint](#Create-endpoint) \n",
    "5. [Validate the endpoint for use](#Validate-the-endpoint-for-use)\n",
    "\n",
    "__Note__: Compare this with the [tensorflow bring your own model example](../tensorflow_iris_byom/tensorflow_BYOM_iris.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thie notebook was last tested on a ml.m5.xlarge instance running the Python 3 (TensorFlow 2.3 Python 3.7 CPU Optimized) kernel in SageMaker Studio."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction \n",
    "\n",
    "We will do a classification task, training locally in the box from where this notebook is being run. We then set up a real-time hosted endpoint in SageMaker.\n",
    "\n",
    "Consider the following model definition for IRIS classification. This mode uses the ``tensorflow.estimator.DNNClassifier`` which is a pre-defined estimator module for its model definition.\n",
    "\n",
    "## Prequisites and Preprocessing\n",
    "### Permissions and environment variables\n",
    "\n",
    "Here we set up the linkage and authentication to AWS services. In this notebook we only need the roles used to give learning and hosting access to your data. The Sagemaker SDK will use S3 defualt buckets when needed. If the ``get_execution_role``  does not return a role with the appropriate permissions, you'll need to specify an IAM role arn that does."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade tensorflow sagemaker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "isConfigCell": true
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "import numpy as np\n",
    "import os\n",
    "import pandas as pd\n",
    "import re\n",
    "import sagemaker\n",
    "from sagemaker.tensorflow import TensorFlowModel\n",
    "from sagemaker.utils import S3DataConfig\n",
    "\n",
    "import shutil\n",
    "import tarfile\n",
    "import tensorflow as tf\n",
    "from tensorflow.python.keras.utils.np_utils import to_categorical\n",
    "from tensorflow.keras.layers import Input, Dense\n",
    "\n",
    "role = sagemaker.get_execution_role()\n",
    "sm_session = sagemaker.Session()\n",
    "bucket_name = sm_session.default_bucket()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Definitions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For this example, we'll use a very simple network architecture, with three densely-connected layers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def iris_mlp(metrics):\n",
    "    ### Setup loss and output node activation\n",
    "    output_activation = \"softmax\"\n",
    "    loss = \"sparse_categorical_crossentropy\"\n",
    "\n",
    "    input = Input(shape=(4,), name=\"input\")\n",
    "\n",
    "    x = Dense(\n",
    "        units=10,\n",
    "        kernel_regularizer=tf.keras.regularizers.l2(0.001),\n",
    "        activation=\"relu\",\n",
    "        name=\"dense_layer1\",\n",
    "    )(input)\n",
    "\n",
    "    x = Dense(\n",
    "        units=20,\n",
    "        kernel_regularizer=tf.keras.regularizers.l2(0.001),\n",
    "        activation=\"relu\",\n",
    "        name=\"dense_layer2\",\n",
    "    )(x)\n",
    "\n",
    "    x = Dense(\n",
    "        units=10,\n",
    "        activation=\"relu\",\n",
    "        kernel_regularizer=tf.keras.regularizers.l2(0.001),\n",
    "        name=\"dense_layer3\",\n",
    "    )(x)\n",
    "\n",
    "    output = Dense(units=3, activation=output_activation)(x)\n",
    "\n",
    "    ### Compile the model\n",
    "    model = tf.keras.Model(input, output)\n",
    "\n",
    "    model.compile(optimizer=\"adam\", loss=loss, metrics=metrics)\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll use the pre-processed iris training and test data stored in a public S3 bucket for this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_bucket = S3DataConfig(\n",
    "    sm_session, \"example-notebooks-data-config\", \"config/data_config.json\"\n",
    ").get_data_bucket()\n",
    "print(f\"Using data from {data_bucket}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download iris test and train data sets from S3\n",
    "SOURCE_DATA_BUCKET = data_bucket\n",
    "SOURCE_DATA_PREFIX = \"datasets/tabular/iris\"\n",
    "sm_session.download_data(\".\", bucket=SOURCE_DATA_BUCKET, key_prefix=SOURCE_DATA_PREFIX)\n",
    "\n",
    "# Load the training and test data from .csv to a Pandas data frame.\n",
    "train_df = pd.read_csv(\n",
    "    \"iris_train.csv\",\n",
    "    header=0,\n",
    "    names=[\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\", \"class\"],\n",
    ")\n",
    "test_df = pd.read_csv(\n",
    "    \"iris_test.csv\",\n",
    "    header=0,\n",
    "    names=[\"sepal_length\", \"sepal_width\", \"petal_length\", \"petal_width\", \"class\"],\n",
    ")\n",
    "\n",
    "# Pop the record labels into N x 1 Numpy arrays\n",
    "train_labels = np.array(train_df.pop(\"class\"))\n",
    "test_labels = np.array(test_df.pop(\"class\"))\n",
    "\n",
    "# Save the remaining features as Numpy arrays\n",
    "train_np = np.array(train_df)\n",
    "test_np = np.array(test_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training the Network Locally"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we train the network using the Tensorflow .fit method, just like if we were using our local computers. This should only take a few seconds because the model is so simple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EPOCHS = 50\n",
    "BATCH_SIZE = 32\n",
    "\n",
    "EARLY_STOPPING = tf.keras.callbacks.EarlyStopping(\n",
    "    monitor=\"val_loss\", mode=\"auto\", restore_best_weights=True\n",
    ")\n",
    "\n",
    "# Instantiate classifier\n",
    "classifier = iris_mlp(metrics=[\"accuracy\", \"binary_accuracy\"])\n",
    "\n",
    "# Fit classifier\n",
    "history = classifier.fit(\n",
    "    x=train_np,\n",
    "    y=train_labels,\n",
    "    validation_data=(test_np, test_labels),\n",
    "    callbacks=[EARLY_STOPPING],\n",
    "    batch_size=BATCH_SIZE,\n",
    "    epochs=EPOCHS,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up hosting for the model\n",
    "\n",
    "### Export the model from tensorflow\n",
    "\n",
    "In order to set up hosting, we have to import the model from training to hosting. We will begin by exporting the model from TensorFlow and saving it to our file system. We also need to convert the model into a form that is readable by ``sagemaker.tensorflow.model.TensorFlowModel``. There is a small difference between a SageMaker model and a TensorFlow model. The conversion is easy and fairly trivial. Simply move the tensorflow exported model into a directory ``export\\Servo\\`` and tar the entire directory. SageMaker will recognize this as a loadable TensorFlow model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier.save(\"export/Servo/1\")\n",
    "with tarfile.open(\"model.tar.gz\", \"w:gz\") as tar:\n",
    "    tar.add(\"export\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Open a new sagemaker session and upload the model on to the default S3 bucket. We can use the ``sagemaker.Session.upload_data`` method to do this. We need the location of where we exported the model from TensorFlow and where in our default bucket we want to store the model(``/model``). The default S3 bucket can be found using the ``sagemaker.Session.default_bucket`` method."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we upload the model to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_response = sm_session.upload_data(\"model.tar.gz\", bucket=bucket_name, key_prefix=\"model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import model into SageMaker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the ``sagemaker.tensorflow.model.TensorFlowModel`` to import the model into SageMaker that can be deployed. We need the location of the S3 bucket where we have the model and the role for authentication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sagemaker_model = TensorFlowModel(\n",
    "    model_data=f\"s3://{bucket_name}/model/model.tar.gz\",\n",
    "    role=role,\n",
    "    framework_version=\"2.3\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create endpoint\n",
    "\n",
    "Now the model is ready to be deployed at a SageMaker endpoint. We can use the ``sagemaker.tensorflow.model.TensorFlowModel.deploy`` method to do this. Unless you have created or prefer other instances, we recommend using a single ``'ml.m5.2xlarge'`` instance for this example. These are supplied as arguments. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "predictor = sagemaker_model.deploy(initial_instance_count=1, instance_type=\"ml.m5.2xlarge\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Validate the endpoint for use\n",
    "\n",
    "We can now use this endpoint to classify an example to ensure that it works. The output from `predict` will be an array of probabilities for each of the 3 classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample = [6.4, 3.2, 4.5, 1.5]\n",
    "predictor.predict(sample)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete all temporary directories so that we are not affecting the next run. Also, optionally delete the end points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "os.remove(\"model.tar.gz\")\n",
    "os.remove(\"iris_test.csv\")\n",
    "os.remove(\"iris_train.csv\")\n",
    "os.remove(\"iris.data\")\n",
    "shutil.rmtree(\"export\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you do not want to continue using the endpoint, you can remove it. Remember, open endpoints are charged. If this is a simple test or practice, it is recommended to delete them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_endpoint()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Notebook CI Test Results\n",
    "\n",
    "This notebook was tested in multiple regions. The test results are as follows, except for us-west-2 which is shown at the top of the notebook.\n",
    "\n",
    "![This us-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This us-east-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-east-2/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This us-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/us-west-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ca-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ca-central-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This sa-east-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/sa-east-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This eu-west-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This eu-west-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-2/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This eu-west-3 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-west-3/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This eu-central-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-central-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This eu-north-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/eu-north-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ap-southeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ap-southeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-southeast-2/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ap-northeast-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ap-northeast-2 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-northeast-2/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n",
    "\n",
    "![This ap-south-1 badge failed to load. Check your device's internet connectivity, otherwise the service is currently unavailable](https://prod.us-west-2.tcx-beacon.docs.aws.dev/sagemaker-nb/ap-south-1/advanced_functionality|tensorflow_iris_byom|tensorflow_BYOM_iris.ipynb)\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.m5.xlarge",
  "kernelspec": {
   "display_name": "conda_tensorflow2_p310",
   "language": "python",
   "name": "conda_tensorflow2_p310"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  },
  "notice": "Copyright 2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.  Licensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}