recommender_system_tutorial.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyP79e6DHaHuojjJWk1f90eH",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "ea5b93232e074516a4080b0ad9a2cdab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_900f4ad555054a6ca77ed884a45fbf35",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_0963310a6d084fdc8208f1b363a29211",
              "IPY_MODEL_0580069a5c57483c9f6846a9aa6db291"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "900f4ad555054a6ca77ed884a45fbf35": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "0963310a6d084fdc8208f1b363a29211": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_3b378b731e1f4b8e8fb9816ba81e8b74",
            "_dom_classes": [],
            "description": "Dl Completed...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_b236d50d19be4c76bae026d0da730bba"
          },
          "model_module_version": "1.5.0"
        },
        "0580069a5c57483c9f6846a9aa6db291": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_9795286e7edf491aafd1e22d521d952f",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [08:08&lt;00:00, 488.42s/ url]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_5977c1713b764e66871ced349d9da0b4"
          },
          "model_module_version": "1.5.0"
        },
        "3b378b731e1f4b8e8fb9816ba81e8b74": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "b236d50d19be4c76bae026d0da730bba": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "9795286e7edf491aafd1e22d521d952f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "5977c1713b764e66871ced349d9da0b4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "9326e8a7c4294b179dade4caa38ed166": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_d4555d4b75b843cb92fb2747a2dd42cd",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_1a440a70159647a787d829c78b0a8e70",
              "IPY_MODEL_6ae3e6fefc354de1a3a4b37543fa73d0"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "d4555d4b75b843cb92fb2747a2dd42cd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "1a440a70159647a787d829c78b0a8e70": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_7438901fa0824aa6a646f02e76f5b3cf",
            "_dom_classes": [],
            "description": "Dl Size...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8ed75df013804400bf92f02d1523346b"
          },
          "model_module_version": "1.5.0"
        },
        "6ae3e6fefc354de1a3a4b37543fa73d0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_31eb5e5c9d4145758a411156d54821b8",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 4/4 [08:08&lt;00:00, 122.10s/ MiB]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_b544800abecc4a27a3ead1140fff34cb"
          },
          "model_module_version": "1.5.0"
        },
        "7438901fa0824aa6a646f02e76f5b3cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "8ed75df013804400bf92f02d1523346b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "31eb5e5c9d4145758a411156d54821b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "b544800abecc4a27a3ead1140fff34cb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "acda6ebe40d14e0c900b0dfbe5ce5e4b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_2666e29dc6d24ad1aa099a75e823c416",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_8ed01c6cb3ab4bdfa00d15cd6f259d37",
              "IPY_MODEL_7017189a8391408aa20dd60715dc8706"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "2666e29dc6d24ad1aa099a75e823c416": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "8ed01c6cb3ab4bdfa00d15cd6f259d37": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_57cab894bd8b439a95cfb4504d5f7a4c",
            "_dom_classes": [],
            "description": "Extraction completed...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_1cf5f270aca64dae97420a160561e805"
          },
          "model_module_version": "1.5.0"
        },
        "7017189a8391408aa20dd60715dc8706": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_3be2b83f46ab48af8130060a9e4c3a44",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [08:08&lt;00:00, 488.36s/ file]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_d1650d1d8e394c98aeff889f716c5882"
          },
          "model_module_version": "1.5.0"
        },
        "57cab894bd8b439a95cfb4504d5f7a4c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "1cf5f270aca64dae97420a160561e805": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "3be2b83f46ab48af8130060a9e4c3a44": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "d1650d1d8e394c98aeff889f716c5882": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "c85db6d37ffa4aa999d4cc83c976b6f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_19a07375053b41f992cdba81e8bf2a45",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_c11f63b8515b4deeb5aa087137daf740",
              "IPY_MODEL_28a2126547e54be387e39aa49fac5f06"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "19a07375053b41f992cdba81e8bf2a45": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "c11f63b8515b4deeb5aa087137daf740": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_0286026a05c443659e30ffb06d7d173d",
            "_dom_classes": [],
            "description": "Generating splits...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_94c48efe95eb46569bc2b202c5af4a04"
          },
          "model_module_version": "1.5.0"
        },
        "28a2126547e54be387e39aa49fac5f06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_8a8275032aa748899183e1bd2462ad9a",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [01:53&lt;00:00, 113.36s/ splits]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_2f73b9cf76c7461d9501bfbb97b0d8ae"
          },
          "model_module_version": "1.5.0"
        },
        "0286026a05c443659e30ffb06d7d173d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "94c48efe95eb46569bc2b202c5af4a04": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "8a8275032aa748899183e1bd2462ad9a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "2f73b9cf76c7461d9501bfbb97b0d8ae": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "58ed1e1811ff4457a32719fdf78acdab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_961d3dc212884b3ca94e50011ea51f9a",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_5b497e43f2ca4d85968694cdb26b2abe",
              "IPY_MODEL_e80f9f65078441f49124a6fa9f40a3c1"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "961d3dc212884b3ca94e50011ea51f9a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "5b497e43f2ca4d85968694cdb26b2abe": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_9780af5babd34f4e8b76603c2f264b38",
            "_dom_classes": [],
            "description": "Generating train examples...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 100000,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 100000,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8ef465ec4cc6476fb1e778d791b1be16"
          },
          "model_module_version": "1.5.0"
        },
        "e80f9f65078441f49124a6fa9f40a3c1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_861c39af868e424ba6b3bdf032691bf4",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 100000/100000 [01:52&lt;00:00, 1004.49 examples/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_c91f7c69a8764ae38d4e728fb38e1439"
          },
          "model_module_version": "1.5.0"
        },
        "9780af5babd34f4e8b76603c2f264b38": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "8ef465ec4cc6476fb1e778d791b1be16": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "861c39af868e424ba6b3bdf032691bf4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "c91f7c69a8764ae38d4e728fb38e1439": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "a76bdd5541c94629a93544f9b6d3bae1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_301f2591588f41fb8b008258f2fe7456",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_0754bec9f95442e59ce6eaf8186da0b2",
              "IPY_MODEL_83401ea6578044a1b9d279170258a065"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "301f2591588f41fb8b008258f2fe7456": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "0754bec9f95442e59ce6eaf8186da0b2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_cfc5f8517e974f46a1469465ed8be3c9",
            "_dom_classes": [],
            "description": "Shuffling movielens-train.tfrecord...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 100000,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 100000,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_07976623a03643888c66bdbf7b2cef01"
          },
          "model_module_version": "1.5.0"
        },
        "83401ea6578044a1b9d279170258a065": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_8f2cd32039cb4fd78b0ed7a48369014e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 100000/100000 [00:00&lt;00:00, 15.23 examples/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_0edcca2c747f44048d35c8ac27d745a2"
          },
          "model_module_version": "1.5.0"
        },
        "cfc5f8517e974f46a1469465ed8be3c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "07976623a03643888c66bdbf7b2cef01": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "8f2cd32039cb4fd78b0ed7a48369014e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "0edcca2c747f44048d35c8ac27d745a2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "5836d1a2e06d4417a79a306987ca2a8d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_4c7f3f0497a442498e29140b13da3718",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_5eac2b4797f342589fdf1a729d69b4c4",
              "IPY_MODEL_82bc0ba55b52415ca3c1d092d402f0d8"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "4c7f3f0497a442498e29140b13da3718": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "5eac2b4797f342589fdf1a729d69b4c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_4cead5ce2ec549389930e286008dbd43",
            "_dom_classes": [],
            "description": "Dl Completed...: ",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 0,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_43a37170841b406ead6ed8a0061290f5"
          },
          "model_module_version": "1.5.0"
        },
        "82bc0ba55b52415ca3c1d092d402f0d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_f2df5ab944484b9aa5571b1f92fb31ae",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 0/0 [00:00&lt;?, ? url/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_78a235be12ae486cb85db74c9e0dc696"
          },
          "model_module_version": "1.5.0"
        },
        "4cead5ce2ec549389930e286008dbd43": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "43a37170841b406ead6ed8a0061290f5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "f2df5ab944484b9aa5571b1f92fb31ae": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "78a235be12ae486cb85db74c9e0dc696": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "37d9b4c872a042bf83509aabf01e49cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_048b5344d1f94581b549a774e27d5b54",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_c8cca219c1194d36bc93f0ce16664154",
              "IPY_MODEL_3dd0280309064c14970cd7186b7ba0d6"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "048b5344d1f94581b549a774e27d5b54": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "c8cca219c1194d36bc93f0ce16664154": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_d370bf61eff04e778a1810ed74ce2d45",
            "_dom_classes": [],
            "description": "Dl Size...: ",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 0,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_76b9fe27088a42eeb6c861aa61b4218c"
          },
          "model_module_version": "1.5.0"
        },
        "3dd0280309064c14970cd7186b7ba0d6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_cb907be1778148688a8b3d344378259e",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 0/0 [00:00&lt;?, ? MiB/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_be1e18a683bf474b83414922115b8cb3"
          },
          "model_module_version": "1.5.0"
        },
        "d370bf61eff04e778a1810ed74ce2d45": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "76b9fe27088a42eeb6c861aa61b4218c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "cb907be1778148688a8b3d344378259e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "be1e18a683bf474b83414922115b8cb3": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "017c5f33da3a497097df21de70bfd873": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_0bbb513d33884d6ab920498cf07a74a1",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_f5994d027b694fd68d9534f4685b8448",
              "IPY_MODEL_e42f9d4198c549da9a0c3abdffc30cd7"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "0bbb513d33884d6ab920498cf07a74a1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "f5994d027b694fd68d9534f4685b8448": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_cae09be9a5f346a2bb7663396474f7ab",
            "_dom_classes": [],
            "description": "Extraction completed...: ",
            "_model_name": "FloatProgressModel",
            "bar_style": "success",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 0,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_3ce95d4466bc43019bb09538fe551f2d"
          },
          "model_module_version": "1.5.0"
        },
        "e42f9d4198c549da9a0c3abdffc30cd7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_7122e14af06b4b0e9c9091cf3d325122",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 0/0 [00:00&lt;?, ? file/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_e1fa286fbc844c6e821b1aa2479a5c61"
          },
          "model_module_version": "1.5.0"
        },
        "cae09be9a5f346a2bb7663396474f7ab": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "3ce95d4466bc43019bb09538fe551f2d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "7122e14af06b4b0e9c9091cf3d325122": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "e1fa286fbc844c6e821b1aa2479a5c61": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "1d1e79c03bb145738ef6fee464a5274e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_cea1babe04e34a929978ae0ee86d07da",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_e6964e7aaa8f42e293e41ca62f8bb1d3",
              "IPY_MODEL_c4a5eb5a8e654be4b269af08a31add41"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "cea1babe04e34a929978ae0ee86d07da": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "e6964e7aaa8f42e293e41ca62f8bb1d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_b8425150aca94b619551ee8fe8349738",
            "_dom_classes": [],
            "description": "Generating splits...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 1,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_1b30b087eda94047a350ee51248512d5"
          },
          "model_module_version": "1.5.0"
        },
        "c4a5eb5a8e654be4b269af08a31add41": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_df215de96d744afca92ae23c3ef16263",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1/1 [00:00&lt;00:00,  1.15 splits/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_8ab2b2f1658b4a728bbb31b0817a2973"
          },
          "model_module_version": "1.5.0"
        },
        "b8425150aca94b619551ee8fe8349738": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "1b30b087eda94047a350ee51248512d5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "df215de96d744afca92ae23c3ef16263": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "8ab2b2f1658b4a728bbb31b0817a2973": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "380f4d4102b34e60abfed1eb9bdbadd2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_00f1ec84c2c54d36b5a69eb76f546613",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_bd1f4889292c471992e68a83ab764858",
              "IPY_MODEL_d883f0e109ff46f9a449a411a1404025"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "00f1ec84c2c54d36b5a69eb76f546613": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "bd1f4889292c471992e68a83ab764858": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_9d5e2fb4988e46fea9cd924e82e13d5d",
            "_dom_classes": [],
            "description": "Generating train examples...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 1682,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1682,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_a7318131bcd347c7a3c28fe94bcce41b"
          },
          "model_module_version": "1.5.0"
        },
        "d883f0e109ff46f9a449a411a1404025": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_c0ad928e95664faea5f00efb245e75ec",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1682/1682 [00:00&lt;00:00, 1950.46 examples/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_7fafd911808f4381b88aa2956beb3d1c"
          },
          "model_module_version": "1.5.0"
        },
        "9d5e2fb4988e46fea9cd924e82e13d5d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "a7318131bcd347c7a3c28fe94bcce41b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "c0ad928e95664faea5f00efb245e75ec": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "7fafd911808f4381b88aa2956beb3d1c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "a31a11356ec04a97b6cc7957985fce93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "state": {
            "_view_name": "HBoxView",
            "_dom_classes": [],
            "_model_name": "HBoxModel",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "box_style": "",
            "layout": "IPY_MODEL_8ca8e193f3604436a2b76b062a1b2bfa",
            "_model_module": "@jupyter-widgets/controls",
            "children": [
              "IPY_MODEL_5641187ab3124a2c97613794ff3a2140",
              "IPY_MODEL_87c44927b6fd4fa188a3f2bba1968d26"
            ]
          },
          "model_module_version": "1.5.0"
        },
        "8ca8e193f3604436a2b76b062a1b2bfa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "5641187ab3124a2c97613794ff3a2140": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "state": {
            "_view_name": "ProgressView",
            "style": "IPY_MODEL_bc2f6fcc89734b43a83c72226f5d6bc0",
            "_dom_classes": [],
            "description": "Shuffling movielens-train.tfrecord...: 100%",
            "_model_name": "FloatProgressModel",
            "bar_style": "",
            "max": 1682,
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": 1682,
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "orientation": "horizontal",
            "min": 0,
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_f51b4a5b99e040fd9f9681c5f2451c74"
          },
          "model_module_version": "1.5.0"
        },
        "87c44927b6fd4fa188a3f2bba1968d26": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "state": {
            "_view_name": "HTMLView",
            "style": "IPY_MODEL_dc80ecc2d8974e0bb23120bff7d0e215",
            "_dom_classes": [],
            "description": "",
            "_model_name": "HTMLModel",
            "placeholder": "​",
            "_view_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "value": " 1682/1682 [00:00&lt;00:00, 50934.75 examples/s]",
            "_view_count": null,
            "_view_module_version": "1.5.0",
            "description_tooltip": null,
            "_model_module": "@jupyter-widgets/controls",
            "layout": "IPY_MODEL_0ee218a7eed5400cb5b79a338effae5d"
          },
          "model_module_version": "1.5.0"
        },
        "bc2f6fcc89734b43a83c72226f5d6bc0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "ProgressStyleModel",
            "description_width": "initial",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "bar_color": null,
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "f51b4a5b99e040fd9f9681c5f2451c74": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        },
        "dc80ecc2d8974e0bb23120bff7d0e215": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_view_name": "StyleView",
            "_model_name": "DescriptionStyleModel",
            "description_width": "",
            "_view_module": "@jupyter-widgets/base",
            "_model_module_version": "1.5.0",
            "_view_count": null,
            "_view_module_version": "1.2.0",
            "_model_module": "@jupyter-widgets/controls"
          },
          "model_module_version": "1.5.0"
        },
        "0ee218a7eed5400cb5b79a338effae5d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "state": {
            "_view_name": "LayoutView",
            "grid_template_rows": null,
            "right": null,
            "justify_content": null,
            "_view_module": "@jupyter-widgets/base",
            "overflow": null,
            "_model_module_version": "1.2.0",
            "_view_count": null,
            "flex_flow": null,
            "width": null,
            "min_width": null,
            "border": null,
            "align_items": null,
            "bottom": null,
            "_model_module": "@jupyter-widgets/base",
            "top": null,
            "grid_column": null,
            "overflow_y": null,
            "overflow_x": null,
            "grid_auto_flow": null,
            "grid_area": null,
            "grid_template_columns": null,
            "flex": null,
            "_model_name": "LayoutModel",
            "justify_items": null,
            "grid_row": null,
            "max_height": null,
            "align_content": null,
            "visibility": null,
            "align_self": null,
            "height": null,
            "min_height": null,
            "padding": null,
            "grid_auto_rows": null,
            "grid_gap": null,
            "max_width": null,
            "order": null,
            "_view_module_version": "1.2.0",
            "grid_template_areas": null,
            "object_position": null,
            "object_fit": null,
            "grid_auto_columns": null,
            "margin": null,
            "display": null,
            "left": null
          },
          "model_module_version": "1.2.0"
        }
      }
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/xei/recommender-system-tutorial/blob/5-wrong-attempt-to-convert-a-value-with-an-unsupported-type-to-a-tensor/recommender_system_tutorial.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "l-8TmjXRTezi"
      },
      "source": [
        "# Build a recommendation system with TensorFlow and Keras\n",
        "\n",
        "\n",
        "\n",
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://colab.research.google.com/github/xei/recommender-system-tutorial/blob/main/recommender_system_tutorial.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
        "  </td>\n",
        "  <td>\n",
        "    <a target=\"_blank\" href=\"https://github.com/xei/recommender-system-tutorial/blob/main/recommender_system_tutorial.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
        "  </td>\n",
        "</table>\n",
        "\n",
        "<p>&nbsp;</p>\n",
        "<p>&nbsp;</p>\n",
        "\n",
        "Here, we are going to learn the fundamentals of information retrieval and recommendation systems and build a practical movie recommender service using [TensorFlow Recommenders](https://www.tensorflow.org/recommenders) and [Keras](https://keras.io/) and deploy it using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving).\n",
        "\n",
        "This step-by-step tutorial is recommended to both academia and industry enthusiasts."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JR7yjVX9UUxs"
      },
      "source": [
        "## Introduction\n",
        "\n",
        "### What is a recommendation system?!\n",
        "Online services usually provide thousands, millions or even billions of items like products, video clips, movies, musics, news, articles, blog posts, advertisements, etc. For example, the [Google Play Store](https://play.google.com/store) provides millions of apps and [YouTube](https://www.youtube.com/) provides billions of videos. [[1]](https://developers.google.com/machine-learning/recommendation/overview)\n",
        "\n",
        "However, users prefer to see a handful shortlist of likely items instead of struggling with the full corpora. They usually can search or filter the list to find the best handful items, but sometimes they even don't know what they really want (e.g. a birthday gift). In a physical store an expert seller would help in this case by useful recommendations. So, why not in an online store?!\n",
        "\n",
        "A recommedation system can retrieve, filter and recommend best personalized results for the user - results which the user is likely to buy. So it is one of the major requirements of modern businesses in order to increase their `conversion rate`. On September 21, 2009, Netflix gave a grand prize of $1,000,000 to a team which bested Netflix's own algorithm for predicting ratings by 10.06%. [[2]](https://web.archive.org/web/20090924184639/http://www.netflixprize.com/community/viewtopic.php?id=1537)\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"634\" height=\"416\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/netflix_prize.jpeg\">\n",
        "</p>\n",
        "\n",
        "A recommendation system ia a system that gives a `query (context)` which is what we know about the liking list, and filter the corpus (full catalog of items) to a shortlist of `candidates` (items, documents). A query (context) can be a combination of ***user id***, ***user's geographical location***, ***user's history of previous purchases***, etc and the resulting candidates can be some new items that we guess are interesting for the user.\n",
        "\n",
        "The query can also be a combination of ***item id***, ***its image***, ***its textual description***, etc and the candidates can be some similar or related items from the corpus.\n",
        "<p>&nbsp;</p>\n",
        "<p align=\"center\">\n",
        "  <img width=\"556\" height=\"82\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/recsys_io.png\">\n",
        "</p>\n",
        "<p>&nbsp;</p>\n",
        "\n",
        "### Recommendation stages (tasks)\n",
        "In practice, dealing with a large corpus and filter it to a shortlist is an intractable and inefficient task. So practical recommender systems has two (or three) filterng phases:\n",
        "1. Retrieval (Candidate Generation)\n",
        "2. Ranking (Scoring)\n",
        "3. Re-ranking or optimazation or ...\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"700\" height=\"394\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/retrieval_ranking.png\">\n",
        "</p>\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"700\" height=\"394\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/deepmind_forgoogle_recsys.png\">\n",
        "</p>\n",
        "\n",
        "### Content-based Filtering vs Collaborative Filtering\n",
        "Filtering items is based on similarities. we can filter the list based on similar candidates (`content-based filtering`) or based on the similarity between queries and candidates (`collaborative filtering`). Collaborative filtering algorithms usually perform better than content-based methods.\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"700\" height=\"330\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/cbf_cf.png\">\n",
        "</p>\n",
        "\n",
        "### Representation of a query or a candidate\n",
        "A query or a candidate has lots of different features. For example a query can be constructed by these features:\n",
        "- user_id\n",
        "- user_previous_history\n",
        "- user_job\n",
        "- etc.\n",
        "\n",
        "And a candidate can have features like:\n",
        "- item_description\n",
        "- item_image\n",
        "- item_price\n",
        "- posted_time\n",
        "- etc.\n",
        "\n",
        "These obviouse features can be `numerical variables`, `categorical variables`, `bitmaps` or `raw texts`. However, these low-level features are not enough for any prediction and we should extract some more abstract `latent features` from these obvious features to represent the query or the candidate as a numerical high-dimensional vector - known as `Embedding Vector`.\n",
        "\n",
        "`Matrix Factorization` (MF) is a classic collaborative filtering method to learn some `latent factors` (latent features) from `user_id`, `item_id` and `rating` features and represent **users** and **items** by latent (embedding) vectors.\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"826\" height=\"398\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/mf.png\">\n",
        "</p>\n",
        "\n",
        "Matrix Factorization method only uses `user_id` and `candidate_id` features collaboratively to learn the `latent features`. In fact it doesn't care about other side-features like `candidate_description`, `price`, `user_comment`, etc.\n",
        "\n",
        "To involve side-features as well as ids while learning latent features (embeddings), we can use deep neural network (DNN) architectures like `softmax` or `two-tower` neural models.\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"540\" height=\"340\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/tensorflow_two_tower.gif\">\n",
        "</p>\n",
        "\n",
        "YouTube two-tower neural model uses side-features to represent queries and candidates in an abstract high-dimentional embedding vector.\n",
        "\n",
        "<p align=\"center\">\n",
        "  <img width=\"540\" height=\"340\" src=\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/youtube_retrieval.png\">\n",
        "</p>\n",
        "\n",
        "### Movielens dataset\n",
        "\n",
        "The `Movielens` dataset is a benchmark dataset in the field of recommender system research containing a set of *ratings* given to *movies* by a set of *users*, collected from the [MovieLens website](http://movielens.org/) - a movie recommendation service.\n",
        "\n",
        "There are 5 different versions of Movielens available for different purposes: \"25m\", \"latest-small\", \"100k\", \"1m\" and \"20m\". In this tutorial we are going to work with \"100k\" version. For more information about different versions visit the [official website](https://grouplens.org/datasets/movielens/).\n",
        "\n",
        "**movielens/100k-ratings**\n",
        "\n",
        "The oldest version of the MovieLens dataset containing 100,000 ratings from 943 users on 1,682 movies. Each user has rated at least 20 movies. Ratings are in whole-star increments. This dataset contains demographic data of users in addition to data on movies and ratings.\n",
        "\n",
        "**movielens/100k-movies**\n",
        "\n",
        "This dataset contains data of 1,682 movies rated in the ***movielens/100k-ratings*** dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NDrq_pdbB6Pd"
      },
      "source": [
        "## Make an input pipeline with TensorFlow Datasets\n",
        "TensorFlow Datasets ([TFDS library](https://www.tensorflow.org/datasets/overview)) provides a collection of ready-to-use datasets for use with TensorFlow, Jax, and other Machine Learning frameworks.\n",
        "\n",
        "It handles downloading and preparing the data deterministically and constructing a `tf.data.Dataset` (or `np.array`) in order to enabling easy-to-use and high-performance input pipelines."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Sciomsbda9NT",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "073a70a6-d0fb-49f7-95ca-d89b89e42994"
      },
      "source": [
        "# Make sure the latest version of TFDS library is installed.\n",
        "# Older versions are not supported here.\n",
        "!pip install -q --upgrade tensorflow-datasets\n",
        " \n",
        "import tensorflow_datasets as tfds"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 3.7MB 12.1MB/s \n",
            "\u001b[?25h"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IWY6OFnhLQGx",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 581,
          "referenced_widgets": [
            "ea5b93232e074516a4080b0ad9a2cdab",
            "900f4ad555054a6ca77ed884a45fbf35",
            "0963310a6d084fdc8208f1b363a29211",
            "0580069a5c57483c9f6846a9aa6db291",
            "3b378b731e1f4b8e8fb9816ba81e8b74",
            "b236d50d19be4c76bae026d0da730bba",
            "9795286e7edf491aafd1e22d521d952f",
            "5977c1713b764e66871ced349d9da0b4",
            "9326e8a7c4294b179dade4caa38ed166",
            "d4555d4b75b843cb92fb2747a2dd42cd",
            "1a440a70159647a787d829c78b0a8e70",
            "6ae3e6fefc354de1a3a4b37543fa73d0",
            "7438901fa0824aa6a646f02e76f5b3cf",
            "8ed75df013804400bf92f02d1523346b",
            "31eb5e5c9d4145758a411156d54821b8",
            "b544800abecc4a27a3ead1140fff34cb",
            "acda6ebe40d14e0c900b0dfbe5ce5e4b",
            "2666e29dc6d24ad1aa099a75e823c416",
            "8ed01c6cb3ab4bdfa00d15cd6f259d37",
            "7017189a8391408aa20dd60715dc8706",
            "57cab894bd8b439a95cfb4504d5f7a4c",
            "1cf5f270aca64dae97420a160561e805",
            "3be2b83f46ab48af8130060a9e4c3a44",
            "d1650d1d8e394c98aeff889f716c5882",
            "c85db6d37ffa4aa999d4cc83c976b6f9",
            "19a07375053b41f992cdba81e8bf2a45",
            "c11f63b8515b4deeb5aa087137daf740",
            "28a2126547e54be387e39aa49fac5f06",
            "0286026a05c443659e30ffb06d7d173d",
            "94c48efe95eb46569bc2b202c5af4a04",
            "8a8275032aa748899183e1bd2462ad9a",
            "2f73b9cf76c7461d9501bfbb97b0d8ae",
            "58ed1e1811ff4457a32719fdf78acdab",
            "961d3dc212884b3ca94e50011ea51f9a",
            "5b497e43f2ca4d85968694cdb26b2abe",
            "e80f9f65078441f49124a6fa9f40a3c1",
            "9780af5babd34f4e8b76603c2f264b38",
            "8ef465ec4cc6476fb1e778d791b1be16",
            "861c39af868e424ba6b3bdf032691bf4",
            "c91f7c69a8764ae38d4e728fb38e1439",
            "a76bdd5541c94629a93544f9b6d3bae1",
            "301f2591588f41fb8b008258f2fe7456",
            "0754bec9f95442e59ce6eaf8186da0b2",
            "83401ea6578044a1b9d279170258a065",
            "cfc5f8517e974f46a1469465ed8be3c9",
            "07976623a03643888c66bdbf7b2cef01",
            "8f2cd32039cb4fd78b0ed7a48369014e",
            "0edcca2c747f44048d35c8ac27d745a2"
          ]
        },
        "outputId": "7298e407-5334-4e7a-a116-cf59941d3553"
      },
      "source": [
        "# Download the data, save them as `tfrecord` files, load the `tfrecord` files\n",
        "# and create the `tf.data.Dataset` object containing the dataset.\n",
        "ratings_dataset, ratings_dataset_info = tfds.load(\n",
        "    name='movielens/100k-ratings',\n",
        "    # MovieLens dataset is not splitted into `train` and `test` sets by default.\n",
        "    # So TFDS has put it all into `train` split. We load it completely and split\n",
        "    # it manually.\n",
        "    split='train',\n",
        "    # `with_info=True` makes the `load` function return a `tfds.core.DatasetInfo`\n",
        "    # object containing dataset metadata like version, description, homepage,\n",
        "    # citation, etc.\n",
        "    with_info=True\n",
        ")\n",
        "\n",
        "# Calling the `tfds.load()` function in old versions of TFDS won't return an\n",
        "# instance of `tf.data.Dataset` type. So we can make sure about it.\n",
        "import tensorflow as tf\n",
        "assert isinstance(ratings_dataset, tf.data.Dataset)\n",
        "\n",
        "print(\n",
        "    \"ratings_dataset size: %d\" % ratings_dataset.__len__()\n",
        ")\n",
        "\n",
        "# Use `tfds.as_dataframe()` to convert `tf.data.Dataset` to `pandas.DataFrame`.\n",
        "# Add the `tfds.core.DatasetInfo` as second argument of `tfds.as_dataframe` to\n",
        "# visualize images, audio, texts, videos, etc. `pandas.DataFrame` will load the\n",
        "# full dataset in-memory, and can be very expensive to display. So use it only\n",
        "# with take() function.\n",
        "print(\n",
        "    tfds.as_dataframe(ratings_dataset.take(5), ratings_dataset_info)\n",
        ")\n",
        "\n",
        "## Feature selection\n",
        "ratings_dataset = ratings_dataset.map(\n",
        "    lambda rating: {\n",
        "        # `user_id` is useful as a user identifier.\n",
        "        'user_id': rating['user_id'],\n",
        "        # `movie_id` is useful as a movie identifier.\n",
        "        'movie_id': rating['movie_id'],\n",
        "        # `movie_title` is useful as a textual information about the movie.\n",
        "        'movie_title': rating['movie_title'],\n",
        "        # `user_rating` shows the user's level of interest to a movie.\n",
        "        'user_rating': rating['user_rating'],\n",
        "        # `timestamp` will allow us to model the effect of time.\n",
        "        'timestamp': rating['timestamp']\n",
        "    }\n",
        ")\n",
        "\n",
        "print(\n",
        "    tfds.as_dataframe(ratings_dataset.take(5), ratings_dataset_info)\n",
        ")\n",
        "\n",
        "## Split dataset randomly (80% for train and 20% for test)\n",
        "trainset_size = 0.8 * ratings_dataset.__len__().numpy()\n",
        "# In an industrial recommender system, this would most likely be done by time:\n",
        "# The data up to time T would be used to predict interactions after T.\n",
        "\n",
        "# set the global seed:\n",
        "tf.random.set_seed(42)\n",
        "# More info: https://www.tensorflow.org/api_docs/python/tf/random/set_seed\n",
        "\n",
        "# Shuffle the elements of the dataset randomly.\n",
        "ratings_dataset_shuffled = ratings_dataset.shuffle(\n",
        "    # the new dataset will be sampled from a buffer window of first `buffer_size`\n",
        "    # elements of the dataset\n",
        "    buffer_size=100_000,\n",
        "    # set the random seed that will be used to create the distribution.\n",
        "    seed=42,\n",
        "    # `list(dataset.as_numpy_iterator()` yields different result for each call\n",
        "    # Because reshuffle_each_iteration defaults to True.\n",
        "    reshuffle_each_iteration=False\n",
        ")\n",
        "# More info: https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle\n",
        " \n",
        "ratings_trainset = ratings_dataset_shuffled.take(trainset_size)\n",
        "ratings_testset = ratings_dataset_shuffled.skip(trainset_size)\n",
        "\n",
        "print(\n",
        "    \"ratings_trainset size: %d\" % ratings_trainset.__len__()\n",
        ")\n",
        "print(\n",
        "    \"ratings_testset size: %d\" % ratings_testset.__len__()\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[1mDownloading and preparing dataset 4.70 MiB (download: 4.70 MiB, generated: 32.41 MiB, total: 37.10 MiB) to /root/tensorflow_datasets/movielens/100k-ratings/0.1.0...\u001b[0m\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ea5b93232e074516a4080b0ad9a2cdab",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Dl Completed...', max=1.0, style=Progre…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "9326e8a7c4294b179dade4caa38ed166",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Dl Size...', max=1.0, style=ProgressSty…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "acda6ebe40d14e0c900b0dfbe5ce5e4b",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Extraction completed...', max=1.0, styl…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c85db6d37ffa4aa999d4cc83c976b6f9",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Generating splits...', max=1.0, style=ProgressStyle(descr…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "58ed1e1811ff4457a32719fdf78acdab",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Generating train examples...', max=100000.0, style=Progre…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a76bdd5541c94629a93544f9b6d3bae1",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Shuffling movielens-train.tfrecord...', max=100000.0, sty…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\r\u001b[1mDataset movielens downloaded and prepared to /root/tensorflow_datasets/movielens/100k-ratings/0.1.0. Subsequent calls will reuse this data.\u001b[0m\n",
            "ratings_dataset size: 100000\n",
            "   bucketized_user_age movie_genres  ... user_rating user_zip_code\n",
            "0                 45.0          [7]  ...         4.0      b'53211'\n",
            "1                 25.0      [4, 14]  ...         2.0      b'80525'\n",
            "2                 18.0          [4]  ...         4.0      b'55439'\n",
            "3                 50.0       [5, 7]  ...         4.0      b'06472'\n",
            "4                 50.0     [10, 16]  ...         3.0      b'75094'\n",
            "\n",
            "[5 rows x 12 columns]\n",
            "  movie_id                                movie_title  ...  user_id user_rating\n",
            "0   b'357'  b\"One Flew Over the Cuckoo's Nest (1975)\"  ...   b'138'         4.0\n",
            "1   b'709'                b'Strictly Ballroom (1992)'  ...    b'92'         2.0\n",
            "2   b'412'             b'Very Brady Sequel, A (1996)'  ...   b'301'         4.0\n",
            "3    b'56'                     b'Pulp Fiction (1994)'  ...    b'60'         4.0\n",
            "4   b'895'                         b'Scream 2 (1997)'  ...   b'197'         3.0\n",
            "\n",
            "[5 rows x 5 columns]\n",
            "ratings_trainset size: 80000\n",
            "ratings_testset size: 20000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2JOGVWBEWiZZ"
      },
      "source": [
        "To make a custom `tf.data.Dataset` object including your own dataset visit [this link](https://www.tensorflow.org/datasets/add_dataset).\n",
        "\n",
        "For more information about `tf.data.Dataset` visit [this link](https://www.tensorflow.org/api_docs/python/tf/data/Dataset)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GnlXCP02UV8K"
      },
      "source": [
        "## Preprocess raw features and make embeddings with Keras preprocessing layers\n",
        "\n",
        "Raw features are usually not be immediately usable in a machine learning model and should be preprocessed in the first place.\n",
        "- **Numerical features** (ratings, prices, timestamps, etc) can be far away in terms of scale and need to be `normalized` so that their values lie in a small interval around 0.\n",
        "- **Categorical features** (ids, usernames/emails, titles, etc) are usually string features and have to be translated into `embedding vectors` (numerical feature representations) that are adjusted during training the model.\n",
        "- **Text features** (descriptions, comments, etc) need to be at first, `tokenized` (split into smaller parts such as individual words known as word pieces) and then translated into embeddings.\n",
        "\n",
        "[Keras preprocessing layers](https://keras.io/guides/preprocessing_layers/) let us build `end-to-end` portable models that accept raw features (raw images or raw structured data) as input; models that handle feature normalization or feature value indexing on their own."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EHvSx5Vq1dFO"
      },
      "source": [
        "Let's first have a look at what features we can use from the MovieLens dataset:\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iIiKVYmASodk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c8286164-3127-41e9-ded5-934b326c65c8"
      },
      "source": [
        "from pprint import pprint\n",
        "\n",
        "for rating in ratings_trainset.take(1).as_numpy_iterator():\n",
        "  pprint(rating)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'movie_id': b'898',\n",
            " 'movie_title': b'Postman, The (1997)',\n",
            " 'timestamp': 885409515,\n",
            " 'user_id': b'681',\n",
            " 'user_rating': 4.0}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vAIqWlEjsvVO"
      },
      "source": [
        "For more information about each field visit [this link](https://www.tensorflow.org/datasets/catalog/movielens)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "82VLAC1qJHMD"
      },
      "source": [
        "### Normalize numerical features\n",
        "`timestamp` values are far too large to be used directly in a machine learning model. However, it can be normalized in a small interval around 0. Standardization (`Z-score Normalization`) is a common preprocessing transformation that rescales features to normalize their range by subtracting the feature's `mean` and dividing by its `standard deviation`.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ga9_p0_1GpUC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "13cf6e35-b23c-4ed5-fbde-e2ac000f7bdb"
      },
      "source": [
        "# Make a Keras Normalization layer to standardize a numerical feature.\n",
        "timestamp_normalization_layer = \\\n",
        "    tf.keras.layers.experimental.preprocessing.Normalization(axis=None)\n",
        "\n",
        "# Normalization layer is a non-trainable layer and its state (mean and std of\n",
        "# feature set) must be set before training in a step called \"adaptation\".\n",
        "timestamp_normalization_layer.adapt(\n",
        "    ratings_trainset.map(\n",
        "        lambda x: x['timestamp']\n",
        "    )\n",
        ")\n",
        "\n",
        "for rating in ratings_trainset.take(3).as_numpy_iterator():\n",
        "  print(\n",
        "      f\"Raw timestamp: {rating['timestamp']} ->\",\n",
        "      f\"Normalized timestamp: {timestamp_normalization_layer(rating['timestamp'])}\"\n",
        "  )"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Raw timestamp: 885409515 -> Normalized timestamp: 0.3526017963886261\n",
            "Raw timestamp: 883388887 -> Normalized timestamp: -0.026022713631391525\n",
            "Raw timestamp: 891249586 -> Normalized timestamp: 1.4468868970870972\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Q_2VmQ4KpOpo"
      },
      "source": [
        "### Turning categorical features into embeddings\n",
        "\n",
        "A categorical feature is a feature that does not express a continuous quantity, but rather takes on one of a set of fixed values. Most deep learning models express these feature by turning them into high-dimensional embedding vectors which will be adjusted during model training.\n",
        "\n",
        "Here we represent each user and each movie by an embedding vector. Initially, these embeddings will take on random values, but during training, we will adjust them so that embeddings of users and the movies they watch end up closer together.\n",
        "\n",
        "Taking raw categorical features and turning them into embeddings is normally a two-step process:\n",
        "\n",
        "\n",
        "1.   Build a mapping (called a `\"vocabulary\"`) that maps each raw values for example \"Postman, The (1997)\" to unique integers (say, 15).\n",
        "2.   Turn these integers into embedding vectors.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "2yfWHKFfpPSx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d48ddf78-6c1d-4f8f-f1b2-38b9e6962209"
      },
      "source": [
        "# Make a Keras StringLookup layer as the mapping (lookup)\n",
        "user_id_lookup_layer = \\\n",
        "    tf.keras.layers.experimental.preprocessing.StringLookup(mask_token=None)\n",
        "\n",
        "# StringLookup layer is a non-trainable layer and its state (the vocabulary)\n",
        "# must be constructed and set before training in a step called \"adaptation\".\n",
        "user_id_lookup_layer.adapt(\n",
        "    ratings_trainset.map(\n",
        "        lambda x: x['user_id']\n",
        "    )\n",
        ")\n",
        "\n",
        "print(\n",
        "    f\"Vocabulary[:10] -> {user_id_lookup_layer.get_vocabulary()[:10]}\"\n",
        "    # Vocabulary: ['[UNK]', '405', '655', '13', ...]\n",
        "    # The vocabulary includes one (or more!) unknown (or \"out of vocabulary\", OOV)\n",
        "    # tokens. So the layer can handle categorical values that are not in the\n",
        "    # vocabulary and the model can continue to learn about and make\n",
        "    # recommendations even using features that have not been seen during\n",
        "    # vocabulary construction.\n",
        ")\n",
        "\n",
        "print(\n",
        "    \"Mapped integer for user ids: ['-2', '13', '655', 'xxx']\\n\",\n",
        "    user_id_lookup_layer(\n",
        "        ['-2', '13', '655', 'xxx']\n",
        "    )\n",
        ")\n",
        "\n",
        "user_id_embedding_dim = 32\n",
        "# The larger it is, the higher the capacity of the model, but the slower it is\n",
        "# to fit and serve and more prone to overfitting.\n",
        "\n",
        "user_id_embedding_layer = tf.keras.layers.Embedding(\n",
        "    # Size of the vocabulary\n",
        "    input_dim=user_id_lookup_layer.vocab_size(),\n",
        "    # Dimension of the dense embedding\n",
        "    output_dim=user_id_embedding_dim\n",
        ")\n",
        " \n",
        "# A model that takes raw string feature values (user_id) in and yields embeddings\n",
        "user_id_model = tf.keras.Sequential(\n",
        "    [\n",
        "        user_id_lookup_layer,\n",
        "        user_id_embedding_layer\n",
        "    ]\n",
        ")\n",
        " \n",
        "print(\n",
        "    \"Embeddings for user ids: ['-2', '13', '655', 'xxx']\\n\",\n",
        "    user_id_model(\n",
        "        ['-2', '13', '655', 'xxx']\n",
        "    )\n",
        ")\n",
        "\n",
        "\n",
        "movie_id_lookup_layer = \\\n",
        "    tf.keras.layers.experimental.preprocessing.StringLookup(mask_token=None)\n",
        "movie_id_lookup_layer.adapt(\n",
        "    ratings_trainset.map(\n",
        "        lambda x: x['movie_id']\n",
        "    )\n",
        ")\n",
        "\n",
        "# Same as user_id_embedding_dim to be able to measure the similarity\n",
        "movie_id_embedding_dim = 32\n",
        "\n",
        "movie_id_embedding_layer = tf.keras.layers.Embedding(\n",
        "    input_dim=movie_id_lookup_layer.vocab_size(),\n",
        "    output_dim=movie_id_embedding_dim\n",
        ")\n",
        " \n",
        "movie_id_model = tf.keras.Sequential(\n",
        "    [\n",
        "        movie_id_lookup_layer,\n",
        "        movie_id_embedding_layer\n",
        "    ]\n",
        ")\n",
        "\n",
        "print(\n",
        "    f\"Embedding for the movie 898:\\n {movie_id_model('898')}\"\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vocabulary[:10] -> ['[UNK]', '405', '655', '13', '450', '276', '303', '416', '537', '234']\n",
            "Mapped integer for user ids: ['-2', '13', '655', 'xxx']\n",
            " tf.Tensor([0 3 2 0], shape=(4,), dtype=int64)\n",
            "WARNING:tensorflow:Layers in a Sequential model should only have a single input tensor, but we receive a <class 'list'> input: ['-2', '13', '655', 'xxx']\n",
            "Consider rewriting this model with the Functional API.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layers in a Sequential model should only have a single input tensor, but we receive a <class 'list'> input: ['-2', '13', '655', 'xxx']\n",
            "Consider rewriting this model with the Functional API.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Embeddings for user ids: ['-2', '13', '655', 'xxx']\n",
            " tf.Tensor(\n",
            "[[ 0.01645621 -0.00589932 -0.01471175 -0.00355174 -0.04663396  0.01846724\n",
            "   0.02401174  0.03724445 -0.02736737 -0.02768031 -0.01896119  0.02223358\n",
            "  -0.03668128  0.00480639  0.00746088  0.03996835 -0.04905364  0.00212307\n",
            "   0.01345445 -0.03006717  0.02294225  0.00458346 -0.03924345  0.01767061\n",
            "   0.01602763 -0.01630496  0.01014177 -0.02893742  0.03527372 -0.00593783\n",
            "   0.04485276 -0.02624741]\n",
            " [ 0.04355587 -0.04048269 -0.04138212  0.01247839 -0.01294935  0.00139042\n",
            "   0.01233207  0.03024682 -0.03334862 -0.02790955  0.01242272 -0.04128085\n",
            "   0.04214266  0.04348017  0.01045523 -0.00205957 -0.03556986 -0.01739997\n",
            "   0.04255753  0.02757342  0.0136765   0.01282351 -0.01459817 -0.00855327\n",
            "  -0.03894869 -0.0358853  -0.03112409  0.01894793  0.02213276 -0.02511839\n",
            "   0.00912381 -0.00024097]\n",
            " [ 0.00016055 -0.03171784 -0.03682018  0.01463613  0.04559476  0.01670735\n",
            "  -0.01924447 -0.01310781 -0.0052641  -0.03164054  0.00288255  0.02052755\n",
            "   0.0398633  -0.01861371  0.01233826  0.04681553 -0.03879207 -0.02040946\n",
            "   0.04356605 -0.03658737 -0.01806207 -0.0237723  -0.04685124  0.04004553\n",
            "   0.01409379  0.00821855 -0.02908291  0.02173609 -0.0136477  -0.04532908\n",
            "  -0.03502221  0.03436175]\n",
            " [ 0.01645621 -0.00589932 -0.01471175 -0.00355174 -0.04663396  0.01846724\n",
            "   0.02401174  0.03724445 -0.02736737 -0.02768031 -0.01896119  0.02223358\n",
            "  -0.03668128  0.00480639  0.00746088  0.03996835 -0.04905364  0.00212307\n",
            "   0.01345445 -0.03006717  0.02294225  0.00458346 -0.03924345  0.01767061\n",
            "   0.01602763 -0.01630496  0.01014177 -0.02893742  0.03527372 -0.00593783\n",
            "   0.04485276 -0.02624741]], shape=(4, 32), dtype=float32)\n",
            "WARNING:tensorflow:Layers in a Sequential model should only have a single input tensor, but we receive a <class 'str'> input: 898\n",
            "Consider rewriting this model with the Functional API.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:Layers in a Sequential model should only have a single input tensor, but we receive a <class 'str'> input: 898\n",
            "Consider rewriting this model with the Functional API.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "Embedding for the movie 898:\n",
            " [-0.00188267 -0.01525087 -0.00824153 -0.04668652  0.005212   -0.02621797\n",
            "  0.03498086 -0.03825183  0.02820224 -0.03877431  0.02133146 -0.0108206\n",
            "  0.02659546 -0.00487442 -0.02332041 -0.02766312 -0.03729247 -0.02090429\n",
            " -0.02203454  0.00766915  0.03768411 -0.00749893 -0.03042971 -0.01427402\n",
            " -0.0393108  -0.04658396 -0.00478858  0.01963672 -0.04822909 -0.00989123\n",
            "  0.02923337  0.02679086]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U3hdi7pZS2oC"
      },
      "source": [
        "### Tokenize textual features and translate them into embeddings\n",
        "Candidates textual description and users' reviews can be useful especially in a `cold-start` or `long-tail` scenario.\n",
        "\n",
        "While the MovieLens dataset does not give us rich textual features, we can still use movie titles. This may help us capture the fact that movies with very similar titles are likely to belong to the same series (for example \"Harry Potter and the Philosopher's Stone\" and \"Harry Potter and the Chamber of Secrets\")."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "mGF8c2ZKOtuG",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3941d2e4-cbc3-442e-d4cb-45210a26ad5f"
      },
      "source": [
        "# Keras TextVectorization layer transforms the raw texts into `word pieces` and\n",
        "# map these pieces into tokens.\n",
        "movie_title_vectorization_layer = \\\n",
        "    tf.keras.layers.experimental.preprocessing.TextVectorization()\n",
        "movie_title_vectorization_layer.adapt(\n",
        "    ratings_trainset.map(\n",
        "        lambda rating: rating['movie_title']\n",
        "    )\n",
        ")\n",
        "\n",
        "# Verify that the tokenization is done correctly\n",
        "print(\n",
        "    \"Vocabulary[40:50] -> \",\n",
        "    movie_title_vectorization_layer.get_vocabulary()[40:50]\n",
        ")\n",
        "\n",
        "print(\n",
        "    \"Vectorized title for 'Postman, The (1997)'\\n\",\n",
        "    movie_title_vectorization_layer('Postman, The (1997)')\n",
        ")\n",
        "\n",
        "movie_title_embedding_dim = 32\n",
        "movie_title_embedding_layer = tf.keras.layers.Embedding(\n",
        "    input_dim=len(movie_title_vectorization_layer.get_vocabulary()),\n",
        "    output_dim=movie_title_embedding_dim,\n",
        "    # Whether or not the input value 0 is a MASK token.\n",
        "    # Keras TextVectorization layer builds the vocabulary with MASK token.\n",
        "    mask_zero=True\n",
        ")\n",
        "\n",
        "movie_title_model = tf.keras.Sequential(\n",
        "    [\n",
        "       movie_title_vectorization_layer,\n",
        "       movie_title_embedding_layer,\n",
        "       # each title contains multiple words, so we will get multiple embeddings\n",
        "       # for each title that should be compressed into a single embedding for\n",
        "       # the text. Models like RNNs, Transformers or Attentions are useful here.\n",
        "       # However, averaging all the words' embeddings together is also a good\n",
        "       # starting point.\n",
        "       tf.keras.layers.GlobalAveragePooling1D()\n",
        "    ]\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Vocabulary[40:50] ->  ['first', 'contact', '1971', '1977', 'monty', '1983', 'love', 'on', 'last', 'men']\n",
            "Vectorized title for 'Postman, The (1997)'\n",
            " tf.Tensor([1120    2    4], shape=(3,), dtype=int64)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PEmOyPxuOcHx"
      },
      "source": [
        "For more information about feature preprocessing visit [this link](https://www.tensorflow.org/recommenders/examples/featurization?hl=lt)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UChreol5-Aj2"
      },
      "source": [
        "## Query and Candidate representation\n",
        "We are building a [two-tower retrieval model](https://research.google/pubs/pub48840/), a model including two seperate models (towers) one for transforming query raw features to query representation (query tower) and one another for transforming candidate raw features to the same dimensionality candidate representation.\n",
        "\n",
        "The output tensors of the two models will multiply together (inner product) to give a query-candidate `affinity score` (similarity measure). Higher scores express a better match between the candidate and the query."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "H1jx9RDfoJIq"
      },
      "source": [
        "# Query tower\n",
        "query_model = user_id_model\n",
        " \n",
        "# Candidate tower\n",
        "candidate_model = movie_id_model\n",
        " \n",
        " \n",
        "# Here we only used query and candidate identifiers to buid the towers. This\n",
        "# corresponds exactly to a classic matrix factorization approach.\n",
        "# https://ieeexplore.ieee.org/abstract/document/4781121\n",
        "# However, we can extend `tf.keras.Model` class to an arbitrarily complex model\n",
        "# including other features and return the final embedding vector at the end.\n",
        "# For example, by using movie metadata in the candidate tower, we can alleviate\n",
        "# cold-start problem.\n",
        "# return tf.concat([\n",
        "#         self.user_embedding(inputs[\"user_id\"]),\n",
        "#         self.timestamp_embedding(inputs[\"timestamp\"]),\n",
        "#         self.normalized_timestamp(inputs[\"timestamp\"])\n",
        "#     ], axis=1)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-X0nS9g18q4v"
      },
      "source": [
        "## Build the Retrieval (Candidate Generation) task\n",
        "\n",
        "It is about selecting an initial set of hundreds of candidates from all possible candidates. The main objective of this model is to efficiently weed out all candidates that the user is not interested in. Because the retrieval model may be dealing with millions of candidates, **it has to be computationally efficient**.\n",
        "\n",
        "A retrieval system is a model that predicts a set of movies from the catalogue that the user is likely to watch. So the train set should be expressesing which movies the users watched, and which they did not. for example:\n",
        "```\n",
        "[\n",
        "  (('user1', 'Star Wars'), POSITIVE),\n",
        "  (('user1', 'Harry Potter'), NEGATIVE),\n",
        "  ...\n",
        "]\n",
        "```\n",
        "So we treat Movielens as an `implicit feedback dataset`, where users' watches tell us which things they prefer to see and which they'd rather not see. This means that every movie a user rated (so watched!), no matter the given rating, is an **implicit positive example**, and every movie they have not rated (not seen!) is an **implicit negative example**."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QP5VaGW69c2B"
      },
      "source": [
        "# We don't need rating field for the retrieval task\n",
        "retrieval_ratings_trainset = ratings_trainset.map(\n",
        "    lambda rating: {\n",
        "        'user_id': rating['user_id'],\n",
        "        'movie_id': rating['movie_id'],\n",
        "    }\n",
        ")\n",
        " \n",
        "retrieval_ratings_testset = ratings_testset.map(\n",
        "    lambda rating: {\n",
        "        'user_id': rating['user_id'],\n",
        "        'movie_id': rating['movie_id'],\n",
        "    }\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SqeFh9s99bLw"
      },
      "source": [
        "The similarity between the query representation (query embedding vector) and the candiate representation (candidate embedding vector) a.k.a. `affinity score` can be calcualted by dot-product (or other similarity measures). The K-nearest candidates (candidates with higher affinity scores) will be chosen for the final list.\n",
        "\n",
        "In our training data we only have positive (user, movie) pairs. To figure out how good our model is, we need to compare the affinity score that the model calculates for this positive pair to the scores of all the other possible candidates: if the score for the positive pair is higher than for all other possible candidates, our model is highly accurate.\n",
        "\n",
        "To measure the performance of a retrieval task, `factorized top-K categorical accuracy` metrics over a corpus of candidates can be used. These metrics measure how good the model is at picking the true candidate out of **all possible candidates** in the system.\n",
        "\n",
        "For example, a `top-5 categorical accuracy` metric of `0.2` would tell us that, on average, the true positive is in the top 5 retrieved items 20% of the time."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DgwkIUn71_0y",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 356,
          "referenced_widgets": [
            "5836d1a2e06d4417a79a306987ca2a8d",
            "4c7f3f0497a442498e29140b13da3718",
            "5eac2b4797f342589fdf1a729d69b4c4",
            "82bc0ba55b52415ca3c1d092d402f0d8",
            "4cead5ce2ec549389930e286008dbd43",
            "43a37170841b406ead6ed8a0061290f5",
            "f2df5ab944484b9aa5571b1f92fb31ae",
            "78a235be12ae486cb85db74c9e0dc696",
            "37d9b4c872a042bf83509aabf01e49cf",
            "048b5344d1f94581b549a774e27d5b54",
            "c8cca219c1194d36bc93f0ce16664154",
            "3dd0280309064c14970cd7186b7ba0d6",
            "d370bf61eff04e778a1810ed74ce2d45",
            "76b9fe27088a42eeb6c861aa61b4218c",
            "cb907be1778148688a8b3d344378259e",
            "be1e18a683bf474b83414922115b8cb3",
            "017c5f33da3a497097df21de70bfd873",
            "0bbb513d33884d6ab920498cf07a74a1",
            "f5994d027b694fd68d9534f4685b8448",
            "e42f9d4198c549da9a0c3abdffc30cd7",
            "cae09be9a5f346a2bb7663396474f7ab",
            "3ce95d4466bc43019bb09538fe551f2d",
            "7122e14af06b4b0e9c9091cf3d325122",
            "e1fa286fbc844c6e821b1aa2479a5c61",
            "1d1e79c03bb145738ef6fee464a5274e",
            "cea1babe04e34a929978ae0ee86d07da",
            "e6964e7aaa8f42e293e41ca62f8bb1d3",
            "c4a5eb5a8e654be4b269af08a31add41",
            "b8425150aca94b619551ee8fe8349738",
            "1b30b087eda94047a350ee51248512d5",
            "df215de96d744afca92ae23c3ef16263",
            "8ab2b2f1658b4a728bbb31b0817a2973",
            "380f4d4102b34e60abfed1eb9bdbadd2",
            "00f1ec84c2c54d36b5a69eb76f546613",
            "bd1f4889292c471992e68a83ab764858",
            "d883f0e109ff46f9a449a411a1404025",
            "9d5e2fb4988e46fea9cd924e82e13d5d",
            "a7318131bcd347c7a3c28fe94bcce41b",
            "c0ad928e95664faea5f00efb245e75ec",
            "7fafd911808f4381b88aa2956beb3d1c",
            "a31a11356ec04a97b6cc7957985fce93",
            "8ca8e193f3604436a2b76b062a1b2bfa",
            "5641187ab3124a2c97613794ff3a2140",
            "87c44927b6fd4fa188a3f2bba1968d26",
            "bc2f6fcc89734b43a83c72226f5d6bc0",
            "f51b4a5b99e040fd9f9681c5f2451c74",
            "dc80ecc2d8974e0bb23120bff7d0e215",
            "0ee218a7eed5400cb5b79a338effae5d"
          ]
        },
        "outputId": "5f0d33f7-88fb-4502-8a7d-cdee8a763306"
      },
      "source": [
        "# To calculate the factorized top-k categorical accuracy we need the dataset of\n",
        "# all possible candidates that are used as implicit negatives for evaluation.\n",
        "movies_dataset, movies_dataset_info = tfds.load(\n",
        "    name='movielens/100k-movies',\n",
        "    split='train',\n",
        "    with_info=True\n",
        ")\n",
        "\n",
        "print(\n",
        "    tfds.as_dataframe(movies_dataset.take(5), movies_dataset_info)\n",
        ")\n",
        " \n",
        "# We are using just `movie_id` feature for making the candidates representation\n",
        "candidates_corpus_dataset = movies_dataset.map(\n",
        "    lambda movie: movie['movie_id']\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[1mDownloading and preparing dataset 4.70 MiB (download: 4.70 MiB, generated: 150.35 KiB, total: 4.84 MiB) to /root/tensorflow_datasets/movielens/100k-movies/0.1.0...\u001b[0m\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "5836d1a2e06d4417a79a306987ca2a8d",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Dl Completed...', max=1.0, style=Progre…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "37d9b4c872a042bf83509aabf01e49cf",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Dl Size...', max=1.0, style=ProgressSty…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "017c5f33da3a497097df21de70bfd873",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Extraction completed...', max=1.0, styl…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "1d1e79c03bb145738ef6fee464a5274e",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Generating splits...', max=1.0, style=ProgressStyle(descr…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "380f4d4102b34e60abfed1eb9bdbadd2",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Generating train examples...', max=1682.0, style=Progress…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "display_data",
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a31a11356ec04a97b6cc7957985fce93",
              "version_minor": 0,
              "version_major": 2
            },
            "text/plain": [
              "HBox(children=(FloatProgress(value=0.0, description='Shuffling movielens-train.tfrecord...', max=1682.0, style…"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "\r\u001b[1mDataset movielens downloaded and prepared to /root/tensorflow_datasets/movielens/100k-movies/0.1.0. Subsequent calls will reuse this data.\u001b[0m\n",
            "  movie_genres movie_id                      movie_title\n",
            "0          [4]  b'1681'           b'You So Crazy (1994)'\n",
            "1       [4, 7]  b'1457'   b'Love Is All There Is (1996)'\n",
            "2       [1, 3]   b'500'          b'Fly Away Home (1996)'\n",
            "3          [0]   b'838'  b'In the Line of Duty 2 (1987)'\n",
            "4          [7]  b'1648'       b'Niagara, Niagara (1997)'\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P0aPHewDuQZf"
      },
      "source": [
        "[TensorFlow Recommenders](https://www.tensorflow.org/recommenders) (TFRS) is a library to facilitate building and evaluating flexible recommendation models.\n",
        "\n",
        "It can calculate the `factorized top-k categorical accuracy` through `FactorizedTopK` metrics using dataset of all possible candidate embeddings."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YPO73Ihx_xQS",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "429718b5-5533-49a2-cc06-ed6379ce37b4"
      },
      "source": [
        "!pip install -q scann tensorflow-recommenders\n",
        "# We also installed `ScaNN` package as a dependency for TFRS library\n",
        "# We will describe ScaNN in future but it has to be installed before importing\n",
        "# the TFRS\n",
        "import tensorflow_recommenders as tfrs\n",
        "\n",
        "factorized_top_k_metrics = tfrs.metrics.FactorizedTopK(\n",
        "    # dataset of candidate embeddings from which candidates should be retrieved\n",
        "    candidates=candidates_corpus_dataset.batch(128).map(\n",
        "        candidate_model\n",
        "    )\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\u001b[K     |████████████████████████████████| 61kB 4.1MB/s \n",
            "\u001b[K     |████████████████████████████████| 394.7MB 43kB/s \n",
            "\u001b[K     |████████████████████████████████| 11.1MB 11.1MB/s \n",
            "\u001b[?25h"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ie_v22Od-zY8"
      },
      "source": [
        "`in-batch softmax loss` can be used as a `loss function` in order to train the system.\n",
        "\n",
        "TFRS proposes a Keras layer named `tfrs.tasks.Retrieval` that takes the query and candidate embeddings as arguments, and returns the computed loss."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4E8eOyErNqHY"
      },
      "source": [
        "retrieval_task_layer = tfrs.tasks.Retrieval(\n",
        "    metrics=factorized_top_k_metrics\n",
        ")\n",
        "\n",
        "# The task computes the metrics and return the in-batch softmax loss.\n",
        "# Because the metrics range over the entire candidate set, they are usually much\n",
        "# slower to compute. Consider setting `compute_metrics=False` in Retrieval\n",
        "# costructor during training to save the time in computing the metrics."
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lVFXh69bPYL9"
      },
      "source": [
        "### Create the training loop\n",
        "To create an appropriate training loop and train the models we can extend the class `tf.keras.Model` and override the `train_step` and `test_step` functions. [See how](https://keras.io/guides/customizing_what_happens_in_fit/).\n",
        "\n",
        "However, to keep the focus on modelling and abstract away some of the boilerplate, TFRS exposes `tfrs.models.Model` base class which allows us to compute both training and test losses using the same method. All we need to do is to set up the components in the `__init__` method, and implement the `compute_loss` method, taking in the raw features and returning a loss value. The base model will then take care of creating the appropriate training loop to fit the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EGN_rnLNN07m"
      },
      "source": [
        "class RetrievalModel(tfrs.models.Model):\n",
        "  \"\"\"MovieLens candidate generation model\"\"\"\n",
        " \n",
        "  def __init__(self, query_model, candidate_model, retrieval_task_layer):\n",
        "    super().__init__()\n",
        "    self.query_model: tf.keras.Model = query_model\n",
        "    self.candidate_model: tf.keras.Model = candidate_model\n",
        "    self.retrieval_task_layer: tf.keras.layers.Layer = retrieval_task_layer\n",
        " \n",
        " #def compute_loss(self, features: Dict[Text, tf.Tensor], training=False):\n",
        "  def compute_loss(self, features, training=False) -> tf.Tensor:\n",
        "    query_embeddings = self.query_model(features['user_id'])\n",
        "    positive_candidate_embeddings = self.candidate_model(features[\"movie_id\"])\n",
        "\n",
        "    loss = self.retrieval_task_layer(\n",
        "        query_embeddings,\n",
        "        positive_candidate_embeddings\n",
        "        # ,compute_metrics=not training  # To speed up training\n",
        "    )\n",
        "    return loss"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ylJqYSN8GH68"
      },
      "source": [
        "### Fit the model using standard Keras routine"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HoyYTDhB9CAP"
      },
      "source": [
        "movielens_retrieval_model = RetrievalModel(\n",
        "    query_model,\n",
        "    candidate_model,\n",
        "    retrieval_task_layer\n",
        ")\n",
        "\n",
        "optimizer_step_size = 0.1\n",
        "movielens_retrieval_model.compile(\n",
        "    optimizer=tf.keras.optimizers.Adagrad(\n",
        "        learning_rate=optimizer_step_size\n",
        "    )\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZNw02KWxE3bI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0282f6e9-af6a-49b9-e726-a3cfe593df10"
      },
      "source": [
        "# Shuffle the training data for each epoch.\n",
        "# Batch and cache both the training and evaluation data.\n",
        "# `cache()` method caches the elements in the dataset in memory. To caches data\n",
        "# in a file pass the `filename` argument to the method: cache(filename='')\n",
        "# The first time the dataset is iterated over, its elements will be cached\n",
        "# either in the specified file or in memory. Subsequent iterations will use the\n",
        "# cached data.\n",
        "retrieval_cached_ratings_trainset = \\\n",
        "  retrieval_ratings_trainset.shuffle(100_000).batch(8192).cache()\n",
        "retrieval_cached_ratings_testset = \\\n",
        "  retrieval_ratings_testset.batch(4096).cache()\n",
        " \n",
        "num_epochs = 5 \n",
        "history = movielens_retrieval_model.fit(\n",
        "    retrieval_cached_ratings_trainset,\n",
        "    validation_data=retrieval_cached_ratings_testset,\n",
        "    validation_freq=1,\n",
        "    epochs=num_epochs\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "10/10 [==============================] - 33s 3s/step - factorized_top_k/top_1_categorical_accuracy: 1.7500e-04 - factorized_top_k/top_5_categorical_accuracy: 0.0025 - factorized_top_k/top_10_categorical_accuracy: 0.0064 - factorized_top_k/top_50_categorical_accuracy: 0.0655 - factorized_top_k/top_100_categorical_accuracy: 0.1396 - loss: 69844.1065 - regularization_loss: 0.0000e+00 - total_loss: 69844.1065 - val_factorized_top_k/top_1_categorical_accuracy: 0.0024 - val_factorized_top_k/top_5_categorical_accuracy: 0.0148 - val_factorized_top_k/top_10_categorical_accuracy: 0.0314 - val_factorized_top_k/top_50_categorical_accuracy: 0.1550 - val_factorized_top_k/top_100_categorical_accuracy: 0.2723 - val_loss: 28808.9102 - val_regularization_loss: 0.0000e+00 - val_total_loss: 28808.9102\n",
            "Epoch 2/5\n",
            "10/10 [==============================] - 29s 3s/step - factorized_top_k/top_1_categorical_accuracy: 0.0016 - factorized_top_k/top_5_categorical_accuracy: 0.0143 - factorized_top_k/top_10_categorical_accuracy: 0.0320 - factorized_top_k/top_50_categorical_accuracy: 0.1571 - factorized_top_k/top_100_categorical_accuracy: 0.2805 - loss: 67482.7500 - regularization_loss: 0.0000e+00 - total_loss: 67482.7500 - val_factorized_top_k/top_1_categorical_accuracy: 0.0015 - val_factorized_top_k/top_5_categorical_accuracy: 0.0125 - val_factorized_top_k/top_10_categorical_accuracy: 0.0260 - val_factorized_top_k/top_50_categorical_accuracy: 0.1339 - val_factorized_top_k/top_100_categorical_accuracy: 0.2493 - val_loss: 28339.3379 - val_regularization_loss: 0.0000e+00 - val_total_loss: 28339.3379\n",
            "Epoch 3/5\n",
            "10/10 [==============================] - 30s 3s/step - factorized_top_k/top_1_categorical_accuracy: 0.0024 - factorized_top_k/top_5_categorical_accuracy: 0.0200 - factorized_top_k/top_10_categorical_accuracy: 0.0427 - factorized_top_k/top_50_categorical_accuracy: 0.1851 - factorized_top_k/top_100_categorical_accuracy: 0.3122 - loss: 66307.1009 - regularization_loss: 0.0000e+00 - total_loss: 66307.1009 - val_factorized_top_k/top_1_categorical_accuracy: 0.0012 - val_factorized_top_k/top_5_categorical_accuracy: 0.0097 - val_factorized_top_k/top_10_categorical_accuracy: 0.0216 - val_factorized_top_k/top_50_categorical_accuracy: 0.1235 - val_factorized_top_k/top_100_categorical_accuracy: 0.2349 - val_loss: 28256.8340 - val_regularization_loss: 0.0000e+00 - val_total_loss: 28256.8340\n",
            "Epoch 4/5\n",
            "10/10 [==============================] - 30s 3s/step - factorized_top_k/top_1_categorical_accuracy: 0.0023 - factorized_top_k/top_5_categorical_accuracy: 0.0241 - factorized_top_k/top_10_categorical_accuracy: 0.0487 - factorized_top_k/top_50_categorical_accuracy: 0.2023 - factorized_top_k/top_100_categorical_accuracy: 0.3328 - loss: 65604.8991 - regularization_loss: 0.0000e+00 - total_loss: 65604.8991 - val_factorized_top_k/top_1_categorical_accuracy: 0.0010 - val_factorized_top_k/top_5_categorical_accuracy: 0.0071 - val_factorized_top_k/top_10_categorical_accuracy: 0.0179 - val_factorized_top_k/top_50_categorical_accuracy: 0.1152 - val_factorized_top_k/top_100_categorical_accuracy: 0.2237 - val_loss: 28279.2637 - val_regularization_loss: 0.0000e+00 - val_total_loss: 28279.2637\n",
            "Epoch 5/5\n",
            "10/10 [==============================] - 30s 3s/step - factorized_top_k/top_1_categorical_accuracy: 0.0026 - factorized_top_k/top_5_categorical_accuracy: 0.0261 - factorized_top_k/top_10_categorical_accuracy: 0.0543 - factorized_top_k/top_50_categorical_accuracy: 0.2155 - factorized_top_k/top_100_categorical_accuracy: 0.3483 - loss: 65096.5256 - regularization_loss: 0.0000e+00 - total_loss: 65096.5256 - val_factorized_top_k/top_1_categorical_accuracy: 6.5000e-04 - val_factorized_top_k/top_5_categorical_accuracy: 0.0057 - val_factorized_top_k/top_10_categorical_accuracy: 0.0151 - val_factorized_top_k/top_50_categorical_accuracy: 0.1055 - val_factorized_top_k/top_100_categorical_accuracy: 0.2146 - val_loss: 28346.2773 - val_regularization_loss: 0.0000e+00 - val_total_loss: 28346.2773\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lvDnN_hQJ5fi"
      },
      "source": [
        "`factorized_top_k/top_10_categorical_accuracy: 0.0538` would tell us that, on average, the true positive is in the top 10 retrieved items from the entire candidate set 5% of the time.\n",
        "\n",
        "If the candidate set is a large set, turn metric calculation off in training, and only run it in evaluation. Because this can be quite slow!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3djrQCxhBZCV",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "outputId": "52d7a26b-63da-4e7b-9412-3651a3df4215"
      },
      "source": [
        "# Plot changes in model loss during training\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.plot(history.history[\"loss\"])\n",
        "plt.plot(history.history[\"val_loss\"])\n",
        "plt.title(\"Model losses during training\")\n",
        "plt.xlabel(\"epoch\")\n",
        "plt.ylabel(\"loss\")\n",
        "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEWCAYAAACnlKo3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xdZX3v8c93bplbksmNWyaYIFQBqxGGSIscEQ4QLgU8UooVQWvFFm3xVU8FTkWKl1N7TiuUVkREFFDEFKVExAIK6PHCJWDkLgkBmgSEIWGSzEwymcvv/LGemeyZzEz2rMzek2S+79drv2at53nW2s9ayd7fvZ619tqKCMzMzPKomOgOmJnZ7sshYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8R2KZLmSwpJVUW0/aCkn+/senYlkl6Q9N93YvknJR0zjl0aN5L2l9QuqXI829rEcohYbukNb6uk2UPKf53ewOdPTM8mr4g4NCLuH+/1jhbYxYqI/4qIxojoHc+2NrEcIrazngfe1z8j6feB+onrzuS0Kxxx+ahhcnKI2M66CTi3YP484MbCBpKmS7pRUqukFyV9WlJFqquU9E+SXpO0CjhlmGW/LullSWslfT7Pm5Wk/SQtlbRe0kpJHymoWyRpmaSNkl6R9KVUXivpW5LWSWqT9LCkvXfUL0kHSvqppA1pu747Sr8+kPbJOkl/N6Tum5I+XzB/jKQ1BfMvSLpI0mNAh6SqwuEwSX8vaUna95vSUFdLwfKHpaPGTZL+XdJ3C5+voN3BwDXAH6QhpraC/n1F0p2SOoB3SzolrXOjpNWS/r5gPYOGGCXdL+lzkn6R+nB3/1HtWNqm+nML9uOl2slhQSueQ8R21gPANEkHpzfRs4FvDWnzr8B04ADgXWSh86FU9xHgVODtQAtw5pBlvwn0AAemNicAf56jn7cAa4D90nP8b0nHprp/Af4lIqYBbwSWpPLzUr/nAbOAvwA2F9GvzwF3AzOA5rT925F0CPAV4AOpX7NS+7F4H1nwNkVEzzD1p5FtexOwFPi39Nw1wG1pO2YC3wHeM9wTRMTTZNv+qzTE1FRQ/afAF4CpwM+BDrJ/36bUr7+UdMYo/f9Tsv8LewE1wP8ca9u0H68G3g/sS/ZvNneU9dg4cojYeOg/GjkeeBpY219RECyXRMSmiHgB+GeyN06As4ArI2J1RKwH/qFg2b2Bk4FPRERHRLwKXJHWVzRJ84CjgIsiYktELAeuY9sRVDdwoKTZEdEeEQ8UlM8CDoyI3oh4JCI2FtGvbuANwH7p+UY6l3AmcEdE/CwiuoBLgb6xbBtwVdp3m0eo/3lE3JnOLdwEvC2VHwlUpeW7I+L7wENjfG6A2yPiFxHRl7b1/oh4PM0/RhZO7xpl+W9ExLOp/0uAhTnangn8ICJ+HhFbgc8AvilgmThEbDzcRPYp8YMMGcoCZgPVwIsFZS+y7ZPifsDqIXX93pCWfTkNJ7UBXyX7JDoW+wHrI2LTCH34MPB7wDNpyOrUgu26C7hF0kuS/o+k6iL69SlAwENpCOnPRunXwLZHRAewbozbtnoH9b8rmO4EatMQ0X7A2hh8B9YdrWuHzy/pHZLuUzZ0uYHsCGb28IsO27/GHG2H7sdOxr4fLSeHiO20iHiR7AT7ycD3h1S/xrZP5v32Z9vRystkw0WFdf1WA13A7IhoSo9pEXHoGLv4EjBT0tTh+hARKyLifWQh8I/ArZIa0if0yyPiEOAPyYbdzt1RvyLidxHxkYjYD/gocLWkA4fp16Btl1RPduTTr4PBFynsM8w68n7ifhmYK0kFZfNGajzK8wwtv5ls2GxeREwnO5ei7ZYaXy9TMAwoqY7B+9FKyCFi4+XDwLHp0/SANIyyBPiCpKmS3gD8DdvOmywB/lpSs6QZwMUFy75Mdm7hnyVNk1Qh6Y2SRhse2U5ErAZ+CfxDOln+1tTfbwFIOkfSnIjoA9rSYn2S3i3p99OQ3EayMOzbUb8k/bGk/je118neaIcbproVOFXSO9M5is8y+DW5HDhZ0kxJ+wCfGMt278CvgF7g4+mE/OnAolHavwI0p36OZirZUd8WSYvIjlBL7VbgjyT9Yerf31P64LLEIWLjIiKei4hlI1T/Fdmn6lVkJ19vBq5PdV8jGzL6DfAo2x/JnEt2EvUpsjfkW8lOno7V+4D5ZEcltwGXRcSPU91i4ElJ7WQn2c9O4+77pOfbSHau56dkQ1w76tcRwINpfUuBCyNi1dAORcSTwMfI9sfLaT1rCprcRLZfXiALrRGv8hqrdO7gf5CFaRtwDnAH2RHWcO4FngR+J+m1UVZ9AfBZSZvIzk0sGaXtuEj78a/ILiB4GWgHXmXkbbFxJP8olZkBSHoQuCYivjHRfdkZkhrJgvGgiHh+ovuzp/ORiNkkJeldkvZJw1nnAW8F/nOi+5WHpD+SVC+pAfgn4HGyIzgrMYeI2eT1JrLhsjbgk8CZ6XzP7uh0sqHKl4CDyIYkPcxSBh7OMjOz3HwkYmZmuU34TdvKbfbs2TF//vyJ7oaZ2W7jkUceeS0i5gxXN+lCZP78+SxbNtKVqGZmNpSkF0eq83CWmZnl5hAxM7PcHCJmZpbbpDsnYmY2Vt3d3axZs4YtW7ZMdFdKqra2lubmZqqrq4texiFiZrYDa9asYerUqcyfP5/BNz7ec0QE69atY82aNSxYsKDo5TycZWa2A1u2bGHWrFl7bIAASGLWrFljPtpyiJiZFWFPDpB+ebbRw1lFuuonK5jRUMMbZzdwwJxG9p42ZVL8pzIzG41DpAg9vX187f+tYtOWnoGyhppKFsxpYMHsRg6Y3cABcxp445xGFsxuoGGKd6uZjZ+2tjZuvvlmLrjggjEtd/LJJ3PzzTfT1NRUop45RIpSVVnBY5edwO82bmFVawerXutgVWs7q1o7WL76de547CUK72O5z7RaDpiTBcsBsxtZMKeBN85uZO6MOiorfPRiZmPT1tbG1VdfvV2I9PT0UFU18tv4nXfeWequOUSKJYl9p9ex7/Q6jjpw9qC6Ld29vLiuMwuW1zpS0LTzg9+8zIbN3QPtaqoqmD+rngNmN6aQSX9nN9BUv6NfHTWzyeriiy/mueeeY+HChVRXV1NbW8uMGTN45plnePbZZznjjDNYvXo1W7Zs4cILL+T8888Htt3mqb29nZNOOol3vvOd/PKXv2Tu3Lncfvvt1NXV7XTfHCLjoLa6kjftM5U37TN1UHlEsL5j66Ajl+daO3j21U38+OlX6Onbdvgys6FmYFjsgDn9Q2SN7D+znpoqX/9gtqu4/AdP8tRLG8d1nYfsN43L/ujQEeu/+MUv8sQTT7B8+XLuv/9+TjnlFJ544omBS3Gvv/56Zs6cyebNmzniiCN473vfy6xZswatY8WKFXznO9/ha1/7GmeddRbf+973OOecc3a67w6REpLErMYpzGqcwhHzZw6q6+ntY/XrmwfCZdVr7TzX2sG9z7SyZNm2n9murBDzZtQNCpb+obI5jT65bzYZLVq0aNB3Oa666ipuu+02AFavXs2KFSu2C5EFCxawcOFCAA4//HBeeOGFcemLQ2SCVFVWsGB2AwtmN3DcwYPrNm7p5vkULKtaO9IRTDu/WPkaXT19A+2mTqnigDnZOrYNjWUn9+tqKsu8RWaTw2hHDOXS0NAwMH3//ffz4x//mF/96lfU19dzzDHHDPtdjylTpgxMV1ZWsnnz5nHpi0NkFzSttpq3zWvibfMGX1HR1xe8tGEzz/efd0nnYB5+4XX+Y/lLg9rObaobON9yQLpq7IA5Dew3vY4Kn9w3261MnTqVTZs2DVu3YcMGZsyYQX19Pc888wwPPPBAWfvmENmNVFSI5hn1NM+o5+iDBv8+zOatvVm4vNaejmKykPn+o2vZ1LXt0uTa6grmz8ouRx56Bdm02uLvl2Nm5TNr1iyOOuoo3vKWt1BXV8fee+89ULd48WKuueYaDj74YN70pjdx5JFHlrVvk+431ltaWmIy/ShVRNDa3jUwLLaqtT2FTQf/tb6T3oKT+7Mbp6TvuzQMuoJs3ow6qip9ct8mr6effpqDDz54xw33AMNtq6RHIqJluPY+EtnDSWKvqbXsNbWWIw8YfKJta08f/7W+8NLk7BzMXU++wvqO1QPtqivF/jPrWTC7MQuYgivIZjbU+OS+2STmEJnEaqoqOHCvRg7cq3G7urbOrdu+81JwBdnPnm1la++2k/vT66oHhsQKz8G8YVY9tdU+uW+2p3OI2LCa6ms4bP8aDtt/xqDy3r7gpbbNPFcQLKtaO/jFytf43qPbLk2uUPbdl+l11TTV19BUV830+mqa6mpoqq+mqb6a6XXVg+qb6quZWlvtb/Wb7UYcIjYmlRVi3sx65s2s55g3Da7r6OoZON+yqrWd1k1dtG3uZkNnN69s2sJvX9nEhs7uQSf6h5Kyq9Oa6qtT8GwLmKa6aqYNCZ0sjLKw8pcyzcrPIWLjpmFKFW+ZO523zJ0+arvu3j42bu6mbXM3bZ3dbNi8lQ1pOpvvpq1z60D96vWdtHVmbfpGuQ6koaaSpvr+o5/BAdMfQv1l/fVNdTXUVlf4vI5ZTg4RK7vqyoqBb/KPRV9fsKmrhw2d3bRt3pqFzuZuNnQWTBcE04pX2lMQbaW7d+T0qamqKDjaqUlHOwWhM+hoqCaVVTN1SpXDxyY9h4jtNioqNHAeZX/qi14uItjc3TtwpNO2eWsKoiHzaXpt22aeemkDbZu76dzaO+J6K1N/tg2z9QfP4KOhprqadD4oq5tWW+VLpm1M8t4KHuDKK6/k/PPPp76++NfMWDhEbI8nifqaKuprqtivaWx3Le3q6WVDOq8zEDqdBcNv6Yhow+Zu1rVv5bnWdto6uwf99sxwpk6pyoKlIGSm1WZ9bJhSRUNNJfXpbzZfRf2UShpqqmhIf+unVFJT6aG4yWCkW8EX48orr+Scc85xiJhNhClVlew1tZK9ptaOabme3j42bukZdH5nQ+fgcz2FdWvbNrNpSw+dW3tGPfoZqqpC1NdU0jilaiB0BoJoSppOgdTYP5/+Nk6poj6FVH1Nf0BV+QKFXVDhreCPP/549tprL5YsWUJXVxfvec97uPzyy+no6OCss85izZo19Pb2cumll/LKK6/w0ksv8e53v5vZs2dz3333jXvfHCJmJVBVWcHMhhpmNtQADTtsX6ivLxt+69jaQ0dXLx1dWbBk8z10dmXTnVuzuo6uHjq29tJZ0H5t2+aB+bEGU3WlBsKnYbtwqtx2pDRofnAgDQ21PSqYfnQx/O7x8V3nPr8PJ31xxOrCW8Hffffd3HrrrTz00ENEBKeddho/+9nPaG1tZb/99uOHP/whkN1Ta/r06XzpS1/ivvvuY/bs2SOuf2c4RMx2MRUVSkcSVTB1x+2L0ZuCqTMFTn/49IdTZ1cv7V3ZkVDH1qxdewqg/vnXOzenZbJw2txdfDDVVFYMDMfVDz0yGpjfdkRUPyUFUc3IQ3vVk/S80t13383dd9/N29/+dgDa29tZsWIFRx99NJ/85Ce56KKLOPXUUzn66KPL0h+HiNkkUFkhGtMb9Xjp7YuBo5yBQOrqGTiCKjwSat9uPmu/rr2Tzq3byrZ09+34iZPqSlFbVcmU6kpqqyuoq66kNk3XDkxXUltVMVBeV93fPrWrqqSuZtv0lOrKtJ7CdVQM+vnr0Y4YyiEiuOSSS/joRz+6Xd2jjz7KnXfeyac//WmOO+44PvOZz5S8Pw4RM8ulskJMrc3uMjBeCoOpfdDQ3bahuv4jo87uXrYMPPoGTW/a0kPrpi66erLyzQV1eXzttH3pXbuBCkGFhNLfCmUXbgyaBlSh7dputwxpuiJbZmjbwgsmCm8Ff+KJJ3LppZfy/ve/n8bGRtauXUt1dTU9PT3MnDmTc845h6amJq677rpBy3o4y8z2eIXBtPeOm49ZRNDV00dXd9+2YOnZFkKbu3vp6h48v6W7j2m1HcxurCEiO2fVl9bVF9AXQW9fH91pOtLfvsja5DU4eKp56+GL+L03H8K7jjueE097Ly2L3gFAQ0Mj/3rt9bz4/HN87jN/R0VFBdXV1fzTlVfR1rmVD3zow5xw4onsu+9+/Oyn94/PjizgW8Gbme1A3lvBxzCh0jcQNsOVFbQllfcVlm+bjoA+Bq9jtPfzqooKDtlvWq5t9a3gzcwmgPqPJijPd3kGB9PggCmVkl7eIOkFSY9LWi5pWSqbKekeSSvS3xmpXJKukrRS0mOSDitYz3mp/QpJ5xWUH57WvzIt629dmdmkJYnKClFVWUFNVXZhQN3A94ZKc8xQjmvk3h0RCwsOhS4GfhIRBwE/SfMAJwEHpcf5wFcgCx3gMuAdwCLgsv7gSW0+UrDc4tJvjplNRpNh6D/PNk7EhdanAzek6RuAMwrKb4zMA0CTpH2BE4F7ImJ9RLwO3AMsTnXTIuKByLb8xoJ1mZmNm9raWtatW7dHB0lEsG7dOmprx3Z3hlKfEwngbkkBfDUirgX2joiXU/3vYOAijLnA6oJl16Sy0crXDFO+HUnnkx3dsP/+++/M9pjZJNTc3MyaNWtobW2d6K6UVG1tLc3NzWNaptQh8s6IWCtpL+AeSc8UVkZEpIApqRRe10J2dVapn8/M9izV1dUsWLBgoruxSyrpcFZErE1/XwVuIzun8UoaiiL9fTU1XwvMK1i8OZWNVt48TLmZmZVJyUJEUoOkqf3TwAnAE8BSoP8Kq/OA29P0UuDcdJXWkcCGNOx1F3CCpBnphPoJwF2pbqOkI9NVWecWrMvMzMqglMNZewO3patuq4CbI+I/JT0MLJH0YeBF4KzU/k7gZGAl0Al8CCAi1kv6HPBwavfZiFifpi8AvgnUAT9KDzMzKxN/Y93MzEY12jfWJ+e9lM3MbFw4RMzMLDeHiJmZ5eYQMTOz3BwiZmaWm0PEzMxyc4iYmVluDhEzM8vNIWJmZrk5RMzMLDeHiJmZ5eYQMTOz3BwiZmaWm0PEzMxyc4iYmVluDhEzM8vNIWJmZrk5RMzMLDeHiJmZ5eYQMTOz3BwiZmaWm0PEzMxyc4iYmVluDhEzM8vNIWJmZrk5RMzMLDeHiJmZ5eYQMTOz3BwiZmaWm0PEzMxyK3mISKqU9GtJd6T5b0p6XtLy9FiYyiXpKkkrJT0m6bCCdZwnaUV6nFdQfrikx9MyV0lSqbfHzMy2KceRyIXA00PK/jYiFqbH8lR2EnBQepwPfAVA0kzgMuAdwCLgMkkz0jJfAT5SsNziUm6ImZkNVtIQkdQMnAJcV0Tz04EbI/MA0CRpX+BE4J6IWB8RrwP3AItT3bSIeCAiArgROKM0W2JmZsMp9ZHIlcCngL4h5V9IQ1ZXSJqSyuYCqwvarEllo5WvGaZ8O5LOl7RM0rLW1tbcG2NmZoOVLEQknQq8GhGPDKm6BHgzcAQwE7ioVH3oFxHXRkRLRLTMmTOn1E9nZjZplPJI5CjgNEkvALcAx0r6VkS8nIasuoBvkJ3nAFgLzCtYvjmVjVbePEy5mZmVSclCJCIuiYjmiJgPnA3cGxHnpHMZpCupzgCeSIssBc5NV2kdCWyIiJeBu4ATJM1IJ9RPAO5KdRslHZnWdS5we6m2x8zMtlc1Ac/5bUlzAAHLgb9I5XcCJwMrgU7gQwARsV7S54CHU7vPRsT6NH0B8E2gDvhRepiZWZkou7Bp8mhpaYlly5ZNdDfMzHYbkh6JiJbh6vyNdTMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW4OETMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW4OETMzy80hYmZmuTlEzMwsN4eImZnlVlSISLpQ0jRlvi7pUUknlLpzZma2ayv2SOTPImIjcAIwA/gA8MWS9crMzHYLxYaI0t+TgZsi4smCMjMzm6SKDZFHJN1NFiJ3SZoK9JWuW2ZmtjuoKrLdh4GFwKqI6JQ0E/hQ6bplZma7g2KPRP4A+G1EtEk6B/g0sKF03TIzs91BsSHyFaBT0tuATwLPATeWrFdmZrZbKDZEeiIigNOBf4uILwNTS9ctMzPbHRR7TmSTpEvILu09WlIFUF26bpmZ2e6g2CORPwG6yL4v8jugGfi/JeuVmZntFooKkRQc3wamSzoV2BIRRZ0TkVQp6deS7kjzCyQ9KGmlpO9KqknlU9L8ylQ/v2Adl6Ty30o6saB8cSpbKeniorfazMzGRbG3PTkLeAj4Y+As4EFJZxb5HBcCTxfM/yNwRUQcCLxOdvkw6e/rqfyK1A5JhwBnA4cCi4GrUzBVAl8GTgIOAd6X2pqZWZkUO5z1d8AREXFeRJwLLAIu3dFCkpqBU4Dr0ryAY4FbU5MbgDPS9OlpnlR/XGp/OnBLRHRFxPPAyvT8i4CVEbEqIrYCt6S2ZmZWJsWGSEVEvFowv67IZa8EPsW2b7fPAtoioifNrwHmpum5wGqAVL8htR8oH7LMSOXbkXS+pGWSlrW2thbRbTMzK0axIfKfku6S9EFJHwR+CNw52gLp3MmrEfHITvZxp0XEtRHREhEtc+bMmejumJntMYq6xDci/lbSe4GjUtG1EXHbDhY7CjhN0slALTAN+BegSVJVOtpoBtam9muBecAaSVXAdLIjnv7yfoXLjFRuZmZlUPSPUkXE9yLib9JjRwFCRFwSEc0RMZ/sxPi9EfF+4D6g/6T8ecDtaXppmifV35u+4LgUODtdvbUAOIjsJP/DwEHpaq+a9BxLi90eMzPbeaMeiUjaBMRwVUBExLQcz3kRcIukzwO/Br6eyr8O3CRpJbCeLBSIiCclLQGeAnqAj0VEb+rfx4G7gErg+nSLejMzKxNlH/Ynj5aWlli2bNlEd8PMbLch6ZGIaBmuzr+xbmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW4OETMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW4OETMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW4OETMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHIrWYhIqpX0kKTfSHpS0uWp/JuSnpe0PD0WpnJJukrSSkmPSTqsYF3nSVqRHucVlB8u6fG0zFWSVKrtMTOz7VWVcN1dwLER0S6pGvi5pB+lur+NiFuHtD8JOCg93gF8BXiHpJnAZUALEMAjkpZGxOupzUeAB4E7gcXAjzAzs7Io2ZFIZNrTbHV6xCiLnA7cmJZ7AGiStC9wInBPRKxPwXEPsDjVTYuIByIigBuBM0q1PWZmtr2SnhORVClpOfAqWRA8mKq+kIasrpA0JZXNBVYXLL4mlY1WvmaY8uH6cb6kZZKWtba27vR2mZlZpqQhEhG9EbEQaAYWSXoLcAnwZuAIYCZwUSn7kPpxbUS0RETLnDlzSv10ZmaTRlmuzoqINuA+YHFEvJyGrLqAbwCLUrO1wLyCxZpT2WjlzcOUm5lZmZTy6qw5kprSdB1wPPBMOpdBupLqDOCJtMhS4Nx0ldaRwIaIeBm4CzhB0gxJM4ATgLtS3UZJR6Z1nQvcXqrtMTOz7ZXy6qx9gRskVZKF1ZKIuEPSvZLmAAKWA3+R2t8JnAysBDqBDwFExHpJnwMeTu0+GxHr0/QFwDeBOrKrsnxllplZGSm7sGnyaGlpiWXLlk10N8zMdhuSHomIluHq/I11MzPLzSFiZma5OUTMzCw3h4iZmeXmEDEzs9wcImZmlptDxMzMcnOImJlZbg4RMzPLzSFiZma5OUTMzCw3h4iZmeXmEDEzs9wcImZmlptDxMzMcnOImJlZbg4RMzPLzSFiZma5OUTMzCw3h4iZmeXmEDEzs9wcImZmlptDxMzMcnOImJlZbg4RMzPLzSFiZma5OUTMzCw3h4iZmeXmEDEzs9wcImZmllvJQkRSraSHJP1G0pOSLk/lCyQ9KGmlpO9KqknlU9L8ylQ/v2Bdl6Ty30o6saB8cSpbKeniUm2LmZkNr5RHIl3AsRHxNmAhsFjSkcA/AldExIHA68CHU/sPA6+n8itSOyQdApwNHAosBq6WVCmpEvgycBJwCPC+1NbMzMqkZCESmfY0W50eARwL3JrKbwDOSNOnp3lS/XGSlMpviYiuiHgeWAksSo+VEbEqIrYCt6S2ZmZWJiU9J5KOGJYDrwL3AM8BbRHRk5qsAeam6bnAaoBUvwGYVVg+ZJmRyofrx/mSlkla1traOh6bZmZmlDhEIqI3IhYCzWRHDm8u5fON0o9rI6IlIlrmzJkzEV0wM9sjleXqrIhoA+4D/gBoklSVqpqBtWl6LTAPINVPB9YVlg9ZZqRyMzMrk1JenTVHUlOargOOB54mC5MzU7PzgNvT9NI0T6q/NyIilZ+drt5aABwEPAQ8DByUrvaqITv5vrRU22NmZtur2nGT3PYFbkhXUVUASyLiDklPAbdI+jzwa+Drqf3XgZskrQTWk4UCEfGkpCXAU0AP8LGI6AWQ9HHgLqASuD4inizh9piZ2RDKPuxPHi0tLbFs2bKJ7oaZ2W5D0iMR0TJcnb+xbmZmuTlEzMwst1KeE9mztD4L1bVQ05g9qmomukdmZhPOIVKsa98F3Z3b5iuqoaYBpkzN/tY0bAuYmgaY0jhM2Shtqxug0v8cZrZ78btWsc64GrraYWsHbN2U/R2Yb0+PDuhcB12pfmsH9Gwu/jmqarcFS01jQRA1QE1BAE1pHBxCg9oWlFfXQ4VHLM2sdBwixTr0PfmW6+2B7o6C0CkMnvR3uDDqb7tlI2x8aXDbvu4in1wFIVQQMCOG02hHVWm6qhakfPvCzPY4DpFSq6yCyulQO3381tmzdXDgbO0YfPSztWB6uOBqf7WgbaqPvuKeW5XDBNEIgVNZA6oY/KioLJhX+lu5fTtVZEdR25UPt/zQ9e7gMdBWI6y3YP3DrrdweQdq2fR/HWHgawmR/b8deAydT/+nR2szsI4ook1/3XDPM7SMHdQPs64R2xT8HXWbd9CmpmCOMrUAAAbzSURBVAHe9alx/2dxiOyOqmqgaibUzxyf9UVAz5ZhAqfwqGhI2aAjqA7YuGZw2+6O8enbLm+0INPYAhKRvfsw5I2yYHqgbrh2FNdu1HXszHONdR1FtrN8+v9P9f//atzbIWIlIkF1XfZgnG5Q2dcHfT0Fn4Z6h//01FdYvoPHQNsYYb2FbUdaT+/2nySH7UMMv+6+YZYfafvGum2F/x7ZRMG8tq8ruh3DtBvrOopsN2i+DM9VURDAwx1JFk5v12aYZWD0NgPrUBFtCo5Wh/vAMFqbEbdHRbTp397ycIhYaVRUQIUvgzbb0/nSHTMzy80hYmZmuTlEzMwsN4eImZnl5hAxM7PcHCJmZpabQ8TMzHJziJiZWW6T7udxJbUCL+ZcfDbw2jh2Z7y4X2Pjfo2N+zU2e2K/3hARw97OYtKFyM6QtGyk3xmeSO7X2LhfY+N+jc1k65eHs8zMLDeHiJmZ5eYQGZtrJ7oDI3C/xsb9Ghv3a2wmVb98TsTMzHLzkYiZmeXmEDEzs9wcIsOQtFjSbyWtlHTxMPVTJH031T8oaf4u0q8PSmqVtDw9/rwMfbpe0quSnhihXpKuSn1+TNJhpe5Tkf06RtKGgn31mTL1a56k+yQ9JelJSRcO06bs+6zIfpV9n0mqlfSQpN+kfl0+TJuyvx6L7FfZX48Fz10p6deS7himbnz3V0T4UfAAKoHngAOAGuA3wCFD2lwAXJOmzwa+u4v064PAv5V5f/034DDgiRHqTwZ+RPZ7pkcCD+4i/ToGuGMC/n/tCxyWpqcCzw7z71j2fVZkv8q+z9I+aEzT1cCDwJFD2kzE67GYfpX99Vjw3H8D3Dzcv9d47y8fiWxvEbAyIlZFxFbgFuD0IW1OB25I07cCx0kl/1HjYvpVdhHxM2D9KE1OB26MzANAk6R9d4F+TYiIeDkiHk3Tm4CngblDmpV9nxXZr7JL+6A9zVanx9Crgcr+eiyyXxNCUjNwCnDdCE3GdX85RLY3F1hdML+G7V9MA20iogfYAMzaBfoF8N40BHKrpHkl7lMxiu33RPiDNBzxI0mHlvvJ0zDC28k+xRaa0H02Sr9gAvZZGppZDrwK3BMRI+6vMr4ei+kXTMzr8UrgU0DfCPXjur8cInuWHwDzI+KtwD1s+7Rh23uU7H5AbwP+FfiPcj65pEbge8AnImJjOZ97NDvo14Tss4jojYiFQDOwSNJbyvG8O1JEv8r+epR0KvBqRDxS6ufq5xDZ3lqg8BNDcyobto2kKmA6sG6i+xUR6yKiK81eBxxe4j4Vo5j9WXYRsbF/OCIi7gSqJc0ux3NLqiZ7o/52RHx/mCYTss921K+J3GfpOduA+4DFQ6om4vW4w35N0OvxKOA0SS+QDXkfK+lbQ9qM6/5yiGzvYeAgSQsk1ZCdeFo6pM1S4Lw0fSZwb6SzVBPZryHj5qeRjWtPtKXAuemKoyOBDRHx8kR3StI+/ePAkhaRvRZK/saTnvPrwNMR8aURmpV9nxXTr4nYZ5LmSGpK03XA8cAzQ5qV/fVYTL8m4vUYEZdERHNEzCd7j7g3Is4Z0mxc91dV3gX3VBHRI+njwF1kV0RdHxFPSvossCwilpK92G6StJLs5O3Zu0i//lrSaUBP6tcHS90vSd8hu2pntqQ1wGVkJxmJiGuAO8muNloJdAIfKnWfiuzXmcBfSuoBNgNnl+GDAGSfFD8APJ7G0wH+F7B/Qd8mYp8V06+J2Gf7AjdIqiQLrSURccdEvx6L7FfZX48jKeX+8m1PzMwsNw9nmZlZbg4RMzPLzSFiZma5OUTMzCw3h4iZmeXmEDHbTSi7i+52d2U1m0gOETMzy80hYjbOJJ2TfmtiuaSvphv1tUu6Iv32xE8kzUltF0p6IN2k7zZJM1L5gZJ+nG52+KikN6bVN6ab+T0j6dtluHu02agcImbjSNLBwJ8AR6Wb8/UC7wcayL4xfCjwU7Jv0APcCFyUbtL3eEH5t4Evp5sd/iHQf9uTtwOfAA4h+22Zo0q+UWaj8G1PzMbXcWQ32ns4HSTUkd0qvA/4bmrzLeD7kqYDTRHx01R+A/DvkqYCcyPiNoCI2AKQ1vdQRKxJ88uB+cDPS79ZZsNziJiNLwE3RMQlgwqlS4e0y3u/oa6C6V78GrYJ5uEss/H1E+BMSXsBSJop6Q1kr7UzU5s/BX4eERuA1yUdnco/APw0/bLgGklnpHVMkVRf1q0wK5I/xZiNo4h4StKngbslVQDdwMeADrIfLvo02fDWn6RFzgOuSSGxim137P0A8NV099Vu4I/LuBlmRfNdfM3KQFJ7RDROdD/MxpuHs8zMLDcfiZiZWW4+EjEzs9wcImZmlptDxMzMcnOImJlZbg4RMzPL7f8DNRNqSIO6NRoAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "Fs0nwFpAL7YI",
        "outputId": "8b590906-bc47-4e2a-89c6-e3b2c133332f"
      },
      "source": [
        "# Plot changes in model accuracy during training\n",
        "plt.plot(history.history[\"factorized_top_k/top_100_categorical_accuracy\"])\n",
        "plt.plot(history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"])\n",
        "plt.title(\"Model accuracies during training\")\n",
        "plt.xlabel(\"epoch\")\n",
        "plt.ylabel(\"accuracy\")\n",
        "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9bn48c+TPSEkIQlrQkiAgCwqSEBAUVxYVC7W1qqorfbaS1u19v60i/aqbe1ybe21dtGqtXazgktrS1UUUNQqIATBhU2SsCRhCwkkhCQkmXl+f5wTMhkGmEAmM0me9+t1Xsyc8z0zz5ww55nvcr5HVBVjjDHGX1S4AzDGGBOZLEEYY4wJyBKEMcaYgCxBGGOMCcgShDHGmIAsQRhjjAnIEoRpNxHJFREVkZggyt4sIu92RlydTURqRWRoiN/jLRH58mnsv1hEburImDpSe45hZxxv05YliG5ORLaLSKOIZPqtX+ee5HPDE1nXp6rJqloS7jhORFUvU9U/dfTrish0ESk73ddpzzHsCse7u7EE0TNsA+a1PBGRM4Gk8IUTGYKpAXVV4gjr97s7H9+ewhJEz/AX4Is+z28C/uxbQERSReTPIlIhIjtE5N6WE4yIRIvIz0Vkv4iUAFcE2Pf3IrJbRMpF5EciEh1MYCLygojsEZFqEXlHRMb4bEsUkf9z46kWkXdFJNHddr6IrBCRgyJSKiI3u+vbNMn4N3G5tabbRGQrsNVd90v3NWpEZK2ITPMpHy0i3xWRYhE55G4f7PNaw93H8e4x2ikie0XkcZ9YM0XkZTfWKhH59/FO3iIyQ0Q2u5/3N4D4bPu+iDzj87xNU5/72X8sIu8BdcBQ3+PRcizcOA+IyDYRuczn9fLcv8EhEVkmIo/6vp9PuV7AYmCQ2+xTKyKD3PheFJFnRKQGuFlEJonISvez7xaR34hInN/fo+UY/tF9z1fcGN4XkWGnWHamiGxxj+NjIvK2nEZTXU9lCaJnWAWkiMgo98R9HeD/xf81kAoMBS7ESShfcrf9FzAHGA8UAFf77ftHoBkY7paZCQT7ZVwM5AP9gA+Av/ps+zkwAZgKpAPfBrwiMsTd79dAX2AcsD7I9wP4DHAuMNp9vsZ9jXTgWeAFEUlwt92JU/u6HEgB/hPn5OvvQWCE+zrDgSzgfnfbXUCZG2t/4LvAMXPciNMM+HfgXiATKAbOa8fnAvgCMB/oDewIsP1cYIv7+j8Dfi8iLUnoWWA1kAF8332tY6jqYeAyYJfb7JOsqrvczVcCLwJpOH9LD/D/3PebAlwC3HqC+K8DfgD0AYqAH7e3rHscXwTucT/LFpz/Q6a9VNWWbrwA24FLcU46/wvMBpYCMTgnqVwgGmgERvvs9xXgLffxm8BXfbbNdPeNwTnhHQESfbbPA5a7j28G3g0y1jT3dVNxfrzUA2cHKHcP8NJxXuMt4Ms+z9u8v/v6F58kjgMt74tzcrnyOOUUJxkIcBgY5rNtCrDNffwA8E9g+Ene94vAKp/ngpNYvuw+/z7wjM/23Ja/g89nf+B4x8M9FkU+25Lc/QcAOThJPsln+zO+7+f3utOBMr913wfeOcln/G/fv13LMXQf/xF4ymfb5cDm9pZ1j+NKv+NY6vv/wpbgFmsj7Dn+ArwD5OHXvITz6y6Wtr84d+D8CgYYhPMF893WYoi77+7WH6JE+ZUPyK3N/Bj4PM6va69PPPFAAs6vaH+Dj7M+WG1iE5FvArfgfE7FqSm0dOoH8159cU62a32OgeAkXoCHcE6eS9ztT6rqgwFep81xVlUVkZMeRz8nK7/H5/Xr3HiScT5vlar61o5KcT7/Kb+/iIwAHsapeSbh/KhYG0x8ODW15FMoG+g4nnaHek9kTUw9hKruwOmsvhynGcPXfqAJ52TfIgcodx/vpu2JIsfncSlODSJTVdPcJUVVx3By1+M0SVyKU2vIddeLG1MDMCzAfqXHWQ/OL3nfDvgBAcocbd5x+xu+DVwD9FHVNKCa1rb/E71Xi/04tZ0xPscgVVWTAVT1kKrepapDgbnAnSJySYDXaXOc3aYf3+Pers/WTruBdBHxff0TJYfjvY//+t8Cm4F8VU3BaV6TY/bqWLuB7JYn7nHMPn5xczyWIHqWW3CaVw77rlRVD/A88GMR6e228d9Jaz/F88AdIpItIn2Au3323Q0sAf5PRFJEJEpEhonIhUHE0xsnuVTinPh+4vO6XuBp4GG3AzRaRKaISDxO2/alInKNiMSISIaIjHN3XQ98VkSS3A7NW4KIoRmoAGJE5H6cGkSLp4Afiki+OM4SkQy/4+cFfgf8QkT6AYhIlojMch/PEZHh7omqGqdd3suxXgHGiMhn3Y7nO2ibBNYDF4hIjoik4jS1dQj3B0Qh8H0RiRORKcB/nGCXvUCGG8eJ9AZqgFoROQP4WocEfGKvAGeKyGfc43gbgZOpOQlLED2IqharauFxNn8d5xdqCfAuTofl0+623wGvAx/idCT710C+CMQBG3Ha718EBgYR0p9xmqvK3X1X+W3/JvAxTidyFfBTIEpVd+LUhO5y168Hznb3+QVOf8pe4E+07fQO5HXgNeBTN5YG2jaTPIyTIJfgnOh+DyQGeJ3v4HSUrnJH8CwDRrrb8t3ntcBK4DFVXe7/Aqq6H6e57UGcpJkPvOezfSnwHPARTjPNyyf5bO11A07fSSXwI/e9jgQqqKqbgQVAiTtCadBxXvObODXFQzj/j57r4JgDxdZyHH+G81lG4yS/gJ/FHJ+4nTjGGNOGiDyH0/H7vXDHcjrEGVJcBtwQKDGb47MahDEGABGZ6DYPRonIbJz+oX+EO65TISKzRCTNbZJs6ffwr6Gak7BRTMaYFgNwmg8zcH5xf01V14U3pFM2BaeZtKXp8zOqWh/ekLoea2IyxhgTkDUxGWOMCajbNDFlZmZqbm5uuMMwxpguZe3atftVtW+gbd0mQeTm5lJYeLwRnMYYYwIRkUBzdgHWxGSMMeY4LEEYY4wJyBKEMcaYgELaB+FebPNLnFktn/KfwVJEvoozT4oHZxqC+aq6UZzbYG7CmWoZnCmQvxrKWI0xPVNTUxNlZWU0NDSEO5SQSkhIIDs7m9jY2KD3CVmCcKdyfhSYgXPRzRoRWaSqG32KPauqj7vl5+LMezPb3VasquMwxpgQKisro3fv3uTm5uIzXXu3oqpUVlZSVlZGXl5e0PuFsolpEs7NSUpUtRFYiHPp/lGqWuPztBenPlWxMcackoaGBjIyMrptcgAQETIyMtpdSwplgsii7ayYZbTegOYoce4PXIwz8+IdPpvyRGSdey/Zaf77GWNMR+nOyaHFqXzGsHdSq+qjqjoMZ7rke93Vu4EcVR2Pc1+CZ0UkxX9fEZkvIoUiUlhRUdF5QRtjTJh5vMrhI83srz1CZW1oZjIPZYIop+0dqbJpvUNZIAtxbiaPqh5R1Ur38VqcWz6O8N9BVZ9U1QJVLejbN+CFgMYYE9EOHjzIY489dsIyzR4vtQ1NVBxqYGdlHVv2HGL6pTNZV1TOroP1HKhrCklsoRzFtAbIF5E8nMRwHc6NQ44SkXxV3eo+vQLY6q7vi3N/XI+IDMW5cUpJCGM1xpiwaEkQt956K6pKs1epb/RwqP4ITV6hoclDo6f1BoRx0VEkxEbz4j/+RWJsNImx0cREh6aJLGQJQlWbReR2nDt2RQNPq+oGEXkAKFTVRcDtInIpzv2QDwA3ubtfADwgIk04t2b8qqpWhSpWY4zpbKpKo8fLnd/6NsXFxYweexZRMTHExcWTkprGtuKtLFmxjm/ccgN7dpfTeKSBO+64g6991Rnx3zK9UFVtLZdddhnnn38+K1asICsri3/+858kJga68WH7dJvpvgsKCtTmYjLGtNemTZsYNWoUAD/41wY27qo5yR7tM3pQCvfPGc2RZi/1jR7qm5ylodGDR5Xy0p18/ebrePWd91n//nt8ad7nWL12PWeMGE50lFBVVUV6ejr19fVMnDiRt99+m4yMjKMJora2luHDh1NYWMi4ceO45pprmDt3LjfeeOMJP2sLEVmrqgWBYu82k/UZY0yk8Kri9Speheq6JjbsqsHr/hiPEiEhNpq0pFgS46KJrUsiITaKEf17syslgUmTJjHmjPyjr/WrX/2Kl156CYDS0lK2bt1KRkZGm/fLy8tj3DjnsrEJEyawffv2DvkcliCMMcb1vf8Y067yHq/S0ORpUzM40uRF3Uu6oqOcZJAYG01inPNvfExUmyGnNXFtT8O9evU6+vitt95i2bJlrFy5kqSkJKZPnx7wWob4+Pijj6Ojo6mv75ib51mCMMaYIDR7vEeTQH2jh4YmL0eaPUe3x0RFkRgXTUpCDImx0STERRMXHXXS6w969+7NoUOHAm6rrq6mT58+JCUlsXnzZlat6tzbaluCMMYYH6pKk8etGfjUDpr8RhIlxrnNRG7tIDb61K4ayMjI4LzzzmPs2LEkJibSv3//o9tmz57N448/zqhRoxg5ciSTJ08+7c/XHtZJbYzpsbxeZcPGjQwemt+mZtDsbU0G8TEtzUNRTs0gNpqYU0wG4Wad1MYYE0CTx0vRvlo27Krhk/JqNu6qYePuGn4xqy9aVYeIkBATRUpCDAluf0FCbDTRUd1/Go7jsQRhjOl2Gpo8bNpdw4ZdNWzYVc2GXTVs3nOIxmanZpAYG83oQSl89pws+iQp+f2SiY+NJqoHzMnUHpYgjDFdWk1DExt9agWf7KqmuOIwHq/TfJ6aGMuYQSncPDWXMYNSGDMolbzMXkdrBps2bSIxzk6FgdhRMcZ0GRWHjhytEbT8u6Oy7uj2/inxjBmUyqwxAxgzKJUxg1LI7pPYI2ZrDQVLEMaYiKOqlB+s55PyGja6ieCTXdXsrWmdtTQnPYmxWSlcUzD4aM2gb+/4E7yqaS9LEMaYsPN4lQ27qllRXMnK4ko+LDvIQXeG0iiB4f2SmTos82giGD0ohdTE4G+daU6NJQhjTKfzepUtew+xsriSFcWVvL+tkkMNzQDk90tm9pgBjMlKZeygFM4YkEJiXHSYIw6dgwcP8uyzz3Lrrbe2e99HHnmE+fPnk5SUFILILEEYYzqBqlKy/zAriitZVVzJypJKqg43AjAkI4k5Zw1k8tAMpgzLoF/vhDBH27l8p/tur0ceeYQbb7zREoQxpmsprapzawj7WVlSebT/YGBqAtNH9mXqsEymDMsgK+30p6Xuyu6++26Ki4sZN24cM2bMoF+/fjz//PMcOXKEq666ih/84AccPnyYa665hrKyMjweD/fddx979+5l165dXHTRRWRmZrJ8+fIOj80ShDGmQ+ytaTiaEFYUV1J2wJkwLjM5jslDM5g6LJOpwzIYkpEUuaOKFt8Nez7u2NcccCZc9uBxNz/44IN88sknrF+/niVLlvDiiy+yevVqVJW5c+fyzjvvUFFRwaBBg3jllVcAZ46m1NRUHn74YZYvX05mZmbHxuyyBGGMOSWVtUdYVVLFyhInIZRUHAac6w4mD03ny+fnMXV4Jvn9kiM3IUSYJUuWsGTJEsaPHw9AbW0tW7duZdq0adx111185zvfYc6cOUybNq1T4rEEYYwJSnV9E6u3VTlNRsWVbN7jzEDaKy6aSXnpzJuYw5RhGYwamNJ1p6c4wS/9zqCq3HPPPXzlK185ZtsHH3zAq6++yr333ssll1zC/fffH/J4LEEYYwKqa2xmzfYDRxPCJ+XVeBXiY6IoyO3Dt2aNZMqwDM7MSj3lmUxN2+m+Z82axX333ccNN9xAcnIy5eXlxMbG0tzcTHp6OjfeeCNpaWk89dRTbfa1JiZjTEg1NHn4YOcBVrlDT9eXHqTZq8RGC+MH9+H2i/OZOiyD8TlpxMd032Gnnc13uu/LLruM66+/nilTpgCQnJzMM888Q1FREd/61reIiooiNjaW3/72twDMnz+f2bNnM2jQoJB0Utt038b0UE0eLx+VHWRFkTPstHDHARqbvUQJnJmdxtRhGUwdlsGEIX1I6sZzFQWaAru7sum+jTEBebzKxl01R0cZrdleRV2jc0e00QNT+MLkIUwdlsHEvHRSEuwqZWMJwphuy+tVPt3nc7VySSU17tXKw/slc/WEbKYMzeDcoRmk94oLc7QmElmCMKabUFW2uVcrryxxrliudK9WzklP4vIzBzJlWAZThmbQL6VnXa18Mqra7Yfinkp3giUIY7qwsgN1Rye4W1lcyZ6aBgAGpCRw4Yi+TkIYlkF2n9BMxdAdJCQkUFlZSUZGRrdNEqpKZWUlCQnt+2FgCcKYLmRfTQMrSypZUVTJipL9lFY5Vytn9Io7mgymDsskN5KvVo4w2dnZlJWVUVFREe5QQiohIYHs7Ox27WMJwpgIVnW4kVUllUensCh2r1ZOSYhh8tAMbjkvjynDMhnR365WPlWxsbHk5eWFO4yIZAnCmAhS09DE6pKqo/0Im3bXAM7VyhPz0rl24mCmDsvs2lcrmy7DEoQxYVTX2Ezh9gNuP8J+Pva7WvmbM0cwZVgmZ2Xb1cqm81mCMKYTNTR5WLfzICtLnISwvvQgTR4lJkoYn5PG7RfnM2Woc7VyQqxdrWzCyxKEMSGkqmzYVcNbW/Y5VytvP8CRlquVs1K55fyhTB2WQUFu975a2XRN9j/SmBCorm/in+vLWbC69Gg/wqiBKdxwrnO18qShdrWyiXyWIIzpIKpK4Y4DLFi9k1c/3k1Dk5cxg1L44WfGcvnYAWQkx4c7RGPaxRKEMaep6nAjf1tbxsI1OymuOExyfAyfOyebeZNyGJuVGu7wjDllliCMOQVer7KiuJIFa3ayZMMemjzKOTlp/Ozqs5hz1kDrTzDdgv0vNqYd9tU08MLaMp5bU8rOqjrSkmK5cfIQ5k3KYUT/3uEOz5gOZQnCmJNo9nh5+9MKFqwuZfmWfXi8ypShGdw1cwSzxgyw4aim27IEYcxxlB2o4/k1pTxfWMaemgYyk+P5r2lDuXbiYPIye4U7PGNCzhKEMT4am728sWkvC9aU8u+tzuRtF+T35ftzR3PJqP52NbPpUSxBGAOUVNTy3JpS/vZBGftrGxmYmsAdF+dzzcTBZKUlhjs8Y8IipAlCRGYDvwSigadU9UG/7V8FbgM8QC0wX1U3utvuAW5xt92hqq+HMlbT8zQ0eXjtkz0sWL2T97dVER0lXHJGP+ZNyuGCEX1tMjzT44UsQYhINPAoMAMoA9aIyKKWBOB6VlUfd8vPBR4GZovIaOA6YAwwCFgmIiNU1ROqeE3PsWXPIRas3slL68qprm8iJz2Jb80ayecnZNud1ozxEcoaxCSgSFVLAERkIXAlcDRBqGqNT/leQMs98a4EFqrqEWCbiBS5r7cyhPGabuzwkWZe/mgXC9eUsm7nQeKio5g1dgDzJg5m8tAMoqy2YMwxQpkgsoBSn+dlwLn+hUTkNuBOIA642GffVX77ZgXYdz4wHyAnJ6dDgjbdh6rycXk1C1aX8q8Pd1F7pJnh/ZK594pRfPacbNJ7xYU7RGMiWtg7qVX1UeBREbkeuBe4qR37Pgk8CVBQUND+O3KbbqmmoYl/rnMmytu4u4aE2CiuOHMQ8yYNZsKQPnbnNWOCFMoEUQ4M9nme7a47noXAb09xX9PDtUyUt3B1Ka98vKvNRHlzzx5EaqLNnGpMe4UyQawB8kUkD+fkfh1wvW8BEclX1a3u0yuAlseLgGdF5GGcTup8YHUIYzVdVNXhRv7+QRkL15RStK+W5PgYPntONvMm5nBmtk2UZ8zpCFmCUNVmEbkdeB1nmOvTqrpBRB4AClV1EXC7iFwKNAEHcJuX3HLP43RoNwO32Qgm08LrVVaWVLJg9U6WbNhLo8fL+Jw0fva5s7jirIH0ig97y6kx3YKodo+m+4KCAi0sLAx3GCaE/CfKS02M5arxWcyblMPIATZRnjGnQkTWqmpBoG32U8tENI9XefvTfSxYXcqbm52J8iYPTbeJ8ozpBJYgTEQqO1DH84VlvFBYyu7qBjKT4/jytDyum5hjE+UZ00ksQZiI0eTxsmxj24nypuX35f45zkR5cTE2UZ4xnckShAm7bfsPs3DNTv62tnWivK9fnM/nJ2QzOD0p3OEZ02NZgjBh0dDk4fUNzkR5q0qcifIuPqMf8yYN5sIR/WyiPGMigCUI06n8J8obnJ7It2aN5OoJ2fS3ifKMiSiWIEzI1TU28/KHu1mwZifrdh4kNlqYNWYA103MYeowmyjPmEhlCcKEzMdl1SxYs5NF652J8ob17WUT5RnThViCMB2qZaK8hWtK2bCrhviYKK44ayDzJuVQYBPlGdOlWIIwp01VWbvjAAt8JsobNTCFB64cw5XjsmyiPGO6KEsQ5pS1TJT33JpStu6rpVdcNFeNz2bepMGcmZVqtQVjujhLEKbdPth5gD+8t53XP9lDo8fLuMFp/PRzZzLnrEE2UZ4x3Yh9m027FO2r5ZrHV5IUF8315+Zw3aTBnDEgJdxhGWNCwBKEaZefvLqJxNho3rhrOn17x4c7HGNMCNnkNiZob39awZub9/H1S4ZbcjCmB7AEYYLS7PHyo5c3MiQjiZum5oY7HGNMJ7AEYYKyYPVOtu6r5buXjyI+xu7BYExPYAnCnFR1XRMPL/2UKUMzmDm6f7jDMcZ0EksQ5qR+9eZWDtY3cd+c0XZtgzE9iCUIrxdeuwc2/QsaasIdTcQpqajlTyu2c93EwYweZMNZjelJbJhrdSmsewZWPQZRMZAzBfJnwPAZ0G8U9PBfzD95dRMJsdHcOWNkuEMxxnQySxB9hsC3S6D0fdi6FIqWwdL7nSUlG/IvdZLF0Ashvne4o+1U/95awbJN+7j7sjNsWKsxPZCoarhj6BAFBQVaWFjYMS9WXe4kiqKlUPwWNB6CqFjImQz5M50aRt8zunXtotnj5YpfvUt9k4eld15gI5eM6aZEZK2qFgTaZjWIQFKzYMJNztLc6NQuipbC1mWw9D5nSR0Mwy91kkXehRCfHO6oO9TCNaVs2XuIx288x5KDMT2U1SDaq7rMqV1sXQolb0FjrVO7GDLFqV0MnwF9R3bp2kV1fRMX/fwt8vsls3D+ZBu5ZEw3ZjWIjpSaDRNudpbmRihd1dp3seReZ0kd3NrRnXdBl6td/ObNrRyoa7Rhrcb0cJYgTkdMnJMA8i6AmT+Eg6Vu38Uy+Oh5KHwaouPckVFu30XmiIiuXWzff5g/rtjO5ydkMzYrNdzhGGPCKKgmJhH5O/B7YLGqekMe1SnotCamYDU3ws6VrX0XFZuc9ak5TqLId2sXcb3CG6ef+X8u5L2i/Sz/5nT6pSSEOxxjTIh1RBPTY8CXgF+JyAvAH1R1S0cF2C3FxDlDY4deCDN/5NYu3GTx4UIo/L1TuxgytbXvIjM/rLWLFUX7WbJxL9+aNdKSgzGmfZ3UIpIKzAP+BygFfgc8o6pNoQkveBFXgziR5iNO7aKl76Jis7M+Lac1WeRN69TahcerXPGrf3OooZk37rqQhFgbuWRMT9AhndQikgHcCHwBWAf8FTgfuAmYfvph9iAx8TB0urPM+jEc3NmaLNYvgDVPQXR8a+0ifwZkDA9p7eL5wlI27znEo9efY8nBGAME3wfxEjAS+AvwR1Xd7bOt8HjZpzN1qRrEiTQfgR0rWofS7ndb8tKGuH0XMyF3GsQlddhb1jQ0cdFDbzG0by+e/8oUG7lkTA/SETWIX6nq8kAbIiE5dCsx8TDsImeZ9WM4sKO172L9s621i9zznKao/JmQMey0ahePLi+iqq6RP86ZZMnBGHNUsAlitIisU9WDACLSB5inqo+FLjQDOHNFTfyyszQ1wM4VTrIoWgqv3+MsfXJbk0Xu+e2qXeyoPMwf3t3O587J5sxsG9ZqjGkVbBPTelUd57dunaqOD1lk7dRtmpja48D21r6Lbe9AU51buzi/tTkqY9gJX+Krf1nLO1srWP7N6fS3kUvG9Dgd0cQULSKibjYRkWggrqMCNKeoTy5M+i9naWqAHe+19l28drez9Mlrvarbr3axsriS1zbs4ZszR1hyMMYcI9gaxEPAEOAJd9VXgFJVvSuEsbVLj6xBnEjVttZkse0daK6HmAQnSQyfgWf4DP7jr7uorm+yYa3G9GAnqkEEmyCicJLCJe6qpcBTqurpsChPkyWIE2hqgB3vtvZdVBYBsM3bHx0+g6FTrnI6vWMTwxyoMaaznXaCOI03ng38EojGSSgP+m2/E/gy0AxUAP+pqjvcbR7gY7foTlWde6L3sgQRvNrdn/LY7x7nkpiPOMf7CXK0djHNbY669KR9F8aY7uG0+yBEJB/4X2A0cLSxWlWHnmCfaOBRYAZQBqwRkUWqutGn2DqgQFXrRORrwM+Aa91t9f4d46ZjPPqhl9/WXcKs2+5HBsTD9vfcobRLYfG3nULpw3z6Lqx2YUxPFGwn9R+A7wG/AC7CmZcp6iT7TAKKVLUEQEQWAlcCRxOE37UVq3Cu1DYhVFpVx+//vY3PnpPF2YPTnJX5lzrLZT+FyuLWvou1f4T3H4eYRGfqj+HuJIPpeWH9DMaYzhFsgkhU1TfckUw7gO+LyFrg/hPsk4UzX1OLMuDcE5S/BVjs8zxBRApxmp8eVNV/+O8gIvOB+QA5OTnBfZIe7n8XbyI6Svj2rDMCF8gY5iznfgWa6mH7u06y2LrEWRbjTPsx+FzoP8ZdxkKvzE79HMaY0As2QRxxO6q3isjtQDnQYXfBEZEbgQLgQp/VQ1S1XESGAm+KyMeqWuy7n6o+CTwJTh9ER8XTXb1fUsmrH+/hzhkjGJAaxLDW2MTWqcn5mVO72LoUit9w5436a2vZ5P5tE0b/Mc69L2LiQ/Z5jDGhFWyC+AaQBNwB/BCnmemmk+xTDgz2eZ7trmtDRC7FmR32QlU90rJeVcvdf0tE5C1gPFDsv78JjserPPDyRgalJvBf047bdXRiLbWLyV91ntdWwL4NsLdl+QTefxI87p8xKsZJEv6Jo/fAiL5pkjHGcdIE4XY2X6uq3wRqcfofgrEGyBeRPJzEcB1wvd9rj8e5tmK2qu7zWd8HqFPVIyKSCZyH04FtTtHfPihjw64afnndOBLjOhyuDxwAABe/SURBVOiah+S+kDzdmZW2hacZqoqdZNGSOHaugo9faC2T2Kc1WbQsfUd16ASExpjTd9IEoaoeETm/vS+sqs1uc9TrOMNcn1bVDSLyAFCoqouAh3Caql5wJ4lrGc46CnhCRLw4neEP+o1+Mu1Qe6SZh17fwvicNOaePSi0bxYdA31HOsvYz7Wurz8A+zY5CWPPx86/H/zZmR4EAHFqJ741jf5jnFlsrbZhTFgEe6Hcb3E6nV8ADresV9W/hy609rHrII7vodc38+jyYv5+61TOyekT7nBaeb1wYFvbJqq9G5x1LeJ6t61p9B8L/UZBQkr44jamG+mIuZgSgErgYp91CkRMgjCBlVbV8bt/b+Mz4wZFVnIAiIpq7dcY7XMd5JFat7bh00z18YvObVpbpA3xa6Ya6wy/jbIpQ4zpKEElCFUNtt/BRJifvraZKIFvzz7OsNZIFJ8Mgyc6SwtVqC5rW9PYuwE+XQzqdcrEJDq1C/9mqqT08HwOY7q4YK+k/gNOjaENVf3PDo/IdJjC7VW8/NFuvnFJPoPSuviV0CKQNthZRs5uXd9UDxVb2jZTbXkV1v2ltUzvQQGG4OZDdGznfw5jupBgm5he9nmcAFwF7Or4cExH8brDWgekJPCVC09xWGtXEJsIg8Y5SwtVqN3btqaxdwOUvAXeJqdMVCz0PePY/o3kftYpbowr2Camv/k+F5EFwLshich0iJfWlfNRWTW/uPZskuKC/R3QTYhA7wHOMvzS1vXNjVC5tW0z1ba34aOFrWWSMtvWNAaMhcyREGv3yzA9z6meOfKBfh0ZiOk4h48087PXN3P24DSuPDsr3OFEjpi41toC17SuP1zpc8GfmzgKfw/NDc52iXaapPybqVKyrLZhurVg+yAO0bYPYg/wnZBEZE7bE28Xs7fmCI/dMIGoKDuBnVSvDMi7wFlaeD1QVdK2map0DXziU5lOSD12JFW/URDXq/M/gzEhEGwTU+9QB2I6RvnBep54p4S5Zw9iwpAIG9balUS5tYbMfBhzVev6hmrYu7Ft4lj/LDTWugXEGW4b6II/G4JruphgaxBXAW+qarX7PA2YHmiGVRNeP128GYDvXNaFhrV2JQmpMGSKs7TweuHgjmMv+Nv0Mkcr3tFxTpJIH+okkPShzv3C0/Oc9TF2i3cTeYLtg/ieqr7U8kRVD4rI9wBLEBFk7Y4qFn24izsuHk5WVx/W2pVERbkn/TwYNad1feNh2LfZSRiVRc4V4lXbnCnUmw63lpMoSMmG9FyfxOEmkj55znUhxoRBsAki0M2BetjQmMjmDGvdRP+UeL5yod0uNCLE9YLsCc7iSxVq97UmjKqS1scbF0F9Vdvyvfq1Jgv/GkhSunWUm5AJ9iRfKCIP49xCFOA2YG1oQjKn4p8flvNh6UH+7/Nn0yvecndEE4He/Z0lZ/Kx2+sPtiaMA24CqdoO2//ddkguQHyKT/LIa1sD6T3Qqd0Yc4qCPZN8HbgPeA6nUXUpTpIwEaCusZmfLt7CWdmpXDXehrV2eYlpkDgeBo0/dltTPRzY4ZM43H/3fASbXwZvc2vZ6Hjok9u2uarlcVqOXUluTirYUUyHgbtDHIs5RU+8XcKemgZ+c/14G9ba3cUmQr8znMWfpxmqS/2arrY7/5a8Bc31rWUlGlKzj+0wTx/qJBUbqmsIfhTTUuDzqnrQfd4HWKiqs0IZnDm5XQfreeKdYuacNZCCXJuUrkeLjmntLPfvhmqZfqSl1uFbA9nwknO/Dl/JAwL0e+S19nuYHiHYJqbMluQAoKoHRMSupI4AP3ttM16Fu21YqzkR3+lHhkw9dnv9Ab8O8+1uzWM5fPhs27IJqQFqHe7j5AHW79GNBJsgvCKSo6o7AUQklwCzu5rO9cHOA/xj/S5uv2g42X3sdp3mNCT2gaw+kHXOsdsa65ymKv9RV7s+gI3/BPW0lo1J9Ov38HmcmuPUckyXEexf63+Ad0XkbUCAacD8kEVlTkpVeeBfG+nbO56vTbdhrSaE4pKg/2hn8edpcvo9jjZdbXcfl0DxG63zWQFExUDq4OP3e8TatTuRJthO6tdEpAAnKazDuUCu/sR7mVBa9OEu1pce5KGrz7JhrSZ8omPdGkKAKeW9Xqjdc+y1HlUlUL7WmbbEV+9Brf0cfYY4kyGmZjkXEaZmWQIJg2A7qb8MfAPIBtYDk4GVtL0Fqekk9Y0eHly8mbFZKXzunOxwh2NMYFFRkDLIWXLPO3Z7XZVPh7lPEila6nSo+0tMdxJF6uBjk0dKlvM+NnS3QwX70/MbwERglapeJCJnAD8JXVjmRJ58p4Td1Q08cu04G9Zquq6kdGfxv9IcoKkBDu2C6nKoKXeasVoeH9gBO947tgaC2xEfKHmkDnYe9+pnnejtEGyCaFDVBhFBROJVdbOIjAxpZCag3dX1PP52MZefOYBzh2aEOxxjQiM24fhNVy2OHIKaXW2TR3U51JQ5M+5+uqTttR/g3EkwZaBf8sh2lpbHiX1s+hJXsAmizJ3B9R/AUhE5AOwIXVjmeB56bQser3LPZaPCHYox4RXfG/qOdJZAVJ3hu9VlbvIoa00i1WVQ+j7U7G69DW2LmMRjax7+tZL4nnEHhGA7qVsmxP++iCwHUoHXQhaVCejD0oP8fV05X5s+jMHpNqzVmBMSaW3GGnhW4DJeLxze11rzqC7zeVzujMQ6tIdjRvXHp7o1jwDJI8VdusFtats9/EVV3w5FIObEVJUHXt5IZnI8t9qwVmM6RlRU6wWEBOgLAWco76HdxyaPllpJ+Vqoqzx2v159W5utjiYRn1pJ8oCIvy4ksqMzR7380W7W7jjATz93Jr0TbKSGMZ0mOtaZ3DAt5/hlGuuc/hD/5FFT7twLpORtaDzUdh+JcmbcDZQ8WhJLUmZYO9UtQXQBDU3OsNbRA1O4esLgcIdjjPEXlwSZw53leBqq2yYP376R3R/C5lfBc6TtPtFx7lDhbL8mLZ9aSUJayDrVLUF0AU/9u4Tyg/X8/PNnE23DWo3pmhJSnSXQFengdKrXVR6bPFo61ne859RSfKc2AYjtBUMvhHkLOjxkSxARbm9NA4+9VczsMQOYMsyGtRrTbYlAr0xnGTQucBmvx+k0908eSaE5N1iCiHAPvb6FZo9yz+U2W6sxPV5UtHs1eRYMnhT6twv5O5hT9nFZNS+uLeNL5+cyJMNu4GKM6VyWICKUM6x1A5nJcdx+0Qk6vowxJkQsQUSoVz/ew5rtB7hr5kgb1mqMCQtLEBGoocnDT17dxBkDenNNgQ1rNcaEhyWICPT7d7dRfrCe+/9jtA1rNcaEjSWICLOvpoHHlhcxc3R/pg7LDHc4xpgezBJEhPn5ki00erx893KbrdUYE16WICLIJ+XVvLC2jC+dl0dupg1rNcaElyWICNEyW2t6Uhy3X2zDWo0x4RfSBCEis0Vki4gUicjdAbbfKSIbReQjEXlDRIb4bLtJRLa6y02hjDMSvPbJHlZvq+LOmSNIsWGtxpgIELIEISLRwKPAZcBoYJ6I+M9StQ4oUNWzgBeBn7n7pgPfA84FJgHfE5E+oYo13BqaPPxk8SZG9u/NtTas1RgTIUJZg5gEFKlqiao2AguBK30LqOpyVa1zn64Cst3Hs4ClqlqlqgeApcDsEMYaVn94bzulVfXcO2cUMdHW6meMiQyhPBtlAaU+z8vcdcdzC7C4PfuKyHwRKRSRwoqKitMMNzz2HWrg0eVFXDqqH9Py+4Y7HGOMOSoifq6KyI1AAfBQe/ZT1SdVtUBVC/r27Zon14eXfEpDk8eGtRpjIk4oE0Q54Nugnu2ua0NELgX+B5irqkfas29Xt2FXNc8VlnLT1FyG9k0OdzjGGNNGKBPEGiBfRPJEJA64DljkW0BExgNP4CSHfT6bXgdmikgft3N6pruu21BVfvjyRtISY7nj4vxwh2OMMccI2Q2DVLVZRG7HObFHA0+r6gYReQAoVNVFOE1KycAL4txTdaeqzlXVKhH5IU6SAXhAVatCFWs4LNm4l1UlVfzwyjGkJtmwVmNM5BFVDXcMHaKgoEALCwvDHUZQjjR7mPmLd4iLjmLxN6bZyCVjTNiIyFpVLQi0zc5MYfCnFdvZUVnHfXNGW3IwxkQsOzt1sv21R/j1G0VcfEY/LhjRNUdeGWN6BksQnezhpZ9Sb8NajTFdgCWITrRpdw0LV+/kC1OGMLyfDWs1xkQ2SxCdRFX50SsbSUmM5RuX2LBWY0zkswTRSZZt2sd7RZX8v0tHkJYUF+5wjDHmpCxBdILGZi8/fmUjw/slc/25OeEOxxhjgmIJohP8eeV2tlfWce8Vo4i1Ya3GmC7CzlYhVll7hF++sZXpI/syfWS/cIdjjDFBswQRYr9Y9il1jR7uvcKGtRpjuhZLECG0Zc8hnn1/J1+YPITh/XqHOxxjjGkXSxAh0jJba+8EG9ZqjOmaLEGEyJub9/Fu0X7++9J8+vSyYa3GmK7HEkQIOMNaNzG0by9unDwk3OEYY8wpsQQRAn9ZtYOS/YdtWKsxpkuzs1cHqzrcyC+Xfcq0/EwusmGtxpguzBJEB3tk2afUHmnmvjmjce+SZ4wxXZIliA60de8h/vr+Tm44dwgj+tuwVmNM12YJogP96JVNJMVF8/9mjAh3KMYYc9osQXSQ5Vv28fanFXzjknzSbVirMaYbsATRAZo8Xn708kbyMnvxxSm54Q7HGGM6hCWIDvDXVTsorjjM/1w+irgYO6TGmO7Bzman6WBdI79YtpXzh2dyySgb1mqM6T4sQZymR5Zt5VBDE/fOGWXDWo0x3YoliNNQtO8Qf1m1g3mTcjhjQEq4wzHGmA5lCeI0/PiVTSTFRnOnDWs1xnRDliBO0Vtb9rF8SwV3XJJPRnJ8uMMxxpgOZwniFDR7vPzolU3kZiRx09TccIdjjDEhYQniFDy7eidF+2r5rg1rNcZ0Y3Z2a6fquiYeXvopU4dlMGN0/3CHY4wxIWMJop1++cZWauqbbLZWY0y3ZwmiHYoravnzyu1cOzGHUQNtWKsxpnuzBNEOP3llEwmx0dw104a1GmO6P0sQQXrn0wre2LyP2y8eTqYNazXG9ACWIILgDGvdSE56El86Lzfc4RhjTKewBBGEBWtK+XRvLd+9/AziY6LDHY4xxnQKSxAnUV3fxMNLtnBuXjqzxgwIdzjGGNNpLEGcxK/f2MpBG9ZqjOmBQpogRGS2iGwRkSIRuTvA9gtE5AMRaRaRq/22eURkvbssCmWcx1NSUcsfV2znmgmDGZuVGo4QjDEmbGJC9cIiEg08CswAyoA1IrJIVTf6FNsJ3Ax8M8BL1KvquFDFF4yfvLqZ+Jgo7pplw1qNMT1PKGsQk4AiVS1R1UZgIXClbwFV3a6qHwHeEMZxSt4r2s+yTXu57eLh9OudEO5wjDGm04UyQWQBpT7Py9x1wUoQkUIRWSUinwlUQETmu2UKKyoqTifWNjxe5YcvbyS7TyL/eV5eh72uMcZ0JZHcST1EVQuA64FHRGSYfwFVfVJVC1S1oG/fvh32xs+tKWXznkN89/JRJMTasFZjTM8UygRRDgz2eZ7trguKqpa7/5YAbwHjOzK446lpaOL/lmxhUm46l421Ya3GmJ4rlAliDZAvInkiEgdcBwQ1GklE+ohIvPs4EzgP2HjivTrGo28WUVXXaMNajTE9XsgShKo2A7cDrwObgOdVdYOIPCAicwFEZKKIlAGfB54QkQ3u7qOAQhH5EFgOPOg3+ikktu8/zNPvbePqc7I5M9uGtRpjeraQDXMFUNVXgVf91t3v83gNTtOT/34rgDNDGVsg/7t4E7HRUXxr1sjOfmtjjIk4kdxJ3alWFO/n9Q17ue2i4fRLsWGtxhhjCYKWYa2byEpL5JbzbVirMcaAJQgAXigsZdPuGu65/Awb1mqMMa4enyAONTTx8yVbKBjShyvOHBjucIwxJmKEtJO6K6hv9DBhSB9unT7chrUaY4yPHp8g+qUk8MQXCsIdhjHGRJwe38RkjDEmMEsQxhhjArIEYYwxJiBLEMYYYwKyBGGMMSYgSxDGGGMCsgRhjDEmIEsQxhhjAhJVDXcMHUJEKoAdp/ESmcD+DgqnI1lc7WNxtY/F1T7dMa4hqhrwns3dJkGcLhEpdO+BHVEsrvaxuNrH4mqfnhaXNTEZY4wJyBKEMcaYgCxBtHoy3AEch8XVPhZX+1hc7dOj4rI+CGOMMQFZDcIYY0xAliCMMcYE1KMShIjMFpEtIlIkIncH2B4vIs+5298XkdwIietmEakQkfXu8uVOiutpEdknIp8cZ7uIyK/cuD8SkXMiJK7pIlLtc7zu76S4BovIchHZKCIbROQbAcp0+jELMq5OP2YikiAiq0XkQzeuHwQo0+nfySDjCst30n3vaBFZJyIvB9jWscdLVXvEAkQDxcBQIA74EBjtV+ZW4HH38XXAcxES183Ab8JwzC4AzgE+Oc72y4HFgACTgfcjJK7pwMthOF4DgXPcx72BTwP8LTv9mAUZV6cfM/cYJLuPY4H3gcl+ZcLxnQwmrrB8J933vhN4NtDfq6OPV0+qQUwCilS1RFUbgYXAlX5lrgT+5D5+EbhEQn+j6mDiCgtVfQeoOkGRK4E/q2MVkCYiAyMgrrBQ1d2q+oH7+BCwCcjyK9bpxyzIuDqdewxq3aex7uI/aqbTv5NBxhUWIpINXAE8dZwiHXq8elKCyAJKfZ6XceyX5GgZVW0GqoGMCIgL4HNuk8SLIjI4xDEFK9jYw2GK20SwWETGdPabu1X78Ti/Pn2F9ZidIC4IwzFzm0vWA/uApap63OPVid/JYOKC8HwnHwG+DXiPs71Dj1dPShBd2b+AXFU9C1hK6y8EE9gHOPPLnA38GvhHZ765iCQDfwP+W1VrOvO9T+QkcYXlmKmqR1XHAdnAJBEZ2xnvezJBxNXp30kRmQPsU9W1oX6vFj0pQZQDvlk+210XsIyIxACpQGW441LVSlU94j59CpgQ4piCFcwx7XSqWtPSRKCqrwKxIpLZGe8tIrE4J+G/qurfAxQJyzE7WVzhPGbuex4ElgOz/TaF4zt50rjC9J08D5grIttxmqIvFpFn/Mp06PHqSQliDZAvInkiEofTgbPIr8wi4Cb38dXAm+r29oQzLr826rk4bciRYBHwRXdkzmSgWlV3hzsoERnQ0u4qIpNw/p+H/KTivufvgU2q+vBxinX6MQsmrnAcMxHpKyJp7uNEYAaw2a9Yp38ng4krHN9JVb1HVbNVNRfnPPGmqt7oV6xDj1fMqe7Y1ahqs4jcDryOM3LoaVXdICIPAIWqugjnS/QXESnC6QS9LkLiukNE5gLNblw3hzouABFZgDO6JVNEyoDv4XTYoaqPA6/ijMopAuqAL0VIXFcDXxORZqAeuK4TEj04v/C+AHzstl8DfBfI8YktHMcsmLjCccwGAn8SkWichPS8qr4c7u9kkHGF5TsZSCiPl021YYwxJqCe1MRkjDGmHSxBGGOMCcgShDHGmIAsQRhjjAnIEoQxxpiALEEYEwHEmU31mNk5jQknSxDGGGMCsgRhTDuIyI3uvQLWi8gT7qRutSLyC/feAW+ISF+37DgRWeVO6PaSiPRx1w8XkWXuxHgfiMgw9+WT3YnfNovIXzthJmFjTsgShDFBEpFRwLXAee5Ebh7gBqAXzpWsY4C3ca7sBvgz8B13QrePfdb/FXjUnRhvKtAy1cZ44L+B0Tj3Bzkv5B/KmBPoMVNtGNMBLsGZlG2N++M+EWc6aC/wnFvmGeDvIpIKpKnq2+76PwEviEhvIEtVXwJQ1QYA9/VWq2qZ+3w9kAu8G/qPZUxgliCMCZ4Af1LVe9qsFLnPr9ypzl9zxOexB/t+mjCzJiZjgvcGcLWI9AMQkXQRGYLzPbraLXM98K6qVgMHRGSau/4LwNvuHd3KROQz7mvEi0hSp34KY4Jkv1CMCZKqbhSRe4ElIhIFNAG3AYdxbipzL06T07XuLjcBj7sJoITWmVu/ADzhzsLZBHy+Ez+GMUGz2VyNOU0iUquqyeGOw5iOZk1MxhhjArIahDHGmICsBmGMMSYgSxDGGGMCsgRhjDEmIEsQxhhjArIEYYwxJqD/DwAkidXwyET+AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X25xwNezOn3v"
      },
      "source": [
        "As we can see, the model is `overfitted` on the training dataset. It works well for training data, simply because it memorized them. However, the performance on training data is not important. The model should be able to `generalize` to new unseen data.\n",
        "\n",
        "The overfitting phenomenon is especially strong when models have many parameters. It can be mediated by `model regularization` and use of user and movie features that help the model generalize better to unseen data.\n",
        "\n",
        "Moreover, The model is re-recommending some of users' already watched movies. These known-positive watches can crowd out test movies out of top K recommendations and reduce the model accuracy for test data. This can be tackled by excluding previously seen movies from test recommendations in a third stage. This approach is relatively common in the recommender systems literature, but we don't follow it in these tutorials. If not recommending past watches is important, we should expect appropriately specified models to learn this behaviour automatically from past user history and contextual information. Additionally, it is often appropriate to recommend the same item multiple times (say, an evergreen TV series or a regularly purchased item)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vu1I9SGxXIGy"
      },
      "source": [
        "### Making predictions\n",
        "To make recommendations, at first, we have to find the representation embedding vectors for all candidates in the corpus and `index` them all for future retrieval.\n",
        "\n",
        "Then, we can get a `query`, pass it through the query tower and find its representation embedding vector.\n",
        "\n",
        "Finally, we can calculate the affinity score between the query and all the candidates, sort them and find `k-nearest` candidates to the query. It is called a `brute-force` search and TFRS exposes a layer named `tfrs.layers.factorized_top_k.BruteForce` to do this."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_OqB4kwwS1mR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "e7823633-d280-4413-ac9c-c9ef12bd393c"
      },
      "source": [
        "brute_force_layer = tfrs.layers.factorized_top_k.BruteForce(\n",
        "    movielens_retrieval_model.query_model\n",
        ")\n",
        "\n",
        "brute_force_layer.index_from_dataset(\n",
        "  tf.data.Dataset.zip(\n",
        "      (\n",
        "          candidates_corpus_dataset.batch(100),\n",
        "          candidates_corpus_dataset.batch(100).map(\n",
        "              movielens_retrieval_model.candidate_model\n",
        "          )\n",
        "      )\n",
        "  )\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow_recommenders.layers.factorized_top_k.BruteForce at 0x7fd1975e6400>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ccdAoIjAS0bC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6f9933ca-9624-4752-833b-343bfb0b3f80"
      },
      "source": [
        "user_id = '42'\n",
        "afinity_scores, movie_ids = brute_force_layer(\n",
        "    tf.constant([user_id])\n",
        ")\n",
        "\n",
        "print(f\"Recommendations for user {user_id} using BruteForce: {movie_ids[0, :5]}\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Recommendations for user 42 using BruteForce: [b'468' b'1043' b'560' b'944' b'140']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVlilfUEYPMa"
      },
      "source": [
        "Performing a brute-force search in a large corpus can be too slow and impractical in a production environment. In practice we can speed this up by using an `Approximate Nearest Neighbor` search instead of the brute-force search. Approximate Nearest Neighbor (`ANN`) search can greatly outperform brute-force search speed while sacrificing little in terms of accuracy.\n",
        "\n",
        "We can use Google [ScaNN](https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html), Facebook [Faiss](https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/), Spotify [Annoy](https://github.com/spotify/annoy), or other approximate vector similarity search libraries to build an `approximate retrieval index`. However, TFRS uses `ScaNN` library and exposes the layer `tfrs.layers.factorized_top_k.ScaNN` letting us make the model really end-to-end.\n",
        "\n",
        "Note: to use `tfrs.layers.factorized_top_k.ScaNN` layer, `ScaNN` library should be installed using `pip install scann` before importing the `TFRS` library.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2_c77jsdiPp9",
        "outputId": "d0a4fd6f-8a27-44dd-84a8-32ebf38ef3ce"
      },
      "source": [
        "scann_layer = tfrs.layers.factorized_top_k.ScaNN(\n",
        "    movielens_retrieval_model.query_model\n",
        ")\n",
        "\n",
        "scann_layer.index_from_dataset(\n",
        "  tf.data.Dataset.zip(\n",
        "      (\n",
        "          candidates_corpus_dataset.batch(100),\n",
        "          candidates_corpus_dataset.batch(100).map(\n",
        "              movielens_retrieval_model.candidate_model\n",
        "          )\n",
        "      )\n",
        "  )\n",
        ")\n",
        "\n",
        "user_id = '42'\n",
        "afinity_scores, movie_ids = scann_layer(\n",
        "    tf.constant([user_id])\n",
        ")\n",
        "\n",
        "print(f\"Recommendations for user {user_id} using ScaNN: {movie_ids[0, :5]}\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Recommendations for user 42 using ScaNN: [b'1043' b'944' b'356' b'140' b'560']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iumhoS_lucaV"
      },
      "source": [
        "### Save the model to use in future\n",
        "To make it possible to serve the Keras model using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving), export it to a `SavedModel` format."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Fowdeb85j6OS",
        "outputId": "1b301bff-5db0-4460-f4c8-f7e9e443a172"
      },
      "source": [
        "import os\n",
        "import tempfile\n",
        "\n",
        "with tempfile.TemporaryDirectory() as tmp_dir:\n",
        "  retrieval_model_path = os.path.join(tmp_dir, \"retrieval_model\")\n",
        "\n",
        "\n",
        "scann_layer.save(\n",
        "    retrieval_model_path,\n",
        "    options=tf.saved_model.SaveOptions(namespace_whitelist=[\"Scann\"])\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:absl:Found untraced functions such as query_with_exclusions, embedding_layer_call_and_return_conditional_losses, embedding_layer_call_fn, embedding_layer_call_fn, embedding_layer_call_and_return_conditional_losses while saving (showing 5 of 6). These functions will not be directly callable after loading.\n",
            "WARNING:absl:Found untraced functions such as query_with_exclusions, embedding_layer_call_and_return_conditional_losses, embedding_layer_call_fn, embedding_layer_call_fn, embedding_layer_call_and_return_conditional_losses while saving (showing 5 of 6). These functions will not be directly callable after loading.\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmplyr89r93/retrieval_model/assets\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "INFO:tensorflow:Assets written to: /tmp/tmplyr89r93/retrieval_model/assets\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UNET0awMnJ-X"
      },
      "source": [
        "# Reload the saved model to confirm that it works correctly\n",
        "reloaded_model = tf.keras.models.load_model(retrieval_model_path)\n",
        "afinity_scores, movie_ids = reloaded_model(\n",
        "    tf.constant([user_id])\n",
        ")\n",
        "\n",
        "print(f\"Recommendations for user {user_id} using reloaded saved model: {movie_ids[0, :5]}\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tnRXsM-3xsGU"
      },
      "source": [
        "If the saved model works correctly, back it up from the temporary directory to your Google Drive, download it from there, copy to your server (`retrieval_model/1/saved_model.pb`) and serve it."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FZznewEYzYh1"
      },
      "source": [
        "### Serve the saved model using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)\n",
        "\n",
        "We can use TensorFlow Serving Docker image to run a Docker container serving the saved model. Note that `tensorflow/serving` image doesn't support `ScaNN` layer, so it's important to use `google/tf-serving-scann` custom image instead.\n",
        "\n",
        "TF Serving uses the port 8500 for gRPC and the port 8501 for REST APIs.\n",
        "\n",
        "```\n",
        "docker run -p 8501:8501 \\\n",
        "--mount type=bind,source=PATH_TO_RETRIEVAL_DIR/retrieval,target=/models/retrieval \\\n",
        "-e MODEL_NAME=retrieval -t google/tf-serving-scann &\n",
        "```\n",
        "\n",
        "Now we can send the `query` as an HTTP request to the server and get the recommended candidates.\n",
        "```\n",
        "curl --location --request POST 'http://SERVER_IP_ADDRESS:8501/v1/models/retrieval:predict' \\\n",
        "--header 'Content-Type: application/json' \\\n",
        "--data-raw '{\n",
        "    \"instances\":\n",
        "        [\n",
        "            \"42\"\n",
        "        ]\n",
        "}'\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3Ez1GmXvB8L-"
      },
      "source": [
        "## Build the Ranking (Scoring) task\n",
        "\n",
        "The `ranking model` takes the outputs of the retrieval model and fine-tunes them to select the best possible handful of recommendations.\n",
        "\n",
        " Its task is to narrow down the set of items the user may be interested in to a shortlist of likely candidates. So the traing set should be expressesing ***how much the users liked the movies they did watch***. This is a form of `explicit feedback` - given that a user watched a movie, we can tell roughly how much they liked by looking at the rating they have given.\n",
        "```\n",
        "[\n",
        "  (('user1', 'Star Wars'), 4.0),\n",
        "  (('user1', 'Harry Potter'), 5.0),\n",
        "  ...\n",
        "]\n",
        "```\n",
        " This time we are going to predict `user_rating` value as the objective. So like the other `regression` problems we can use `MSE (Mean Squared Error)` as loss function and `RMSE (Root Mean Squared Error)` as an accuracy metric. The state of the art (`SOTA`) RMSE value for `MovieLens/100k` is equal to `0.909`.\n",
        "\n",
        "`tfrs.tasks.Ranking` layer gets the predicted ratings and the `ground truth` as input, calculates the metrics and returns the loss value.\n",
        "\n",
        "Ranking models do not face the same efficiency constraints as retrieval models do, and so we have a little bit more freedom in our choice of architectures. A model composed of multiple stacked `Dense` layers is a relatively common architecture for ranking tasks.\n",
        "\n",
        "In most cases, a ranking model can be substantially improved by using more features rather than just user and candidate identifiers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TLwaxI00k1xX"
      },
      "source": [
        "class RankingModel(tfrs.models.Model):\n",
        "  \"\"\"MovieLens ranking model\"\"\"\n",
        "\n",
        "  def __init__(self, query_model, candidate_model):\n",
        "    super().__init__()\n",
        "\n",
        "    self.query_model: tf.keras.Model = query_model\n",
        "    self.candidate_model: tf.keras.Model = candidate_model\n",
        "    self.rating_model = tf.keras.Sequential(\n",
        "        [\n",
        "            tf.keras.layers.Dense(256, activation='relu'),\n",
        "            tf.keras.layers.Dense(64, activation='relu'),\n",
        "            tf.keras.layers.Dense(1)\n",
        "        ]\n",
        "    )\n",
        "    self.ranking_task_layer: tf.keras.layers.Layer = tfrs.tasks.Ranking(\n",
        "        loss=tf.keras.losses.MeanSquaredError(),\n",
        "        metrics=[\n",
        "            tf.keras.metrics.RootMeanSquaredError()\n",
        "        ]\n",
        "    )\n",
        "\n",
        "\n",
        "  def compute_loss(self, features, training=False) -> tf.Tensor:\n",
        "    query_embeddings = self.query_model(features['user_id'])\n",
        "    candidate_embeddings = self.candidate_model(features[\"movie_id\"])\n",
        "    rating_predictions = self.rating_model(\n",
        "        tf.concat(\n",
        "            [query_embeddings, candidate_embeddings],\n",
        "            axis=1\n",
        "        )\n",
        "        # We could use `tf.keras.layers.Concatenate(axis=1)([x, y])`\n",
        "    )\n",
        "\n",
        "    loss = self.ranking_task_layer(\n",
        "        predictions=rating_predictions,\n",
        "        labels=features[\"user_rating\"]\n",
        "    )\n",
        "    return loss"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "it4JTHYF1eh-"
      },
      "source": [
        "### Fit the ranking model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BijEncJG1iaW"
      },
      "source": [
        "movielens_ranking_model = RankingModel(query_model, candidate_model)\n",
        "\n",
        "optimizer_step_size = 0.1\n",
        "movielens_ranking_model.compile(\n",
        "    optimizer=tf.keras.optimizers.Adagrad(\n",
        "        learning_rate=optimizer_step_size\n",
        "    )\n",
        ")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JXZykvH6-DvF",
        "outputId": "8ed2414d-5cb9-4fbe-b4df-20189d1b909e"
      },
      "source": [
        "ranking_ratings_trainset = ratings_trainset.shuffle(100_000).batch(8192).cache()\n",
        "ranking_ratings_testset = ratings_testset.batch(4096).cache()\n",
        "\n",
        "history = movielens_ranking_model.fit(\n",
        "    ranking_ratings_trainset,\n",
        "    validation_data=ranking_ratings_testset,\n",
        "    validation_freq=1,\n",
        "    epochs=5\n",
        ")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/5\n",
            "10/10 [==============================] - 2s 144ms/step - root_mean_squared_error: 2.7292 - loss: 6.7427 - regularization_loss: 0.0000e+00 - total_loss: 6.7427 - val_root_mean_squared_error: 1.0694 - val_loss: 1.1080 - val_regularization_loss: 0.0000e+00 - val_total_loss: 1.1080\n",
            "Epoch 2/5\n",
            "10/10 [==============================] - 1s 77ms/step - root_mean_squared_error: 1.0666 - loss: 1.1324 - regularization_loss: 0.0000e+00 - total_loss: 1.1324 - val_root_mean_squared_error: 1.0556 - val_loss: 1.0822 - val_regularization_loss: 0.0000e+00 - val_total_loss: 1.0822\n",
            "Epoch 3/5\n",
            "10/10 [==============================] - 1s 61ms/step - root_mean_squared_error: 1.0394 - loss: 1.0736 - regularization_loss: 0.0000e+00 - total_loss: 1.0736 - val_root_mean_squared_error: 1.0142 - val_loss: 0.9957 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.9957\n",
            "Epoch 4/5\n",
            "10/10 [==============================] - 1s 62ms/step - root_mean_squared_error: 1.0131 - loss: 1.0225 - regularization_loss: 0.0000e+00 - total_loss: 1.0225 - val_root_mean_squared_error: 0.9991 - val_loss: 0.9650 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.9650\n",
            "Epoch 5/5\n",
            "10/10 [==============================] - 1s 61ms/step - root_mean_squared_error: 1.0068 - loss: 1.0112 - regularization_loss: 0.0000e+00 - total_loss: 1.0112 - val_root_mean_squared_error: 0.9986 - val_loss: 0.9639 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.9639\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 295
        },
        "id": "1-uNjmoY-gsA",
        "outputId": "611bee0c-cce8-4a2a-8f6e-77d828180abe"
      },
      "source": [
        "# Plot changes in model loss during training\n",
        "plt.plot(history.history[\"loss\"])\n",
        "plt.plot(history.history[\"val_loss\"])\n",
        "plt.title(\"Model losses during training\")\n",
        "plt.xlabel(\"epoch\")\n",
        "plt.ylabel(\"loss\")\n",
        "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEWCAYAAABxMXBSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3xUZfb48c9JISGQBEJCDU1BipQAIYBlBV0VG6AiTQRBBHSLfl1ddVfXdVd/q+5a1xUQRUARCIiKiogFxEILSO9FIHRBOoGU8/vj3ugQkjBJpqSc9+s1L2fufe6dMxcnZ57nPvdcUVWMMcYYb4UEOwBjjDFliyUOY4wxRWKJwxhjTJFY4jDGGFMkljiMMcYUiSUOY4wxRWKJwwSdiDQSERWRMC/a3iki35Z0P6WJiPwoIr8twfZrRKSrD0PyGRFpICLHRSTUl21NcFniMEXi/pE7IyLxeZb/4P7RbhScyCouVb1YVef5er+FJWlvqeoOVa2qqtm+bGuCyxKHKY5tQP/cFyLSGogKXjgVU2noWVnvoGKyxGGK421gkMfrwcBEzwYiEisiE0XkgIhsF5HHRCTEXRcqIv8RkZ9EZCtwQz7bvikie0Rkl4g8VZw/UCJSV0RmisghEdksInd7rEsRkTQROSoi+0TkBXd5pIi8IyIHReSwiCwRkVrni0tEmojI1yJyxP1cUwuJ6w73mBwUkb/mWTdeRJ7yeN1VRNI9Xv8oIg+LyErghIiEeQ51icjfRSTVPfbH3GGsZI/t27u9w2MiMk1Epnq+n0e7FsBooIs7fHTYI75RIjJLRE4A3UTkBnefR0Vkp4j83WM/Zw0fisg8EfmniHznxjAnt/dalLbu+kEex/FxKeGQn/GeJQ5THAuBGBFp4f7h7Ae8k6fNf4FY4ALgCpxEM8RddzdwI9AOSAZ659l2PJAFNHHbXAMMK0acU4B0oK77Hv9PRK50170MvKyqMcCFQKq7fLAbd32gBjASOOVFXP8E5gDVgUT3859DRFoCo4A73LhquO2Loj9Osq2mqln5rO+B89mrATOBV933rgS8736OOGAycHN+b6Cq63A++wJ3+Kiax+oBwNNANPAtcALn37eaG9c9ItKrkPgH4Py/UBOoBDxY1LbucXwNuB2og/NvVq+Q/RgfssRhiiu313E1sA7YlbvCI5k8qqrHVPVH4HmcP5YAfYCXVHWnqh4C/uWxbS3geuB+VT2hqvuBF939eU1E6gOXAg+raoaqLgfe4NeeUibQRETiVfW4qi70WF4DaKKq2aq6VFWPehFXJtAQqOu+X0HnBnoDH6vqfFU9DTwO5BTlswGvuMfuVAHrv1XVWe65greBtu7yzkCYu32mqs4AFhfxvQE+VNXvVDXH/azzVHWV+3olTkK6opDt31LVjW78qUBSMdr2Bj5S1W9V9QzwN8AK7wWIJQ5TXG/j/Bq8kzzDVEA8EA5s91i2nV9/EdYFduZZl6uhu+0ed6joMDAG5xdnUdQFDqnqsQJiuAu4CFjvDkfd6PG5PgOmiMhuEXlORMK9iOvPgACL3eGhoYXE9ctnV9UTwMEifrad51m/1+P5SSDSHf6pC+zSsyubnm9f531/EekkInPFGZY8gtNTic9/03zjq1qMtnmP40mKfhxNMVniMMWiqttxTpJfD8zIs/onfv0FnqsBv/ZK9uAMBXmuy7UTOA3Eq2o19xGjqhcXMcTdQJyIROcXg6puUtX+OH/4nwWmi0gV95f4k6raErgEZ0ht0PniUtW9qnq3qtYFRgCviUiTfOI667OLSBRODyfXCc6eaFA7n30U95f1HqCeiIjHsvoFNS7kffIufxdnSKy+qsbinBuRc7byrT14DPGJSGXOPo7GjyxxmJK4C7jS/dX8C3eIJBV4WkSiRaQh8AC/ngdJBf4oIokiUh14xGPbPTjnCp4XkRgRCRGRC0WksKGPc6jqTuB74F/uCe82brzvAIjIQBFJUNUc4LC7WY6IdBOR1u5w21GcBJhzvrhE5DYRyf1D9jPOH9f8hqCmAzeKyGXuOYd/cPb3cDlwvYjEiUht4P6ifO7zWABkA793T6r3BFIKab8PSHTjLEw0Tu8uQ0RScHqi/jYduElELnHj+zv+T1bGZYnDFJuqblHVtAJW/wHn1/NWnBOo7wLj3HVjcYaDVgDLOLfHMgjnROhanD/C03FOgBZVf6ARTu/jfeAJVf3CXdcdWCMix3FOlPdzx9Fru+93FOfczdc4w1fni6sjsMjd30zgPlXdmjcgVV0D/A7neOxx95Pu0eRtnOPyI06iKnB2VlG55wJuwUmgh4GBwMc4Pan8fAWsAfaKyE+F7Ppe4B8icgznXENqIW19wj2Of8CZBLAHOA7sp+DPYnxI7EZOxlRcIrIIGK2qbwU7lpIQkao4ybCpqm4LdjzlnfU4jKlAROQKEantDlUNBtoAs4MdV3GIyE0iEiUiVYD/AKtwemrGzyxxGFOxNMMZCjsM/Ano7Z6/KYt64gxD7gaa4gw32hBKANhQlTHGmCKxHocxxpgiCXqRtECIj4/XRo0aBTsMY4wpU5YuXfqTqibkXV4hEkejRo1ISyto1qgxxpj8iMj2/JbbUJUxxpgiscRhjDGmSCxxGGOMKZIKcY7DGGOKKjMzk/T0dDIyMoIdit9FRkaSmJhIeHi4V+0tcRhjTD7S09OJjo6mUaNGnF1QuHxRVQ4ePEh6ejqNGzf2ahsbqjLGmHxkZGRQo0aNcp00AESEGjVqFKlnZYnDGGMKUN6TRq6ifk5LHIVYtuNnxny9BSvLYowxv7LEUYj3l+3iX5+u54mZa8jOseRhjAmsw4cP89prrxV5u+uvv57Dhw+fv2ExWeIoxJM9LubuyxszccF2fv/uMjIys4MdkjGmAikocWRlZRW63axZs6hWrZq/wrJZVYUJCRH+ekNLasVE8tQn6zh4fDFjByUTG+XdlDVjjCmJRx55hC1btpCUlER4eDiRkZFUr16d9evXs3HjRnr16sXOnTvJyMjgvvvuY/jw4cCvZZaOHz/Oddddx2WXXcb3339PvXr1+PDDD6lcuXKJ4rLE4YVhl19AzZhIHkxdQe/R3zNhaAp1q5XswBtjyo4nP1rD2t1HfbrPlnVjeOKmiwtt88wzz7B69WqWL1/OvHnzuOGGG1i9evUv02bHjRtHXFwcp06domPHjtx6663UqFHjrH1s2rSJyZMnM3bsWPr06cN7773HwIEDSxS734aqRGSciOwXkdUFrG8uIgtE5LSIPJhn3Y8iskpElotImsfyOBH5XEQ2uf+t7q/48+rRti7jh3Zk75EMbnnte9bv9e3/RMYYcz4pKSlnXWvxyiuv0LZtWzp37szOnTvZtGnTOds0btyYpKQkADp06MCPP/5Y4jj82eMYD7wKTCxg/SHgj0CvAtZ3U9Wf8ix7BPhSVZ8RkUfc1w/7IFavXHJhPKkjuzB43GJuG72A1+9IpsuFNc6/oTGmTDtfzyBQqlSp8svzefPm8cUXX7BgwQKioqLo2rVrvtdiRERE/PI8NDSUU6dOlTgOv/U4VHU+TnIoaP1+VV0CZBZhtz2BCe7zCRScdPymRZ0YZtx7CbViIhk8bjGfrCyrd900xpR20dHRHDt2LN91R44coXr16kRFRbF+/XoWLlwYsLhK66wqBeaIyFIRGe6xvJbH/ZH3ArUK2oGIDBeRNBFJO3DggE+DS6wexfSRXWiTGMvvJy/jre+2+XT/xhgDUKNGDS699FJatWrFQw89dNa67t27k5WVRYsWLXjkkUfo3LlzwOLy6z3HRaQR8LGqtiqkzd+B46r6H49l9VR1l4jUBD4H/qCq80XksKpW82j3s6qe9zxHcnKy+uNGThmZ2dw35Qc+W7OPEb+5gIe7NyckpGJcaWpMebdu3TpatGgR7DACJr/PKyJLVTU5b9tS2eNQ1V3uf/cD7wMp7qp9IlIHwP3v/uBE6IgMD+W12ztwR+eGjJm/lQdSl3MmKyeYIRljjN+VusQhIlVEJDr3OXANkDszayYw2H0+GPgw8BGeLTRE+EfPi3no2mZ8sHw3Q8cv4VhGUU7bGGNM2eK3WVUiMhnoCsSLSDrwBBAOoKqjRaQ2kAbEADkicj/QEogH3neLboUB76rqbHe3zwCpInIXsB3o46/4i0JE+F23JtSMjuCRGavoO2Yh44d0pGZMZLBDM8YYn/Nb4lDV/udZvxdIzGfVUaBtAdscBK4qeXT+cVtyfRKiI7h30jJuGeVcKHhhQtVgh2WMMT5V6oaqyrquzWoy+e7OnDqTza2jvmfp9p+DHZIxxviUJQ4/aFu/GjPuvYTYyuHc/sZCPl+7L9ghGWOMz1ji8JOGNarw3j2XcFGtaEa8nca7i3YEOyRjTBlT3LLqAC+99BInT570cUQOSxx+FF81gsl3d+Y3FyXwl/dX8cLnG+2mUMYYr5XWxGHVcf2sSkQYYwcl85cZq3jly03sO5LB0ze3IizUcrYxpnCeZdWvvvpqatasSWpqKqdPn+bmm2/mySef5MSJE/Tp04f09HSys7N5/PHH2bdvH7t376Zbt27Ex8czd+5cn8ZliSMAwkNDeK53G2rHRvLfrzZz4PhpXh3QjqhKdviNKRM+fQT2rvLtPmu3huueKbSJZ1n1OXPmMH36dBYvXoyq0qNHD+bPn8+BAweoW7cun3zyCeDUsIqNjeWFF15g7ty5xMfH+zZubKgqYESEP13TjKd6tWLehv30H7uIg8dPBzssY0wZMWfOHObMmUO7du1o374969evZ9OmTbRu3ZrPP/+chx9+mG+++YbY2Fi/x2I/eQNsYOeGJERH8MfJP9B79AImDEmhQY2oYIdljCnMeXoGgaCqPProo4wYMeKcdcuWLWPWrFk89thjXHXVVfztb3/zayzW4wiCay+uzaRhnTh04gy3jPqe1buOBDskY0wp5FlW/dprr2XcuHEcP34cgF27drF//352795NVFQUAwcO5KGHHmLZsmXnbOtrljiCJLlRHO/d04WIsBD6jlnA/I2+Lf1ujCn7PMuqf/755wwYMIAuXbrQunVrevfuzbFjx1i1ahUpKSkkJSXx5JNP8thjjwEwfPhwunfvTrdu3Xwel1/LqpcW/iqr7gv7jmYweNxiNu8/znO923BL+/yqsBhjAs3KqpexsuoVSa2YSFJHdqFjozgeSF3BqHlb7FoPY0ypZomjFIiJDGf80I70aFuXZ2ev5+8z15CdY8nDGFM62ayqUiIiLJSX+iZRKyaCsd9sY/+x07zYN4nI8NBgh2ZMhaWquLd4KNeKOsphPY5SJCRE+OsNLXnshhZ8unovg95czJGTdlMoY4IhMjKSgwcPlvuhY1Xl4MGDREZ6f/8g63GUQsMuv4CaMZE8mLqC3qOd+3rUrVY52GEZU6EkJiaSnp7OgQPlf8ZjZGQkiYneT8zx5x0AxwE3AvtVtVU+65sDbwHtgb+q6n/c5fWBiUAtQIHXVfVld93fgbuB3H/Jv6jqLH99hmDq0bYu8VUrMWLiUm55zUkezWpHBzssYyqM8PBwGjduHOwwSiV/DlWNB7oXsv4Q8EfgP3mWZwF/UtWWQGfgdyLS0mP9i6qa5D7KZdLIdcmF8aSO7EKOKr1Hf8/CrQeDHZIxxvgvcajqfJzkUND6/aq6BMjMs3yPqi5znx8D1gH1/BVnadeiTgwz7r2EWjGRDHpzMZ+s3BPskIwxFVypPjkuIo2AdsAij8W/F5GVIjJORKoXsu1wEUkTkbSyPkaZWD2K6SO70CYxlt9PXsZb320LdkjGmAqs1CYOEakKvAfcr6pH3cWjgAuBJGAP8HxB26vq66qarKrJCQkJfo/X36pFVeKdYZ24ukUtnvxoLf/6dB05dq2HMSYISmXiEJFwnKQxSVVn5C5X1X2qmq2qOcBYICVYMQZDZHgoowZ2YGDnBoz5eisPpC7nTFZOsMMyxlQwpW46rjhX27wJrFPVF/Ksq6OquYP8NwOrAx1fsIWGCP/s2Yo6sZX592cb+On4GUYNbE90ZHiwQzPGVBB+K3IoIpOBrkA8sA94AggHUNXRIlIbSANigBzgONASaAN8A6xyl4M77VZE3sYZplLgR2CERyIpUGkuclgS09J28siMVTSrFc34oR2pGe39BTzGGHM+BRU5tOq4Zdy8Dfu5d9Iy4qpUYsLQFC5MqBrskIwx5YRVxy2nujaryeS7O3PqTDa9R33Psh0/BzskY0w5Z4mjHGhbvxoz7r2EmMrhDBi7kC/W7gt2SMaYcswSRznRsEYV3rvnEi6qFc3wt9OYvHhHsEMyxpRTljjKkfiqEUy+uzO/uSiBR2es4sXPN5b7yp7GmMCzxFHOVIkIY+ygZG7rkMjLX27i0RmryMq2az2MMb5T6q7jMCUXHhrCc73bUDs2kv9+tZkDx07z3wHtiKpk/9zGmJKzHkc5JSL86ZpmPNWrFXM37GfA2EUcPH462GEZY8oBSxzl3MDODRk1sAPr9hyl9+gF7Dx0MtghGWPKOEscFcC1F9dm0rBOHDpxhptf+57Vu44EOyRjTBlmiaOCSG4Ux3v3dCEiLIS+YxYwf2PZLjVvjAkeSxyFyc6EcjSdtUnNaGbcewn146IYOn4JM5alBzskY0wZZImjMF8/B691hq//DYe2Bjsan6gVE0nqyC50bBTHA6krGDVvi13rYYwpEkschanZHCpXh7lPwSvtYOyVsOA1OLY32JGVSExkOOOHdqRH27o8O3s9T360lmy7KZQxxktWHdcbh3fCmhmwajrsXQkINLoMWveGFj0gKs5nsQZSTo7yr0/XMfabbVzfujYv9EkiMjw02GEZY0oJK6vuq7LqBzbC6ulOEjm0BULCoclvnSTS7DqoVMU37xNAb3yzlac+WUdK4zjG3pFMbJTdFMoYY4nD9/fjUIU9y50EsnoGHNsN4VFO8mjV20kmYZV8+55+NHPFbh5MXUGj+CjGD0mhbrXKwQ7JGBNkljj8eSOnnBzY8b2TRNZ+CKcOQWQ1aNnDSSKNLoOQ0j8E9P2WnxgxcSlVIsKYMDSFZrWjgx2SMSaIgnIjJxEZJyL7RSTfe4OLSHMRWSAip0XkwTzruovIBhHZLCKPeCxvLCKL3OVTRST4P+tDQpzkcNNL8OBGGDANLrrW6YlM7AEvtITZj0L60lI9vfeSC+NJHdmFHFV6j/6ehVsPBjskY0wp5Nceh4j8Bude4hNVtVU+62sCDYFewM+q+h93eSiwEbgaSAeWAP1Vda2IpAIzVHWKiIwGVqjqqMLiCNqtY8+chI2zYfV7sGkOZJ+B6o2h1a3OOZGaLQIfkxfSfz7JnW8tYcfBk7zUL4nrW9cJdkjGmCAISo9DVecDhwpZv19VlwCZeValAJtVdauqngGmAD1FRIArgeluuwk4Sad0qhQFrW6BfpPgwU3Q839QvRF8+4Jzfchrl8A3L8DP24Md6VkSq0cxfWQX2iTG8rt3lzH+u23BDskYU4qU1us46gE7PV6nu8tqAIdVNSvP8nOIyHARSRORtAMHSkF5jcrVoN1AGPQB/GkDXPecMwPryyfh5TbwxtWwaAwc3x/sSAGoFlWJd4Z14uoWtfj7R2v516fryLFrPYwxlN7EUWKq+rqqJqtqckJCQrDDOVvVmtBpBAz7HO5bCVc9AWdOwKd/huebwcRe8MM7cOpwUMOMDA9l1MAODOzcgDFfb+WB1OWcybKbQhlT0ZXWO/vsAup7vE50lx0EqolImNvryF1edlVvCJc/4Dz2r3On906HD38HH/8fNL3GOSdyUXdn6CvAQkOEf/ZsRZ3Yyvz7sw0cPHGGUQM7UDWitP6vY4zxt9La41gCNHVnUFUC+gEz1TmTPxfo7bYbDHwYpBh9r2YLuOpx+ONyGPYVdBwG6Utg+hD4T1OYMRw2znGKLwaQiPC7bk34d+82fL/lIH3HLGD/sYyAxmCMKT38PatqMtAViAf2AU8A4QCqOlpEagNpQAyQgzMDq6WqHhWR64GXgFBgnKo+7e7zApyT5XHAD8BAVS301nZBm1XlCznZ8OO3Ti9k7UzIOAyV46BlT2h9GzTo4kwHDpB5G/Zz76RlxFWpxMShKVyQUDVg722MCSy7ALCsJg5PWWdgy5fOcNaGWZB5EmLqwcU3O9N76ySBiN/DWLHzMEPHLyFHlTfv7Ej7BtX9/p7GmMCzxFEeEoenMydgw6dOEtn8BeRkQo0mzpXqrXtDfFO/vv32gycYNG4x+45m8Gr/9vy2ZS2/vp8xJvAscZS3xOHp5CFYN9NJIj9+CyjUbuMkkFa3QmyiX972p+OnGTp+Cat3HeH/3dyafikN/PI+xpjgsMRRnhOHp6N7YM37zjmRXUudZQ0ugda3QsteUCXep2934nQWv3t3GfM2HOD+3zblvquaIgEYLjPG+J8ljoqSODwd3OLUy1o9HQ6sBwmFC7s5w1nNb4DIGJ+8TWZ2Dn+ZsYppS9Ppn1Kfp3q1JjTEkocxZZ0ljoqYOHKpwr417n1E3oMjOyAs0inE2Kq3c61IeGQJ30J5fs5GXp27meta1ealfklEhJX+isDGmIJZ4qjIicOTqnNtyKppzpDWiQMQEQPNb3SGsxp3hdDiX9z35rfb+OfHa7m8aTxj7uhAVCW7UNCYssoShyWOc2VnwY/znV7Iuo/g9BGokuCcC2ndGxJTinWNSGraTh55byXtGlRn3J0dia1sdxQ0piyyxGGJo3CZGbD5c2dm1sbZkJUBsfV/LQFfq1WRrhGZvXoPf5j8A01qRjNxaAoJ0RF+DN4Y4w+WOCxxeO/0MVg/yzknsuUryMmC+Ga/Tu+tcaFXu5m/8QAj3l5K7dhI3r4rhcTqga+1ZYwpPkscljiK58RBWPuBczOq7d85y+q2d5LIxbdATOE3eVq6/RBD3lpClYgw3r6rE01qWokSY8oKSxyWOEruyC5YM8MZztqzHBDnlrmtbnVqZ0XF5bvZ2t1HGTRuETkKE4em0KpebGDjNsYUiyUOSxy+9dNmd3rvdDi4CULCoclV0H4wNL/+nObbfjrBwDcWcfRUJm/e2ZGUxvknGWNM6RGUW8eaciy+CXR9BH6/BEbMh873wN7VMKU/LH/3nOaN46sw/Z4u1IyJ4I43FzF3Q+m406ExpugscZiSEYE6beGaf8J9y6HxFTDzj7Dtm3Oa1omtTOqILjStVZW7J6Tx0YrdQQjYGFNSljiM74SGQ5+JEHcBTB0IP206p0mNqhG8e3dn2jeozh+n/MDkxTuCEKgxpiQscRjfqlwNbk+FkDCYdJszKyuPmMhwJgxNoetFCTw6YxWjv94ShECNMcVlicP4XvVG0H8yHN0NU2+HrHNv0Fi5Uihj7kjmprZ1eebT9Tw3ez0VYaKGMeWB3xKHiIwTkf0isrqA9SIir4jIZhFZKSLt3eXdRGS5xyNDRHq568aLyDaPdUn+it+UUP0UuHk07FgAH/7OqZGVR6WwEF7qm8SATg14bd4WHv9wNTk5ljyMKe38WYFuPPAqMLGA9dcBTd1HJ2AU0ElV5wJJACISB2wG5nhs95CqTvdTzMaXWt0Ch7bCV/90znt0+8s5TUJDhKd7tSImMpzRX2/hWEYW/7mtLeGh1hk2prTyW+JQ1fki0qiQJj2BieqMTywUkWoiUkdV93i06Q18qqon/RWn8bPL/+Qkj6+fdZJH237nNBERHrmuOTGVw3hu9gaOZ2Txv9vbExluZdmNKY2C+bOuHrDT43W6u8xTP2BynmVPu0NbL4pIgZXzRGS4iKSJSNqBAwd8E7EpOhG48SVodDl8+Hv48bsCm97btQlP9WrFVxv2M3jcYo5lZAYwUGOMt0rteICI1AFaA595LH4UaA50BOKAhwvaXlVfV9VkVU1OSEjwa6zmPMIqQd+3Ia6xc7L8YMGzqAZ2bshLfZNYuv1nbn9jEYdOnAlgoMYYbwQzcewC6nu8TnSX5eoDvK+qv/zsVNU96jgNvAWkBCRSU3KVq8OAVJAQmNQbTh4qsGnPpHq8PqgDG/Yeo8+YBew9khHAQI0x5xPMxDETGOTOruoMHMlzfqM/eYap3F4IIiJALyDfGVumlIprDP0mO8USp+Q/TTfXlc1rMWFoCnuPZNB79Pf8+NOJAAZqjCmMP6fjTgYWAM1EJF1E7hKRkSIy0m0yC9iKM2tqLHCvx7aNcHojX+fZ7SQRWQWsAuKBp/wVv/GTBp2g12uw43uY+Yd8p+nm6nxBDSbf3ZkTp7O4bcwC1u89GsBAjTEFseq4Jji+/jfMfQq6PuoUSyzEpn3HuOPNxZzKzOatIR1p36B6gII0pmKz6rimdPnNg9B2AMz7F6xMLbRp01rRTBvZhWpR4Qx8YxHfbvopQEEaY/JjicMEhwjc9DI0vMy5snz794U2rx8XxbSRXWgQF8XQ8UuYvXpvgAI1xuRlicMET+403WoNnJPlhUzTBagZHcmU4Z25uF4M905ayvSl6QEK1BjjyavEISL3iUiMOwPqTRFZJiLX+Ds4UwFExcHt05zn7/YpdJouQLWoSrxzVycuuTCeB6et4K3vtgUgSGOMJ297HENV9ShwDVAduAN4xm9RmYol7gLo9y4c3gFT74Cswi/6qxIRxpt3JtP94to8+dFaXv5ik1XWNSaAvE0c4v73euBtVV3jscyYkmvYBXq+Btu/hY/+WOg0XYCIsFBeHdCO3h0SefGLjfzz43VWWdeYAPG2yOFSEZkDNAYeFZFoIMd/YZkKqc1tTkHEef8P4i6EKx4qtHlYaAjP3dqG6Mgwxn23jaMZmTxzS2vCrLKuMX7lbeK4C6fU+VZVPemWOx/iv7BMhXXFn53kMfcp50rz1r0LbR4SIvztxpbEVg7npS82cTwji5f7JxERZpV1jfEXb3+adQE2qOphERkIPAYc8V9YpsISgR6vQMNL4YN7YMdCLzYR7v/tRfztxpbMXrOXYRPSOHkmKwDBGlMxeZs4RgEnRaQt8CdgCwXfoMmYkgmLgL7vQGx9mDLA6YF4Yehljfl37zZ8t/knBr6xiCMnrSy7Mf7gbeLIcm+41BN4VVX/B0T7LyxT4eVO09UcmNQHTv3s1Wa3Jdfntdvbs3rXUfq+voD9x6yyrjG+5qLtu4EAAB1cSURBVG3iOCYij+JMw/1EREKAcP+FZQxQ40J3mu52r6bp5ureqg7j7uzI9oMn6TN6Aek/2w0kjfElbxNHX+A0zvUce3HunfFvv0VlTK6Gl0CPV+HHb+Dj+887TTfXZU3jeWdYJw6dOEPvUQvYvP+YnwM1puLwKnG4yWISECsiNwIZqmrnOExgtO0LVzwCyyfBN897vVmHhtWZOqILWTlKnzELWZVu8zmM8QVvS470ARYDt+HcmW+RiBQ+T9IYX+r6CLTuA1/9E1a/5/VmLerEMH1kFyqHh9J/7EIWbT3oxyCNqRi8Har6K9BRVQer6iCcW7Y+7r+wjMlDBHq+Cg26wPv3wI5FXm/aKL4K0+/pQq2YCAaNW8zc9fv9GKgx5Z+3iSNEVT2/bQe92VZExonIfhHJ9xavbtHEV0Rks4isFJH2HuuyRWS5+5jpsbyxiCxyt5kqIpW8/AymrAuLgL6TIKYuTOkPh7wvcFgntjKpI7pwUa1o7p6YxswVu/0YqDHlm7eJY7aIfCYid4rIncAnOLd+PZ/xQPdC1l8HNHUfw3GuF8l1SlWT3EcPj+XPAi+qahPgZ5yr2k1FUaUG3D4dcrKdarpeTtMFqFE1gnfv7kT7htW5b8oPTFq03Y+BGlN+eXty/CHgdaCN+3hdVR/2Yrv5QGF1snsCE9WxEKgmInUKaiwiAlwJTHcXTQB6efMZTDkS38SZpntoG6QO8nqaLkB0ZDgTh6bQrVlN/vr+akbNK/weIMaYc3ldDU5V31PVB9zH+z56/3rATo/X6e4ygEgRSRORhSKSmxxqAIdVNSuf9mcRkeHu9mkHDhzwUbim1Gh0KfT4L2ybD5/8n9fTdAEiw0MZc0cHerSty7Oz1/Ps7PVWlt2YIii0yKGIHAPy+0YJoKoa45eoHA1VdZeIXAB8JSKrKEJ9LFV9HaeXRHJysv1VKI+S+jvlSOY/51TTvfwBrzcNDw3hxb5JREeGMWreFo6eyuQfPVsRGmJ3CzDmfApNHKrq77Iiu4D6Hq8T3WWoau5/t4rIPKAd8B7OcFaY2+v4pb2poLr9xUkeXz7pVNO9+GavNw0NEZ7q1YrYyuG8Nm8LxzKyeL5PW8KtLLsxhQr2N2QmMMidXdUZOKKqe0SkuohEAIhIPHApsNatlzUXyL2GZDDwYTACN6WECPT8H9TvDDNGwM4lRdxc+HP35jzcvTkzV+xmxNtLycjM9lOwxpQPfk0cIjIZWAA0E5F0EblLREaKyEi3ySxgK7AZGAvc6y5vAaSJyAqcRPGMqq511z0MPCAim3HOebzpz89gyoDwSOg3CWLqwOR+8POPRd7FPV0v5OmbWzF3w34GjVvMsQyrrGtMQaQinBRMTk7WtLS0YIdh/O3ARnjzt1C1Ntw1BypXK/IuZq7YzQNTl9O8TjQThqRQo2qEHwI1pmwQkaWqmpx3ebCHqozxnYSLnAsED211pulmF73X0KNtXcYOSmbTvuP0GbOAPUdO+SFQY8o2SxymfGl8uXMHwW1fwycPFGmabq5uzWsycWgK+46epveoBfz40wk/BGpM2WWJw5Q/SQPg8gdh2UT47uVi7aLTBTWYfHdnTmVm03v0AtbtOerjII0puyxxmPKp21/h4lvgiydgzQfF2kXrxFhSR3QhLEToO2YBS7d7X97EmPLMEocpn0JCoNcoSEyB90dAevEmRzSpWZVpI7sQV6USA99YxDebrAqBMZY4TPkVHgn9J0PVWu403eIVNawfF0XqyC40rBHFXePTmL16j48DNaZsscRhyrcq8XD7NKcQ4rt9IKN4dwGsGR3J1OFdaFUvhnsnLWNa2s7zb2RMOWWJw5R/Cc2g79twcDOkDi7WNF2A2Khw3hnWiUubxPPQ9JWM+9b7+4EYU55Y4jAVwwVXwI0vwda5MOvBYk3TBYiqFMYbg5PpfnFt/vHxWl78fKNV1jUVjiUOU3G0vwMuewCWjofv/1vs3USEhfLqgHbc1iGRl7/cxJMfrSUnx5KHqTgKrY5rTLlz5ePOleWf/w2qN4KWPc67SX7CQkN49tY2REeGM+67bRzLyOLZW1sTZpV1TQVgicNULCEhcPNoOLoLZgyH2HpQr0MxdyU8fmMLYiuH8+IXGzl+OpNX+rcjIizUx0EbU7rYzyNT8YRXhn6ToWoCvNsPDu8o9q5EhPt+25QnbmrJZ2v2cdf4NE6czjr/hsaUYZY4TMVUNQEGTIOs0/Bu32JP08015NLGPH9bWxZsPcjANxdx+KT390E3pqyxxGEqrprNoc8E+GkjTBsC2SXrKdzaIZHXbm/Pml1H6ff6QvYfy/BRoMaULpY4TMV2YTe44QXY8iV8+lCxp+nmuvbi2oy7syM7Dp3kttEL2HnopI8CNab0sMRhTIfBcOn9kDYOFvyvxLu7rGk87wzrxOGTmfQe/T2b9h3zQZDGlB5+SxwiMk5E9ovI6gLWi4i8IiKbRWSliLR3lyeJyAIRWeMu7+uxzXgR2SYiy91Hkr/iNxXMVU9Ay54w5zFY93GJd9e+QXWmjuhMjkKfMQtYmX7YB0EaUzr4s8cxHuheyPrrgKbuYzgwyl1+Ehikqhe7278kIp73AH1IVZPcx3Lfh20qpJAQuHkM1GsPM+6GXctKvMvmtWOYNqILVSLCGDB2EQu3HvRBoMYEn98Sh6rOBw4V0qQnMFEdC4FqIlJHVTeq6iZ3H7uB/UCCv+I05hfhlaH/FIiKd6rpHi55IcNG8VWYPvISasdGMnjcYr5ct88HgRoTXME8x1EP8PxmprvLfiEiKUAlYIvH4qfdIawXRSSioJ2LyHARSRORtAMH7B4KxktVa8LtqZB5yp2mW/I7/9WOjSR1RBea1Y5mxNtL+XD5Lh8EakzwlNqT4yJSB3gbGKKqOe7iR4HmQEcgDni4oO1V9XVVTVbV5IQE67CYIqjZwpmme2A9TC/5NF2AuCqVmDSsEx0aVuf+qct5e2Hx7g1iTGkQzMSxC6jv8TrRXYaIxACfAH91h7EAUNU97tDWaeAtICWA8ZqK5MIr4YbnYfMXMPvhEk/TBYiODGfC0BSubFaTxz9YzWvzNvsgUGMCL5iJYyYwyJ1d1Rk4oqp7RKQS8D7O+Y/pnhu4vRBERIBeQL4ztozxieQhcMkfYckbsHDU+dt7ITI8lNF3dKBnUl2em72Bpz5ey3ErUWLKGL8VORSRyUBXIF5E0oEngHAAVR0NzAKuBzbjzKQa4m7aB/gNUENE7nSX3enOoJokIgmAAMuBkf6K3xgAfvsk/LwNPvuLU023+fUl3mV4aAgv9kkiJjKcN77dxruLd3BTm7r0S6lPUv1qOL+LjCm9pCLchCY5OVnT0tKCHYYpq86chPE3OOc8hsyCuu18sltV5Yedh5myeAcfrdjDqcxsmteOpl/H+tzcLpHYqHCfvI8xxSUiS1U1+ZzlljiM8cKxffDGVc5tZ+/+EmITfbv7jExmrtjNlMU7WbXrCJXCQri+VW36pTSgU+M464WYoLDEYYnDlNS+tfDmNVC9IQydDRHRfnmb1buOMHXJTj74YRfHTmdxQXwV+nasz60dEomvWuAMdGN8zhKHJQ7jC5u/gEl9oMlVzj09Qv13L7RTZ7L5ZNUepizeQdr2nwkPFa5uWYt+HRtwWZN4QkKsF2L8yxKHJQ7jK0vehE8egJThcP2/A/KWm/YdY8qSncxYls7PJzOpV60yfTvWp09yfWrHRgYkBlPxWOKwxGF86bO/woJXofuz0Dlwk/tOZ2Xz2Zp9TFm8g++3HCREoFuzmvRLaUC3Zgl2z3PjU5Y4LHEYX8rJhtRBsGGWM2TVrLB6nv6x/eAJpi7ZybSl6Rw4dppaMRHc1qE+fTvWp35cVMDjMeWPJQ5LHMbXzpyAt66HnzbB0E+hTtughJGZncNX6/czZfEOvt54gByFy5vG07djfa5pWZtKYdYLMcVjicMSh/GHY3th7FWg2TDsS4itd/5t/Gj34VOkpu0kdclOdh/JIK5KJW5tX49+KQ24MKFqUGMzZY8lDkscxl/2roZx3SGuEQyZDRHB/wOdnaN8s+kAUxbv5It1+8jKUVIaxdEvpT7Xt65DZHhosEM0ZYAlDkscxp82fQ7v9oGm10C/dyGk9Pxh3n8sg/eW7mLqkh38ePAkMZFh3NyuHn07NqBl3Zhgh2dKMUscljiMvy0eC7MehE4j4bpngx3NOVSVBVsPMmXxTmav3suZ7BzaJsbSL6UBN7WtS9UI/12TYsomSxyWOEwgzH4UFr4G1/0bOg0PdjQF+vnEGd7/YRdTluxg477jVKkUyk1t69IvpQFtE2OtxIkBLHFY4jCBkZMNUwfCxtnObWgvujbYERVKVVm2wym0+PHKXwst9k9pQK+kelZosYKzxGGJwwTK6eMw/no4uMWpaVW7dbAj8srRjExmLt/NlCU7WL3rKBFhIVzfug79OtYnxQotVkiWOCxxmEA6useppqvqVNONqRvsiIpk9a4jTFmygw9/2O0UWkyoQr+O9bm1fSI1rNBihWGJwxKHCbS9q9xpuhfAkE9LxTTdojp5JotPVu5hypKdLHULLV7Tsjb9Uupz6YVWaLG8s8RhicMEw8Y5MLkvNL0W+k0qVdN0i2rjvmNMWbyTGT+kc/hkJonVK9M3uT63WaHFcqugxOHXWgQiMk5E9otIvvcGd+83/oqIbBaRlSLS3mPdYBHZ5D4GeyzvICKr3G1eERt4NaXZRdc4hRA3fgpzHgt2NCVyUa1o/nZTSxY+ehUv90uiQVwUz3++kUue+ZJhE5bwxdp9ZGXnBDtMEwD+nrg9HngVmFjA+uuApu6jEzAK6CQicTj3KE8GFFgqIjNV9We3zd3AIpz7lncHPvXjZzCmZDoNh0NbnGm6cRdAyt3BjqhEIsND6ZlUj55J9fjxpxNMTdvJtLR0vliXRq2YCPokO+XerdBi+eX3oSoRaQR8rKqt8lk3BpinqpPd1xuArrkPVR3h2c59zFXV5u7y/p7tCmJDVSbocrJhygDYNAcGpELTq4MdkU9lZufw5br9TFniFFoEuKxJPP06NuDqlrWs0GIZVdBQVbAvFa0H7PR4ne4uK2x5ej7LzyEiw4HhAA0aNPBdxMYUR0go3PomvNUdpt0JQz+D2uf8liqzwkND6N6qNt1b1WbX4VOkLtnJtLSd/O7dZdSoUolbOyTSt2N9K7RYTpTbnwGq+rqqJqtqckJCQrDDMcaZVTUgFSJinLpWR/cEOyK/qFetMv939UV88/CVvDWkI8mNqvPmt9u46vmv6TNmAe//kE5GZnawwzQlEOzEsQuo7/E60V1W2PLEfJYbUzbE1IUBU+HUYWe21ZkTwY7Ib0JDhG7NajLmjmQWPHolf+7ejH1HM/i/qStIefoLnvhwNev2HA12mKYYgp04ZgKD3NlVnYEjqroH+Ay4RkSqi0h14BrgM3fdURHp7M6mGgR8GLTojSmOOm2g9zjnOo9pQ+Do7mBH5Hc1oyO5t2sT5v6pK+8O68QVzWoyefFOrnv5G3r+7zumLN7BidNZwQ7TeMmvJ8dFZDLOie54YB/OTKlwAFUd7f7xfxVnZtRJYIiqprnbDgX+4u7qaVV9y12ejDNbqzLObKo/6Hk+hJ0cN6XS4rEw6yGQEGh+A3QcBo1/AxVkhvmhE2eYsSydKUt2snm/U2ixR1Jd+nVsQBsrtFgq2AWAljhMaXRoK6S9BT+8Dad+hviLIPkuaNsPKlcLdnQB4RRa/JnJi3fy8crdZGTm0KJODP1T6tMzqR6xla3QYrBY4rDEYUqzzFOw5gNY8gbsSoPwKGh9m9MLqdMm2NEFzNGMTD5cvpspi3ewZrdTaPGG1nXol9KAjo2qWy8kwCxxWOIwZcXu5ZD2JqycBlmnIDHFSSAte0J4xSntsSr9CJOX7GDm8t0cP53FhQlV6JVUj5jK4eSokqNOb+XX55Cj6i7jl+Xkee3NNqr6y7q82+So4uxWycnJsw35bJOTZ5t89uu8Vz7vo2dv82tM3m8zdlAyv7moeDNLLXFY4jBlzamfYcUUpxdycDNUjoP2d0DyUKjeKNjRBczJM1l8vHIPUxbvYNmOw0XePkRARM76b4gIISIIzimlkBBxl3m0xeN1iMc2AsLZr0PcNmdtI3m2kXy2KSy2PMvO2gb3/fLbxo0hJMR53Se5+NfPWOKwxGHKKlXY9rWTQNbPAs1xrjzvOAya/LZMF04sqiMnM8lW9eqPc24bU3yl9cpxY8z5iMAFXZ3HkV2wbAIsHe9cRFitgdMDaXcHVIkPapiBYHckLB2sx2FMWZSdCes/hiVvwo/fQGgluPhmpxeS2LHCTOk1/mU9DmPKk9BwJ1FcfDPsX++cTF8+GVZOdW5V23GYMyurUpVgR2rKIetxGFNenD4Oq1KdXsi+1U5NrLb9oeNdkNAs2NGZMshOjlviMBWFKuxc7JxMX/sBZJ+BRpc7vZDmNzi9FWO8YInDEoepiI4fcK5KT3sLjuyAqrWhw53QYbBTcNGYQljisMRhKrKcbNj0udML2fxFha2PZYrGTo4bU5GFhEKz7s7j0DZY+hYsexvWzYQaTZ3zIG37V5j6WKZkrMdhTEWVmeGcA1nyBqQvcetj9XbrY7UNdnSmFLAehzHmbOGRThXetv3Oro+1bKJzLUjHYdCyV4Wqj2W8Yz0OY8yvTh2GFZPPrY/VYQjENQ52dCbA7OS4JQ5jvGf1sQwFJw6/3jpWRLqLyAYR2Swij+SzvqGIfCkiK0Vknogkusu7ichyj0eGiPRy140XkW0e65L8+RmMqZBy62P1fQfuXwVX/Bn2rHTqY72cBN+8ACd+CnaUJkj81uMQkVBgI3A1kA4sAfqr6lqPNtOAj1V1gohciXPr2Dvy7CcO2AwkqupJERnvbjPd21isx2GMD2RnwvpPnF5Ibn2slr2cXkj9FJvSWw4F4+R4CrBZVbe6AUwBegJrPdq0BB5wn88FPshnP72BT1X1pB9jNcacT2g4XNzLeexfD2njnPMhq1KhVmtnSm/r2yCiePd+MGWHP4eq6gE7PV6nu8s8rQBucZ/fDESLSI08bfoBk/Mse9od3npRRCLye3MRGS4iaSKSduDAgeJ9AmNM/mo2h+ufgwfWwY0vAQof3w8vtIBZf4YDG4IdofEjv57j8MKDwBUi8gNwBbALyM5dKSJ1gNbAZx7bPAo0BzoCccDD+e1YVV9X1WRVTU5IKN5tE40x5xFRFZKHwMhvYegcuKi7c3Hh/1Jg/I2w5n1niMuUK/4cqtoF1Pd4negu+4Wq7sbtcYhIVeBWVfW8N2Qf4H1VzfTYZo/79LSIvIWTfIwxwSQCDTo5j+7/cupjLRkH0+5062MNhvaDITbvoIMpi/zZ41gCNBWRxiJSCWfIaaZnAxGJF5HcGB4FxuXZR3/yDFO5vRDEuSdkL2C1H2I3xhRXlXi47P/gvuUwIBXqtIGvn4OXWsPUgbB1njPd15RZfutxqGqWiPweZ5gpFBinqmtE5B9AmqrOBLoC/xIRBeYDv8vdXkQa4fRYvs6z60kikoBzv/jlwEh/fQZjTAmEhMJF1zqPs+pjfWT1sco4uwDQGBM4eetjhVWGNrdB8l1Q1y7JKm2sVpUxJvjOVx8r+S7ndrhWH6tUsx6HMSa4Th2GFVPc+libnPpY7QZC8lCrjxVkVqvKEocxpZsqbJvv1sf6xKmP1eS3zpXpTa+2+lhBYENVxpjSTQQuuMJ5HN0NSyfA0vEwuS/ENnCuF2k3EKokWHmTILMehzGm9MpbHyuXhEBImMcj1M+vg7jNOZ81DEICc+229TiMMWWPZ32sAxtgwyxnZlZOlscju2ivNcdJSJmnir+PoBPvE9JNL0HDS3z67pY4jDFlQ0Iz5xFsqk7yKTTReJOMipGwvGqTJ7ZKvi86aYnDGGOKQgQk1D1Zn2+N1XIv2EUOjTHGlDGWOIwxxhSJJQ5jjDFFYonDGGNMkVjiMMYYUySWOIwxxhSJJQ5jjDFFYonDGGNMkVSIWlUicgDYXszN44GffBiOr1hcRWNxFY3FVTSlNS4oWWwNVTUh78IKkThKQkTS8ivyFWwWV9FYXEVjcRVNaY0L/BObDVUZY4wpEkscxhhjisQSx/m9HuwACmBxFY3FVTQWV9GU1rjAD7HZOQ5jjDFFYj0OY4wxRWKJwxhjTJFY4nCJSHcR2SAim0XkkXzWR4jIVHf9IhFpVEriulNEDojIcvcxLAAxjROR/SKyuoD1IiKvuDGvFJH2/o7Jy7i6isgRj2P1twDFVV9E5orIWhFZIyL35dMm4MfMy7gCfsxEJFJEFovICjeuJ/NpE/Dvo5dxBfz76PHeoSLyg4h8nM863x4vVa3wDyAU2AJcAFQCVgAt87S5FxjtPu8HTC0lcd0JvBrg4/UboD2wuoD11wOfAgJ0BhaVkri6Ah8H4f+vOkB793k0sDGff8eAHzMv4wr4MXOPQVX3eTiwCOicp00wvo/exBXw76PHez8AvJvfv5evj5f1OBwpwGZV3aqqZ4ApQM88bXoCE9zn04GrRERKQVwBp6rzgUOFNOkJTFTHQqCaiNQpBXEFharuUdVl7vNjwDqgXp5mAT9mXsYVcO4xOO6+DHcfeWfxBPz76GVcQSEiicANwBsFNPHp8bLE4agH7PR4nc65X6Bf2qhqFnAEqFEK4gK41R3emC4i9f0ckze8jTsYurhDDZ+KyMWBfnN3iKAdzq9VT0E9ZoXEBUE4Zu6wy3JgP/C5qhZ4vAL4ffQmLgjO9/El4M9ATgHrfXq8LHGUfR8BjVS1DfA5v/6qMOdahlN7py3wX+CDQL65iFQF3gPuV9WjgXzvwpwnrqAcM1XNVtUkIBFIEZFWgXjf8/EiroB/H0XkRmC/qi7193vlssTh2AV4/jJIdJfl20ZEwoBY4GCw41LVg6p62n35BtDBzzF5w5vjGXCqejR3qEFVZwHhIhIfiPcWkXCcP86TVHVGPk2CcszOF1cwj5n7noeBuUD3PKuC8X08b1xB+j5eCvQQkR9xhrOvFJF38rTx6fGyxOFYAjQVkcYiUgnn5NHMPG1mAoPd572Br9Q90xTMuPKMg/fAGacOtpnAIHemUGfgiKruCXZQIlI7d1xXRFJw/v/3+x8b9z3fBNap6gsFNAv4MfMmrmAcMxFJEJFq7vPKwNXA+jzNAv599CauYHwfVfVRVU1U1UY4fyO+UtWBeZr59HiFFXfD8kRVs0Tk98BnODOZxqnqGhH5B5CmqjNxvmBvi8hmnBOw/UpJXH8UkR5AlhvXnf6OS0Qm48y2iReRdOAJnBOFqOpoYBbOLKHNwElgiL9j8jKu3sA9IpIFnAL6BSD5g/OL8A5glTs+DvAXoIFHbME4Zt7EFYxjVgeYICKhOIkqVVU/Dvb30cu4Av59LIg/j5eVHDHGGFMkNlRljDGmSCxxGGOMKRJLHMYYY4rEEocxxpgiscRhjDGmSCxxGFPKiVOh9pyKp8YEiyUOY4wxRWKJwxgfEZGB7v0alovIGLcg3nERedG9f8OXIpLgtk0SkYVuMbz3RaS6u7yJiHzhFhVcJiIXuruv6hbNWy8ikwJQmdmYAlniMMYHRKQF0Be41C2Clw3cDlTBuXr3YuBrnKvZASYCD7vF8FZ5LJ8E/M8tKngJkFt2pB1wP9AS5/4sl/r9QxlTACs5YoxvXIVT0G6J2xmojFN6OweY6rZ5B5ghIrFANVX92l0+AZgmItFAPVV9H0BVMwDc/S1W1XT39XKgEfCt/z+WMeeyxGGMbwgwQVUfPWuhyON52hW3xs9pj+fZ2HfXBJENVRnjG18CvUWkJoCIxIlIQ5zvWG+3zQDgW1U9AvwsIpe7y+8AvnbvwpcuIr3cfUSISFRAP4UxXrBfLcb4gKquFZHHgDkiEgJkAr8DTuDc8OcxnKGrvu4mg4HRbmLYyq/VcO8AxriVTTOB2wL4MYzxilXHNcaPROS4qlYNdhzG+JINVRljjCkS63EYY4wpEutxGGOMKRJLHMYYY4rEEocxxpgiscRhjDGmSCxxGGOMKZL/DzXCcw0dA5LjAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Haid763z4PL2"
      },
      "source": [
        "## Next Step\n",
        "Of course, making a powerful recommender system requires much more effort. You can read these materials to dive in deeper.\n",
        "\n",
        "<p>&nbsp;</p>\n",
        "\n",
        "[Side Features] https://www.tensorflow.org/recommenders/examples/featurization\n",
        "\n",
        "[Context Features] https://www.tensorflow.org/recommenders/examples/context_features\n",
        "\n",
        "[Deep Recommenders] https://www.tensorflow.org/recommenders/examples/deep_recommenders\n",
        "\n",
        "[Multi-task Recommenders] https://www.tensorflow.org/recommenders/examples/multitask\n",
        "\n",
        "[DCN] https://www.tensorflow.org/recommenders/examples/dcn\n",
        "\n",
        "<p>&nbsp;</p>\n",
        "\n",
        "Papers:\n",
        "\n",
        "[Two-tower neural network] https://storage.googleapis.com/pub-tools-public-publication-data/pdf/6c8a86c981a62b0126a11896b7f6ae0dae4c3566.pdf\n",
        "\n",
        "[SOTA] https://arxiv.org/pdf/1905.01395v1.pdf\n",
        "\n",
        "[SSL] https://arxiv.org/abs/2007.12865\n",
        "\n",
        "[Multi-task] http://www.jiaqima.com/papers/SNR.pdf\n",
        "\n",
        "<p>&nbsp;</p>\n",
        "\n",
        "References:\n",
        "\n",
        "[1] https://developers.google.com/machine-learning/recommendation/overview\n",
        "\n",
        "[2] https://www.tensorflow.org/recommenders"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7TqxYMXOGcrC"
      },
      "source": [
        "## Donation\n",
        "Give a ⭐ if this tutorial helped you!\n",
        "\n",
        "https://github.com/xei/recommender-system-tutorial"
      ]
    }
  ]
}