9. Understanding VideoBERT, BART, and more/9.05. Performing text summarization with BART .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mp5Dlmk-kxNF"
   },
   "source": [
    "# Performing text summarization with BART \n",
    "First, let us import the BartTokenizer for tokenizing and BartForConditionalGeneration for text summarization from the transformers library: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true,
    "id": "q94H6mbVk0WM"
   },
   "outputs": [],
   "source": [
    "%%capture \n",
    "!pip install transformers==3.5.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "id": "7ojbv2X2kxNR"
   },
   "outputs": [],
   "source": [
    "from transformers import BartTokenizer, BartForConditionalGeneration "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dk16hE0rkxNS"
   },
   "source": [
    "\n",
    "We will use the BART-large model. bart-large-cnnis the pre-trained BART large model for the text summarization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "id": "8ByB3AomkxNT"
   },
   "outputs": [],
   "source": [
    "model = BartForConditionalGeneration.from_pretrained('facebook/bart-large-cnn')\n",
    "tokenizer = BartTokenizer.from_pretrained('facebook/bart-large-cnn')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1ifOcuOYkxNT"
   },
   "source": [
    "\n",
    "Now, define the text which we want to summarize: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true,
    "id": "pW7xWN-zkxNU"
   },
   "outputs": [],
   "source": [
    "text = \"\"\"Machine learning (ML) is the study of computer algorithms that improve automatically through experience.It is seen as a subset of artificial intelligence. Machine learning algorithms build a mathematical model based on sample data, known as training data, in order to make predictions or decisions without being explicitly programmed to do so.Machine learning algorithms are used in a wide variety of applications, such as email filtering and computer vision, where it is difficult or infeasible to develop conventional algorithms to perform the needed tasks.\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kwHQBxY1kxNU"
   },
   "source": [
    "Tokenize the text: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2NGXBIC0kxNU",
    "outputId": "f53a5944-0ac0-4b01-b892-ef32b31d56da"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Truncation was not explicitly activated but `max_length` is provided a specific value, please use `truncation=True` to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to `truncation`.\n"
     ]
    }
   ],
   "source": [
    "inputs = tokenizer([text], max_length=1024, return_tensors='pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nP03Jo7YkxNV"
   },
   "source": [
    "\n",
    "Get the summary ids which are the ids of the tokens generated by the model: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true,
    "id": "xXg_vBmTkxNV"
   },
   "outputs": [],
   "source": [
    "summary_ids = model.generate(inputs['input_ids'], num_beams=4, max_length=100, early_stopping=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wYBOPkUrkxNV"
   },
   "source": [
    "\n",
    "Now decode the summary id and get the corresponding token (word): "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true,
    "id": "7tIxIQoMkxNW"
   },
   "outputs": [],
   "source": [
    "summary = ([tokenizer.decode(i, skip_special_tokens=True, clean_up_tokenization_spaces=False) for i in summary_ids])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8J6MiWErkxNW"
   },
   "source": [
    "That's it. Now, let's print the summary of our given text: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "KQQG8FR3kxNW",
    "outputId": "195bbfb5-a2cb-4121-b2d0-e76bf670ed45"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Machine learning is the study of computer algorithms that improve automatically through experience. It is seen as a subset of artificial intelligence. Machine learning algorithms are used in a wide variety of applications, such as email filtering and computer vision, where it is difficult or infeasible to develop conventional algorithms to perform the needed tasks.']\n"
     ]
    }
   ],
   "source": [
    "print(summary)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6Wdk_6ZMkxNW"
   },
   "source": [
    "\n",
    "\n",
    "As we can observe, we now have the summarized text. In this way, we can use the BART model for text summarization. "
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "9.05. Performing text summarization with BART .ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}