examples/Finetuning_GettingStarted.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting Started with Finetuning\n",
    "\n",
    "Finetuning lets you take a model that has been trained on a very broad task and adapt it to your specific niche.\n",
    "\n",
    "For general models this works really well. To intuitively understand why, let's look at a text generation model.\n",
    "\n",
    "T5, an open-source model by Google, has been trained on around 750GB of text. The task was simply to predict some masked words within sentences. Take this sentence as an example: \"The man went to the `___`, he bought a gallon of `___`.\" In order to correctly fill in the gaps, the model must understand enough about language and the world.\n",
    "\n",
    "It turns out that this knowledge is transferrable to other tasks, hence the term transfer learning.\n",
    "\n",
    "In this tutorial we'll be going over Backprop's basic flow of going from data to a model in production."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Run the cell below if you don't have Backprop yet (e.g on Google Colab - ensure your runtime has an Nvidia GPU (Runtime -> Change runtime type))\n",
    "\n",
    "> You will need to restart your runtime afterwards! (Runtime -> Restart runtime)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run if you do not have backprop\n",
    "!pip install backprop\n",
    "!pip install datasets\n",
    "# DO NOT RUN THIS CELL AGAIN AFTER RESTARTING"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finetuning a text generation model\n",
    "\n",
    "Text generation is one of the many tasks that Backprop supports. Even though it is more technical than other tasks like classification, it is well worth learning as there are no limits to the tasks you can solve.\n",
    "\n",
    "The basic idea is very simple. You give the model some text as input and tell it what you expect as output.\n",
    "\n",
    "For example, to solve a classification task, your input and output data might look something like this:\n",
    "\n",
    "```python\n",
    "inp = [\"I really liked the service I received!\", \"Meh, it was not impressive.\"]\n",
    "out = [\"positive\", \"negative\"]\n",
    "```\n",
    "\n",
    "The expected behaviour of a model trained with data like this is to output \"positive\" for positive text and \"negative\" for negative text.\n",
    "\n",
    "Now, let's look at an example that shows the finetuning process in detail."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating questions\n",
    "\n",
    "One of the tasks that Backprop supports right out of the box is question answering based on context.\n",
    "\n",
    "What if instead you wanted to do the reverse? That is generate questions based on context. That's certainly possible with text generation.\n",
    "\n",
    "With a minimal amount of code, we'll build a model that can take any paragraph of text (such as something from Elon Musk's wikipedia page) and generate questions that can be answered based on the paragraph.\n",
    "\n",
    "<img src=\"assets/texttask.png\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparing the data\n",
    "\n",
    "Finetuning a model needs data. The good news is that you need a lot less of it. For our proof of concept, we will use less than 1000 examples.\n",
    "\n",
    "Our examples are from the SQuAD dataset.\n",
    "\n",
    "This dataset has multiple questions and answers on different paragraphs of text. We'll get a section of text as input and a list of questions as output. Pretty simple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Reusing dataset squad (/home/kristo/.cache/huggingface/datasets/squad/plain_text/1.0.0/1244d044b266a5e4dbd4174d23cb995eead372fbca31a03edc3f8a132787af41)\n"
     ]
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "dataset = load_dataset(\"squad\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'answers': {'answer_start': [515], 'text': ['Saint Bernadette Soubirous']},\n",
       " 'context': 'Architecturally, the school has a Catholic character. Atop the Main Building\\'s gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.',\n",
       " 'id': '5733be284776f41900661182',\n",
       " 'question': 'To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?',\n",
       " 'title': 'University_of_Notre_Dame'}"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset[\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just so you can see what the data looks like, each item in the list is a dictionary with context, question and answer. All we'll do is group it up."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "input_data = []\n",
    "output_data = []\n",
    "\n",
    "last_context = \"\"\n",
    "\n",
    "# Limit to 5000 items for proof of concept\n",
    "for i in range(5000):\n",
    "    context = dataset[\"train\"][i][\"context\"]\n",
    "    question = dataset[\"train\"][i][\"question\"]\n",
    "    if context != last_context:\n",
    "        input_data.append(context)\n",
    "        last_context = context\n",
    "        output_data.append([])\n",
    "\n",
    "    output_index = len(input_data) - 1\n",
    "    output_data[output_index].append(question)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Architecturally, the school has a Catholic character. Atop the Main Building\\'s gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?',\n",
       " 'What is in front of the Notre Dame Main Building?',\n",
       " 'The Basilica of the Sacred heart at Notre Dame is beside to which structure?',\n",
       " 'What is the Grotto at Notre Dame?',\n",
       " 'What sits on top of the Main Building at Notre Dame?']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(820, 820)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(input_data), len(output_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! Seems like we got 820 examples for training. That's just a small portion of the data, but it's enough to achieve some promising results. There is just 1 final step before finetuning.\n",
    "\n",
    "The T5 model that we are planning on using has already been finetuned on some tasks such as translation. For multiple tasks, it is useful to add a prefix to the input that can let the model know what it should do.\n",
    "\n",
    "For example, `generate questions: Some paragraph of text.`\n",
    "\n",
    "Additionally, our `output_data` is currently a list of strings. Our model expects just a string for an output example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_data = [f\"generate questions: {i}\" for i in input_data]\n",
    "output_data = [\"; \".join(o) for o in output_data]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'generate questions: Architecturally, the school has a Catholic character. Atop the Main Building\\'s gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "input_data[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?; What is in front of the Notre Dame Main Building?; The Basilica of the Sacred heart at Notre Dame is beside to which structure?; What is the Grotto at Notre Dame?; What sits on top of the Main Building at Notre Dame?'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "output_data[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For output data, we chose `;` as a separator that's not too common in text. Anything similar should work fine."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Finetuning\n",
    "\n",
    "We are ready to finetune. All we'll need to do is pass the list of input and output strings to Backprop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import backprop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see which model we can use for this purpose."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Name                     t5-small\n",
      "Description              This is the small T5 model by Google.\n",
      "Supported tasks          ['text-generation']\n",
      "Finetunable tasks        ['text-generation']\n",
      "----------\n",
      "Name                     t5-base\n",
      "Description              This is the base T5 model by Google.\n",
      "Supported tasks          ['text-generation']\n",
      "Finetunable tasks        ['text-generation']\n",
      "----------\n",
      "Name                     t5-base-qa-summary-emotion\n",
      "Description              This is the T5 base model by Google, and has been finetuned further for Q&A, Summarisation, and Sentiment analysis (emotion detection).\n",
      "Supported tasks          ['text-generation', 'emotion', 'summarisation', 'qa']\n",
      "Finetunable tasks        ['text-generation', 'emotion', 'summarisation', 'qa']\n",
      "----------\n",
      "Name                     gpt2-large\n",
      "Description              A large (774M parameter) version of OpenAI's GPT-2 model. This is a general-use model, and has not been further finetuned on any specific languages or tasks.\n",
      "Supported tasks          ['text-generation']\n",
      "Finetunable tasks        []\n",
      "----------\n",
      "Name                     gpt2-medium\n",
      "Alias                    gpt2\n",
      "Description              A medium version of OpenAI's GPT-2 model. This is a general-use model, and has not been further finetuned on any specific languages or tasks.\n",
      "Supported tasks          ['text-generation']\n",
      "Finetunable tasks        []\n",
      "----------\n",
      "Name                     distilgpt2\n",
      "Description              A distilled (82M parameter) version of OpenAI's GPT-2 model.\n",
      "Supported tasks          ['text-generation']\n",
      "Finetunable tasks        []\n",
      "----------\n"
     ]
    }
   ],
   "source": [
    "backprop.TextGeneration.list_models(display=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "GPU available: True, used: True\n",
      "TPU available: None, using: 0 TPU cores\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing data...\n",
      "Finding the optimal batch size...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/kristo/.local/lib/python3.8/site-packages/transformers/optimization.py:557: UserWarning: This overload of add_ is deprecated:\n",
      "\tadd_(Number alpha, Tensor other)\n",
      "Consider using one of the following signatures instead:\n",
      "\tadd_(Tensor other, *, Number alpha) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:882.)\n",
      "  exp_avg_sq_row.mul_(beta2t).add_(1.0 - beta2t, update.mean(dim=-1))\n",
      "Batch size 2 succeeded, trying batch size 4\n",
      "Batch size 4 succeeded, trying batch size 8\n",
      "Batch size 8 succeeded, trying batch size 16\n",
      "Batch size 16 succeeded, trying batch size 32\n",
      "Batch size 32 failed, trying batch size 16\n",
      "Finished batch size finder, will continue with full run using batch size 16\n",
      "Restored states from the checkpoint file at /home/kristo/Documents/backprop/examples/scale_batch_size_temp_model.ckpt\n",
      "GPU available: True, used: True\n",
      "TPU available: None, using: 0 TPU cores\n",
      "\n",
      "  | Name  | Type             | Params\n",
      "-------------------------------------------\n",
      "0 | model | HFSeq2SeqTGModel | 60.5 M\n",
      "-------------------------------------------\n",
      "60.5 M    Trainable params\n",
      "0         Non-trainable params\n",
      "60.5 M    Total params\n",
      "242.026   Total estimated model params size (MB)\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validation sanity check', layout=Layout…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "be2dbc9ffcfc465ab327196611298ec3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Training', layout=Layout(flex='2'), max…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', description='Validating', layout=Layout(flex='2'), m…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Training finished! Save your model for later with backprop.save or upload it with backprop.upload\n"
     ]
    }
   ],
   "source": [
    "# Start a local text generation task with t5-small\n",
    "tg = backprop.TextGeneration(\"t5-small\")\n",
    "# Length here refers to number of tokens (1 token ~ 1 word)\n",
    "tg.finetune({\"input_text\": input_data, \"output_text\": output_data}, max_input_length=256, max_output_length=256)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "context = \"\"\"generate questions: Tesla, Inc. (originally Tesla Motors) was incorporated in July 2003 by Martin Eberhard and Marc Tarpenning, who financed the company until the Series A round of funding.[90] Both men played active roles in the company's early development prior to Musk's involvement.[91] Musk led the Series A round of investment in 2004, joining Tesla's board of directors as its chairman.[92][93][94][95] Musk took an active role within the company and oversaw Roadster product design but was not deeply involved in day-to-day business operations.[96] Following a series of escalating conflicts in 2007 and the 2008 financial crisis, Eberhard was ousted from the firm.[97][98] Musk assumed leadership of the company as CEO and product architect in 2008, positions he still holds today. A 2009 lawsuit settlement with Eberhard designated Musk as a Tesla co-founder, along with Tarpenning and two others.[4][5] As of 2019, Musk is the longest tenured CEO of any automotive manufacturer globally.[99]\"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'What company was founded in 2003 by which two men?; Who financed the company until the Series A round of funding?; What is the name of the company that was incorporated by which men?; Which two men were involved in the'"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Have a look at the text generation notebook to understand these parameters\n",
    "tg(context, max_length=50, min_length=30, temperature=0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not bad.\n",
    "\n",
    "We used less than 1000 training examples, and less than 1 minute for training. This is the power of transfer learning! Check out our [finetuning docs](https://backprop.readthedocs.io/en/latest/Finetuning.html) for other tasks you can finetune.\n",
    "\n",
    "Where to from here? You can save this model for later with `tg.save(\"my-model\")` and load ready for text generation with `tg = backprop.TextGeneration(\"my-model\")`.\n",
    "\n",
    "For production, Backprop lets you deploy your finetuned models with a single line of code.\n",
    "\n",
    "You can get an API Key by signing up on the [dashboard](https://backprop.co).\n",
    "\n",
    "The model will be private, always available, and scale to support thousands of requests a second if needed. The best part is that you only pay for what you use. If it is idle, you pay nothing at all. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving model...\n",
      "Testing that the model can be loaded...\n",
      "Creating zip...\n",
      "Getting upload url...\n",
      "Uploading to Backprop, this may take a few minutes...\n",
      "Successfully uploaded the model to Backprop. See the build process at https://dashboard.backprop.co\n"
     ]
    }
   ],
   "source": [
    "# Name the model\n",
    "name = \"t5-question-generation\"\n",
    "# Give a description\n",
    "description = \"This T5 small model was partly finetuned on SQuAD to generate questions based on given context.\"\n",
    "\n",
    "# Requires API Key from backprop.co\n",
    "tg.upload(name=name, description=description, api_key=\"abc\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"assets/dashboard-models.png\" style=\"max-width: 600px\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After the build has completed, *only* you can use the model anywhere via our API. See our [API docs](https://docs.backprop.co/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'What company is the brain-based company that integrates the human brain with AI?; What company created the technology that could be implanted in the human brain?; What company is Ne'"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using backprop\n",
    "import backprop\n",
    "\n",
    "context = \"generate questions: In 2016, Musk co-founded Neuralink, a neurotechnology start-up company to integrate the human brain with AI. The company is centered on creating devices that can be implanted in the human brain, with the eventual purpose of helping human beings merge with software and keep pace with advancements in artificial intelligence. These enhancements could improve memory or allow more direct interfacing with computing devices.[142][143]\"\n",
    "\n",
    "tg = backprop.TextGeneration(\"t5-question-generation\", api_key=\"abc\")\n",
    "tg(context, temperature=0.5, max_length=40, min_length=15)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'output': 'In what year was Neuralink founded?; What is the name of the company that is in charge of integrating the human brain with AI?; What is the purpose of the company?'}"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Using plain http\n",
    "import requests\n",
    "\n",
    "body = {\n",
    "    \"model\": \"t5-question-generation\",\n",
    "    \"temperature\": 0.5,\n",
    "    \"max_length\": 40,\n",
    "    \"min_length\": 15,\n",
    "    \"text\": context\n",
    "}\n",
    "\n",
    "# Requires API Key\n",
    "requests.post(\"https://api.backprop.co/text-generation\", json=body,\n",
    "              headers={\"x-api-key\": \"abc\"}).json()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}