Ch12_Optimization_Algorithms/Momentum.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Momentum\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In gradient descent, we mentioned that the gradient of the objective function's independent variable represents the direction of the objective function's fastest descend at the current position of the independent variable. Therefore, gradient descent is also called steepest descent. In each iteration, the gradient descends according to the current position of the independent variable while updating the latter along the current position of the gradient. However, this can lead to problems if the iterative direction of the independent variable relies exclusively on the current position of the independent variable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises with Gradient Descent\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we will consider an objective function $f(\\boldsymbol{x})=0.1x_1^2+2x_2^2$, whose input and output are a two-dimensional vector $\\boldsymbol{x} = [x_1, x_2]$ and a scalar, respectively. In contrast to :numref:chapter_gd, here, the coefficient $x_1^2$ is reduced from $1$ to $0.1$. We are going to implement gradient descent based on this objective function, and demonstrate the iterative trajectory of the independent variable using the learning rate $0.4$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 20, x1 -0.943467, x2 -0.000073\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"184pt\" version=\"1.1\" viewBox=\"0 0 243 184\" width=\"243pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.360018 \n",
       "L 243.120313 184.360018 \n",
       "L 243.120313 -0 \n",
       "L 0 -0 \n",
       "L 0 184.360018 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 235.920313 148.803768 \n",
       "L 235.920313 9.303768 \n",
       "L 40.620313 9.303768 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_1\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 99.781827 \n",
       "L 220.6625 100.137415 \n",
       "L 205.404688 100.219474 \n",
       "L 190.146875 100.028004 \n",
       "L 174.889063 99.562528 \n",
       "L 162.682813 98.944114 \n",
       "L 150.476563 98.135418 \n",
       "L 138.270313 97.136441 \n",
       "L 129.115625 96.20797 \n",
       "L 116.909375 94.749147 \n",
       "L 107.754688 93.457251 \n",
       "L 95.100875 91.362592 \n",
       "L 83.342188 88.941052 \n",
       "L 80.290625 88.226121 \n",
       "L 71.135938 85.839402 \n",
       "L 61.98125 82.858314 \n",
       "L 55.878125 80.421415 \n",
       "L 50.486044 77.686121 \n",
       "L 46.409725 74.950827 \n",
       "L 43.559891 72.215533 \n",
       "L 41.880132 69.480239 \n",
       "L 41.320212 66.744945 \n",
       "L 41.880132 64.009651 \n",
       "L 43.671875 61.164945 \n",
       "L 46.723438 58.324147 \n",
       "L 50.486044 55.803768 \n",
       "L 55.878125 53.068474 \n",
       "L 62.752171 50.33318 \n",
       "L 71.307374 47.597886 \n",
       "L 81.908321 44.862592 \n",
       "L 92.496875 42.642177 \n",
       "L 98.6 41.522654 \n",
       "L 111.96875 39.392004 \n",
       "L 123.0125 37.985281 \n",
       "L 135.21875 36.632924 \n",
       "L 147.425 35.586377 \n",
       "L 159.63125 34.730111 \n",
       "L 171.8375 34.064126 \n",
       "L 184.04375 33.615063 \n",
       "L 199.301563 33.31418 \n",
       "L 214.559375 33.286827 \n",
       "L 229.817188 33.533004 \n",
       "L 235.920313 33.708062 \n",
       "L 235.920313 33.708062 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_2\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 113.788096 \n",
       "L 232.86875 113.854525 \n",
       "L 229.817188 113.913138 \n",
       "L 226.765625 113.963936 \n",
       "L 223.714063 114.00692 \n",
       "L 220.6625 114.042088 \n",
       "L 217.610938 114.069441 \n",
       "L 214.559375 114.088978 \n",
       "L 211.507813 114.100701 \n",
       "L 208.45625 114.104609 \n",
       "L 205.404688 114.100701 \n",
       "L 202.353125 114.088978 \n",
       "L 199.301563 114.069441 \n",
       "L 196.25 114.042088 \n",
       "L 193.198438 114.00692 \n",
       "L 190.146875 113.963936 \n",
       "L 187.095313 113.913138 \n",
       "L 184.04375 113.854525 \n",
       "L 180.992188 113.788096 \n",
       "L 177.940625 113.713852 \n",
       "L 174.889063 113.631794 \n",
       "L 171.8375 113.54192 \n",
       "L 168.785938 113.444231 \n",
       "L 165.734375 113.338726 \n",
       "L 163.208944 113.244945 \n",
       "L 162.682813 113.224223 \n",
       "L 159.63125 113.095747 \n",
       "L 156.579688 112.958982 \n",
       "L 153.528125 112.813929 \n",
       "L 150.476563 112.660587 \n",
       "L 147.425 112.498956 \n",
       "L 144.373438 112.329036 \n",
       "L 141.321875 112.150827 \n",
       "L 138.270313 111.96433 \n",
       "L 135.21875 111.769544 \n",
       "L 132.167188 111.566469 \n",
       "L 129.115625 111.355105 \n",
       "L 126.064063 111.135453 \n",
       "L 123.0125 110.907512 \n",
       "L 119.960938 110.671282 \n",
       "L 117.943803 110.509651 \n",
       "L 116.909375 110.421415 \n",
       "L 113.857813 110.152298 \n",
       "L 110.80625 109.874357 \n",
       "L 107.754688 109.587592 \n",
       "L 104.703125 109.292004 \n",
       "L 101.651563 108.987592 \n",
       "L 98.6 108.674357 \n",
       "L 95.548438 108.352298 \n",
       "L 92.496875 108.021415 \n",
       "L 90.277557 107.774357 \n",
       "L 89.445313 107.67532 \n",
       "L 86.39375 107.302754 \n",
       "L 83.342188 106.920756 \n",
       "L 80.290625 106.529326 \n",
       "L 77.239063 106.128464 \n",
       "L 74.1875 105.71817 \n",
       "L 71.135938 105.298444 \n",
       "L 69.291587 105.039062 \n",
       "L 68.084375 104.85671 \n",
       "L 65.032813 104.385631 \n",
       "L 61.98125 103.904422 \n",
       "L 58.929688 103.413082 \n",
       "L 55.878125 102.911612 \n",
       "L 52.826563 102.40001 \n",
       "L 52.263653 102.303768 \n",
       "L 49.775 101.844239 \n",
       "L 46.723438 101.269827 \n",
       "L 43.671875 100.684474 \n",
       "L 40.620313 100.08818 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 40.620313 33.40171 \n",
       "L 43.671875 32.805415 \n",
       "L 46.723438 32.220063 \n",
       "L 49.775 31.645651 \n",
       "L 52.263653 31.186121 \n",
       "L 52.826563 31.089879 \n",
       "L 55.878125 30.578278 \n",
       "L 58.929688 30.076808 \n",
       "L 61.98125 29.585468 \n",
       "L 65.032813 29.104259 \n",
       "L 68.084375 28.63318 \n",
       "L 69.291587 28.450827 \n",
       "L 71.135938 28.191446 \n",
       "L 74.1875 27.77172 \n",
       "L 77.239063 27.361426 \n",
       "L 80.290625 26.960564 \n",
       "L 83.342188 26.569133 \n",
       "L 86.39375 26.187136 \n",
       "L 89.445313 25.81457 \n",
       "L 90.277557 25.715533 \n",
       "L 92.496875 25.468474 \n",
       "L 95.548438 25.137592 \n",
       "L 98.6 24.815533 \n",
       "L 101.651563 24.502298 \n",
       "L 104.703125 24.197886 \n",
       "L 107.754688 23.902298 \n",
       "L 110.80625 23.615533 \n",
       "L 113.857813 23.337592 \n",
       "L 116.909375 23.068474 \n",
       "L 117.943803 22.980239 \n",
       "L 119.960938 22.818608 \n",
       "L 123.0125 22.582378 \n",
       "L 126.064063 22.354437 \n",
       "L 129.115625 22.134784 \n",
       "L 132.167188 21.923421 \n",
       "L 135.21875 21.720346 \n",
       "L 138.270313 21.52556 \n",
       "L 141.321875 21.339063 \n",
       "L 144.373438 21.160854 \n",
       "L 147.425 20.990934 \n",
       "L 150.476563 20.829303 \n",
       "L 153.528125 20.675961 \n",
       "L 156.579688 20.530907 \n",
       "L 159.63125 20.394143 \n",
       "L 162.682813 20.265667 \n",
       "L 163.208944 20.244945 \n",
       "L 165.734375 20.151163 \n",
       "L 168.785938 20.045659 \n",
       "L 171.8375 19.94797 \n",
       "L 174.889063 19.858096 \n",
       "L 177.940625 19.776037 \n",
       "L 180.992188 19.701794 \n",
       "L 184.04375 19.635365 \n",
       "L 187.095313 19.576752 \n",
       "L 190.146875 19.525953 \n",
       "L 193.198438 19.48297 \n",
       "L 196.25 19.447802 \n",
       "L 199.301563 19.420449 \n",
       "L 202.353125 19.400911 \n",
       "L 205.404688 19.389189 \n",
       "L 208.45625 19.385281 \n",
       "L 211.507813 19.389189 \n",
       "L 214.559375 19.400911 \n",
       "L 217.610938 19.420449 \n",
       "L 220.6625 19.447802 \n",
       "L 223.714063 19.48297 \n",
       "L 226.765625 19.525953 \n",
       "L 229.817188 19.576752 \n",
       "L 232.86875 19.635365 \n",
       "L 235.920313 19.701794 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_3\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 124.500998 \n",
       "L 232.86875 124.555068 \n",
       "L 229.817188 124.602777 \n",
       "L 226.765625 124.644124 \n",
       "L 223.714063 124.67911 \n",
       "L 220.6625 124.707736 \n",
       "L 217.610938 124.73 \n",
       "L 214.559375 124.745902 \n",
       "L 211.507813 124.755444 \n",
       "L 208.45625 124.758625 \n",
       "L 205.404688 124.755444 \n",
       "L 202.353125 124.745902 \n",
       "L 199.301563 124.73 \n",
       "L 196.25 124.707736 \n",
       "L 193.198438 124.67911 \n",
       "L 190.146875 124.644124 \n",
       "L 187.095313 124.602777 \n",
       "L 184.04375 124.555068 \n",
       "L 180.992188 124.500998 \n",
       "L 177.940625 124.440567 \n",
       "L 174.889063 124.373775 \n",
       "L 171.8375 124.300622 \n",
       "L 168.785938 124.221108 \n",
       "L 167.542708 124.186121 \n",
       "L 165.734375 124.13275 \n",
       "L 162.682813 124.036014 \n",
       "L 159.63125 123.932606 \n",
       "L 156.579688 123.822527 \n",
       "L 153.528125 123.705777 \n",
       "L 150.476563 123.582355 \n",
       "L 147.425 123.452262 \n",
       "L 144.373438 123.315497 \n",
       "L 141.321875 123.172061 \n",
       "L 138.270313 123.021953 \n",
       "L 135.21875 122.865174 \n",
       "L 132.167188 122.701724 \n",
       "L 129.115625 122.531602 \n",
       "L 126.064063 122.354809 \n",
       "L 123.0125 122.171344 \n",
       "L 119.960938 121.981207 \n",
       "L 116.909375 121.7844 \n",
       "L 113.857813 121.58092 \n",
       "L 111.96875 121.450827 \n",
       "L 110.80625 121.366664 \n",
       "L 107.754688 121.138723 \n",
       "L 104.703125 120.903768 \n",
       "L 101.651563 120.6618 \n",
       "L 98.6 120.412818 \n",
       "L 95.548438 120.156823 \n",
       "L 92.496875 119.893814 \n",
       "L 89.445313 119.623791 \n",
       "L 86.39375 119.346755 \n",
       "L 83.342188 119.062705 \n",
       "L 80.290625 118.771642 \n",
       "L 79.716213 118.715533 \n",
       "L 77.239063 118.460485 \n",
       "L 74.1875 118.138904 \n",
       "L 71.135938 117.809929 \n",
       "L 68.084375 117.473562 \n",
       "L 65.032813 117.129802 \n",
       "L 61.98125 116.778649 \n",
       "L 58.929688 116.420104 \n",
       "L 55.878125 116.054166 \n",
       "L 55.273855 115.980239 \n",
       "L 52.826563 115.663726 \n",
       "L 49.775 115.261247 \n",
       "L 46.723438 114.850953 \n",
       "L 43.671875 114.432844 \n",
       "L 40.620313 114.00692 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 40.620313 19.48297 \n",
       "L 43.671875 19.057046 \n",
       "L 46.723438 18.638936 \n",
       "L 49.775 18.228642 \n",
       "L 52.826563 17.826163 \n",
       "L 55.273855 17.509651 \n",
       "L 55.878125 17.435724 \n",
       "L 58.929688 17.069786 \n",
       "L 61.98125 16.711241 \n",
       "L 65.032813 16.360088 \n",
       "L 68.084375 16.016328 \n",
       "L 71.135938 15.679961 \n",
       "L 74.1875 15.350986 \n",
       "L 77.239063 15.029404 \n",
       "L 79.716213 14.774357 \n",
       "L 80.290625 14.718248 \n",
       "L 83.342188 14.427185 \n",
       "L 86.39375 14.143135 \n",
       "L 89.445313 13.866099 \n",
       "L 92.496875 13.596076 \n",
       "L 95.548438 13.333067 \n",
       "L 98.6 13.077072 \n",
       "L 101.651563 12.82809 \n",
       "L 104.703125 12.586121 \n",
       "L 107.754688 12.351167 \n",
       "L 110.80625 12.123225 \n",
       "L 111.96875 12.039062 \n",
       "L 113.857813 11.908969 \n",
       "L 116.909375 11.70549 \n",
       "L 119.960938 11.508682 \n",
       "L 123.0125 11.318546 \n",
       "L 126.064063 11.135081 \n",
       "L 129.115625 10.958288 \n",
       "L 132.167188 10.788166 \n",
       "L 135.21875 10.624715 \n",
       "L 138.270313 10.467936 \n",
       "L 141.321875 10.317829 \n",
       "L 144.373438 10.174392 \n",
       "L 147.425 10.037628 \n",
       "L 150.476563 9.907535 \n",
       "L 153.528125 9.784113 \n",
       "L 156.579688 9.667362 \n",
       "L 159.63125 9.557283 \n",
       "L 162.682813 9.453876 \n",
       "L 165.734375 9.35714 \n",
       "L 167.542708 9.303768 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_4\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 133.505659 \n",
       "L 232.86875 133.553108 \n",
       "L 229.817188 133.594975 \n",
       "L 226.765625 133.631259 \n",
       "L 223.714063 133.661962 \n",
       "L 220.6625 133.687082 \n",
       "L 217.610938 133.70662 \n",
       "L 214.559375 133.720575 \n",
       "L 211.507813 133.728948 \n",
       "L 208.45625 133.73174 \n",
       "L 205.404688 133.728948 \n",
       "L 202.353125 133.720575 \n",
       "L 199.301563 133.70662 \n",
       "L 196.25 133.687082 \n",
       "L 193.198438 133.661962 \n",
       "L 190.146875 133.631259 \n",
       "L 187.095313 133.594975 \n",
       "L 184.04375 133.553108 \n",
       "L 180.992188 133.505659 \n",
       "L 177.940625 133.452628 \n",
       "L 174.889063 133.394014 \n",
       "L 171.8375 133.329819 \n",
       "L 168.785938 133.260041 \n",
       "L 165.734375 133.184681 \n",
       "L 162.682813 133.103738 \n",
       "L 159.63125 133.017214 \n",
       "L 156.579688 132.925107 \n",
       "L 153.528125 132.827418 \n",
       "L 150.476563 132.724147 \n",
       "L 147.425 132.615293 \n",
       "L 144.373438 132.500857 \n",
       "L 141.605741 132.392004 \n",
       "L 141.321875 132.380364 \n",
       "L 138.270313 132.249419 \n",
       "L 135.21875 132.112654 \n",
       "L 132.167188 131.97007 \n",
       "L 129.115625 131.821666 \n",
       "L 126.064063 131.667442 \n",
       "L 123.0125 131.507398 \n",
       "L 119.960938 131.341534 \n",
       "L 116.909375 131.169851 \n",
       "L 113.857813 130.992348 \n",
       "L 110.80625 130.809025 \n",
       "L 107.754688 130.619882 \n",
       "L 104.703125 130.42492 \n",
       "L 101.651563 130.224138 \n",
       "L 98.6 130.017536 \n",
       "L 95.548438 129.805114 \n",
       "L 93.473375 129.65671 \n",
       "L 92.496875 129.583768 \n",
       "L 89.445313 129.349749 \n",
       "L 86.39375 129.109651 \n",
       "L 83.342188 128.863474 \n",
       "L 80.290625 128.611219 \n",
       "L 77.239063 128.352886 \n",
       "L 74.1875 128.088474 \n",
       "L 71.135938 127.817984 \n",
       "L 68.084375 127.541415 \n",
       "L 65.032813 127.258768 \n",
       "L 61.98125 126.970043 \n",
       "L 61.477899 126.921415 \n",
       "L 58.929688 126.663789 \n",
       "L 55.878125 126.348912 \n",
       "L 52.826563 126.027674 \n",
       "L 49.775 125.700075 \n",
       "L 46.723438 125.366114 \n",
       "L 43.671875 125.025793 \n",
       "L 40.620313 124.67911 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_5\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 141.440854 \n",
       "L 232.86875 141.483127 \n",
       "L 229.817188 141.520426 \n",
       "L 226.765625 141.552752 \n",
       "L 223.714063 141.580105 \n",
       "L 220.6625 141.602485 \n",
       "L 217.610938 141.619891 \n",
       "L 214.559375 141.632325 \n",
       "L 211.507813 141.639784 \n",
       "L 208.45625 141.642271 \n",
       "L 205.404688 141.639784 \n",
       "L 202.353125 141.632325 \n",
       "L 199.301563 141.619891 \n",
       "L 196.25 141.602485 \n",
       "L 193.198438 141.580105 \n",
       "L 190.146875 141.552752 \n",
       "L 187.095313 141.520426 \n",
       "L 184.04375 141.483127 \n",
       "L 180.992188 141.440854 \n",
       "L 177.940625 141.393608 \n",
       "L 174.889063 141.341389 \n",
       "L 171.8375 141.284196 \n",
       "L 168.785938 141.22203 \n",
       "L 165.734375 141.154891 \n",
       "L 162.682813 141.082779 \n",
       "L 159.63125 141.005694 \n",
       "L 156.579688 140.923635 \n",
       "L 153.528125 140.836603 \n",
       "L 150.476563 140.744597 \n",
       "L 147.425 140.647619 \n",
       "L 145.936433 140.597886 \n",
       "L 144.373438 140.543696 \n",
       "L 141.321875 140.432736 \n",
       "L 138.270313 140.316615 \n",
       "L 135.21875 140.195333 \n",
       "L 132.167188 140.06889 \n",
       "L 129.115625 139.937287 \n",
       "L 126.064063 139.800522 \n",
       "L 123.0125 139.658596 \n",
       "L 119.960938 139.51151 \n",
       "L 116.909375 139.359262 \n",
       "L 113.857813 139.201854 \n",
       "L 110.80625 139.039284 \n",
       "L 107.754688 138.871554 \n",
       "L 104.703125 138.698663 \n",
       "L 101.651563 138.520611 \n",
       "L 98.6 138.337398 \n",
       "L 95.548438 138.149024 \n",
       "L 92.496875 137.955489 \n",
       "L 91.07017 137.862592 \n",
       "L 89.445313 137.752644 \n",
       "L 86.39375 137.540793 \n",
       "L 83.342188 137.323578 \n",
       "L 80.290625 137.101 \n",
       "L 77.239063 136.873059 \n",
       "L 74.1875 136.639755 \n",
       "L 71.135938 136.401087 \n",
       "L 68.084375 136.157056 \n",
       "L 65.032813 135.907661 \n",
       "L 61.98125 135.652903 \n",
       "L 58.929688 135.392782 \n",
       "L 55.878125 135.127298 \n",
       "L 55.878125 135.127298 \n",
       "L 52.826563 134.845395 \n",
       "L 49.775 134.55791 \n",
       "L 46.723438 134.264843 \n",
       "L 43.671875 133.966193 \n",
       "L 40.620313 133.661962 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_6\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 208.45625 148.803768 \n",
       "L 205.404688 148.80145 \n",
       "L 202.353125 148.794496 \n",
       "L 199.301563 148.782906 \n",
       "L 196.25 148.76668 \n",
       "L 193.198438 148.745817 \n",
       "L 190.146875 148.720319 \n",
       "L 187.095313 148.690184 \n",
       "L 184.04375 148.655413 \n",
       "L 180.992188 148.616007 \n",
       "L 177.940625 148.571964 \n",
       "L 174.889063 148.523285 \n",
       "L 171.8375 148.46997 \n",
       "L 168.785938 148.412019 \n",
       "L 165.734375 148.349431 \n",
       "L 162.682813 148.282208 \n",
       "L 159.63125 148.210349 \n",
       "L 156.579688 148.133853 \n",
       "L 153.528125 148.052722 \n",
       "L 150.476563 147.966954 \n",
       "L 147.425 147.87655 \n",
       "L 144.373438 147.78151 \n",
       "L 141.321875 147.681834 \n",
       "L 138.270313 147.577522 \n",
       "L 135.21875 147.468574 \n",
       "L 132.167188 147.35499 \n",
       "L 129.115625 147.236769 \n",
       "L 126.064063 147.113913 \n",
       "L 123.0125 146.98642 \n",
       "L 119.960938 146.854292 \n",
       "L 116.909375 146.717527 \n",
       "L 113.857813 146.576126 \n",
       "L 110.80625 146.430089 \n",
       "L 107.754688 146.279416 \n",
       "L 104.703125 146.124107 \n",
       "L 103.641712 146.068474 \n",
       "L 101.651563 145.960502 \n",
       "L 98.6 145.790146 \n",
       "L 95.548438 145.614991 \n",
       "L 92.496875 145.435038 \n",
       "L 89.445313 145.250285 \n",
       "L 86.39375 145.060734 \n",
       "L 83.342188 144.866384 \n",
       "L 80.290625 144.667236 \n",
       "L 77.239063 144.463289 \n",
       "L 74.1875 144.254542 \n",
       "L 71.135938 144.040997 \n",
       "L 68.084375 143.822654 \n",
       "L 65.032813 143.599511 \n",
       "L 61.98125 143.37157 \n",
       "L 61.477899 143.33318 \n",
       "L 58.929688 143.131763 \n",
       "L 55.878125 142.885587 \n",
       "L 52.826563 142.634437 \n",
       "L 49.775 142.378314 \n",
       "L 46.723438 142.117218 \n",
       "L 43.671875 141.851148 \n",
       "L 40.620313 141.580105 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 235.920313 148.616007 \n",
       "L 232.86875 148.655413 \n",
       "L 229.817188 148.690184 \n",
       "L 226.765625 148.720319 \n",
       "L 223.714063 148.745817 \n",
       "L 220.6625 148.76668 \n",
       "L 217.610938 148.782906 \n",
       "L 214.559375 148.794496 \n",
       "L 211.507813 148.80145 \n",
       "L 208.45625 148.803768 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_7\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 41.320212 148.803768 \n",
       "L 40.620313 148.745817 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p2b6fedb403)\" d=\"M 55.878125 121.450827 \n",
       "L 68.084375 33.921415 \n",
       "L 79.314125 86.439062 \n",
       "L 89.645495 54.928474 \n",
       "L 99.150355 73.834827 \n",
       "L 107.894827 62.491015 \n",
       "L 115.939741 69.297302 \n",
       "L 123.341062 65.21353 \n",
       "L 130.150277 67.663794 \n",
       "L 136.414754 66.193636 \n",
       "L 142.178074 67.07573 \n",
       "L 147.480328 66.546474 \n",
       "L 152.358402 66.864028 \n",
       "L 156.84623 66.673495 \n",
       "L 160.975031 66.787815 \n",
       "L 164.773529 66.719223 \n",
       "L 168.268147 66.760378 \n",
       "L 171.483195 66.735685 \n",
       "L 174.441039 66.750501 \n",
       "L 177.162256 66.741611 \n",
       "L 179.665776 66.746945 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"m380dc778f9\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#p2b6fedb403)\">\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m380dc778f9\" y=\"121.45082720588226\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"68.08437500000014\" xlink:href=\"#m380dc778f9\" y=\"33.921415441176464\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"79.31412500000019\" xlink:href=\"#m380dc778f9\" y=\"86.43906249999995\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"89.64549500000024\" xlink:href=\"#m380dc778f9\" y=\"54.928474264705855\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"99.15035540000027\" xlink:href=\"#m380dc778f9\" y=\"73.83482720588232\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"107.89482696800029\" xlink:href=\"#m380dc778f9\" y=\"62.491015441176444\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"115.93974081056032\" xlink:href=\"#m380dc778f9\" y=\"69.29730249999997\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"123.34106154571555\" xlink:href=\"#m380dc778f9\" y=\"65.21353026470585\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"130.15027662205836\" xlink:href=\"#m380dc778f9\" y=\"67.66379360588232\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"136.41475449229375\" xlink:href=\"#m380dc778f9\" y=\"66.19363560117644\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"142.1780741329103\" xlink:href=\"#m380dc778f9\" y=\"67.07573040399997\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"147.48032820227752\" xlink:href=\"#m380dc778f9\" y=\"66.54647352230585\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"152.35840194609537\" xlink:href=\"#m380dc778f9\" y=\"66.86402765132232\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"156.8462297904078\" xlink:href=\"#m380dc778f9\" y=\"66.67349517391244\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"160.9750314071752\" xlink:href=\"#m380dc778f9\" y=\"66.78781466035836\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"164.77352889460127\" xlink:href=\"#m380dc778f9\" y=\"66.71922296849081\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"168.2681465830332\" xlink:href=\"#m380dc778f9\" y=\"66.76037798361135\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"171.4831948563906\" xlink:href=\"#m380dc778f9\" y=\"66.73568497453903\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"174.4410392678794\" xlink:href=\"#m380dc778f9\" y=\"66.75050077998242\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"177.1622561264491\" xlink:href=\"#m380dc778f9\" y=\"66.74161129671639\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"179.66577563633322\" xlink:href=\"#m380dc778f9\" y=\"66.746944986676\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 235.920313 148.803768 \n",
       "L 235.920313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 40.620313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 235.920313 148.803768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 40.620313 9.303768 \n",
       "L 235.920313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m657dd1a578\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"m8a17cb6e31\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "       <path d=\"M 10.59375 35.5 \n",
       "L 73.1875 35.5 \n",
       "L 73.1875 27.203125 \n",
       "L 10.59375 27.203125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2212\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(48.5070312500001 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −4 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(79.0226562500002 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- −3 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(109.53828125000031 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- −2 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(140.0539062500004 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- −1 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(170.56953125000052 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#m657dd1a578\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#m8a17cb6e31\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(205.27500000000063 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- x1 -->\n",
       "     <defs>\n",
       "      <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-78\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(132.12968750000002 175.08033088235285)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"mbf27c79144\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"148.8037683823528\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"m5ce445b824\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"148.8037683823528\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −3 -->\n",
       "      <g transform=\"translate(21.878125000000004 151.56314338235282)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"121.45082720588226\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"121.45082720588226\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(21.878125000000004 124.21020220588227)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"94.09788602941171\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"94.09788602941171\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(21.878125000000004 96.85726102941172)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"66.74494485294115\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"66.74494485294115\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(30.257812500000004 69.50431985294115)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"39.39200367647058\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"39.39200367647058\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(30.257812500000004 42.151378676470586)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mbf27c79144\" y=\"12.039062500000021\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#m5ce445b824\" y=\"12.039062500000021\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(30.257812500000004 14.798437500000034)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- x2 -->\n",
       "     <g transform=\"translate(14.798437500000004 85.19439338235284)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p2b6fedb403\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"40.620312500000004\" y=\"9.303768382352843\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a9024ef98>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import sys\n",
    "sys.path.insert(0, '..')\n",
    "%matplotlib inline\n",
    "import d2l\n",
    "import numpy as np\n",
    "import torch.optim\n",
    "\n",
    "eta = 0.4\n",
    "\n",
    "def f_2d(x1, x2):\n",
    "    return 0.1 * x1 ** 2 + 2 * x2 ** 2\n",
    "\n",
    "def gd_2d(x1, x2, s1, s2):\n",
    "    return (x1 - eta * 0.2 * x1, x2 - eta * 4 * x2, 0, 0)\n",
    "\n",
    "d2l.show_trace_2d(f_2d, d2l.train_2d(gd_2d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, at the same position, the slope of the objective function has a larger absolute value in the vertical direction ($x_2$ axis direction) than in the horizontal direction ($x_1$ axis direction). Therefore, given the learning rate, using gradient descent for interaction will cause the independent variable to move more in the vertical direction than in the horizontal one. So we need a small learning rate to prevent the independent variable from overshooting the optimal solution for the objective function in the vertical direction. However, it will cause the independent variable to move slower toward the optimal solution in the horizontal direction.\n",
    "\n",
    "Now, we try to make the learning rate slightly larger, so the independent variable will continuously overshoot the optimal solution in the vertical direction and gradually diverge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 20, x1 -0.387814, x2 -1673.365109\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"182pt\" version=\"1.1\" viewBox=\"0 0 262 182\" width=\"262pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 182.25625 \n",
       "L 262.207813 182.25625 \n",
       "L 262.207813 -0 \n",
       "L 0 -0 \n",
       "L 0 182.25625 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 59.707813 146.7 \n",
       "L 255.007812 146.7 \n",
       "L 255.007812 7.2 \n",
       "L 59.707813 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_1\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 96.067244 \n",
       "L 251.95625 96.067244 \n",
       "L 248.904688 96.067244 \n",
       "L 245.853125 96.067244 \n",
       "L 242.801563 96.067245 \n",
       "L 239.75 96.067245 \n",
       "L 236.698438 96.067245 \n",
       "L 233.646875 96.067245 \n",
       "L 230.595313 96.067245 \n",
       "L 227.54375 96.067245 \n",
       "L 224.492188 96.067245 \n",
       "L 221.440625 96.067245 \n",
       "L 218.389063 96.067245 \n",
       "L 215.3375 96.067245 \n",
       "L 212.285938 96.067245 \n",
       "L 209.234375 96.067244 \n",
       "L 206.182813 96.067244 \n",
       "L 203.13125 96.067244 \n",
       "L 200.079688 96.067244 \n",
       "L 197.028125 96.067243 \n",
       "L 193.976562 96.067243 \n",
       "L 190.925 96.067242 \n",
       "L 187.873438 96.067242 \n",
       "L 184.821875 96.067241 \n",
       "L 181.770313 96.067241 \n",
       "L 178.71875 96.06724 \n",
       "L 175.667188 96.06724 \n",
       "L 172.615625 96.067239 \n",
       "L 169.564063 96.067238 \n",
       "L 166.5125 96.067237 \n",
       "L 163.460938 96.067237 \n",
       "L 160.409375 96.067236 \n",
       "L 157.357813 96.067235 \n",
       "L 154.30625 96.067234 \n",
       "L 151.254688 96.067233 \n",
       "L 148.203125 96.067232 \n",
       "L 145.151563 96.067231 \n",
       "L 142.1 96.06723 \n",
       "L 139.048438 96.067229 \n",
       "L 135.996875 96.067228 \n",
       "L 132.945312 96.067227 \n",
       "L 129.89375 96.067225 \n",
       "L 126.842188 96.067224 \n",
       "L 123.790625 96.067223 \n",
       "L 120.739063 96.067222 \n",
       "L 117.6875 96.06722 \n",
       "L 114.635938 96.067219 \n",
       "L 111.584375 96.067217 \n",
       "L 108.532813 96.067216 \n",
       "L 105.48125 96.067214 \n",
       "L 102.429688 96.067213 \n",
       "L 99.378125 96.067211 \n",
       "L 96.326563 96.06721 \n",
       "L 93.275 96.067208 \n",
       "L 90.223438 96.067206 \n",
       "L 87.171875 96.067205 \n",
       "L 84.120313 96.067203 \n",
       "L 81.06875 96.067201 \n",
       "L 78.017188 96.067199 \n",
       "L 74.965625 96.067197 \n",
       "L 71.914062 96.067195 \n",
       "L 68.8625 96.067193 \n",
       "L 65.810937 96.067191 \n",
       "L 62.759375 96.067189 \n",
       "L 59.707813 96.067187 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 59.707813 34.597543 \n",
       "L 62.759375 34.597541 \n",
       "L 65.810937 34.597539 \n",
       "L 68.8625 34.597537 \n",
       "L 71.914062 34.597535 \n",
       "L 74.965625 34.597533 \n",
       "L 78.017188 34.597531 \n",
       "L 81.06875 34.597529 \n",
       "L 84.120313 34.597528 \n",
       "L 87.171875 34.597526 \n",
       "L 90.223438 34.597524 \n",
       "L 93.275 34.597522 \n",
       "L 96.326563 34.597521 \n",
       "L 99.378125 34.597519 \n",
       "L 102.429688 34.597517 \n",
       "L 105.48125 34.597516 \n",
       "L 108.532813 34.597514 \n",
       "L 111.584375 34.597513 \n",
       "L 114.635938 34.597511 \n",
       "L 117.6875 34.59751 \n",
       "L 120.739063 34.597509 \n",
       "L 123.790625 34.597507 \n",
       "L 126.842188 34.597506 \n",
       "L 129.89375 34.597505 \n",
       "L 132.945312 34.597504 \n",
       "L 135.996875 34.597502 \n",
       "L 139.048438 34.597501 \n",
       "L 142.1 34.5975 \n",
       "L 145.151563 34.597499 \n",
       "L 148.203125 34.597498 \n",
       "L 151.254688 34.597497 \n",
       "L 154.30625 34.597496 \n",
       "L 157.357813 34.597495 \n",
       "L 160.409375 34.597494 \n",
       "L 163.460938 34.597494 \n",
       "L 166.5125 34.597493 \n",
       "L 169.564063 34.597492 \n",
       "L 172.615625 34.597491 \n",
       "L 175.667188 34.597491 \n",
       "L 178.71875 34.59749 \n",
       "L 181.770313 34.597489 \n",
       "L 184.821875 34.597489 \n",
       "L 187.873438 34.597488 \n",
       "L 190.925 34.597488 \n",
       "L 193.976562 34.597487 \n",
       "L 197.028125 34.597487 \n",
       "L 200.079688 34.597487 \n",
       "L 203.13125 34.597486 \n",
       "L 206.182813 34.597486 \n",
       "L 209.234375 34.597486 \n",
       "L 212.285938 34.597486 \n",
       "L 215.3375 34.597485 \n",
       "L 218.389063 34.597485 \n",
       "L 221.440625 34.597485 \n",
       "L 224.492188 34.597485 \n",
       "L 227.54375 34.597485 \n",
       "L 230.595313 34.597485 \n",
       "L 233.646875 34.597485 \n",
       "L 236.698438 34.597485 \n",
       "L 239.75 34.597485 \n",
       "L 242.801563 34.597486 \n",
       "L 245.853125 34.597486 \n",
       "L 248.904688 34.597486 \n",
       "L 251.95625 34.597486 \n",
       "L 255.007813 34.597487 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_2\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 108.798048 \n",
       "L 251.95625 108.798048 \n",
       "L 248.904688 108.798049 \n",
       "L 245.853125 108.798049 \n",
       "L 242.801563 108.798049 \n",
       "L 239.75 108.798049 \n",
       "L 236.698438 108.798049 \n",
       "L 233.646875 108.798049 \n",
       "L 230.595313 108.798049 \n",
       "L 227.54375 108.798049 \n",
       "L 224.492188 108.798049 \n",
       "L 221.440625 108.798049 \n",
       "L 218.389063 108.798049 \n",
       "L 215.3375 108.798049 \n",
       "L 212.285938 108.798049 \n",
       "L 209.234375 108.798049 \n",
       "L 206.182813 108.798049 \n",
       "L 203.13125 108.798048 \n",
       "L 200.079688 108.798048 \n",
       "L 197.028125 108.798048 \n",
       "L 193.976562 108.798048 \n",
       "L 190.925 108.798047 \n",
       "L 187.873438 108.798047 \n",
       "L 184.821875 108.798047 \n",
       "L 181.770313 108.798046 \n",
       "L 178.71875 108.798046 \n",
       "L 175.667188 108.798045 \n",
       "L 172.615625 108.798045 \n",
       "L 169.564063 108.798044 \n",
       "L 166.5125 108.798044 \n",
       "L 163.460938 108.798043 \n",
       "L 160.409375 108.798043 \n",
       "L 157.357813 108.798042 \n",
       "L 154.30625 108.798041 \n",
       "L 151.254688 108.798041 \n",
       "L 148.203125 108.79804 \n",
       "L 145.151563 108.798039 \n",
       "L 142.1 108.798039 \n",
       "L 139.048438 108.798038 \n",
       "L 135.996875 108.798037 \n",
       "L 132.945312 108.798036 \n",
       "L 129.89375 108.798035 \n",
       "L 126.842188 108.798035 \n",
       "L 123.790625 108.798034 \n",
       "L 120.739063 108.798033 \n",
       "L 117.6875 108.798032 \n",
       "L 114.635938 108.798031 \n",
       "L 111.584375 108.79803 \n",
       "L 108.532813 108.798029 \n",
       "L 105.48125 108.798028 \n",
       "L 102.429688 108.798026 \n",
       "L 99.378125 108.798025 \n",
       "L 96.326563 108.798024 \n",
       "L 93.275 108.798023 \n",
       "L 90.223438 108.798022 \n",
       "L 87.171875 108.798021 \n",
       "L 84.120313 108.798019 \n",
       "L 81.06875 108.798018 \n",
       "L 78.017188 108.798017 \n",
       "L 74.965625 108.798015 \n",
       "L 71.914062 108.798014 \n",
       "L 68.8625 108.798013 \n",
       "L 65.810937 108.798011 \n",
       "L 62.759375 108.79801 \n",
       "L 59.707813 108.798008 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 59.707813 21.866722 \n",
       "L 62.759375 21.866721 \n",
       "L 65.810938 21.866719 \n",
       "L 68.8625 21.866718 \n",
       "L 71.914062 21.866716 \n",
       "L 74.965625 21.866715 \n",
       "L 78.017188 21.866714 \n",
       "L 81.06875 21.866712 \n",
       "L 84.120313 21.866711 \n",
       "L 87.171875 21.86671 \n",
       "L 90.223438 21.866708 \n",
       "L 93.275 21.866707 \n",
       "L 96.326563 21.866706 \n",
       "L 99.378125 21.866705 \n",
       "L 102.429688 21.866704 \n",
       "L 105.48125 21.866703 \n",
       "L 108.532813 21.866702 \n",
       "L 111.584375 21.866701 \n",
       "L 114.635938 21.8667 \n",
       "L 117.6875 21.866699 \n",
       "L 120.739063 21.866698 \n",
       "L 123.790625 21.866697 \n",
       "L 126.842188 21.866696 \n",
       "L 129.89375 21.866695 \n",
       "L 132.945312 21.866694 \n",
       "L 135.996875 21.866693 \n",
       "L 139.048438 21.866692 \n",
       "L 142.1 21.866692 \n",
       "L 145.151563 21.866691 \n",
       "L 148.203125 21.86669 \n",
       "L 151.254688 21.866689 \n",
       "L 154.30625 21.866689 \n",
       "L 157.357813 21.866688 \n",
       "L 160.409375 21.866688 \n",
       "L 163.460938 21.866687 \n",
       "L 166.5125 21.866686 \n",
       "L 169.564063 21.866686 \n",
       "L 172.615625 21.866685 \n",
       "L 175.667188 21.866685 \n",
       "L 178.71875 21.866684 \n",
       "L 181.770313 21.866684 \n",
       "L 184.821875 21.866684 \n",
       "L 187.873438 21.866683 \n",
       "L 190.925 21.866683 \n",
       "L 193.976562 21.866683 \n",
       "L 197.028125 21.866682 \n",
       "L 200.079688 21.866682 \n",
       "L 203.13125 21.866682 \n",
       "L 206.182813 21.866682 \n",
       "L 209.234375 21.866681 \n",
       "L 212.285938 21.866681 \n",
       "L 215.3375 21.866681 \n",
       "L 218.389063 21.866681 \n",
       "L 221.440625 21.866681 \n",
       "L 224.492188 21.866681 \n",
       "L 227.54375 21.866681 \n",
       "L 230.595313 21.866681 \n",
       "L 233.646875 21.866681 \n",
       "L 236.698438 21.866681 \n",
       "L 239.75 21.866681 \n",
       "L 242.801563 21.866681 \n",
       "L 245.853125 21.866681 \n",
       "L 248.904688 21.866682 \n",
       "L 251.95625 21.866682 \n",
       "L 255.007812 21.866682 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_3\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 118.566738 \n",
       "L 251.95625 118.566738 \n",
       "L 248.904688 118.566738 \n",
       "L 245.853125 118.566739 \n",
       "L 242.801563 118.566739 \n",
       "L 239.75 118.566739 \n",
       "L 236.698438 118.566739 \n",
       "L 233.646875 118.566739 \n",
       "L 230.595313 118.566739 \n",
       "L 227.54375 118.566739 \n",
       "L 224.492188 118.566739 \n",
       "L 221.440625 118.566739 \n",
       "L 218.389063 118.566739 \n",
       "L 215.3375 118.566739 \n",
       "L 212.285938 118.566739 \n",
       "L 209.234375 118.566739 \n",
       "L 206.182813 118.566738 \n",
       "L 203.13125 118.566738 \n",
       "L 200.079688 118.566738 \n",
       "L 197.028125 118.566738 \n",
       "L 193.976562 118.566738 \n",
       "L 190.925 118.566737 \n",
       "L 187.873438 118.566737 \n",
       "L 184.821875 118.566737 \n",
       "L 181.770313 118.566736 \n",
       "L 178.71875 118.566736 \n",
       "L 175.667188 118.566736 \n",
       "L 172.615625 118.566735 \n",
       "L 169.564063 118.566735 \n",
       "L 166.5125 118.566735 \n",
       "L 163.460938 118.566734 \n",
       "L 160.409375 118.566734 \n",
       "L 157.357813 118.566733 \n",
       "L 154.30625 118.566733 \n",
       "L 151.254688 118.566732 \n",
       "L 148.203125 118.566731 \n",
       "L 145.151563 118.566731 \n",
       "L 142.1 118.56673 \n",
       "L 139.048438 118.56673 \n",
       "L 135.996875 118.566729 \n",
       "L 132.945312 118.566728 \n",
       "L 129.89375 118.566728 \n",
       "L 126.842188 118.566727 \n",
       "L 123.790625 118.566726 \n",
       "L 120.739063 118.566725 \n",
       "L 117.6875 118.566725 \n",
       "L 114.635938 118.566724 \n",
       "L 111.584375 118.566723 \n",
       "L 108.532813 118.566722 \n",
       "L 105.48125 118.566721 \n",
       "L 102.429688 118.56672 \n",
       "L 99.378125 118.566719 \n",
       "L 96.326563 118.566718 \n",
       "L 93.275 118.566717 \n",
       "L 90.223438 118.566716 \n",
       "L 87.171875 118.566715 \n",
       "L 84.120313 118.566714 \n",
       "L 81.06875 118.566713 \n",
       "L 78.017188 118.566712 \n",
       "L 74.965625 118.566711 \n",
       "L 71.914062 118.56671 \n",
       "L 68.8625 118.566709 \n",
       "L 65.810937 118.566708 \n",
       "L 62.759375 118.566707 \n",
       "L 59.707813 118.566705 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 59.707813 12.098025 \n",
       "L 62.759375 12.098024 \n",
       "L 65.810937 12.098022 \n",
       "L 68.8625 12.098021 \n",
       "L 71.914062 12.09802 \n",
       "L 74.965625 12.098019 \n",
       "L 78.017188 12.098018 \n",
       "L 81.06875 12.098017 \n",
       "L 84.120313 12.098016 \n",
       "L 87.171875 12.098015 \n",
       "L 90.223438 12.098014 \n",
       "L 93.275 12.098013 \n",
       "L 96.326563 12.098012 \n",
       "L 99.378125 12.098011 \n",
       "L 102.429688 12.09801 \n",
       "L 105.48125 12.098009 \n",
       "L 108.532813 12.098008 \n",
       "L 111.584375 12.098007 \n",
       "L 114.635938 12.098007 \n",
       "L 117.6875 12.098006 \n",
       "L 120.739063 12.098005 \n",
       "L 123.790625 12.098004 \n",
       "L 126.842188 12.098003 \n",
       "L 129.89375 12.098003 \n",
       "L 132.945312 12.098002 \n",
       "L 135.996875 12.098001 \n",
       "L 139.048438 12.098001 \n",
       "L 142.1 12.098 \n",
       "L 145.151563 12.097999 \n",
       "L 148.203125 12.097999 \n",
       "L 151.254688 12.097998 \n",
       "L 154.30625 12.097998 \n",
       "L 157.357813 12.097997 \n",
       "L 160.409375 12.097997 \n",
       "L 163.460938 12.097996 \n",
       "L 166.5125 12.097996 \n",
       "L 169.564063 12.097995 \n",
       "L 172.615625 12.097995 \n",
       "L 175.667188 12.097995 \n",
       "L 178.71875 12.097994 \n",
       "L 181.770313 12.097994 \n",
       "L 184.821875 12.097993 \n",
       "L 187.873438 12.097993 \n",
       "L 190.925 12.097993 \n",
       "L 193.976562 12.097993 \n",
       "L 197.028125 12.097992 \n",
       "L 200.079688 12.097992 \n",
       "L 203.13125 12.097992 \n",
       "L 206.182813 12.097992 \n",
       "L 209.234375 12.097992 \n",
       "L 212.285938 12.097992 \n",
       "L 215.3375 12.097991 \n",
       "L 218.389063 12.097991 \n",
       "L 221.440625 12.097991 \n",
       "L 224.492188 12.097991 \n",
       "L 227.54375 12.097991 \n",
       "L 230.595313 12.097991 \n",
       "L 233.646875 12.097991 \n",
       "L 236.698438 12.097991 \n",
       "L 239.75 12.097991 \n",
       "L 242.801563 12.097992 \n",
       "L 245.853125 12.097992 \n",
       "L 248.904688 12.097992 \n",
       "L 251.95625 12.097992 \n",
       "L 255.007813 12.097992 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_4\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 126.802124 \n",
       "L 251.95625 126.802125 \n",
       "L 248.904688 126.802125 \n",
       "L 245.853125 126.802125 \n",
       "L 242.801563 126.802125 \n",
       "L 239.75 126.802125 \n",
       "L 236.698438 126.802125 \n",
       "L 233.646875 126.802125 \n",
       "L 230.595313 126.802125 \n",
       "L 227.54375 126.802125 \n",
       "L 224.492188 126.802125 \n",
       "L 221.440625 126.802125 \n",
       "L 218.389063 126.802125 \n",
       "L 215.3375 126.802125 \n",
       "L 212.285938 126.802125 \n",
       "L 209.234375 126.802125 \n",
       "L 206.182813 126.802125 \n",
       "L 203.13125 126.802125 \n",
       "L 200.079688 126.802124 \n",
       "L 197.028125 126.802124 \n",
       "L 193.976562 126.802124 \n",
       "L 190.925 126.802124 \n",
       "L 187.873438 126.802124 \n",
       "L 184.821875 126.802123 \n",
       "L 181.770313 126.802123 \n",
       "L 178.71875 126.802123 \n",
       "L 175.667188 126.802122 \n",
       "L 172.615625 126.802122 \n",
       "L 169.564063 126.802122 \n",
       "L 166.5125 126.802121 \n",
       "L 163.460938 126.802121 \n",
       "L 160.409375 126.802121 \n",
       "L 157.357813 126.80212 \n",
       "L 154.30625 126.80212 \n",
       "L 151.254688 126.802119 \n",
       "L 148.203125 126.802119 \n",
       "L 145.151563 126.802118 \n",
       "L 142.1 126.802118 \n",
       "L 139.048438 126.802117 \n",
       "L 135.996875 126.802117 \n",
       "L 132.945312 126.802116 \n",
       "L 129.89375 126.802115 \n",
       "L 126.842188 126.802115 \n",
       "L 123.790625 126.802114 \n",
       "L 120.739063 126.802113 \n",
       "L 117.6875 126.802113 \n",
       "L 114.635938 126.802112 \n",
       "L 111.584375 126.802111 \n",
       "L 108.532813 126.802111 \n",
       "L 105.48125 126.80211 \n",
       "L 102.429688 126.802109 \n",
       "L 99.378125 126.802108 \n",
       "L 96.326563 126.802107 \n",
       "L 93.275 126.802107 \n",
       "L 90.223438 126.802106 \n",
       "L 87.171875 126.802105 \n",
       "L 84.120313 126.802104 \n",
       "L 81.06875 126.802103 \n",
       "L 78.017188 126.802102 \n",
       "L 74.965625 126.802101 \n",
       "L 71.914062 126.8021 \n",
       "L 68.8625 126.802099 \n",
       "L 65.810937 126.802098 \n",
       "L 62.759375 126.802097 \n",
       "L 59.707813 126.802096 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_5\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 134.057646 \n",
       "L 251.95625 134.057646 \n",
       "L 248.904688 134.057646 \n",
       "L 245.853125 134.057646 \n",
       "L 242.801563 134.057646 \n",
       "L 239.75 134.057646 \n",
       "L 236.698438 134.057646 \n",
       "L 233.646875 134.057646 \n",
       "L 230.595313 134.057646 \n",
       "L 227.54375 134.057646 \n",
       "L 224.492188 134.057646 \n",
       "L 221.440625 134.057646 \n",
       "L 218.389063 134.057646 \n",
       "L 215.3375 134.057646 \n",
       "L 212.285938 134.057646 \n",
       "L 209.234375 134.057646 \n",
       "L 206.182813 134.057646 \n",
       "L 203.13125 134.057646 \n",
       "L 200.079688 134.057646 \n",
       "L 197.028125 134.057645 \n",
       "L 193.976562 134.057645 \n",
       "L 190.925 134.057645 \n",
       "L 187.873438 134.057645 \n",
       "L 184.821875 134.057645 \n",
       "L 181.770313 134.057644 \n",
       "L 178.71875 134.057644 \n",
       "L 175.667188 134.057644 \n",
       "L 172.615625 134.057643 \n",
       "L 169.564063 134.057643 \n",
       "L 166.5125 134.057643 \n",
       "L 163.460938 134.057642 \n",
       "L 160.409375 134.057642 \n",
       "L 157.357813 134.057642 \n",
       "L 154.30625 134.057641 \n",
       "L 151.254688 134.057641 \n",
       "L 148.203125 134.05764 \n",
       "L 145.151563 134.05764 \n",
       "L 142.1 134.057639 \n",
       "L 139.048438 134.057639 \n",
       "L 135.996875 134.057638 \n",
       "L 132.945312 134.057638 \n",
       "L 129.89375 134.057637 \n",
       "L 126.842188 134.057637 \n",
       "L 123.790625 134.057636 \n",
       "L 120.739063 134.057636 \n",
       "L 117.6875 134.057635 \n",
       "L 114.635938 134.057634 \n",
       "L 111.584375 134.057634 \n",
       "L 108.532813 134.057633 \n",
       "L 105.48125 134.057632 \n",
       "L 102.429688 134.057632 \n",
       "L 99.378125 134.057631 \n",
       "L 96.326563 134.05763 \n",
       "L 93.275 134.05763 \n",
       "L 90.223438 134.057629 \n",
       "L 87.171875 134.057628 \n",
       "L 84.120313 134.057627 \n",
       "L 81.06875 134.057626 \n",
       "L 78.017188 134.057626 \n",
       "L 74.965625 134.057625 \n",
       "L 71.914062 134.057624 \n",
       "L 68.8625 134.057623 \n",
       "L 65.810937 134.057622 \n",
       "L 62.759375 134.057621 \n",
       "L 59.707813 134.05762 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_6\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 140.617138 \n",
       "L 251.95625 140.617138 \n",
       "L 248.904688 140.617138 \n",
       "L 245.853125 140.617138 \n",
       "L 242.801563 140.617138 \n",
       "L 239.75 140.617138 \n",
       "L 236.698438 140.617139 \n",
       "L 233.646875 140.617139 \n",
       "L 230.595313 140.617139 \n",
       "L 227.54375 140.617139 \n",
       "L 224.492188 140.617139 \n",
       "L 221.440625 140.617139 \n",
       "L 218.389063 140.617139 \n",
       "L 215.3375 140.617138 \n",
       "L 212.285938 140.617138 \n",
       "L 209.234375 140.617138 \n",
       "L 206.182813 140.617138 \n",
       "L 203.13125 140.617138 \n",
       "L 200.079688 140.617138 \n",
       "L 197.028125 140.617138 \n",
       "L 193.976562 140.617138 \n",
       "L 190.925 140.617137 \n",
       "L 187.873438 140.617137 \n",
       "L 184.821875 140.617137 \n",
       "L 181.770313 140.617137 \n",
       "L 178.71875 140.617137 \n",
       "L 175.667188 140.617136 \n",
       "L 172.615625 140.617136 \n",
       "L 169.564063 140.617136 \n",
       "L 166.5125 140.617135 \n",
       "L 163.460938 140.617135 \n",
       "L 160.409375 140.617135 \n",
       "L 157.357813 140.617134 \n",
       "L 154.30625 140.617134 \n",
       "L 151.254688 140.617134 \n",
       "L 148.203125 140.617133 \n",
       "L 145.151563 140.617133 \n",
       "L 142.1 140.617132 \n",
       "L 139.048438 140.617132 \n",
       "L 135.996875 140.617132 \n",
       "L 132.945312 140.617131 \n",
       "L 129.89375 140.617131 \n",
       "L 126.842188 140.61713 \n",
       "L 123.790625 140.61713 \n",
       "L 120.739063 140.617129 \n",
       "L 117.6875 140.617128 \n",
       "L 114.635938 140.617128 \n",
       "L 111.584375 140.617127 \n",
       "L 108.532813 140.617127 \n",
       "L 105.48125 140.617126 \n",
       "L 102.429688 140.617125 \n",
       "L 99.378125 140.617125 \n",
       "L 96.326563 140.617124 \n",
       "L 93.275 140.617123 \n",
       "L 90.223438 140.617123 \n",
       "L 87.171875 140.617122 \n",
       "L 84.120313 140.617121 \n",
       "L 81.06875 140.617121 \n",
       "L 78.017188 140.61712 \n",
       "L 74.965625 140.617119 \n",
       "L 71.914062 140.617118 \n",
       "L 68.8625 140.617117 \n",
       "L 65.810937 140.617117 \n",
       "L 62.759375 140.617116 \n",
       "L 59.707813 140.617115 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_7\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 255.007813 146.649214 \n",
       "L 251.95625 146.649214 \n",
       "L 248.904688 146.649214 \n",
       "L 245.853125 146.649214 \n",
       "L 242.801563 146.649214 \n",
       "L 239.75 146.649214 \n",
       "L 236.698438 146.649214 \n",
       "L 233.646875 146.649214 \n",
       "L 230.595313 146.649214 \n",
       "L 227.54375 146.649214 \n",
       "L 224.492188 146.649214 \n",
       "L 221.440625 146.649214 \n",
       "L 218.389063 146.649214 \n",
       "L 215.3375 146.649214 \n",
       "L 212.285938 146.649214 \n",
       "L 209.234375 146.649214 \n",
       "L 206.182813 146.649214 \n",
       "L 203.13125 146.649214 \n",
       "L 200.079688 146.649214 \n",
       "L 197.028125 146.649214 \n",
       "L 193.976562 146.649213 \n",
       "L 190.925 146.649213 \n",
       "L 187.873438 146.649213 \n",
       "L 184.821875 146.649213 \n",
       "L 181.770313 146.649213 \n",
       "L 178.71875 146.649212 \n",
       "L 175.667188 146.649212 \n",
       "L 172.615625 146.649212 \n",
       "L 169.564063 146.649212 \n",
       "L 166.5125 146.649211 \n",
       "L 163.460938 146.649211 \n",
       "L 160.409375 146.649211 \n",
       "L 157.357813 146.64921 \n",
       "L 154.30625 146.64921 \n",
       "L 151.254688 146.64921 \n",
       "L 148.203125 146.649209 \n",
       "L 145.151563 146.649209 \n",
       "L 142.1 146.649209 \n",
       "L 139.048438 146.649208 \n",
       "L 135.996875 146.649208 \n",
       "L 132.945312 146.649207 \n",
       "L 129.89375 146.649207 \n",
       "L 126.842188 146.649206 \n",
       "L 123.790625 146.649206 \n",
       "L 120.739063 146.649205 \n",
       "L 117.6875 146.649205 \n",
       "L 114.635938 146.649204 \n",
       "L 111.584375 146.649204 \n",
       "L 108.532812 146.649203 \n",
       "L 105.48125 146.649203 \n",
       "L 102.429688 146.649202 \n",
       "L 99.378125 146.649202 \n",
       "L 96.326563 146.649201 \n",
       "L 93.275 146.6492 \n",
       "L 90.223438 146.6492 \n",
       "L 87.171875 146.649199 \n",
       "L 84.120313 146.649198 \n",
       "L 81.06875 146.649198 \n",
       "L 78.017188 146.649197 \n",
       "L 74.965625 146.649196 \n",
       "L 71.914062 146.649195 \n",
       "L 68.8625 146.649195 \n",
       "L 65.810937 146.649194 \n",
       "L 62.759375 146.649193 \n",
       "L 59.707813 146.649192 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p4fa83b147e)\" d=\"M 74.965625 65.429557 \n",
       "L 93.275 65.196296 \n",
       "L 109.38725 65.522862 \n",
       "L 123.56603 65.06567 \n",
       "L 136.043356 65.705739 \n",
       "L 147.023404 64.809642 \n",
       "L 156.685845 66.064177 \n",
       "L 165.188794 64.307828 \n",
       "L 172.671389 66.766717 \n",
       "L 179.256072 63.324272 \n",
       "L 185.050593 68.143695 \n",
       "L 190.149772 61.396503 \n",
       "L 194.637049 70.842573 \n",
       "L 198.585853 57.618075 \n",
       "L 202.060801 76.132372 \n",
       "L 205.118755 50.212356 \n",
       "L 207.809754 86.500378 \n",
       "L 210.177834 35.697147 \n",
       "L 212.261744 106.821671 \n",
       "L 214.095585 7.247337 \n",
       "L 215.709364 146.651404 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"ma38edb0266\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#p4fa83b147e)\">\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"74.96562500000007\" xlink:href=\"#ma38edb0266\" y=\"65.42955736196033\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"93.27500000000012\" xlink:href=\"#ma38edb0266\" y=\"65.19629602286724\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"109.38725000000018\" xlink:href=\"#ma38edb0266\" y=\"65.52286189759756\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"123.56603000000024\" xlink:href=\"#ma38edb0266\" y=\"65.06566967297512\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"136.04335640000028\" xlink:href=\"#ma38edb0266\" y=\"65.70573878744655\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"147.02340363200034\" xlink:href=\"#ma38edb0266\" y=\"64.80964202718656\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"156.68584519616036\" xlink:href=\"#ma38edb0266\" y=\"66.06417749155054\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"165.18879377262118\" xlink:href=\"#ma38edb0266\" y=\"64.30782784144095\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"172.6713885199067\" xlink:href=\"#ma38edb0266\" y=\"66.76671735159438\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"179.25607189751798\" xlink:href=\"#ma38edb0266\" y=\"63.324272037379586\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"185.0505932698159\" xlink:href=\"#ma38edb0266\" y=\"68.14369547728029\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"190.14977207743806\" xlink:href=\"#ma38edb0266\" y=\"61.396502661419305\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"194.63704942814556\" xlink:href=\"#ma38edb0266\" y=\"70.84257260362469\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"198.58585349676818\" xlink:href=\"#ma38edb0266\" y=\"57.61807468453715\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"202.06080107715607\" xlink:href=\"#ma38edb0266\" y=\"76.13237177125971\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"205.1187549478974\" xlink:href=\"#ma38edb0266\" y=\"50.21235584984812\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"207.8097543541498\" xlink:href=\"#ma38edb0266\" y=\"86.50037813982435\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"210.1778338316519\" xlink:href=\"#ma38edb0266\" y=\"35.69714693385763\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"212.26174377185373\" xlink:href=\"#ma38edb0266\" y=\"106.82167062221103\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"214.09558451923135\" xlink:href=\"#ma38edb0266\" y=\"7.24733745851627\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"215.70936437692367\" xlink:href=\"#ma38edb0266\" y=\"146.65140388768893\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 255.007812 146.7 \n",
       "L 255.007812 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 59.707813 146.7 \n",
       "L 59.707813 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 59.707813 146.7 \n",
       "L 255.007813 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 59.707813 7.2 \n",
       "L 255.007813 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"mbeac6b1f2c\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"74.96562500000007\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"m1d26af978f\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"74.96562500000007\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "       <path d=\"M 10.59375 35.5 \n",
       "L 73.1875 35.5 \n",
       "L 73.1875 27.203125 \n",
       "L 10.59375 27.203125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2212\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(67.59453125000007 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"105.48125000000017\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"105.48125000000017\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −4 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(98.11015625000017 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"135.99687500000027\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"135.99687500000027\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- −3 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(128.62578125000027 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"166.5125000000004\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"166.5125000000004\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- −2 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(159.1414062500004 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"197.0281250000005\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"197.0281250000005\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- −1 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(189.6570312500005 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"227.5437500000006\" xlink:href=\"#mbeac6b1f2c\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"227.5437500000006\" xlink:href=\"#m1d26af978f\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(224.3625000000006 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- x1 -->\n",
       "     <defs>\n",
       "      <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-78\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(151.21718750000002 172.9765625)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"m3e4662344c\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"138.22653360392718\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"meece7cf274\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"138.22653360392718\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −1500 -->\n",
       "      <g transform=\"translate(21.878125000000004 140.9859086039272)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"147.412109375\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "       <use x=\"211.03515625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"274.658203125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"113.92847744839753\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"113.92847744839753\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- −1000 -->\n",
       "      <g transform=\"translate(21.878125000000004 116.68785244839754)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"147.412109375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"211.03515625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"274.658203125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"89.63042129286788\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"89.63042129286788\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- −500 -->\n",
       "      <g transform=\"translate(28.240625 92.38979629286789)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "       <use x=\"147.412109375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"211.03515625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"65.33236513733821\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"65.33236513733821\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(49.345312500000006 68.09174013733822)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"41.03430898180855\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"41.03430898180855\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 500 -->\n",
       "      <g transform=\"translate(36.6203125 43.793683981808556)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"127.24609375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"59.7078125\" xlink:href=\"#m3e4662344c\" y=\"16.736252826278893\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"255.0078125\" xlink:href=\"#meece7cf274\" y=\"16.736252826278893\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 1000 -->\n",
       "      <g transform=\"translate(30.2578125 19.495627826278906)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"127.24609375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"190.869140625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- x2 -->\n",
       "     <g transform=\"translate(14.798437500000004 83.09062499999999)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p4fa83b147e\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"59.7078125\" y=\"7.199999999999989\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a915613c8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "eta = 0.6\n",
    "d2l.show_trace_2d(f_2d, d2l.train_2d(gd_2d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Momentum Method\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The momentum method was proposed to solve the gradient descent problem described above. Since mini-batch stochastic gradient descent is more general than gradient descent, the subsequent discussion in this chapter will continue to use the definition for mini-batch stochastic gradient descent $\\mathbf{g}_t$ at time step $t$ given in :numref:chapter_minibatch_sgd. We set the independent variable at time step $t$ to $\\mathbf{x}_t$ and the learning rate to $\\eta_t$. At time step $0$, momentum creates the velocity variable $\\mathbf{v}_0$ and initializes its elements to zero. At time step $t>0$, momentum modifies the steps of each iteration as follows:\n",
    "\n",
    "$$ \\begin{aligned} \\mathbf{v}t &\\leftarrow \\gamma \\mathbf{v}{t-1} + \\eta_t \\mathbf{g}_t, \\ \\mathbf{x}t &\\leftarrow \\mathbf{x}{t-1} - \\mathbf{v}_t, \\end{aligned} $$\n",
    "\n",
    "Here, the momentum hyperparameter $\\gamma$ satisfies $0 \\leq \\gamma < 1$. When $\\gamma=0$, momentum is equivalent to a mini-batch SGD.\n",
    "\n",
    "Before explaining the mathematical principles behind the momentum method, we should take a look at the iterative trajectory of the gradient descent after using momentum in the experiment.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 20, x1 -0.062843, x2 0.001202\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"184pt\" version=\"1.1\" viewBox=\"0 0 243 184\" width=\"243pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 184.360018 \n",
       "L 243.120313 184.360018 \n",
       "L 243.120313 -0 \n",
       "L 0 -0 \n",
       "L 0 184.360018 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 235.920313 148.803768 \n",
       "L 235.920313 9.303768 \n",
       "L 40.620313 9.303768 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_1\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 99.781827 \n",
       "L 220.6625 100.137415 \n",
       "L 205.404688 100.219474 \n",
       "L 190.146875 100.028004 \n",
       "L 174.889063 99.562528 \n",
       "L 162.682813 98.944114 \n",
       "L 150.476563 98.135418 \n",
       "L 138.270313 97.136441 \n",
       "L 129.115625 96.20797 \n",
       "L 116.909375 94.749147 \n",
       "L 107.754688 93.457251 \n",
       "L 95.100875 91.362592 \n",
       "L 83.342188 88.941052 \n",
       "L 80.290625 88.226121 \n",
       "L 71.135938 85.839402 \n",
       "L 61.98125 82.858314 \n",
       "L 55.878125 80.421415 \n",
       "L 50.486044 77.686121 \n",
       "L 46.409725 74.950827 \n",
       "L 43.559891 72.215533 \n",
       "L 41.880132 69.480239 \n",
       "L 41.320212 66.744945 \n",
       "L 41.880132 64.009651 \n",
       "L 43.671875 61.164945 \n",
       "L 46.723438 58.324147 \n",
       "L 50.486044 55.803768 \n",
       "L 55.878125 53.068474 \n",
       "L 62.752171 50.33318 \n",
       "L 71.307374 47.597886 \n",
       "L 81.908321 44.862592 \n",
       "L 92.496875 42.642177 \n",
       "L 98.6 41.522654 \n",
       "L 111.96875 39.392004 \n",
       "L 123.0125 37.985281 \n",
       "L 135.21875 36.632924 \n",
       "L 147.425 35.586377 \n",
       "L 159.63125 34.730111 \n",
       "L 171.8375 34.064126 \n",
       "L 184.04375 33.615063 \n",
       "L 199.301563 33.31418 \n",
       "L 214.559375 33.286827 \n",
       "L 229.817188 33.533004 \n",
       "L 235.920313 33.708062 \n",
       "L 235.920313 33.708062 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_2\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 113.788096 \n",
       "L 232.86875 113.854525 \n",
       "L 229.817188 113.913138 \n",
       "L 226.765625 113.963936 \n",
       "L 223.714063 114.00692 \n",
       "L 220.6625 114.042088 \n",
       "L 217.610938 114.069441 \n",
       "L 214.559375 114.088978 \n",
       "L 211.507813 114.100701 \n",
       "L 208.45625 114.104609 \n",
       "L 205.404688 114.100701 \n",
       "L 202.353125 114.088978 \n",
       "L 199.301563 114.069441 \n",
       "L 196.25 114.042088 \n",
       "L 193.198438 114.00692 \n",
       "L 190.146875 113.963936 \n",
       "L 187.095313 113.913138 \n",
       "L 184.04375 113.854525 \n",
       "L 180.992188 113.788096 \n",
       "L 177.940625 113.713852 \n",
       "L 174.889063 113.631794 \n",
       "L 171.8375 113.54192 \n",
       "L 168.785938 113.444231 \n",
       "L 165.734375 113.338726 \n",
       "L 163.208944 113.244945 \n",
       "L 162.682813 113.224223 \n",
       "L 159.63125 113.095747 \n",
       "L 156.579688 112.958982 \n",
       "L 153.528125 112.813929 \n",
       "L 150.476563 112.660587 \n",
       "L 147.425 112.498956 \n",
       "L 144.373438 112.329036 \n",
       "L 141.321875 112.150827 \n",
       "L 138.270313 111.96433 \n",
       "L 135.21875 111.769544 \n",
       "L 132.167188 111.566469 \n",
       "L 129.115625 111.355105 \n",
       "L 126.064063 111.135453 \n",
       "L 123.0125 110.907512 \n",
       "L 119.960938 110.671282 \n",
       "L 117.943803 110.509651 \n",
       "L 116.909375 110.421415 \n",
       "L 113.857813 110.152298 \n",
       "L 110.80625 109.874357 \n",
       "L 107.754688 109.587592 \n",
       "L 104.703125 109.292004 \n",
       "L 101.651563 108.987592 \n",
       "L 98.6 108.674357 \n",
       "L 95.548438 108.352298 \n",
       "L 92.496875 108.021415 \n",
       "L 90.277557 107.774357 \n",
       "L 89.445313 107.67532 \n",
       "L 86.39375 107.302754 \n",
       "L 83.342188 106.920756 \n",
       "L 80.290625 106.529326 \n",
       "L 77.239063 106.128464 \n",
       "L 74.1875 105.71817 \n",
       "L 71.135938 105.298444 \n",
       "L 69.291587 105.039062 \n",
       "L 68.084375 104.85671 \n",
       "L 65.032813 104.385631 \n",
       "L 61.98125 103.904422 \n",
       "L 58.929688 103.413082 \n",
       "L 55.878125 102.911612 \n",
       "L 52.826563 102.40001 \n",
       "L 52.263653 102.303768 \n",
       "L 49.775 101.844239 \n",
       "L 46.723438 101.269827 \n",
       "L 43.671875 100.684474 \n",
       "L 40.620313 100.08818 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 40.620313 33.40171 \n",
       "L 43.671875 32.805415 \n",
       "L 46.723438 32.220063 \n",
       "L 49.775 31.645651 \n",
       "L 52.263653 31.186121 \n",
       "L 52.826563 31.089879 \n",
       "L 55.878125 30.578278 \n",
       "L 58.929688 30.076808 \n",
       "L 61.98125 29.585468 \n",
       "L 65.032813 29.104259 \n",
       "L 68.084375 28.63318 \n",
       "L 69.291587 28.450827 \n",
       "L 71.135938 28.191446 \n",
       "L 74.1875 27.77172 \n",
       "L 77.239063 27.361426 \n",
       "L 80.290625 26.960564 \n",
       "L 83.342188 26.569133 \n",
       "L 86.39375 26.187136 \n",
       "L 89.445313 25.81457 \n",
       "L 90.277557 25.715533 \n",
       "L 92.496875 25.468474 \n",
       "L 95.548438 25.137592 \n",
       "L 98.6 24.815533 \n",
       "L 101.651563 24.502298 \n",
       "L 104.703125 24.197886 \n",
       "L 107.754688 23.902298 \n",
       "L 110.80625 23.615533 \n",
       "L 113.857813 23.337592 \n",
       "L 116.909375 23.068474 \n",
       "L 117.943803 22.980239 \n",
       "L 119.960938 22.818608 \n",
       "L 123.0125 22.582378 \n",
       "L 126.064063 22.354437 \n",
       "L 129.115625 22.134784 \n",
       "L 132.167188 21.923421 \n",
       "L 135.21875 21.720346 \n",
       "L 138.270313 21.52556 \n",
       "L 141.321875 21.339063 \n",
       "L 144.373438 21.160854 \n",
       "L 147.425 20.990934 \n",
       "L 150.476563 20.829303 \n",
       "L 153.528125 20.675961 \n",
       "L 156.579688 20.530907 \n",
       "L 159.63125 20.394143 \n",
       "L 162.682813 20.265667 \n",
       "L 163.208944 20.244945 \n",
       "L 165.734375 20.151163 \n",
       "L 168.785938 20.045659 \n",
       "L 171.8375 19.94797 \n",
       "L 174.889063 19.858096 \n",
       "L 177.940625 19.776037 \n",
       "L 180.992188 19.701794 \n",
       "L 184.04375 19.635365 \n",
       "L 187.095313 19.576752 \n",
       "L 190.146875 19.525953 \n",
       "L 193.198438 19.48297 \n",
       "L 196.25 19.447802 \n",
       "L 199.301563 19.420449 \n",
       "L 202.353125 19.400911 \n",
       "L 205.404688 19.389189 \n",
       "L 208.45625 19.385281 \n",
       "L 211.507813 19.389189 \n",
       "L 214.559375 19.400911 \n",
       "L 217.610938 19.420449 \n",
       "L 220.6625 19.447802 \n",
       "L 223.714063 19.48297 \n",
       "L 226.765625 19.525953 \n",
       "L 229.817188 19.576752 \n",
       "L 232.86875 19.635365 \n",
       "L 235.920313 19.701794 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_3\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 124.500998 \n",
       "L 232.86875 124.555068 \n",
       "L 229.817188 124.602777 \n",
       "L 226.765625 124.644124 \n",
       "L 223.714063 124.67911 \n",
       "L 220.6625 124.707736 \n",
       "L 217.610938 124.73 \n",
       "L 214.559375 124.745902 \n",
       "L 211.507813 124.755444 \n",
       "L 208.45625 124.758625 \n",
       "L 205.404688 124.755444 \n",
       "L 202.353125 124.745902 \n",
       "L 199.301563 124.73 \n",
       "L 196.25 124.707736 \n",
       "L 193.198438 124.67911 \n",
       "L 190.146875 124.644124 \n",
       "L 187.095313 124.602777 \n",
       "L 184.04375 124.555068 \n",
       "L 180.992188 124.500998 \n",
       "L 177.940625 124.440567 \n",
       "L 174.889063 124.373775 \n",
       "L 171.8375 124.300622 \n",
       "L 168.785938 124.221108 \n",
       "L 167.542708 124.186121 \n",
       "L 165.734375 124.13275 \n",
       "L 162.682813 124.036014 \n",
       "L 159.63125 123.932606 \n",
       "L 156.579688 123.822527 \n",
       "L 153.528125 123.705777 \n",
       "L 150.476563 123.582355 \n",
       "L 147.425 123.452262 \n",
       "L 144.373438 123.315497 \n",
       "L 141.321875 123.172061 \n",
       "L 138.270313 123.021953 \n",
       "L 135.21875 122.865174 \n",
       "L 132.167188 122.701724 \n",
       "L 129.115625 122.531602 \n",
       "L 126.064063 122.354809 \n",
       "L 123.0125 122.171344 \n",
       "L 119.960938 121.981207 \n",
       "L 116.909375 121.7844 \n",
       "L 113.857813 121.58092 \n",
       "L 111.96875 121.450827 \n",
       "L 110.80625 121.366664 \n",
       "L 107.754688 121.138723 \n",
       "L 104.703125 120.903768 \n",
       "L 101.651563 120.6618 \n",
       "L 98.6 120.412818 \n",
       "L 95.548438 120.156823 \n",
       "L 92.496875 119.893814 \n",
       "L 89.445313 119.623791 \n",
       "L 86.39375 119.346755 \n",
       "L 83.342188 119.062705 \n",
       "L 80.290625 118.771642 \n",
       "L 79.716213 118.715533 \n",
       "L 77.239063 118.460485 \n",
       "L 74.1875 118.138904 \n",
       "L 71.135938 117.809929 \n",
       "L 68.084375 117.473562 \n",
       "L 65.032813 117.129802 \n",
       "L 61.98125 116.778649 \n",
       "L 58.929688 116.420104 \n",
       "L 55.878125 116.054166 \n",
       "L 55.273855 115.980239 \n",
       "L 52.826563 115.663726 \n",
       "L 49.775 115.261247 \n",
       "L 46.723438 114.850953 \n",
       "L 43.671875 114.432844 \n",
       "L 40.620313 114.00692 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 40.620313 19.48297 \n",
       "L 43.671875 19.057046 \n",
       "L 46.723438 18.638936 \n",
       "L 49.775 18.228642 \n",
       "L 52.826563 17.826163 \n",
       "L 55.273855 17.509651 \n",
       "L 55.878125 17.435724 \n",
       "L 58.929688 17.069786 \n",
       "L 61.98125 16.711241 \n",
       "L 65.032813 16.360088 \n",
       "L 68.084375 16.016328 \n",
       "L 71.135938 15.679961 \n",
       "L 74.1875 15.350986 \n",
       "L 77.239063 15.029404 \n",
       "L 79.716213 14.774357 \n",
       "L 80.290625 14.718248 \n",
       "L 83.342188 14.427185 \n",
       "L 86.39375 14.143135 \n",
       "L 89.445313 13.866099 \n",
       "L 92.496875 13.596076 \n",
       "L 95.548438 13.333067 \n",
       "L 98.6 13.077072 \n",
       "L 101.651563 12.82809 \n",
       "L 104.703125 12.586121 \n",
       "L 107.754688 12.351167 \n",
       "L 110.80625 12.123225 \n",
       "L 111.96875 12.039062 \n",
       "L 113.857813 11.908969 \n",
       "L 116.909375 11.70549 \n",
       "L 119.960938 11.508682 \n",
       "L 123.0125 11.318546 \n",
       "L 126.064063 11.135081 \n",
       "L 129.115625 10.958288 \n",
       "L 132.167188 10.788166 \n",
       "L 135.21875 10.624715 \n",
       "L 138.270313 10.467936 \n",
       "L 141.321875 10.317829 \n",
       "L 144.373438 10.174392 \n",
       "L 147.425 10.037628 \n",
       "L 150.476563 9.907535 \n",
       "L 153.528125 9.784113 \n",
       "L 156.579688 9.667362 \n",
       "L 159.63125 9.557283 \n",
       "L 162.682813 9.453876 \n",
       "L 165.734375 9.35714 \n",
       "L 167.542708 9.303768 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_4\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 133.505659 \n",
       "L 232.86875 133.553108 \n",
       "L 229.817188 133.594975 \n",
       "L 226.765625 133.631259 \n",
       "L 223.714063 133.661962 \n",
       "L 220.6625 133.687082 \n",
       "L 217.610938 133.70662 \n",
       "L 214.559375 133.720575 \n",
       "L 211.507813 133.728948 \n",
       "L 208.45625 133.73174 \n",
       "L 205.404688 133.728948 \n",
       "L 202.353125 133.720575 \n",
       "L 199.301563 133.70662 \n",
       "L 196.25 133.687082 \n",
       "L 193.198438 133.661962 \n",
       "L 190.146875 133.631259 \n",
       "L 187.095313 133.594975 \n",
       "L 184.04375 133.553108 \n",
       "L 180.992188 133.505659 \n",
       "L 177.940625 133.452628 \n",
       "L 174.889063 133.394014 \n",
       "L 171.8375 133.329819 \n",
       "L 168.785938 133.260041 \n",
       "L 165.734375 133.184681 \n",
       "L 162.682813 133.103738 \n",
       "L 159.63125 133.017214 \n",
       "L 156.579688 132.925107 \n",
       "L 153.528125 132.827418 \n",
       "L 150.476563 132.724147 \n",
       "L 147.425 132.615293 \n",
       "L 144.373438 132.500857 \n",
       "L 141.605741 132.392004 \n",
       "L 141.321875 132.380364 \n",
       "L 138.270313 132.249419 \n",
       "L 135.21875 132.112654 \n",
       "L 132.167188 131.97007 \n",
       "L 129.115625 131.821666 \n",
       "L 126.064063 131.667442 \n",
       "L 123.0125 131.507398 \n",
       "L 119.960938 131.341534 \n",
       "L 116.909375 131.169851 \n",
       "L 113.857813 130.992348 \n",
       "L 110.80625 130.809025 \n",
       "L 107.754688 130.619882 \n",
       "L 104.703125 130.42492 \n",
       "L 101.651563 130.224138 \n",
       "L 98.6 130.017536 \n",
       "L 95.548438 129.805114 \n",
       "L 93.473375 129.65671 \n",
       "L 92.496875 129.583768 \n",
       "L 89.445313 129.349749 \n",
       "L 86.39375 129.109651 \n",
       "L 83.342188 128.863474 \n",
       "L 80.290625 128.611219 \n",
       "L 77.239063 128.352886 \n",
       "L 74.1875 128.088474 \n",
       "L 71.135938 127.817984 \n",
       "L 68.084375 127.541415 \n",
       "L 65.032813 127.258768 \n",
       "L 61.98125 126.970043 \n",
       "L 61.477899 126.921415 \n",
       "L 58.929688 126.663789 \n",
       "L 55.878125 126.348912 \n",
       "L 52.826563 126.027674 \n",
       "L 49.775 125.700075 \n",
       "L 46.723438 125.366114 \n",
       "L 43.671875 125.025793 \n",
       "L 40.620313 124.67911 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_5\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 141.440854 \n",
       "L 232.86875 141.483127 \n",
       "L 229.817188 141.520426 \n",
       "L 226.765625 141.552752 \n",
       "L 223.714063 141.580105 \n",
       "L 220.6625 141.602485 \n",
       "L 217.610938 141.619891 \n",
       "L 214.559375 141.632325 \n",
       "L 211.507813 141.639784 \n",
       "L 208.45625 141.642271 \n",
       "L 205.404688 141.639784 \n",
       "L 202.353125 141.632325 \n",
       "L 199.301563 141.619891 \n",
       "L 196.25 141.602485 \n",
       "L 193.198438 141.580105 \n",
       "L 190.146875 141.552752 \n",
       "L 187.095313 141.520426 \n",
       "L 184.04375 141.483127 \n",
       "L 180.992188 141.440854 \n",
       "L 177.940625 141.393608 \n",
       "L 174.889063 141.341389 \n",
       "L 171.8375 141.284196 \n",
       "L 168.785938 141.22203 \n",
       "L 165.734375 141.154891 \n",
       "L 162.682813 141.082779 \n",
       "L 159.63125 141.005694 \n",
       "L 156.579688 140.923635 \n",
       "L 153.528125 140.836603 \n",
       "L 150.476563 140.744597 \n",
       "L 147.425 140.647619 \n",
       "L 145.936433 140.597886 \n",
       "L 144.373438 140.543696 \n",
       "L 141.321875 140.432736 \n",
       "L 138.270313 140.316615 \n",
       "L 135.21875 140.195333 \n",
       "L 132.167188 140.06889 \n",
       "L 129.115625 139.937287 \n",
       "L 126.064063 139.800522 \n",
       "L 123.0125 139.658596 \n",
       "L 119.960938 139.51151 \n",
       "L 116.909375 139.359262 \n",
       "L 113.857813 139.201854 \n",
       "L 110.80625 139.039284 \n",
       "L 107.754688 138.871554 \n",
       "L 104.703125 138.698663 \n",
       "L 101.651563 138.520611 \n",
       "L 98.6 138.337398 \n",
       "L 95.548438 138.149024 \n",
       "L 92.496875 137.955489 \n",
       "L 91.07017 137.862592 \n",
       "L 89.445313 137.752644 \n",
       "L 86.39375 137.540793 \n",
       "L 83.342188 137.323578 \n",
       "L 80.290625 137.101 \n",
       "L 77.239063 136.873059 \n",
       "L 74.1875 136.639755 \n",
       "L 71.135938 136.401087 \n",
       "L 68.084375 136.157056 \n",
       "L 65.032813 135.907661 \n",
       "L 61.98125 135.652903 \n",
       "L 58.929688 135.392782 \n",
       "L 55.878125 135.127298 \n",
       "L 55.878125 135.127298 \n",
       "L 52.826563 134.845395 \n",
       "L 49.775 134.55791 \n",
       "L 46.723438 134.264843 \n",
       "L 43.671875 133.966193 \n",
       "L 40.620313 133.661962 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_6\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 208.45625 148.803768 \n",
       "L 205.404688 148.80145 \n",
       "L 202.353125 148.794496 \n",
       "L 199.301563 148.782906 \n",
       "L 196.25 148.76668 \n",
       "L 193.198438 148.745817 \n",
       "L 190.146875 148.720319 \n",
       "L 187.095313 148.690184 \n",
       "L 184.04375 148.655413 \n",
       "L 180.992188 148.616007 \n",
       "L 177.940625 148.571964 \n",
       "L 174.889063 148.523285 \n",
       "L 171.8375 148.46997 \n",
       "L 168.785938 148.412019 \n",
       "L 165.734375 148.349431 \n",
       "L 162.682813 148.282208 \n",
       "L 159.63125 148.210349 \n",
       "L 156.579688 148.133853 \n",
       "L 153.528125 148.052722 \n",
       "L 150.476563 147.966954 \n",
       "L 147.425 147.87655 \n",
       "L 144.373438 147.78151 \n",
       "L 141.321875 147.681834 \n",
       "L 138.270313 147.577522 \n",
       "L 135.21875 147.468574 \n",
       "L 132.167188 147.35499 \n",
       "L 129.115625 147.236769 \n",
       "L 126.064063 147.113913 \n",
       "L 123.0125 146.98642 \n",
       "L 119.960938 146.854292 \n",
       "L 116.909375 146.717527 \n",
       "L 113.857813 146.576126 \n",
       "L 110.80625 146.430089 \n",
       "L 107.754688 146.279416 \n",
       "L 104.703125 146.124107 \n",
       "L 103.641712 146.068474 \n",
       "L 101.651563 145.960502 \n",
       "L 98.6 145.790146 \n",
       "L 95.548438 145.614991 \n",
       "L 92.496875 145.435038 \n",
       "L 89.445313 145.250285 \n",
       "L 86.39375 145.060734 \n",
       "L 83.342188 144.866384 \n",
       "L 80.290625 144.667236 \n",
       "L 77.239063 144.463289 \n",
       "L 74.1875 144.254542 \n",
       "L 71.135938 144.040997 \n",
       "L 68.084375 143.822654 \n",
       "L 65.032813 143.599511 \n",
       "L 61.98125 143.37157 \n",
       "L 61.477899 143.33318 \n",
       "L 58.929688 143.131763 \n",
       "L 55.878125 142.885587 \n",
       "L 52.826563 142.634437 \n",
       "L 49.775 142.378314 \n",
       "L 46.723438 142.117218 \n",
       "L 43.671875 141.851148 \n",
       "L 40.620313 141.580105 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 235.920313 148.616007 \n",
       "L 232.86875 148.655413 \n",
       "L 229.817188 148.690184 \n",
       "L 226.765625 148.720319 \n",
       "L 223.714063 148.745817 \n",
       "L 220.6625 148.76668 \n",
       "L 217.610938 148.782906 \n",
       "L 214.559375 148.794496 \n",
       "L 211.507813 148.80145 \n",
       "L 208.45625 148.803768 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_7\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 41.320212 148.803768 \n",
       "L 40.620313 148.745817 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p50d6125b9b)\" d=\"M 55.878125 121.450827 \n",
       "L 68.084375 33.921415 \n",
       "L 85.41725 42.674357 \n",
       "L 103.926808 85.563768 \n",
       "L 121.543942 76.898357 \n",
       "L 137.305493 56.320192 \n",
       "L 150.87833 62.710714 \n",
       "L 162.270982 72.360744 \n",
       "L 171.662129 68.20048 \n",
       "L 179.301232 63.791492 \n",
       "L 185.453185 66.312523 \n",
       "L 190.369407 68.264914 \n",
       "L 194.274465 66.809159 \n",
       "L 197.361537 65.978539 \n",
       "L 199.79265 66.789478 \n",
       "L 201.701295 67.123694 \n",
       "L 203.196013 66.684803 \n",
       "L 204.364192 66.561584 \n",
       "L 205.275645 66.793352 \n",
       "L 205.985821 66.831784 \n",
       "L 206.538543 66.712057 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"m1a28f5d264\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#p50d6125b9b)\">\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m1a28f5d264\" y=\"121.45082720588226\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"68.08437500000014\" xlink:href=\"#m1a28f5d264\" y=\"33.921415441176464\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"85.4172500000002\" xlink:href=\"#m1a28f5d264\" y=\"42.67435661764705\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"103.92680750000025\" xlink:href=\"#m1a28f5d264\" y=\"85.56376838235289\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"121.54394165000032\" xlink:href=\"#m1a28f5d264\" y=\"76.89835661764701\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"137.3054933930004\" xlink:href=\"#m1a28f5d264\" y=\"56.32019191176468\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"150.87832979306043\" xlink:href=\"#m1a28f5d264\" y=\"62.71071426470586\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"162.27098160964567\" xlink:href=\"#m1a28f5d264\" y=\"72.3607443823529\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"171.6621289891667\" xlink:href=\"#m1a28f5d264\" y=\"68.20048019411762\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"179.30123235979391\" xlink:href=\"#m1a28f5d264\" y=\"63.79149155411762\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"185.45318545632406\" xlink:href=\"#m1a28f5d264\" y=\"66.31252251223526\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"190.36940716808328\" xlink:href=\"#m1a28f5d264\" y=\"68.2649137364235\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"194.27446545051626\" xlink:href=\"#m1a28f5d264\" y=\"66.80915913494584\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"197.3615373556915\" xlink:href=\"#m1a28f5d264\" y=\"65.9785389829995\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"199.79265031982385\" xlink:href=\"#m1a28f5d264\" y=\"66.78947829893296\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"201.70129477630417\" xlink:href=\"#m1a28f5d264\" y=\"67.12369444331279\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"203.19601342244005\" xlink:href=\"#m1a28f5d264\" y=\"66.68480317090807\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"204.36419167171283\" xlink:href=\"#m1a28f5d264\" y=\"66.56158422595863\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"205.27564546261226\" xlink:href=\"#m1a28f5d264\" y=\"66.79335175665594\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"205.98582072105305\" xlink:href=\"#m1a28f5d264\" y=\"66.83178447606092\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"206.53854269258923\" xlink:href=\"#m1a28f5d264\" y=\"66.71205743877178\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 235.920313 148.803768 \n",
       "L 235.920313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 40.620313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 40.620313 148.803768 \n",
       "L 235.920313 148.803768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 40.620313 9.303768 \n",
       "L 235.920313 9.303768 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"ma2274c3c45\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"mcd78ca12bf\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "       <path d=\"M 10.59375 35.5 \n",
       "L 73.1875 35.5 \n",
       "L 73.1875 27.203125 \n",
       "L 10.59375 27.203125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2212\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(48.5070312500001 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −4 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(79.0226562500002 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- −3 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(109.53828125000031 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- −2 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(140.0539062500004 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- −1 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(170.56953125000052 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#ma2274c3c45\" y=\"148.80376838235284\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#mcd78ca12bf\" y=\"9.303768382352843\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(205.27500000000063 160.40220588235286)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- x1 -->\n",
       "     <defs>\n",
       "      <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-78\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(132.12968750000002 175.08033088235285)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"mf305832280\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"148.8037683823528\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"mbd3e3627f0\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"148.8037683823528\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −3 -->\n",
       "      <g transform=\"translate(21.878125000000004 151.56314338235282)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"121.45082720588226\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"121.45082720588226\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(21.878125000000004 124.21020220588227)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"94.09788602941171\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"94.09788602941171\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(21.878125000000004 96.85726102941172)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"66.74494485294115\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"66.74494485294115\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(30.257812500000004 69.50431985294115)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"39.39200367647058\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"39.39200367647058\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(30.257812500000004 42.151378676470586)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#mf305832280\" y=\"12.039062500000021\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mbd3e3627f0\" y=\"12.039062500000021\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(30.257812500000004 14.798437500000034)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- x2 -->\n",
       "     <g transform=\"translate(14.798437500000004 85.19439338235284)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p50d6125b9b\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"40.620312500000004\" y=\"9.303768382352843\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a21d08358>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def momentum_2d(x1, x2, v1, v2):\n",
    "    v1 = gamma * v1 + eta * 0.2 * x1\n",
    "    v2 = gamma * v2 + eta * 4 * x2\n",
    "    return x1 - v1, x2 - v2, v1, v2\n",
    "\n",
    "eta, gamma = 0.4, 0.5\n",
    "d2l.show_trace_2d(f_2d, d2l.train_2d(momentum_2d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, when using a smaller learning rate ($\\eta=0.4$) and momentum hyperparameter ($\\gamma=0.5$), momentum moves more smoothly in the vertical direction and approaches the optimal solution faster in the horizontal direction. Now, when we use a larger learning rate ($\\eta=0.6$), the independent variable will no longer diverge."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 20, x1 0.007188, x2 0.002553\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"182pt\" version=\"1.1\" viewBox=\"0 0 243 182\" width=\"243pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 182.25625 \n",
       "L 243.120313 182.25625 \n",
       "L 243.120313 -0 \n",
       "L 0 -0 \n",
       "L 0 182.25625 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 40.620313 146.7 \n",
       "L 235.920313 146.7 \n",
       "L 235.920313 7.2 \n",
       "L 40.620313 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_1\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 235.920313 113.382711 \n",
       "L 235.740809 113.386567 \n",
       "L 232.86875 113.444004 \n",
       "L 229.817188 113.497851 \n",
       "L 226.765625 113.544519 \n",
       "L 223.714063 113.584007 \n",
       "L 220.6625 113.616315 \n",
       "L 217.610938 113.641444 \n",
       "L 214.559375 113.659393 \n",
       "L 211.507813 113.670162 \n",
       "L 208.45625 113.673752 \n",
       "L 205.404688 113.670162 \n",
       "L 202.353125 113.659393 \n",
       "L 199.301563 113.641444 \n",
       "L 196.25 113.616315 \n",
       "L 193.198438 113.584007 \n",
       "L 190.146875 113.544519 \n",
       "L 187.095313 113.497851 \n",
       "L 184.04375 113.444004 \n",
       "L 181.171691 113.386567 \n",
       "L 180.992188 113.382711 \n",
       "L 177.940625 113.309453 \n",
       "L 174.889063 113.228483 \n",
       "L 171.8375 113.139801 \n",
       "L 168.785938 113.043408 \n",
       "L 165.734375 112.939303 \n",
       "L 162.682813 112.827488 \n",
       "L 159.63125 112.70796 \n",
       "L 156.579688 112.580721 \n",
       "L 153.528125 112.445771 \n",
       "L 150.476563 112.303109 \n",
       "L 147.425 112.152736 \n",
       "L 144.373438 111.994652 \n",
       "L 141.321875 111.828856 \n",
       "L 138.270313 111.655348 \n",
       "L 135.21875 111.474129 \n",
       "L 132.478571 111.304478 \n",
       "L 132.167188 111.283657 \n",
       "L 129.115625 111.071284 \n",
       "L 126.064063 110.850582 \n",
       "L 123.0125 110.621552 \n",
       "L 119.960938 110.384194 \n",
       "L 116.909375 110.138507 \n",
       "L 113.857813 109.884493 \n",
       "L 110.80625 109.622149 \n",
       "L 107.754688 109.351478 \n",
       "L 106.342771 109.222388 \n",
       "L 104.703125 109.059442 \n",
       "L 101.651563 108.747128 \n",
       "L 98.6 108.425762 \n",
       "L 95.548438 108.095344 \n",
       "L 92.496875 107.755873 \n",
       "L 89.445313 107.407349 \n",
       "L 87.166297 107.140299 \n",
       "L 86.39375 107.041151 \n",
       "L 83.342188 106.639606 \n",
       "L 80.290625 106.228145 \n",
       "L 77.239063 105.80677 \n",
       "L 74.1875 105.37548 \n",
       "L 71.993118 105.058209 \n",
       "L 71.135938 104.921229 \n",
       "L 68.084375 104.422624 \n",
       "L 65.032813 103.91306 \n",
       "L 61.98125 103.392537 \n",
       "L 59.590335 102.976119 \n",
       "L 58.929688 102.84752 \n",
       "L 55.878125 102.241264 \n",
       "L 52.826563 101.622761 \n",
       "L 49.775 100.992011 \n",
       "L 49.31 100.89403 \n",
       "L 46.723438 100.276343 \n",
       "L 43.671875 99.533731 \n",
       "L 40.760292 98.81194 \n",
       "L 40.620313 98.7719 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 69.702727 \n",
       "L 40.760292 69.662687 \n",
       "L 43.671875 68.940896 \n",
       "L 46.723438 68.198284 \n",
       "L 49.31 67.580597 \n",
       "L 49.775 67.482616 \n",
       "L 52.826563 66.851866 \n",
       "L 55.878125 66.233363 \n",
       "L 58.929688 65.627107 \n",
       "L 59.590335 65.498507 \n",
       "L 61.98125 65.08209 \n",
       "L 65.032813 64.561567 \n",
       "L 68.084375 64.052003 \n",
       "L 71.135938 63.553397 \n",
       "L 71.993118 63.416418 \n",
       "L 74.1875 63.099147 \n",
       "L 77.239063 62.667857 \n",
       "L 80.290625 62.246482 \n",
       "L 83.342188 61.835021 \n",
       "L 86.39375 61.433475 \n",
       "L 87.166297 61.334328 \n",
       "L 89.445313 61.067278 \n",
       "L 92.496875 60.718754 \n",
       "L 95.548438 60.379283 \n",
       "L 98.6 60.048864 \n",
       "L 101.651563 59.727498 \n",
       "L 104.703125 59.415185 \n",
       "L 106.342771 59.252239 \n",
       "L 107.754688 59.123149 \n",
       "L 110.80625 58.852478 \n",
       "L 113.857813 58.590134 \n",
       "L 116.909375 58.336119 \n",
       "L 119.960938 58.090433 \n",
       "L 123.0125 57.853075 \n",
       "L 126.064063 57.624045 \n",
       "L 129.115625 57.403343 \n",
       "L 132.167188 57.19097 \n",
       "L 132.478571 57.170149 \n",
       "L 135.21875 57.000498 \n",
       "L 138.270313 56.819279 \n",
       "L 141.321875 56.645771 \n",
       "L 144.373438 56.479975 \n",
       "L 147.425 56.321891 \n",
       "L 150.476563 56.171517 \n",
       "L 153.528125 56.028856 \n",
       "L 156.579688 55.893905 \n",
       "L 159.63125 55.766667 \n",
       "L 162.682813 55.647139 \n",
       "L 165.734375 55.535323 \n",
       "L 168.785938 55.431219 \n",
       "L 171.8375 55.334826 \n",
       "L 174.889063 55.246144 \n",
       "L 177.940625 55.165174 \n",
       "L 180.992188 55.091915 \n",
       "L 181.171691 55.08806 \n",
       "L 184.04375 55.030623 \n",
       "L 187.095313 54.976776 \n",
       "L 190.146875 54.930108 \n",
       "L 193.198438 54.89062 \n",
       "L 196.25 54.858312 \n",
       "L 199.301563 54.833183 \n",
       "L 202.353125 54.815234 \n",
       "L 205.404688 54.804465 \n",
       "L 208.45625 54.800875 \n",
       "L 211.507813 54.804465 \n",
       "L 214.559375 54.815234 \n",
       "L 217.610938 54.833183 \n",
       "L 220.6625 54.858312 \n",
       "L 223.714063 54.89062 \n",
       "L 226.765625 54.930108 \n",
       "L 229.817188 54.976776 \n",
       "L 232.86875 55.030623 \n",
       "L 235.740809 55.08806 \n",
       "L 235.920313 55.091915 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_2\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 235.920313 125.662887 \n",
       "L 232.86875 125.708266 \n",
       "L 229.817188 125.748307 \n",
       "L 226.765625 125.783008 \n",
       "L 223.714063 125.812371 \n",
       "L 220.6625 125.836395 \n",
       "L 217.610938 125.85508 \n",
       "L 214.559375 125.868427 \n",
       "L 211.507813 125.876435 \n",
       "L 208.45625 125.879104 \n",
       "L 208.45625 125.879104 \n",
       "L 208.45625 125.879104 \n",
       "L 205.404688 125.876435 \n",
       "L 202.353125 125.868427 \n",
       "L 199.301563 125.85508 \n",
       "L 196.25 125.836395 \n",
       "L 193.198438 125.812371 \n",
       "L 190.146875 125.783008 \n",
       "L 187.095313 125.748307 \n",
       "L 184.04375 125.708266 \n",
       "L 180.992188 125.662887 \n",
       "L 177.940625 125.61217 \n",
       "L 174.889063 125.556114 \n",
       "L 171.8375 125.494719 \n",
       "L 168.785938 125.427985 \n",
       "L 165.734375 125.355913 \n",
       "L 162.682813 125.278502 \n",
       "L 159.63125 125.195752 \n",
       "L 156.579688 125.107664 \n",
       "L 153.528125 125.014237 \n",
       "L 150.476563 124.915471 \n",
       "L 147.425 124.811366 \n",
       "L 144.373438 124.701923 \n",
       "L 141.321875 124.587141 \n",
       "L 138.270313 124.467021 \n",
       "L 135.21875 124.341561 \n",
       "L 132.167188 124.210763 \n",
       "L 129.115625 124.074627 \n",
       "L 126.064063 123.933152 \n",
       "L 123.234432 123.797015 \n",
       "L 123.0125 123.78576 \n",
       "L 119.960938 123.625383 \n",
       "L 116.909375 123.459379 \n",
       "L 113.857813 123.287747 \n",
       "L 110.80625 123.110488 \n",
       "L 107.754688 122.927602 \n",
       "L 104.703125 122.739088 \n",
       "L 101.651563 122.544948 \n",
       "L 98.6 122.34518 \n",
       "L 95.548438 122.139784 \n",
       "L 92.496875 121.928762 \n",
       "L 89.484943 121.714925 \n",
       "L 89.445313 121.711951 \n",
       "L 86.39375 121.476972 \n",
       "L 83.342188 121.236045 \n",
       "L 80.290625 120.989168 \n",
       "L 77.239063 120.736343 \n",
       "L 74.1875 120.477569 \n",
       "L 71.135938 120.212846 \n",
       "L 68.084375 119.942175 \n",
       "L 65.032813 119.665554 \n",
       "L 64.679474 119.632836 \n",
       "L 61.98125 119.367843 \n",
       "L 58.929688 119.061839 \n",
       "L 55.878125 118.749525 \n",
       "L 52.826563 118.430902 \n",
       "L 49.775 118.10597 \n",
       "L 46.723438 117.774729 \n",
       "L 44.698569 117.550746 \n",
       "L 43.671875 117.429851 \n",
       "L 40.620313 117.063806 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 51.410821 \n",
       "L 43.671875 51.044776 \n",
       "L 44.698569 50.923881 \n",
       "L 46.723438 50.699898 \n",
       "L 49.775 50.368657 \n",
       "L 52.826563 50.043725 \n",
       "L 55.878125 49.725102 \n",
       "L 58.929688 49.412788 \n",
       "L 61.98125 49.106784 \n",
       "L 64.679474 48.841791 \n",
       "L 65.032813 48.809072 \n",
       "L 68.084375 48.532452 \n",
       "L 71.135938 48.26178 \n",
       "L 74.1875 47.997058 \n",
       "L 77.239063 47.738284 \n",
       "L 80.290625 47.485458 \n",
       "L 83.342188 47.238582 \n",
       "L 86.39375 46.997655 \n",
       "L 89.445313 46.762676 \n",
       "L 89.484943 46.759701 \n",
       "L 92.496875 46.545865 \n",
       "L 95.548438 46.334843 \n",
       "L 98.6 46.129447 \n",
       "L 101.651563 45.929679 \n",
       "L 104.703125 45.735539 \n",
       "L 107.754688 45.547025 \n",
       "L 110.80625 45.364139 \n",
       "L 113.857813 45.18688 \n",
       "L 116.909375 45.015248 \n",
       "L 119.960938 44.849244 \n",
       "L 123.0125 44.688866 \n",
       "L 123.234432 44.677612 \n",
       "L 126.064063 44.541475 \n",
       "L 129.115625 44.4 \n",
       "L 132.167188 44.263863 \n",
       "L 135.21875 44.133065 \n",
       "L 138.270313 44.007606 \n",
       "L 141.321875 43.887486 \n",
       "L 144.373438 43.772704 \n",
       "L 147.425 43.663261 \n",
       "L 150.476563 43.559156 \n",
       "L 153.528125 43.46039 \n",
       "L 156.579688 43.366963 \n",
       "L 159.63125 43.278875 \n",
       "L 162.682813 43.196125 \n",
       "L 165.734375 43.118714 \n",
       "L 168.785938 43.046642 \n",
       "L 171.8375 42.979908 \n",
       "L 174.889063 42.918513 \n",
       "L 177.940625 42.862457 \n",
       "L 180.992188 42.811739 \n",
       "L 184.04375 42.766361 \n",
       "L 187.095313 42.72632 \n",
       "L 190.146875 42.691619 \n",
       "L 193.198438 42.662256 \n",
       "L 196.25 42.638232 \n",
       "L 199.301563 42.619546 \n",
       "L 202.353125 42.6062 \n",
       "L 205.404688 42.598192 \n",
       "L 208.45625 42.595522 \n",
       "L 211.507813 42.598192 \n",
       "L 214.559375 42.6062 \n",
       "L 217.610938 42.619546 \n",
       "L 220.6625 42.638232 \n",
       "L 223.714063 42.662256 \n",
       "L 226.765625 42.691619 \n",
       "L 229.817188 42.72632 \n",
       "L 232.86875 42.766361 \n",
       "L 235.920313 42.811739 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_3\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 235.920313 135.055171 \n",
       "L 232.86875 135.091288 \n",
       "L 229.817188 135.123157 \n",
       "L 226.765625 135.150777 \n",
       "L 223.714063 135.174147 \n",
       "L 220.6625 135.193268 \n",
       "L 217.610938 135.20814 \n",
       "L 214.559375 135.218763 \n",
       "L 211.507813 135.225137 \n",
       "L 208.45625 135.227262 \n",
       "L 205.404688 135.225137 \n",
       "L 202.353125 135.218763 \n",
       "L 199.301563 135.20814 \n",
       "L 196.25 135.193268 \n",
       "L 193.198438 135.174147 \n",
       "L 190.146875 135.150777 \n",
       "L 187.095313 135.123157 \n",
       "L 184.04375 135.091288 \n",
       "L 180.992188 135.055171 \n",
       "L 177.940625 135.014804 \n",
       "L 174.889063 134.970187 \n",
       "L 171.8375 134.921322 \n",
       "L 168.785938 134.868207 \n",
       "L 165.734375 134.810844 \n",
       "L 162.682813 134.749231 \n",
       "L 159.63125 134.683369 \n",
       "L 156.579688 134.613258 \n",
       "L 153.528125 134.538897 \n",
       "L 150.476563 134.460288 \n",
       "L 147.425 134.377429 \n",
       "L 144.373438 134.290321 \n",
       "L 141.605741 134.207463 \n",
       "L 141.321875 134.198603 \n",
       "L 138.270313 134.098928 \n",
       "L 135.21875 133.994824 \n",
       "L 132.167188 133.886289 \n",
       "L 129.115625 133.773325 \n",
       "L 126.064063 133.65593 \n",
       "L 123.0125 133.534106 \n",
       "L 119.960938 133.407852 \n",
       "L 116.909375 133.277167 \n",
       "L 113.857813 133.142053 \n",
       "L 110.80625 133.002509 \n",
       "L 107.754688 132.858534 \n",
       "L 104.703125 132.71013 \n",
       "L 101.651563 132.557296 \n",
       "L 98.6 132.400032 \n",
       "L 95.548438 132.238338 \n",
       "L 93.473375 132.125373 \n",
       "L 92.496875 132.069851 \n",
       "L 89.445313 131.891716 \n",
       "L 86.39375 131.708955 \n",
       "L 83.342188 131.521567 \n",
       "L 80.290625 131.329552 \n",
       "L 77.239063 131.13291 \n",
       "L 74.1875 130.931642 \n",
       "L 71.135938 130.725746 \n",
       "L 68.084375 130.515224 \n",
       "L 65.032813 130.300075 \n",
       "L 61.98125 130.080299 \n",
       "L 61.477899 130.043284 \n",
       "L 58.929688 129.84718 \n",
       "L 55.878125 129.607497 \n",
       "L 52.826563 129.362973 \n",
       "L 49.775 129.113606 \n",
       "L 46.723438 128.859398 \n",
       "L 43.671875 128.600347 \n",
       "L 40.620313 128.336454 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 40.138173 \n",
       "L 43.671875 39.87428 \n",
       "L 46.723438 39.615229 \n",
       "L 49.775 39.36102 \n",
       "L 52.826563 39.111654 \n",
       "L 55.878125 38.867129 \n",
       "L 58.929688 38.627447 \n",
       "L 61.477899 38.431343 \n",
       "L 61.98125 38.394328 \n",
       "L 65.032813 38.174552 \n",
       "L 68.084375 37.959403 \n",
       "L 71.135938 37.748881 \n",
       "L 74.1875 37.542985 \n",
       "L 77.239063 37.341716 \n",
       "L 80.290625 37.145075 \n",
       "L 83.342188 36.95306 \n",
       "L 86.39375 36.765672 \n",
       "L 89.445313 36.58291 \n",
       "L 92.496875 36.404776 \n",
       "L 93.473375 36.349254 \n",
       "L 95.548438 36.236289 \n",
       "L 98.6 36.074595 \n",
       "L 101.651563 35.917331 \n",
       "L 104.703125 35.764497 \n",
       "L 107.754688 35.616092 \n",
       "L 110.80625 35.472118 \n",
       "L 113.857813 35.332574 \n",
       "L 116.909375 35.19746 \n",
       "L 119.960938 35.066775 \n",
       "L 123.0125 34.940521 \n",
       "L 126.064063 34.818696 \n",
       "L 129.115625 34.701302 \n",
       "L 132.167188 34.588338 \n",
       "L 135.21875 34.479803 \n",
       "L 138.270313 34.375699 \n",
       "L 141.321875 34.276024 \n",
       "L 141.605741 34.267164 \n",
       "L 144.373438 34.184306 \n",
       "L 147.425 34.097198 \n",
       "L 150.476563 34.014339 \n",
       "L 153.528125 33.93573 \n",
       "L 156.579688 33.861369 \n",
       "L 159.63125 33.791258 \n",
       "L 162.682813 33.725396 \n",
       "L 165.734375 33.663783 \n",
       "L 168.785938 33.606419 \n",
       "L 171.8375 33.553305 \n",
       "L 174.889063 33.50444 \n",
       "L 177.940625 33.459823 \n",
       "L 180.992188 33.419456 \n",
       "L 184.04375 33.383338 \n",
       "L 187.095313 33.35147 \n",
       "L 190.146875 33.32385 \n",
       "L 193.198438 33.30048 \n",
       "L 196.25 33.281359 \n",
       "L 199.301563 33.266486 \n",
       "L 202.353125 33.255864 \n",
       "L 205.404688 33.24949 \n",
       "L 208.45625 33.247365 \n",
       "L 211.507813 33.24949 \n",
       "L 214.559375 33.255864 \n",
       "L 217.610938 33.266486 \n",
       "L 220.6625 33.281359 \n",
       "L 223.714063 33.30048 \n",
       "L 226.765625 33.32385 \n",
       "L 229.817188 33.35147 \n",
       "L 232.86875 33.383338 \n",
       "L 235.920313 33.419456 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_4\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 235.920313 142.972329 \n",
       "L 232.86875 143.003378 \n",
       "L 229.817188 143.030774 \n",
       "L 226.765625 143.054517 \n",
       "L 223.714063 143.074607 \n",
       "L 220.6625 143.091045 \n",
       "L 217.610938 143.10383 \n",
       "L 214.559375 143.112962 \n",
       "L 211.507813 143.118441 \n",
       "L 208.45625 143.120267 \n",
       "L 205.404688 143.118441 \n",
       "L 202.353125 143.112962 \n",
       "L 199.301563 143.10383 \n",
       "L 196.25 143.091045 \n",
       "L 193.198438 143.074607 \n",
       "L 190.146875 143.054517 \n",
       "L 187.095313 143.030774 \n",
       "L 184.04375 143.003378 \n",
       "L 180.992188 142.972329 \n",
       "L 177.940625 142.937628 \n",
       "L 174.889063 142.899273 \n",
       "L 171.8375 142.857266 \n",
       "L 168.785938 142.811606 \n",
       "L 165.734375 142.762294 \n",
       "L 162.682813 142.709328 \n",
       "L 159.63125 142.65271 \n",
       "L 156.579688 142.592439 \n",
       "L 153.876875 142.535821 \n",
       "L 153.528125 142.52825 \n",
       "L 150.476563 142.458216 \n",
       "L 147.425 142.384396 \n",
       "L 144.373438 142.306791 \n",
       "L 141.321875 142.2254 \n",
       "L 138.270313 142.140224 \n",
       "L 135.21875 142.051262 \n",
       "L 132.167188 141.958514 \n",
       "L 129.115625 141.861981 \n",
       "L 126.064063 141.761662 \n",
       "L 123.0125 141.657558 \n",
       "L 119.960938 141.549668 \n",
       "L 116.909375 141.437992 \n",
       "L 113.857813 141.322531 \n",
       "L 110.80625 141.203284 \n",
       "L 107.754688 141.080251 \n",
       "L 104.703125 140.953433 \n",
       "L 101.651563 140.822829 \n",
       "L 98.6 140.68844 \n",
       "L 95.548438 140.550265 \n",
       "L 93.473375 140.453731 \n",
       "L 92.496875 140.40659 \n",
       "L 89.445313 140.255344 \n",
       "L 86.39375 140.100169 \n",
       "L 83.342188 139.941066 \n",
       "L 80.290625 139.778034 \n",
       "L 77.239063 139.611074 \n",
       "L 74.1875 139.440186 \n",
       "L 71.135938 139.265369 \n",
       "L 68.084375 139.086623 \n",
       "L 65.032813 138.90395 \n",
       "L 61.98125 138.717347 \n",
       "L 58.929688 138.526816 \n",
       "L 56.494602 138.371642 \n",
       "L 55.878125 138.330817 \n",
       "L 52.826563 138.124649 \n",
       "L 49.775 137.914399 \n",
       "L 46.723438 137.700066 \n",
       "L 43.671875 137.481651 \n",
       "L 40.620313 137.259153 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 31.215474 \n",
       "L 43.671875 30.992976 \n",
       "L 46.723438 30.774561 \n",
       "L 49.775 30.560228 \n",
       "L 52.826563 30.349978 \n",
       "L 55.878125 30.14381 \n",
       "L 56.494602 30.102985 \n",
       "L 58.929688 29.94781 \n",
       "L 61.98125 29.75728 \n",
       "L 65.032813 29.570677 \n",
       "L 68.084375 29.388003 \n",
       "L 71.135938 29.209258 \n",
       "L 74.1875 29.034441 \n",
       "L 77.239063 28.863553 \n",
       "L 80.290625 28.696593 \n",
       "L 83.342188 28.533561 \n",
       "L 86.39375 28.374458 \n",
       "L 89.445313 28.219283 \n",
       "L 92.496875 28.068037 \n",
       "L 93.473375 28.020896 \n",
       "L 95.548438 27.924362 \n",
       "L 98.6 27.786187 \n",
       "L 101.651563 27.651798 \n",
       "L 104.703125 27.521194 \n",
       "L 107.754688 27.394376 \n",
       "L 110.80625 27.271343 \n",
       "L 113.857813 27.152096 \n",
       "L 116.909375 27.036635 \n",
       "L 119.960938 26.924959 \n",
       "L 123.0125 26.817069 \n",
       "L 126.064063 26.712965 \n",
       "L 129.115625 26.612646 \n",
       "L 132.167188 26.516113 \n",
       "L 135.21875 26.423365 \n",
       "L 138.270313 26.334403 \n",
       "L 141.321875 26.249227 \n",
       "L 144.373438 26.167836 \n",
       "L 147.425 26.090231 \n",
       "L 150.476563 26.016411 \n",
       "L 153.528125 25.946377 \n",
       "L 153.876875 25.938806 \n",
       "L 156.579688 25.882188 \n",
       "L 159.63125 25.821917 \n",
       "L 162.682813 25.765299 \n",
       "L 165.734375 25.712333 \n",
       "L 168.785938 25.66302 \n",
       "L 171.8375 25.617361 \n",
       "L 174.889063 25.575353 \n",
       "L 177.940625 25.536999 \n",
       "L 180.992188 25.502298 \n",
       "L 184.04375 25.471249 \n",
       "L 187.095313 25.443853 \n",
       "L 190.146875 25.42011 \n",
       "L 193.198438 25.40002 \n",
       "L 196.25 25.383582 \n",
       "L 199.301563 25.370797 \n",
       "L 202.353125 25.361665 \n",
       "L 205.404688 25.356186 \n",
       "L 208.45625 25.35436 \n",
       "L 211.507813 25.356186 \n",
       "L 214.559375 25.361665 \n",
       "L 217.610938 25.370797 \n",
       "L 220.6625 25.383582 \n",
       "L 223.714063 25.40002 \n",
       "L 226.765625 25.42011 \n",
       "L 229.817188 25.443853 \n",
       "L 232.86875 25.471249 \n",
       "L 235.920313 25.502298 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_5\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 71.993118 146.7 \n",
       "L 71.135938 146.655888 \n",
       "L 68.084375 146.49532 \n",
       "L 65.032813 146.331223 \n",
       "L 61.98125 146.163597 \n",
       "L 58.929688 145.992442 \n",
       "L 55.878125 145.817759 \n",
       "L 52.826563 145.639546 \n",
       "L 49.775 145.457804 \n",
       "L 46.723438 145.272534 \n",
       "L 43.671875 145.083734 \n",
       "L 40.620313 144.891405 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 23.583222 \n",
       "L 43.671875 23.390893 \n",
       "L 46.723438 23.202093 \n",
       "L 49.775 23.016823 \n",
       "L 52.826563 22.835081 \n",
       "L 55.878125 22.656868 \n",
       "L 58.929688 22.482184 \n",
       "L 61.98125 22.31103 \n",
       "L 65.032813 22.143404 \n",
       "L 68.084375 21.979307 \n",
       "L 71.135938 21.818739 \n",
       "L 71.993118 21.774627 \n",
       "L 74.1875 21.665402 \n",
       "L 77.239063 21.516926 \n",
       "L 80.290625 21.371862 \n",
       "L 83.342188 21.230212 \n",
       "L 86.39375 21.091975 \n",
       "L 89.445313 20.957151 \n",
       "L 92.496875 20.82574 \n",
       "L 95.548438 20.697743 \n",
       "L 98.6 20.573159 \n",
       "L 101.651563 20.451988 \n",
       "L 104.703125 20.33423 \n",
       "L 107.754688 20.219886 \n",
       "L 110.80625 20.108955 \n",
       "L 113.857813 20.001437 \n",
       "L 116.909375 19.897333 \n",
       "L 119.960938 19.796642 \n",
       "L 123.0125 19.699364 \n",
       "L 123.234432 19.692537 \n",
       "L 126.064063 19.608262 \n",
       "L 129.115625 19.520682 \n",
       "L 132.167188 19.436407 \n",
       "L 135.21875 19.355437 \n",
       "L 138.270313 19.277772 \n",
       "L 141.321875 19.203412 \n",
       "L 144.373438 19.132356 \n",
       "L 147.425 19.064606 \n",
       "L 150.476563 19.00016 \n",
       "L 153.528125 18.939019 \n",
       "L 156.579688 18.881183 \n",
       "L 159.63125 18.826652 \n",
       "L 162.682813 18.775426 \n",
       "L 165.734375 18.727505 \n",
       "L 168.785938 18.682889 \n",
       "L 171.8375 18.641578 \n",
       "L 174.889063 18.603571 \n",
       "L 177.940625 18.56887 \n",
       "L 180.992188 18.537473 \n",
       "L 184.04375 18.509382 \n",
       "L 187.095313 18.484595 \n",
       "L 190.146875 18.463113 \n",
       "L 193.198438 18.444936 \n",
       "L 196.25 18.430064 \n",
       "L 199.301563 18.418497 \n",
       "L 202.353125 18.410235 \n",
       "L 205.404688 18.405277 \n",
       "L 208.45625 18.403625 \n",
       "L 211.507813 18.405277 \n",
       "L 214.559375 18.410235 \n",
       "L 217.610938 18.418497 \n",
       "L 220.6625 18.430064 \n",
       "L 223.714063 18.444936 \n",
       "L 226.765625 18.463113 \n",
       "L 229.817188 18.484595 \n",
       "L 232.86875 18.509382 \n",
       "L 235.920313 18.537473 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_6\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 16.817652 \n",
       "L 43.671875 16.643077 \n",
       "L 46.723438 16.471705 \n",
       "L 49.775 16.303536 \n",
       "L 52.826563 16.138571 \n",
       "L 55.878125 15.976808 \n",
       "L 58.929688 15.818249 \n",
       "L 61.98125 15.662893 \n",
       "L 64.679474 15.528358 \n",
       "L 65.032813 15.511266 \n",
       "L 68.084375 15.366763 \n",
       "L 71.135938 15.225368 \n",
       "L 74.1875 15.08708 \n",
       "L 77.239063 14.951899 \n",
       "L 80.290625 14.819826 \n",
       "L 83.342188 14.690861 \n",
       "L 86.39375 14.565003 \n",
       "L 89.445313 14.442253 \n",
       "L 92.496875 14.322611 \n",
       "L 95.548438 14.206076 \n",
       "L 98.6 14.092649 \n",
       "L 101.651563 13.982329 \n",
       "L 104.703125 13.875117 \n",
       "L 107.754688 13.771012 \n",
       "L 110.80625 13.670016 \n",
       "L 113.857813 13.572126 \n",
       "L 116.909375 13.477345 \n",
       "L 117.943803 13.446269 \n",
       "L 119.960938 13.387427 \n",
       "L 123.0125 13.301428 \n",
       "L 126.064063 13.218446 \n",
       "L 129.115625 13.138482 \n",
       "L 132.167188 13.061535 \n",
       "L 135.21875 12.987605 \n",
       "L 138.270313 12.916694 \n",
       "L 141.321875 12.848799 \n",
       "L 144.373438 12.783923 \n",
       "L 147.425 12.722064 \n",
       "L 150.476563 12.663222 \n",
       "L 153.528125 12.607398 \n",
       "L 156.579688 12.554591 \n",
       "L 159.63125 12.504802 \n",
       "L 162.682813 12.45803 \n",
       "L 165.734375 12.414276 \n",
       "L 168.785938 12.37354 \n",
       "L 171.8375 12.335821 \n",
       "L 174.889063 12.301119 \n",
       "L 177.940625 12.269435 \n",
       "L 180.992188 12.240769 \n",
       "L 184.04375 12.21512 \n",
       "L 187.095313 12.192489 \n",
       "L 190.146875 12.172875 \n",
       "L 193.198438 12.156278 \n",
       "L 196.25 12.1427 \n",
       "L 199.301563 12.132138 \n",
       "L 202.353125 12.124594 \n",
       "L 205.404688 12.120068 \n",
       "L 208.45625 12.118559 \n",
       "L 211.507813 12.120068 \n",
       "L 214.559375 12.124594 \n",
       "L 217.610938 12.132138 \n",
       "L 220.6625 12.1427 \n",
       "L 223.714063 12.156278 \n",
       "L 226.765625 12.172875 \n",
       "L 229.817188 12.192489 \n",
       "L 232.86875 12.21512 \n",
       "L 235.920313 12.240769 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"LineCollection_7\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 40.620313 10.667705 \n",
       "L 43.671875 10.507883 \n",
       "L 46.723438 10.350993 \n",
       "L 49.775 10.197036 \n",
       "L 52.826563 10.046011 \n",
       "L 55.878125 9.897919 \n",
       "L 58.929688 9.752759 \n",
       "L 61.98125 9.610532 \n",
       "L 65.032813 9.471237 \n",
       "L 68.084375 9.334875 \n",
       "L 69.291587 9.28209 \n",
       "L 71.135938 9.203655 \n",
       "L 74.1875 9.076733 \n",
       "L 77.239063 8.952663 \n",
       "L 80.290625 8.831446 \n",
       "L 83.342188 8.71308 \n",
       "L 86.39375 8.597567 \n",
       "L 89.445313 8.484906 \n",
       "L 92.496875 8.375097 \n",
       "L 95.548438 8.26814 \n",
       "L 98.6 8.164036 \n",
       "L 101.651563 8.062784 \n",
       "L 104.703125 7.964384 \n",
       "L 107.754688 7.868836 \n",
       "L 110.80625 7.77614 \n",
       "L 113.857813 7.686296 \n",
       "L 116.909375 7.599305 \n",
       "L 119.960938 7.515166 \n",
       "L 123.0125 7.433879 \n",
       "L 126.064063 7.355444 \n",
       "L 129.115625 7.279861 \n",
       "L 132.167188 7.20713 \n",
       "L 132.478571 7.2 \n",
       "\" style=\"fill:none;stroke:#1f77b4;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p9a240c3001)\" d=\"M 55.878125 125.879104 \n",
       "L 74.1875 25.938806 \n",
       "L 99.454438 115.885075 \n",
       "L 125.168124 84.903582 \n",
       "L 148.019542 67.813791 \n",
       "L 166.697656 98.685349 \n",
       "L 181.047744 79.445842 \n",
       "L 191.511809 81.325619 \n",
       "L 198.777175 89.253574 \n",
       "L 203.571346 81.178526 \n",
       "L 206.554621 84.482092 \n",
       "L 208.274453 85.546406 \n",
       "L 209.156185 82.936741 \n",
       "L 209.513059 84.753283 \n",
       "L 209.564679 84.423228 \n",
       "L 209.457477 83.812006 \n",
       "L 209.283729 84.527133 \n",
       "L 209.097558 84.18913 \n",
       "L 208.927515 84.135769 \n",
       "L 208.785942 84.352795 \n",
       "L 208.675592 84.184152 \n",
       "\" style=\"fill:none;stroke:#ff7f0e;stroke-linecap:square;\"/>\n",
       "    <defs>\n",
       "     <path d=\"M 0 3 \n",
       "C 0.795609 3 1.55874 2.683901 2.12132 2.12132 \n",
       "C 2.683901 1.55874 3 0.795609 3 0 \n",
       "C 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \n",
       "C 1.55874 -2.683901 0.795609 -3 0 -3 \n",
       "C -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \n",
       "C -2.683901 -1.55874 -3 -0.795609 -3 0 \n",
       "C -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \n",
       "C -1.55874 2.683901 -0.795609 3 0 3 \n",
       "z\n",
       "\" id=\"m4c8a578b91\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "    </defs>\n",
       "    <g clip-path=\"url(#p9a240c3001)\">\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m4c8a578b91\" y=\"125.87910447761195\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"74.18750000000014\" xlink:href=\"#m4c8a578b91\" y=\"25.938805970149375\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"99.45443750000024\" xlink:href=\"#m4c8a578b91\" y=\"115.8850746268657\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"125.16812375000035\" xlink:href=\"#m4c8a578b91\" y=\"84.90358208955229\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"148.01954202500042\" xlink:href=\"#m4c8a578b91\" y=\"67.8137910447762\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"166.6976561195005\" xlink:href=\"#m4c8a578b91\" y=\"98.68534925373139\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"181.04774443241055\" xlink:href=\"#m4c8a578b91\" y=\"79.44584238805976\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"191.5118092569764\" xlink:href=\"#m4c8a578b91\" y=\"81.32561946268663\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"198.77717455842222\" xlink:href=\"#m4c8a578b91\" y=\"89.25357352835826\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"203.57134626213454\" xlink:href=\"#m4c8a578b91\" y=\"81.17852633194036\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"206.55462056253464\" xlink:href=\"#m4c8a578b91\" y=\"84.48209177588065\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"208.2744532452306\" xlink:href=\"#m4c8a578b91\" y=\"85.54640647454333\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.156185197151\" xlink:href=\"#m4c8a578b91\" y=\"82.93674052377678\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.51305894945313\" xlink:href=\"#m4c8a578b91\" y=\"84.75328253013534\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.5646787516699\" xlink:href=\"#m4c8a578b91\" y=\"84.42322769979592\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.457477202578\" xlink:href=\"#m4c8a578b91\" y=\"83.81200604392211\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.28372916372274\" xlink:href=\"#m4c8a578b91\" y=\"84.52713294937824\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"209.09755764464848\" xlink:href=\"#m4c8a578b91\" y=\"84.18912956240463\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"208.92751496775358\" xlink:href=\"#m4c8a578b91\" y=\"84.13576915795282\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"208.7859418331758\" xlink:href=\"#m4c8a578b91\" y=\"84.35279521544625\"/>\n",
       "     <use style=\"fill:#ff7f0e;stroke:#000000;stroke-width:0.5;\" x=\"208.67559224590588\" xlink:href=\"#m4c8a578b91\" y=\"84.18415196592807\"/>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 235.920313 146.7 \n",
       "L 235.920313 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 40.620313 146.7 \n",
       "L 40.620313 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 40.620313 146.7 \n",
       "L 235.920313 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 40.620313 7.2 \n",
       "L 235.920313 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m5b25fdbf63\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"m9646d5b9e7\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"55.8781250000001\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- −5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "       <path d=\"M 10.59375 35.5 \n",
       "L 73.1875 35.5 \n",
       "L 73.1875 27.203125 \n",
       "L 10.59375 27.203125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2212\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(48.5070312500001 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_4\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_5\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"86.3937500000002\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- −4 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(79.0226562500002 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"116.90937500000031\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- −3 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(109.53828125000031 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"147.4250000000004\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- −2 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(140.0539062500004 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"177.94062500000052\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- −1 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(170.56953125000052 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#m5b25fdbf63\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"208.45625000000064\" xlink:href=\"#m9646d5b9e7\" y=\"7.199999999999989\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(205.27500000000063 158.29843749999998)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_7\">\n",
       "     <!-- x1 -->\n",
       "     <defs>\n",
       "      <path d=\"M 54.890625 54.6875 \n",
       "L 35.109375 28.078125 \n",
       "L 55.90625 0 \n",
       "L 45.3125 0 \n",
       "L 29.390625 21.484375 \n",
       "L 13.484375 0 \n",
       "L 2.875 0 \n",
       "L 24.125 28.609375 \n",
       "L 4.6875 54.6875 \n",
       "L 15.28125 54.6875 \n",
       "L 29.78125 35.203125 \n",
       "L 44.28125 54.6875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-78\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(132.12968750000002 172.9765625)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"ma2318dbdd0\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"mef1a6985a0\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"146.7\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- −3 -->\n",
       "      <g transform=\"translate(21.878125000000004 149.459375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"125.87910447761195\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"125.87910447761195\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- −2 -->\n",
       "      <g transform=\"translate(21.878125000000004 128.63847947761195)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"105.05820895522392\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"105.05820895522392\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- −1 -->\n",
       "      <g transform=\"translate(21.878125000000004 107.81758395522392)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-2212\"/>\n",
       "       <use x=\"83.7890625\" xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"84.23731343283588\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"84.23731343283588\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0 -->\n",
       "      <g transform=\"translate(30.257812500000004 86.99668843283588)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"63.416417910447834\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_23\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"63.416417910447834\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 1 -->\n",
       "      <g transform=\"translate(30.257812500000004 66.17579291044784)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"42.5955223880598\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"42.5955223880598\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 2 -->\n",
       "      <g transform=\"translate(30.257812500000004 45.35489738805981)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"40.620312500000004\" xlink:href=\"#ma2318dbdd0\" y=\"21.774626865671763\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"235.92031250000002\" xlink:href=\"#mef1a6985a0\" y=\"21.774626865671763\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 3 -->\n",
       "      <g transform=\"translate(30.257812500000004 24.53400186567177)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_15\">\n",
       "     <!-- x2 -->\n",
       "     <g transform=\"translate(14.798437500000004 83.09062499999999)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-78\"/>\n",
       "      <use x=\"59.1796875\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p9a240c3001\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"40.620312500000004\" y=\"7.199999999999989\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a1f1b0860>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "eta = 0.6\n",
    "d2l.show_trace_2d(f_2d, d2l.train_2d(momentum_2d))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Expanding the velocity variable $\\mathbf v_t$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand the momentum method, we can expand the velocity variable over time:\n",
    "\n",
    "$$ \\begin{aligned} \\mathbf{v}t &= \\eta_t \\mathbf{g}t + \\gamma \\mathbf{v}{t-1}, \\ &= \\eta_t \\mathbf{g}t + \\gamma \\eta{t-1} \\mathbf{g}{t-1} + \\gamma\\mathbf{v}{t-1}, \\ &\\ldots\\ &= \\eta_t \\mathbf{g}t + \\gamma \\eta{t-1} \\mathbf{g}{t-1} + \\ldots + \\gamma^{t-1}\\eta_1\\mathbf g_1. \\ \\end{aligned} $$\n",
    "\n",
    "As we can see that $\\mathbf v_t$ is a weighted sum over all past gradients multiplied by the according learning rate, which is the weight update in normal gradient descent. We just call it the scaled gradient. The weights deceases exponentially with the speed controlled by $\\gamma$.\n",
    "\n",
    "The following code block shows the weights for the past 40 time steps under various $\\gamma$s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"187pt\" version=\"1.1\" viewBox=\"0 0 235 187\" width=\"235pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 187.095313 \n",
       "L 235.965625 187.095313 \n",
       "L 235.965625 0 \n",
       "L 0 0 \n",
       "L 0 187.095313 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 27.103125 151.539062 \n",
       "L 222.403125 151.539062 \n",
       "L 222.403125 12.039062 \n",
       "L 27.103125 12.039062 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p7ef975b520)\" d=\"M 27.103125 12.039062 \n",
       "L 31.985625 19.014062 \n",
       "L 36.868125 25.640313 \n",
       "L 41.750625 31.93525 \n",
       "L 46.633125 37.915441 \n",
       "L 51.515625 43.596622 \n",
       "L 56.398125 48.993744 \n",
       "L 61.280625 54.12101 \n",
       "L 66.163125 58.991912 \n",
       "L 71.045625 63.61927 \n",
       "L 75.928125 68.015259 \n",
       "L 80.810625 72.19145 \n",
       "L 85.693125 76.15883 \n",
       "L 90.575625 79.927842 \n",
       "L 95.458125 83.508403 \n",
       "L 100.340625 86.909936 \n",
       "L 105.223125 90.141392 \n",
       "L 110.105625 93.211276 \n",
       "L 114.988125 96.127665 \n",
       "L 119.870625 98.898235 \n",
       "L 124.753125 101.530276 \n",
       "L 129.635625 104.030716 \n",
       "L 134.518125 106.406133 \n",
       "L 139.400625 108.662779 \n",
       "L 144.283125 110.806594 \n",
       "L 149.165625 112.843217 \n",
       "L 154.048125 114.778009 \n",
       "L 158.930625 116.616062 \n",
       "L 163.813125 118.362212 \n",
       "L 168.695625 120.021055 \n",
       "L 173.578125 121.596955 \n",
       "L 178.460625 123.09406 \n",
       "L 183.343125 124.51631 \n",
       "L 188.225625 125.867448 \n",
       "L 193.108125 127.151029 \n",
       "L 197.990625 128.37043 \n",
       "L 202.873125 129.528862 \n",
       "L 207.755625 130.629372 \n",
       "L 212.638125 131.674857 \n",
       "L 217.520625 132.668067 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_2\">\n",
       "    <path clip-path=\"url(#p7ef975b520)\" d=\"M 27.103125 12.039062 \n",
       "L 31.985625 25.989063 \n",
       "L 36.868125 38.544062 \n",
       "L 41.750625 49.843562 \n",
       "L 46.633125 60.013112 \n",
       "L 51.515625 69.165707 \n",
       "L 56.398125 77.403043 \n",
       "L 61.280625 84.816645 \n",
       "L 66.163125 91.488887 \n",
       "L 71.045625 97.493904 \n",
       "L 75.928125 102.89842 \n",
       "L 80.810625 107.762484 \n",
       "L 85.693125 112.140142 \n",
       "L 90.575625 116.080034 \n",
       "L 95.458125 119.625937 \n",
       "L 100.340625 122.81725 \n",
       "L 105.223125 125.689431 \n",
       "L 110.105625 128.274394 \n",
       "L 114.988125 130.600861 \n",
       "L 119.870625 132.694681 \n",
       "L 124.753125 134.579119 \n",
       "L 129.635625 136.275114 \n",
       "L 134.518125 137.801508 \n",
       "L 139.400625 139.175264 \n",
       "L 144.283125 140.411644 \n",
       "L 149.165625 141.524386 \n",
       "L 154.048125 142.525853 \n",
       "L 158.930625 143.427174 \n",
       "L 163.813125 144.238363 \n",
       "L 168.695625 144.968433 \n",
       "L 173.578125 145.625496 \n",
       "L 178.460625 146.216853 \n",
       "L 183.343125 146.749074 \n",
       "L 188.225625 147.228072 \n",
       "L 193.108125 147.659171 \n",
       "L 197.990625 148.047161 \n",
       "L 202.873125 148.396351 \n",
       "L 207.755625 148.710622 \n",
       "L 212.638125 148.993466 \n",
       "L 217.520625 149.248026 \n",
       "\" style=\"fill:none;stroke:#008000;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_3\">\n",
       "    <path clip-path=\"url(#p7ef975b520)\" d=\"M 27.103125 12.039062 \n",
       "L 31.985625 67.839062 \n",
       "L 36.868125 101.319063 \n",
       "L 41.750625 121.407063 \n",
       "L 46.633125 133.459862 \n",
       "L 51.515625 140.691542 \n",
       "L 56.398125 145.030551 \n",
       "L 61.280625 147.633955 \n",
       "L 66.163125 149.195998 \n",
       "L 71.045625 150.133224 \n",
       "L 75.928125 150.695559 \n",
       "L 80.810625 151.032961 \n",
       "L 85.693125 151.235401 \n",
       "L 90.575625 151.356866 \n",
       "L 95.458125 151.429744 \n",
       "L 100.340625 151.473472 \n",
       "L 105.223125 151.499708 \n",
       "L 110.105625 151.51545 \n",
       "L 114.988125 151.524895 \n",
       "L 119.870625 151.530562 \n",
       "L 124.753125 151.533962 \n",
       "L 129.635625 151.536002 \n",
       "L 134.518125 151.537226 \n",
       "L 139.400625 151.537961 \n",
       "L 144.283125 151.538401 \n",
       "L 149.165625 151.538666 \n",
       "L 154.048125 151.538825 \n",
       "L 158.930625 151.53892 \n",
       "L 163.813125 151.538977 \n",
       "L 168.695625 151.539011 \n",
       "L 173.578125 151.539032 \n",
       "L 178.460625 151.539044 \n",
       "L 183.343125 151.539051 \n",
       "L 188.225625 151.539056 \n",
       "L 193.108125 151.539059 \n",
       "L 197.990625 151.53906 \n",
       "L 202.873125 151.539061 \n",
       "L 207.755625 151.539062 \n",
       "L 212.638125 151.539062 \n",
       "L 217.520625 151.539062 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_4\">\n",
       "    <path clip-path=\"url(#p7ef975b520)\" d=\"M 27.103125 12.039062 \n",
       "L 31.985625 151.539062 \n",
       "L 36.868125 151.539062 \n",
       "L 41.750625 151.539062 \n",
       "L 46.633125 151.539062 \n",
       "L 51.515625 151.539062 \n",
       "L 56.398125 151.539062 \n",
       "L 61.280625 151.539062 \n",
       "L 66.163125 151.539062 \n",
       "L 71.045625 151.539062 \n",
       "L 75.928125 151.539062 \n",
       "L 80.810625 151.539062 \n",
       "L 85.693125 151.539062 \n",
       "L 90.575625 151.539062 \n",
       "L 95.458125 151.539062 \n",
       "L 100.340625 151.539062 \n",
       "L 105.223125 151.539062 \n",
       "L 110.105625 151.539062 \n",
       "L 114.988125 151.539062 \n",
       "L 119.870625 151.539062 \n",
       "L 124.753125 151.539062 \n",
       "L 129.635625 151.539062 \n",
       "L 134.518125 151.539062 \n",
       "L 139.400625 151.539062 \n",
       "L 144.283125 151.539062 \n",
       "L 149.165625 151.539062 \n",
       "L 154.048125 151.539062 \n",
       "L 158.930625 151.539062 \n",
       "L 163.813125 151.539062 \n",
       "L 168.695625 151.539062 \n",
       "L 173.578125 151.539062 \n",
       "L 178.460625 151.539062 \n",
       "L 183.343125 151.539062 \n",
       "L 188.225625 151.539062 \n",
       "L 193.108125 151.539062 \n",
       "L 197.990625 151.539062 \n",
       "L 202.873125 151.539062 \n",
       "L 207.755625 151.539062 \n",
       "L 212.638125 151.539062 \n",
       "L 217.520625 151.539062 \n",
       "\" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 222.403125 151.539062 \n",
       "L 222.403125 12.039062 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 27.103125 151.539062 \n",
       "L 27.103125 12.039062 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 27.103125 151.539062 \n",
       "L 222.403125 151.539062 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 27.103125 12.039062 \n",
       "L 222.403125 12.039062 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m27501600ae\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"md25c7b542f\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(23.921875 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"51.515625\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_8\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"51.515625\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(48.334375 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"75.92812500000001\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"75.92812500000001\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 10 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(69.56562500000001 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"100.34062500000002\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"100.34062500000002\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 15 -->\n",
       "      <g transform=\"translate(93.97812500000002 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"124.75312500000001\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_14\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"124.75312500000001\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 20 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(118.39062500000001 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_6\">\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"149.165625\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"149.165625\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_6\">\n",
       "      <!-- 25 -->\n",
       "      <g transform=\"translate(142.803125 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_7\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"173.57812500000003\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"173.57812500000003\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(167.21562500000002 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_8\">\n",
       "     <g id=\"line2d_19\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"197.99062500000002\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_20\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"197.99062500000002\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 35 -->\n",
       "      <g transform=\"translate(191.628125 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_9\">\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m27501600ae\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#md25c7b542f\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 40 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(216.040625 163.1375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_10\">\n",
       "     <!-- time -->\n",
       "     <defs>\n",
       "      <path d=\"M 52 44.1875 \n",
       "Q 55.375 50.25 60.0625 53.125 \n",
       "Q 64.75 56 71.09375 56 \n",
       "Q 79.640625 56 84.28125 50.015625 \n",
       "Q 88.921875 44.046875 88.921875 33.015625 \n",
       "L 88.921875 0 \n",
       "L 79.890625 0 \n",
       "L 79.890625 32.71875 \n",
       "Q 79.890625 40.578125 77.09375 44.375 \n",
       "Q 74.3125 48.1875 68.609375 48.1875 \n",
       "Q 61.625 48.1875 57.5625 43.546875 \n",
       "Q 53.515625 38.921875 53.515625 30.90625 \n",
       "L 53.515625 0 \n",
       "L 44.484375 0 \n",
       "L 44.484375 32.71875 \n",
       "Q 44.484375 40.625 41.703125 44.40625 \n",
       "Q 38.921875 48.1875 33.109375 48.1875 \n",
       "Q 26.21875 48.1875 22.15625 43.53125 \n",
       "Q 18.109375 38.875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.1875 51.21875 25.484375 53.609375 \n",
       "Q 29.78125 56 35.6875 56 \n",
       "Q 41.65625 56 45.828125 52.96875 \n",
       "Q 50 49.953125 52 44.1875 \n",
       "\" id=\"BitstreamVeraSans-Roman-6d\"/>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-65\"/>\n",
       "      <path d=\"M 9.421875 54.6875 \n",
       "L 18.40625 54.6875 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 64.59375 \n",
       "L 9.421875 64.59375 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-69\"/>\n",
       "      <path d=\"M 18.3125 70.21875 \n",
       "L 18.3125 54.6875 \n",
       "L 36.8125 54.6875 \n",
       "L 36.8125 47.703125 \n",
       "L 18.3125 47.703125 \n",
       "L 18.3125 18.015625 \n",
       "Q 18.3125 11.328125 20.140625 9.421875 \n",
       "Q 21.96875 7.515625 27.59375 7.515625 \n",
       "L 36.8125 7.515625 \n",
       "L 36.8125 0 \n",
       "L 27.59375 0 \n",
       "Q 17.1875 0 13.234375 3.875 \n",
       "Q 9.28125 7.765625 9.28125 18.015625 \n",
       "L 9.28125 47.703125 \n",
       "L 2.6875 47.703125 \n",
       "L 2.6875 54.6875 \n",
       "L 9.28125 54.6875 \n",
       "L 9.28125 70.21875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-74\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(113.45703125000001 177.815625)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-74\"/>\n",
       "      <use x=\"39.208984375\" xlink:href=\"#BitstreamVeraSans-Roman-69\"/>\n",
       "      <use x=\"66.9921875\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"164.404296875\" xlink:href=\"#BitstreamVeraSans-Roman-65\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"mac87077203\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_24\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"m99f98b09b9\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"151.5390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2e\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(7.200000000000001 154.2984375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"123.6390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_26\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"123.6390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.2 -->\n",
       "      <g transform=\"translate(7.200000000000001 126.3984375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"95.73906249999999\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"95.73906249999999\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.4 -->\n",
       "      <g transform=\"translate(7.200000000000001 98.4984375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"67.83906249999998\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"67.83906249999998\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_14\">\n",
       "      <!-- 0.6 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(7.200000000000001 70.59843749999999)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"39.93906249999999\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_32\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"39.93906249999999\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_15\">\n",
       "      <!-- 0.8 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "\" id=\"BitstreamVeraSans-Roman-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(7.200000000000001 42.69843750000001)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"27.103125000000002\" xlink:href=\"#mac87077203\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"222.40312500000002\" xlink:href=\"#m99f98b09b9\" y=\"12.0390625\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_16\">\n",
       "      <!-- 1.0 -->\n",
       "      <g transform=\"translate(7.200000000000001 14.798437500000006)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"legend_1\">\n",
       "    <g id=\"patch_7\">\n",
       "     <path d=\"M 83.098125 92.094063 \n",
       "L 216.403125 92.094063 \n",
       "L 216.403125 18.039062 \n",
       "L 83.098125 18.039062 \n",
       "z\n",
       "\" style=\"fill:#ffffff;stroke:#000000;stroke-linejoin:miter;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_35\">\n",
       "     <path d=\"M 91.498125 27.757188 \n",
       "L 108.298125 27.757188 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_36\"/>\n",
       "    <g id=\"text_17\">\n",
       "     <!-- gamma = 0.95 -->\n",
       "     <defs>\n",
       "      <path id=\"BitstreamVeraSans-Roman-20\"/>\n",
       "      <path d=\"M 34.28125 27.484375 \n",
       "Q 23.390625 27.484375 19.1875 25 \n",
       "Q 14.984375 22.515625 14.984375 16.5 \n",
       "Q 14.984375 11.71875 18.140625 8.90625 \n",
       "Q 21.296875 6.109375 26.703125 6.109375 \n",
       "Q 34.1875 6.109375 38.703125 11.40625 \n",
       "Q 43.21875 16.703125 43.21875 25.484375 \n",
       "L 43.21875 27.484375 \n",
       "z\n",
       "M 52.203125 31.203125 \n",
       "L 52.203125 0 \n",
       "L 43.21875 0 \n",
       "L 43.21875 8.296875 \n",
       "Q 40.140625 3.328125 35.546875 0.953125 \n",
       "Q 30.953125 -1.421875 24.3125 -1.421875 \n",
       "Q 15.921875 -1.421875 10.953125 3.296875 \n",
       "Q 6 8.015625 6 15.921875 \n",
       "Q 6 25.140625 12.171875 29.828125 \n",
       "Q 18.359375 34.515625 30.609375 34.515625 \n",
       "L 43.21875 34.515625 \n",
       "L 43.21875 35.40625 \n",
       "Q 43.21875 41.609375 39.140625 45 \n",
       "Q 35.0625 48.390625 27.6875 48.390625 \n",
       "Q 23 48.390625 18.546875 47.265625 \n",
       "Q 14.109375 46.140625 10.015625 43.890625 \n",
       "L 10.015625 52.203125 \n",
       "Q 14.9375 54.109375 19.578125 55.046875 \n",
       "Q 24.21875 56 28.609375 56 \n",
       "Q 40.484375 56 46.34375 49.84375 \n",
       "Q 52.203125 43.703125 52.203125 31.203125 \n",
       "\" id=\"BitstreamVeraSans-Roman-61\"/>\n",
       "      <path d=\"M 45.40625 27.984375 \n",
       "Q 45.40625 37.75 41.375 43.109375 \n",
       "Q 37.359375 48.484375 30.078125 48.484375 \n",
       "Q 22.859375 48.484375 18.828125 43.109375 \n",
       "Q 14.796875 37.75 14.796875 27.984375 \n",
       "Q 14.796875 18.265625 18.828125 12.890625 \n",
       "Q 22.859375 7.515625 30.078125 7.515625 \n",
       "Q 37.359375 7.515625 41.375 12.890625 \n",
       "Q 45.40625 18.265625 45.40625 27.984375 \n",
       "M 54.390625 6.78125 \n",
       "Q 54.390625 -7.171875 48.1875 -13.984375 \n",
       "Q 42 -20.796875 29.203125 -20.796875 \n",
       "Q 24.46875 -20.796875 20.265625 -20.09375 \n",
       "Q 16.0625 -19.390625 12.109375 -17.921875 \n",
       "L 12.109375 -9.1875 \n",
       "Q 16.0625 -11.328125 19.921875 -12.34375 \n",
       "Q 23.78125 -13.375 27.78125 -13.375 \n",
       "Q 36.625 -13.375 41.015625 -8.765625 \n",
       "Q 45.40625 -4.15625 45.40625 5.171875 \n",
       "L 45.40625 9.625 \n",
       "Q 42.625 4.78125 38.28125 2.390625 \n",
       "Q 33.9375 0 27.875 0 \n",
       "Q 17.828125 0 11.671875 7.65625 \n",
       "Q 5.515625 15.328125 5.515625 27.984375 \n",
       "Q 5.515625 40.671875 11.671875 48.328125 \n",
       "Q 17.828125 56 27.875 56 \n",
       "Q 33.9375 56 38.28125 53.609375 \n",
       "Q 42.625 51.21875 45.40625 46.390625 \n",
       "L 45.40625 54.6875 \n",
       "L 54.390625 54.6875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-67\"/>\n",
       "      <path d=\"M 10.984375 1.515625 \n",
       "L 10.984375 10.5 \n",
       "Q 14.703125 8.734375 18.5 7.8125 \n",
       "Q 22.3125 6.890625 25.984375 6.890625 \n",
       "Q 35.75 6.890625 40.890625 13.453125 \n",
       "Q 46.046875 20.015625 46.78125 33.40625 \n",
       "Q 43.953125 29.203125 39.59375 26.953125 \n",
       "Q 35.25 24.703125 29.984375 24.703125 \n",
       "Q 19.046875 24.703125 12.671875 31.3125 \n",
       "Q 6.296875 37.9375 6.296875 49.421875 \n",
       "Q 6.296875 60.640625 12.9375 67.421875 \n",
       "Q 19.578125 74.21875 30.609375 74.21875 \n",
       "Q 43.265625 74.21875 49.921875 64.515625 \n",
       "Q 56.59375 54.828125 56.59375 36.375 \n",
       "Q 56.59375 19.140625 48.40625 8.859375 \n",
       "Q 40.234375 -1.421875 26.421875 -1.421875 \n",
       "Q 22.703125 -1.421875 18.890625 -0.6875 \n",
       "Q 15.09375 0.046875 10.984375 1.515625 \n",
       "M 30.609375 32.421875 \n",
       "Q 37.25 32.421875 41.125 36.953125 \n",
       "Q 45.015625 41.5 45.015625 49.421875 \n",
       "Q 45.015625 57.28125 41.125 61.84375 \n",
       "Q 37.25 66.40625 30.609375 66.40625 \n",
       "Q 23.96875 66.40625 20.09375 61.84375 \n",
       "Q 16.21875 57.28125 16.21875 49.421875 \n",
       "Q 16.21875 41.5 20.09375 36.953125 \n",
       "Q 23.96875 32.421875 30.609375 32.421875 \n",
       "\" id=\"BitstreamVeraSans-Roman-39\"/>\n",
       "      <path d=\"M 10.59375 45.40625 \n",
       "L 73.1875 45.40625 \n",
       "L 73.1875 37.203125 \n",
       "L 10.59375 37.203125 \n",
       "z\n",
       "M 10.59375 25.484375 \n",
       "L 73.1875 25.484375 \n",
       "L 73.1875 17.1875 \n",
       "L 10.59375 17.1875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-3d\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(121.49812500000004 31.957187500000003)scale(0.12 -0.12)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-67\"/>\n",
       "      <use x=\"63.4765625\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"124.755859375\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"222.16796875\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"319.580078125\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"380.859375\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"412.646484375\" xlink:href=\"#BitstreamVeraSans-Roman-3d\"/>\n",
       "      <use x=\"496.435546875\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"528.22265625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      <use x=\"591.845703125\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "      <use x=\"623.6328125\" xlink:href=\"#BitstreamVeraSans-Roman-39\"/>\n",
       "      <use x=\"687.255859375\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_37\">\n",
       "     <path d=\"M 91.498125 45.370938 \n",
       "L 108.298125 45.370938 \n",
       "\" style=\"fill:none;stroke:#008000;stroke-linecap:square;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_38\"/>\n",
       "    <g id=\"text_18\">\n",
       "     <!-- gamma = 0.90 -->\n",
       "     <g transform=\"translate(121.49812500000004 49.570937500000014)scale(0.12 -0.12)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-67\"/>\n",
       "      <use x=\"63.4765625\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"124.755859375\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"222.16796875\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"319.580078125\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"380.859375\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"412.646484375\" xlink:href=\"#BitstreamVeraSans-Roman-3d\"/>\n",
       "      <use x=\"496.435546875\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"528.22265625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      <use x=\"591.845703125\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "      <use x=\"623.6328125\" xlink:href=\"#BitstreamVeraSans-Roman-39\"/>\n",
       "      <use x=\"687.255859375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_39\">\n",
       "     <path d=\"M 91.498125 62.984687 \n",
       "L 108.298125 62.984687 \n",
       "\" style=\"fill:none;stroke:#ff0000;stroke-linecap:square;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_40\"/>\n",
       "    <g id=\"text_19\">\n",
       "     <!-- gamma = 0.60 -->\n",
       "     <g transform=\"translate(121.49812500000004 67.1846875)scale(0.12 -0.12)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-67\"/>\n",
       "      <use x=\"63.4765625\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"124.755859375\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"222.16796875\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"319.580078125\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"380.859375\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"412.646484375\" xlink:href=\"#BitstreamVeraSans-Roman-3d\"/>\n",
       "      <use x=\"496.435546875\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"528.22265625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      <use x=\"591.845703125\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "      <use x=\"623.6328125\" xlink:href=\"#BitstreamVeraSans-Roman-36\"/>\n",
       "      <use x=\"687.255859375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"line2d_41\">\n",
       "     <path d=\"M 91.498125 80.598438 \n",
       "L 108.298125 80.598438 \n",
       "\" style=\"fill:none;stroke:#00bfbf;stroke-linecap:square;\"/>\n",
       "    </g>\n",
       "    <g id=\"line2d_42\"/>\n",
       "    <g id=\"text_20\">\n",
       "     <!-- gamma = 0.00 -->\n",
       "     <g transform=\"translate(121.49812500000004 84.7984375)scale(0.12 -0.12)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-67\"/>\n",
       "      <use x=\"63.4765625\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"124.755859375\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"222.16796875\" xlink:href=\"#BitstreamVeraSans-Roman-6d\"/>\n",
       "      <use x=\"319.580078125\" xlink:href=\"#BitstreamVeraSans-Roman-61\"/>\n",
       "      <use x=\"380.859375\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"412.646484375\" xlink:href=\"#BitstreamVeraSans-Roman-3d\"/>\n",
       "      <use x=\"496.435546875\" xlink:href=\"#BitstreamVeraSans-Roman-20\"/>\n",
       "      <use x=\"528.22265625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      <use x=\"591.845703125\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "      <use x=\"623.6328125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      <use x=\"687.255859375\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p7ef975b520\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"27.103125000000002\" y=\"12.0390625\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a1f15a748>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "gammas = [0.95, 0.9, 0.6, 0]\n",
    "d2l.set_figsize((3.5, 2.5))\n",
    "for gamma in gammas:\n",
    "    x = np.arange(40)\n",
    "    d2l.plt.plot(x, gamma ** x, label='gamma = %.2f'%gamma)\n",
    "d2l.plt.xlabel('time')\n",
    "d2l.plt.legend();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A small $\\gamma$ will let the velocity variable focus on more recent scaled gradients. While a large value will have the velocity variable to include more past scaled gradients. Compared to the plain gradient descent, momentum will make the weight updates be more consistent over time. It might smooth the training progress if $\\mathbf x$ enters the region that the gradient vary, or walk out region that is too flat.\n",
    "\n",
    "Also note that $\\frac{1}{1-\\gamma} = 1 + \\gamma + \\gamma^2 + \\cdots$. So all scaled gradients are similar to each other, e.g. $\\eta_t\\mathbf g_t\\approx \\eta\\mathbf g$ for all $t$s, then the momentum changes the weight updates from $\\eta\\mathbf g$ in normal gradient descent into $\\frac{\\eta}{1-\\gamma} \\mathbf g$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation from Scratch\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compared with mini-batch SGD, the momentum method needs to maintain a velocity variable of the same shape for each independent variable and a momentum hyperparameter is added to the hyperparameter category. In the implementation, we use the state variable states to represent the velocity variable in a more general sense.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def init_momentum_states(feature_dim):\n",
    "    v_w = np.zeros((feature_dim, 1))\n",
    "    v_b = np.zeros(1)\n",
    "    return (v_w, v_b)\n",
    "\n",
    "def sgd_momentum(params, states, hyperparams):\n",
    "    for p, v in zip(params, states):\n",
    "        v[:] = hyperparams['momentum'] * v + hyperparams['lr'] * p.grad\n",
    "        p[:] -= v"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Concise Implementation\n",
    "\n",
    "In Gluon, we only need to use momentum to define the momentum hyperparameter in the Trainer instance to implement momentum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we set the momentum hyperparameter momentum to 0.5, it can be treated as a mini-batch SGD: the mini-batch gradient here is the weighted average of twice the mini-batch gradient of the last two time steps.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 0.245, 0.019 sec/epoch\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"182pt\" version=\"1.1\" viewBox=\"0 0 258 182\" width=\"258pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 182.25625 \n",
       "L 258.595313 182.25625 \n",
       "L 258.595313 0 \n",
       "L 0 0 \n",
       "L 0 182.25625 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "L 48.14375 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p774146b303)\" d=\"M 61.16375 27.117684 \n",
       "L 74.18375 86.137751 \n",
       "L 87.20375 95.146848 \n",
       "L 100.22375 107.302317 \n",
       "L 113.24375 117.653069 \n",
       "L 126.26375 116.774624 \n",
       "L 139.28375 121.276531 \n",
       "L 152.30375 121.740997 \n",
       "L 165.32375 119.871625 \n",
       "L 178.34375 120.943464 \n",
       "L 191.36375 121.472272 \n",
       "L 204.38375 122.252078 \n",
       "L 217.40375 121.407121 \n",
       "L 230.42375 119.502829 \n",
       "L 243.44375 120.118858 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 48.14375 7.2 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m11b3da3909\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m11b3da3909\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"mabea09c371\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mabea09c371\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2e\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(40.192187499999996 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 96.96875 146.7 \n",
       "L 96.96875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#m11b3da3909\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#mabea09c371\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0.5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(89.0171875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 145.79375 146.7 \n",
       "L 145.79375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#m11b3da3909\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#mabea09c371\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 1.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(137.8421875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 194.61875 146.7 \n",
       "L 194.61875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#m11b3da3909\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#mabea09c371\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(186.66718750000004 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m11b3da3909\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#mabea09c371\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 2.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(235.49218750000003 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "\" id=\"BitstreamVeraSans-Roman-6f\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "\" id=\"BitstreamVeraSans-Roman-68\"/>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-65\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "\" id=\"BitstreamVeraSans-Roman-63\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(130.56562499999998 172.97656250000003)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-65\"/>\n",
       "      <use x=\"61.5234375\" xlink:href=\"#BitstreamVeraSans-Roman-70\"/>\n",
       "      <use x=\"125.0\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"186.181640625\" xlink:href=\"#BitstreamVeraSans-Roman-63\"/>\n",
       "      <use x=\"241.162109375\" xlink:href=\"#BitstreamVeraSans-Roman-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"m7995b161d6\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"m998acf4086\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.22 -->\n",
       "      <g transform=\"translate(21.878124999999997 149.459375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 125.238462 \n",
       "L 243.44375 125.238462 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.24 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 127.99783653846151)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 103.776923 \n",
       "L 243.44375 103.776923 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.26 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 106.53629807692306)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 82.315385 \n",
       "L 243.44375 82.315385 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.28 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "\" id=\"BitstreamVeraSans-Roman-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 85.07475961538455)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 60.853846 \n",
       "L 243.44375 60.853846 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 63.6132211538461)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 39.392308 \n",
       "L 243.44375 39.392308 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.32 -->\n",
       "      <g transform=\"translate(21.878124999999997 42.151682692307645)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <path clip-path=\"url(#p774146b303)\" d=\"M 48.14375 17.930769 \n",
       "L 243.44375 17.930769 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m7995b161d6\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m998acf4086\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.34 -->\n",
       "      <g transform=\"translate(21.878124999999997 20.690144230769135)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-6c\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "\" id=\"BitstreamVeraSans-Roman-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437499999997 86.60781250000002)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-6c\"/>\n",
       "      <use x=\"27.783203125\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"88.96484375\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "      <use x=\"141.064453125\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p774146b303\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"48.14375\" y=\"7.200000000000017\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a1f1b8940>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def train_momentum(lr, momentum, num_epochs=2):\n",
    "    d2l.train_ch10(torch.optim.SGD, {'lr': lr, 'momentum': momentum}, data_iter, \n",
    "                   feature_dim, num_epochs)\n",
    "    \n",
    "data_iter, feature_dim = d2l.get_data_ch10(batch_size=10)\n",
    "train_momentum(0.02, 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we increase the momentum hyperparameter momentum to 0.9, it can still be treated as a mini-batch SGD: the mini-batch gradient here will be the weighted average of ten times the mini-batch gradient of the last 10 time steps. Now we keep the learning rate at 0.02."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 0.266, 0.014 sec/epoch\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"182pt\" version=\"1.1\" viewBox=\"0 0 258 182\" width=\"258pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 182.25625 \n",
       "L 258.595313 182.25625 \n",
       "L 258.595313 0 \n",
       "L 0 0 \n",
       "L 0 182.25625 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "L 48.14375 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#pefe36d0506)\" d=\"M 61.16375 105.992106 \n",
       "L 74.18375 108.593199 \n",
       "L 87.20375 92.349152 \n",
       "L 100.22375 116.28144 \n",
       "L 113.24375 114.397096 \n",
       "L 126.26375 103.277295 \n",
       "L 139.28375 112.230484 \n",
       "L 152.30375 119.04863 \n",
       "L 165.32375 113.744855 \n",
       "L 178.34375 111.221929 \n",
       "L 191.36375 118.362034 \n",
       "L 204.38375 110.660106 \n",
       "L 217.40375 93.647065 \n",
       "L 230.42375 111.46888 \n",
       "L 243.44375 97.199159 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 48.14375 7.2 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m9d9a4968dd\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m9d9a4968dd\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"ma6e5b725a7\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#ma6e5b725a7\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2e\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(40.192187499999996 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 96.96875 146.7 \n",
       "L 96.96875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#m9d9a4968dd\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#ma6e5b725a7\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0.5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(89.0171875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 145.79375 146.7 \n",
       "L 145.79375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#m9d9a4968dd\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#ma6e5b725a7\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 1.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(137.8421875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 194.61875 146.7 \n",
       "L 194.61875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#m9d9a4968dd\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#ma6e5b725a7\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(186.66718750000004 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m9d9a4968dd\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#ma6e5b725a7\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 2.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(235.49218750000003 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "\" id=\"BitstreamVeraSans-Roman-6f\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "\" id=\"BitstreamVeraSans-Roman-68\"/>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-65\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "\" id=\"BitstreamVeraSans-Roman-63\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(130.56562499999998 172.97656250000003)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-65\"/>\n",
       "      <use x=\"61.5234375\" xlink:href=\"#BitstreamVeraSans-Roman-70\"/>\n",
       "      <use x=\"125.0\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"186.181640625\" xlink:href=\"#BitstreamVeraSans-Roman-63\"/>\n",
       "      <use x=\"241.162109375\" xlink:href=\"#BitstreamVeraSans-Roman-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"mc68cfeb6d2\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"me861388f62\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.22 -->\n",
       "      <g transform=\"translate(21.878124999999997 149.459375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 125.238462 \n",
       "L 243.44375 125.238462 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.24 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 127.99783653846151)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 103.776923 \n",
       "L 243.44375 103.776923 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.26 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 106.53629807692306)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 82.315385 \n",
       "L 243.44375 82.315385 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.28 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "\" id=\"BitstreamVeraSans-Roman-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 85.07475961538455)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 60.853846 \n",
       "L 243.44375 60.853846 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 63.6132211538461)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 39.392308 \n",
       "L 243.44375 39.392308 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.32 -->\n",
       "      <g transform=\"translate(21.878124999999997 42.151682692307645)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <path clip-path=\"url(#pefe36d0506)\" d=\"M 48.14375 17.930769 \n",
       "L 243.44375 17.930769 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#mc68cfeb6d2\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#me861388f62\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.34 -->\n",
       "      <g transform=\"translate(21.878124999999997 20.690144230769135)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-6c\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "\" id=\"BitstreamVeraSans-Roman-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437499999997 86.60781250000002)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-6c\"/>\n",
       "      <use x=\"27.783203125\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"88.96484375\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "      <use x=\"141.064453125\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"pefe36d0506\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"48.14375\" y=\"7.200000000000017\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a1f1b02e8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_momentum(0.02, 0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the value change of the objective function is not smooth enough at later stages of iteration. Intuitively, ten times the mini-batch gradient is five times larger than two times the mini-batch gradient, so we can try to reduce the learning rate to 1/5 of its original value. Now, the value change of the objective function becomes smoother after its period of decline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 0.245, 0.018 sec/epoch\n"
     ]
    },
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       "  \"https://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Created with matplotlib (https://matplotlib.org/) -->\n",
       "<svg height=\"182pt\" version=\"1.1\" viewBox=\"0 0 258 182\" width=\"258pt\" xmlns=\"https://www.w3.org/2000/svg\" xmlns:xlink=\"https://www.w3.org/1999/xlink\">\n",
       " <defs>\n",
       "  <style type=\"text/css\">\n",
       "*{stroke-linecap:butt;stroke-linejoin:round;stroke-miterlimit:100000;}\n",
       "  </style>\n",
       " </defs>\n",
       " <g id=\"figure_1\">\n",
       "  <g id=\"patch_1\">\n",
       "   <path d=\"M 0 182.25625 \n",
       "L 258.595313 182.25625 \n",
       "L 258.595313 0 \n",
       "L 0 0 \n",
       "L 0 182.25625 \n",
       "z\n",
       "\" style=\"fill:none;\"/>\n",
       "  </g>\n",
       "  <g id=\"axes_1\">\n",
       "   <g id=\"patch_2\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "L 48.14375 7.2 \n",
       "z\n",
       "\" style=\"fill:#ffffff;\"/>\n",
       "   </g>\n",
       "   <g id=\"line2d_1\">\n",
       "    <path clip-path=\"url(#p0dc0bac988)\" d=\"M 61.16375 -8.680518 \n",
       "M 62.349074 -1 \n",
       "L 74.18375 75.684922 \n",
       "L 87.20375 108.460885 \n",
       "L 100.22375 117.515788 \n",
       "L 113.24375 117.569381 \n",
       "L 126.26375 116.528783 \n",
       "L 139.28375 121.36888 \n",
       "L 152.30375 117.04496 \n",
       "L 165.32375 119.061949 \n",
       "L 178.34375 119.176808 \n",
       "L 191.36375 121.148063 \n",
       "L 204.38375 121.951914 \n",
       "L 217.40375 121.950768 \n",
       "L 230.42375 122.01866 \n",
       "L 243.44375 119.839829 \n",
       "\" style=\"fill:none;stroke:#0000ff;stroke-linecap:square;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_3\">\n",
       "    <path d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_4\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_5\">\n",
       "    <path d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"patch_6\">\n",
       "    <path d=\"M 48.14375 7.2 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;\"/>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_1\">\n",
       "    <g id=\"xtick_1\">\n",
       "     <g id=\"line2d_2\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 146.7 \n",
       "L 48.14375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_3\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 -4 \n",
       "\" id=\"m8df6ddecb7\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m8df6ddecb7\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_4\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 0 4 \n",
       "\" id=\"m2e3e6911ac\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2e3e6911ac\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_1\">\n",
       "      <!-- 0.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 66.40625 \n",
       "Q 24.171875 66.40625 20.328125 58.90625 \n",
       "Q 16.5 51.421875 16.5 36.375 \n",
       "Q 16.5 21.390625 20.328125 13.890625 \n",
       "Q 24.171875 6.390625 31.78125 6.390625 \n",
       "Q 39.453125 6.390625 43.28125 13.890625 \n",
       "Q 47.125 21.390625 47.125 36.375 \n",
       "Q 47.125 51.421875 43.28125 58.90625 \n",
       "Q 39.453125 66.40625 31.78125 66.40625 \n",
       "M 31.78125 74.21875 \n",
       "Q 44.046875 74.21875 50.515625 64.515625 \n",
       "Q 56.984375 54.828125 56.984375 36.375 \n",
       "Q 56.984375 17.96875 50.515625 8.265625 \n",
       "Q 44.046875 -1.421875 31.78125 -1.421875 \n",
       "Q 19.53125 -1.421875 13.0625 8.265625 \n",
       "Q 6.59375 17.96875 6.59375 36.375 \n",
       "Q 6.59375 54.828125 13.0625 64.515625 \n",
       "Q 19.53125 74.21875 31.78125 74.21875 \n",
       "\" id=\"BitstreamVeraSans-Roman-30\"/>\n",
       "       <path d=\"M 10.6875 12.40625 \n",
       "L 21 12.40625 \n",
       "L 21 0 \n",
       "L 10.6875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-2e\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(40.192187499999996 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_2\">\n",
       "     <g id=\"line2d_5\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 96.96875 146.7 \n",
       "L 96.96875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_6\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#m8df6ddecb7\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_7\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"96.96875\" xlink:href=\"#m2e3e6911ac\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_2\">\n",
       "      <!-- 0.5 -->\n",
       "      <defs>\n",
       "       <path d=\"M 10.796875 72.90625 \n",
       "L 49.515625 72.90625 \n",
       "L 49.515625 64.59375 \n",
       "L 19.828125 64.59375 \n",
       "L 19.828125 46.734375 \n",
       "Q 21.96875 47.46875 24.109375 47.828125 \n",
       "Q 26.265625 48.1875 28.421875 48.1875 \n",
       "Q 40.625 48.1875 47.75 41.5 \n",
       "Q 54.890625 34.8125 54.890625 23.390625 \n",
       "Q 54.890625 11.625 47.5625 5.09375 \n",
       "Q 40.234375 -1.421875 26.90625 -1.421875 \n",
       "Q 22.3125 -1.421875 17.546875 -0.640625 \n",
       "Q 12.796875 0.140625 7.71875 1.703125 \n",
       "L 7.71875 11.625 \n",
       "Q 12.109375 9.234375 16.796875 8.0625 \n",
       "Q 21.484375 6.890625 26.703125 6.890625 \n",
       "Q 35.15625 6.890625 40.078125 11.328125 \n",
       "Q 45.015625 15.765625 45.015625 23.390625 \n",
       "Q 45.015625 31 40.078125 35.4375 \n",
       "Q 35.15625 39.890625 26.703125 39.890625 \n",
       "Q 22.75 39.890625 18.8125 39.015625 \n",
       "Q 14.890625 38.140625 10.796875 36.28125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-35\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(89.0171875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_3\">\n",
       "     <g id=\"line2d_8\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 145.79375 146.7 \n",
       "L 145.79375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_9\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#m8df6ddecb7\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_10\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"145.79375\" xlink:href=\"#m2e3e6911ac\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_3\">\n",
       "      <!-- 1.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 12.40625 8.296875 \n",
       "L 28.515625 8.296875 \n",
       "L 28.515625 63.921875 \n",
       "L 10.984375 60.40625 \n",
       "L 10.984375 69.390625 \n",
       "L 28.421875 72.90625 \n",
       "L 38.28125 72.90625 \n",
       "L 38.28125 8.296875 \n",
       "L 54.390625 8.296875 \n",
       "L 54.390625 0 \n",
       "L 12.40625 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-31\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(137.8421875 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_4\">\n",
       "     <g id=\"line2d_11\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 194.61875 146.7 \n",
       "L 194.61875 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_12\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#m8df6ddecb7\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_13\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"194.61875000000003\" xlink:href=\"#m2e3e6911ac\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_4\">\n",
       "      <!-- 1.5 -->\n",
       "      <g transform=\"translate(186.66718750000004 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-31\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-35\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"xtick_5\">\n",
       "     <g id=\"line2d_14\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 243.44375 146.7 \n",
       "L 243.44375 7.2 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_15\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m8df6ddecb7\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_16\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m2e3e6911ac\" y=\"7.200000000000017\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_5\">\n",
       "      <!-- 2.0 -->\n",
       "      <defs>\n",
       "       <path d=\"M 19.1875 8.296875 \n",
       "L 53.609375 8.296875 \n",
       "L 53.609375 0 \n",
       "L 7.328125 0 \n",
       "L 7.328125 8.296875 \n",
       "Q 12.9375 14.109375 22.625 23.890625 \n",
       "Q 32.328125 33.6875 34.8125 36.53125 \n",
       "Q 39.546875 41.84375 41.421875 45.53125 \n",
       "Q 43.3125 49.21875 43.3125 52.78125 \n",
       "Q 43.3125 58.59375 39.234375 62.25 \n",
       "Q 35.15625 65.921875 28.609375 65.921875 \n",
       "Q 23.96875 65.921875 18.8125 64.3125 \n",
       "Q 13.671875 62.703125 7.8125 59.421875 \n",
       "L 7.8125 69.390625 \n",
       "Q 13.765625 71.78125 18.9375 73 \n",
       "Q 24.125 74.21875 28.421875 74.21875 \n",
       "Q 39.75 74.21875 46.484375 68.546875 \n",
       "Q 53.21875 62.890625 53.21875 53.421875 \n",
       "Q 53.21875 48.921875 51.53125 44.890625 \n",
       "Q 49.859375 40.875 45.40625 35.40625 \n",
       "Q 44.1875 33.984375 37.640625 27.21875 \n",
       "Q 31.109375 20.453125 19.1875 8.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-32\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(235.49218750000003 158.29843750000003)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_6\">\n",
       "     <!-- epoch -->\n",
       "     <defs>\n",
       "      <path d=\"M 18.109375 8.203125 \n",
       "L 18.109375 -20.796875 \n",
       "L 9.078125 -20.796875 \n",
       "L 9.078125 54.6875 \n",
       "L 18.109375 54.6875 \n",
       "L 18.109375 46.390625 \n",
       "Q 20.953125 51.265625 25.265625 53.625 \n",
       "Q 29.59375 56 35.59375 56 \n",
       "Q 45.5625 56 51.78125 48.09375 \n",
       "Q 58.015625 40.1875 58.015625 27.296875 \n",
       "Q 58.015625 14.40625 51.78125 6.484375 \n",
       "Q 45.5625 -1.421875 35.59375 -1.421875 \n",
       "Q 29.59375 -1.421875 25.265625 0.953125 \n",
       "Q 20.953125 3.328125 18.109375 8.203125 \n",
       "M 48.6875 27.296875 \n",
       "Q 48.6875 37.203125 44.609375 42.84375 \n",
       "Q 40.53125 48.484375 33.40625 48.484375 \n",
       "Q 26.265625 48.484375 22.1875 42.84375 \n",
       "Q 18.109375 37.203125 18.109375 27.296875 \n",
       "Q 18.109375 17.390625 22.1875 11.75 \n",
       "Q 26.265625 6.109375 33.40625 6.109375 \n",
       "Q 40.53125 6.109375 44.609375 11.75 \n",
       "Q 48.6875 17.390625 48.6875 27.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-70\"/>\n",
       "      <path d=\"M 30.609375 48.390625 \n",
       "Q 23.390625 48.390625 19.1875 42.75 \n",
       "Q 14.984375 37.109375 14.984375 27.296875 \n",
       "Q 14.984375 17.484375 19.15625 11.84375 \n",
       "Q 23.34375 6.203125 30.609375 6.203125 \n",
       "Q 37.796875 6.203125 41.984375 11.859375 \n",
       "Q 46.1875 17.53125 46.1875 27.296875 \n",
       "Q 46.1875 37.015625 41.984375 42.703125 \n",
       "Q 37.796875 48.390625 30.609375 48.390625 \n",
       "M 30.609375 56 \n",
       "Q 42.328125 56 49.015625 48.375 \n",
       "Q 55.71875 40.765625 55.71875 27.296875 \n",
       "Q 55.71875 13.875 49.015625 6.21875 \n",
       "Q 42.328125 -1.421875 30.609375 -1.421875 \n",
       "Q 18.84375 -1.421875 12.171875 6.21875 \n",
       "Q 5.515625 13.875 5.515625 27.296875 \n",
       "Q 5.515625 40.765625 12.171875 48.375 \n",
       "Q 18.84375 56 30.609375 56 \n",
       "\" id=\"BitstreamVeraSans-Roman-6f\"/>\n",
       "      <path d=\"M 54.890625 33.015625 \n",
       "L 54.890625 0 \n",
       "L 45.90625 0 \n",
       "L 45.90625 32.71875 \n",
       "Q 45.90625 40.484375 42.875 44.328125 \n",
       "Q 39.84375 48.1875 33.796875 48.1875 \n",
       "Q 26.515625 48.1875 22.3125 43.546875 \n",
       "Q 18.109375 38.921875 18.109375 30.90625 \n",
       "L 18.109375 0 \n",
       "L 9.078125 0 \n",
       "L 9.078125 75.984375 \n",
       "L 18.109375 75.984375 \n",
       "L 18.109375 46.1875 \n",
       "Q 21.34375 51.125 25.703125 53.5625 \n",
       "Q 30.078125 56 35.796875 56 \n",
       "Q 45.21875 56 50.046875 50.171875 \n",
       "Q 54.890625 44.34375 54.890625 33.015625 \n",
       "\" id=\"BitstreamVeraSans-Roman-68\"/>\n",
       "      <path d=\"M 56.203125 29.59375 \n",
       "L 56.203125 25.203125 \n",
       "L 14.890625 25.203125 \n",
       "Q 15.484375 15.921875 20.484375 11.0625 \n",
       "Q 25.484375 6.203125 34.421875 6.203125 \n",
       "Q 39.59375 6.203125 44.453125 7.46875 \n",
       "Q 49.3125 8.734375 54.109375 11.28125 \n",
       "L 54.109375 2.78125 \n",
       "Q 49.265625 0.734375 44.1875 -0.34375 \n",
       "Q 39.109375 -1.421875 33.890625 -1.421875 \n",
       "Q 20.796875 -1.421875 13.15625 6.1875 \n",
       "Q 5.515625 13.8125 5.515625 26.8125 \n",
       "Q 5.515625 40.234375 12.765625 48.109375 \n",
       "Q 20.015625 56 32.328125 56 \n",
       "Q 43.359375 56 49.78125 48.890625 \n",
       "Q 56.203125 41.796875 56.203125 29.59375 \n",
       "M 47.21875 32.234375 \n",
       "Q 47.125 39.59375 43.09375 43.984375 \n",
       "Q 39.0625 48.390625 32.421875 48.390625 \n",
       "Q 24.90625 48.390625 20.390625 44.140625 \n",
       "Q 15.875 39.890625 15.1875 32.171875 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-65\"/>\n",
       "      <path d=\"M 48.78125 52.59375 \n",
       "L 48.78125 44.1875 \n",
       "Q 44.96875 46.296875 41.140625 47.34375 \n",
       "Q 37.3125 48.390625 33.40625 48.390625 \n",
       "Q 24.65625 48.390625 19.8125 42.84375 \n",
       "Q 14.984375 37.3125 14.984375 27.296875 \n",
       "Q 14.984375 17.28125 19.8125 11.734375 \n",
       "Q 24.65625 6.203125 33.40625 6.203125 \n",
       "Q 37.3125 6.203125 41.140625 7.25 \n",
       "Q 44.96875 8.296875 48.78125 10.40625 \n",
       "L 48.78125 2.09375 \n",
       "Q 45.015625 0.34375 40.984375 -0.53125 \n",
       "Q 36.96875 -1.421875 32.421875 -1.421875 \n",
       "Q 20.0625 -1.421875 12.78125 6.34375 \n",
       "Q 5.515625 14.109375 5.515625 27.296875 \n",
       "Q 5.515625 40.671875 12.859375 48.328125 \n",
       "Q 20.21875 56 33.015625 56 \n",
       "Q 37.15625 56 41.109375 55.140625 \n",
       "Q 45.0625 54.296875 48.78125 52.59375 \n",
       "\" id=\"BitstreamVeraSans-Roman-63\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(130.56562499999998 172.97656250000003)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-65\"/>\n",
       "      <use x=\"61.5234375\" xlink:href=\"#BitstreamVeraSans-Roman-70\"/>\n",
       "      <use x=\"125.0\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"186.181640625\" xlink:href=\"#BitstreamVeraSans-Roman-63\"/>\n",
       "      <use x=\"241.162109375\" xlink:href=\"#BitstreamVeraSans-Roman-68\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "   <g id=\"matplotlib.axis_2\">\n",
       "    <g id=\"ytick_1\">\n",
       "     <g id=\"line2d_17\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 146.7 \n",
       "L 243.44375 146.7 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_18\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L 4 0 \n",
       "\" id=\"m2ca2c9356c\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_19\">\n",
       "      <defs>\n",
       "       <path d=\"M 0 0 \n",
       "L -4 0 \n",
       "\" id=\"m3b4377a1fb\" style=\"stroke:#000000;stroke-width:0.5;\"/>\n",
       "      </defs>\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"146.70000000000002\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_7\">\n",
       "      <!-- 0.22 -->\n",
       "      <g transform=\"translate(21.878124999999997 149.459375)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_2\">\n",
       "     <g id=\"line2d_20\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 125.238462 \n",
       "L 243.44375 125.238462 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_21\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_22\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"125.23846153846154\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_8\">\n",
       "      <!-- 0.24 -->\n",
       "      <defs>\n",
       "       <path d=\"M 37.796875 64.3125 \n",
       "L 12.890625 25.390625 \n",
       "L 37.796875 25.390625 \n",
       "z\n",
       "M 35.203125 72.90625 \n",
       "L 47.609375 72.90625 \n",
       "L 47.609375 25.390625 \n",
       "L 58.015625 25.390625 \n",
       "L 58.015625 17.1875 \n",
       "L 47.609375 17.1875 \n",
       "L 47.609375 0 \n",
       "L 37.796875 0 \n",
       "L 37.796875 17.1875 \n",
       "L 4.890625 17.1875 \n",
       "L 4.890625 26.703125 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-34\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 127.99783653846151)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_3\">\n",
       "     <g id=\"line2d_23\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 103.776923 \n",
       "L 243.44375 103.776923 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_24\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_25\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"103.77692307692308\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_9\">\n",
       "      <!-- 0.26 -->\n",
       "      <defs>\n",
       "       <path d=\"M 33.015625 40.375 \n",
       "Q 26.375 40.375 22.484375 35.828125 \n",
       "Q 18.609375 31.296875 18.609375 23.390625 \n",
       "Q 18.609375 15.53125 22.484375 10.953125 \n",
       "Q 26.375 6.390625 33.015625 6.390625 \n",
       "Q 39.65625 6.390625 43.53125 10.953125 \n",
       "Q 47.40625 15.53125 47.40625 23.390625 \n",
       "Q 47.40625 31.296875 43.53125 35.828125 \n",
       "Q 39.65625 40.375 33.015625 40.375 \n",
       "M 52.59375 71.296875 \n",
       "L 52.59375 62.3125 \n",
       "Q 48.875 64.0625 45.09375 64.984375 \n",
       "Q 41.3125 65.921875 37.59375 65.921875 \n",
       "Q 27.828125 65.921875 22.671875 59.328125 \n",
       "Q 17.53125 52.734375 16.796875 39.40625 \n",
       "Q 19.671875 43.65625 24.015625 45.921875 \n",
       "Q 28.375 48.1875 33.59375 48.1875 \n",
       "Q 44.578125 48.1875 50.953125 41.515625 \n",
       "Q 57.328125 34.859375 57.328125 23.390625 \n",
       "Q 57.328125 12.15625 50.6875 5.359375 \n",
       "Q 44.046875 -1.421875 33.015625 -1.421875 \n",
       "Q 20.359375 -1.421875 13.671875 8.265625 \n",
       "Q 6.984375 17.96875 6.984375 36.375 \n",
       "Q 6.984375 53.65625 15.1875 63.9375 \n",
       "Q 23.390625 74.21875 37.203125 74.21875 \n",
       "Q 40.921875 74.21875 44.703125 73.484375 \n",
       "Q 48.484375 72.75 52.59375 71.296875 \n",
       "\" id=\"BitstreamVeraSans-Roman-36\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 106.53629807692306)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-36\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_4\">\n",
       "     <g id=\"line2d_26\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 82.315385 \n",
       "L 243.44375 82.315385 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_27\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_28\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"82.31538461538457\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_10\">\n",
       "      <!-- 0.28 -->\n",
       "      <defs>\n",
       "       <path d=\"M 31.78125 34.625 \n",
       "Q 24.75 34.625 20.71875 30.859375 \n",
       "Q 16.703125 27.09375 16.703125 20.515625 \n",
       "Q 16.703125 13.921875 20.71875 10.15625 \n",
       "Q 24.75 6.390625 31.78125 6.390625 \n",
       "Q 38.8125 6.390625 42.859375 10.171875 \n",
       "Q 46.921875 13.96875 46.921875 20.515625 \n",
       "Q 46.921875 27.09375 42.890625 30.859375 \n",
       "Q 38.875 34.625 31.78125 34.625 \n",
       "M 21.921875 38.8125 \n",
       "Q 15.578125 40.375 12.03125 44.71875 \n",
       "Q 8.5 49.078125 8.5 55.328125 \n",
       "Q 8.5 64.0625 14.71875 69.140625 \n",
       "Q 20.953125 74.21875 31.78125 74.21875 \n",
       "Q 42.671875 74.21875 48.875 69.140625 \n",
       "Q 55.078125 64.0625 55.078125 55.328125 \n",
       "Q 55.078125 49.078125 51.53125 44.71875 \n",
       "Q 48 40.375 41.703125 38.8125 \n",
       "Q 48.828125 37.15625 52.796875 32.3125 \n",
       "Q 56.78125 27.484375 56.78125 20.515625 \n",
       "Q 56.78125 9.90625 50.3125 4.234375 \n",
       "Q 43.84375 -1.421875 31.78125 -1.421875 \n",
       "Q 19.734375 -1.421875 13.25 4.234375 \n",
       "Q 6.78125 9.90625 6.78125 20.515625 \n",
       "Q 6.78125 27.484375 10.78125 32.3125 \n",
       "Q 14.796875 37.15625 21.921875 38.8125 \n",
       "M 18.3125 54.390625 \n",
       "Q 18.3125 48.734375 21.84375 45.5625 \n",
       "Q 25.390625 42.390625 31.78125 42.390625 \n",
       "Q 38.140625 42.390625 41.71875 45.5625 \n",
       "Q 45.3125 48.734375 45.3125 54.390625 \n",
       "Q 45.3125 60.0625 41.71875 63.234375 \n",
       "Q 38.140625 66.40625 31.78125 66.40625 \n",
       "Q 25.390625 66.40625 21.84375 63.234375 \n",
       "Q 18.3125 60.0625 18.3125 54.390625 \n",
       "\" id=\"BitstreamVeraSans-Roman-38\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 85.07475961538455)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-38\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_5\">\n",
       "     <g id=\"line2d_29\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 60.853846 \n",
       "L 243.44375 60.853846 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_30\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_31\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"60.85384615384612\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_11\">\n",
       "      <!-- 0.30 -->\n",
       "      <defs>\n",
       "       <path d=\"M 40.578125 39.3125 \n",
       "Q 47.65625 37.796875 51.625 33 \n",
       "Q 55.609375 28.21875 55.609375 21.1875 \n",
       "Q 55.609375 10.40625 48.1875 4.484375 \n",
       "Q 40.765625 -1.421875 27.09375 -1.421875 \n",
       "Q 22.515625 -1.421875 17.65625 -0.515625 \n",
       "Q 12.796875 0.390625 7.625 2.203125 \n",
       "L 7.625 11.71875 \n",
       "Q 11.71875 9.328125 16.59375 8.109375 \n",
       "Q 21.484375 6.890625 26.8125 6.890625 \n",
       "Q 36.078125 6.890625 40.9375 10.546875 \n",
       "Q 45.796875 14.203125 45.796875 21.1875 \n",
       "Q 45.796875 27.640625 41.28125 31.265625 \n",
       "Q 36.765625 34.90625 28.71875 34.90625 \n",
       "L 20.21875 34.90625 \n",
       "L 20.21875 43.015625 \n",
       "L 29.109375 43.015625 \n",
       "Q 36.375 43.015625 40.234375 45.921875 \n",
       "Q 44.09375 48.828125 44.09375 54.296875 \n",
       "Q 44.09375 59.90625 40.109375 62.90625 \n",
       "Q 36.140625 65.921875 28.71875 65.921875 \n",
       "Q 24.65625 65.921875 20.015625 65.03125 \n",
       "Q 15.375 64.15625 9.8125 62.3125 \n",
       "L 9.8125 71.09375 \n",
       "Q 15.4375 72.65625 20.34375 73.4375 \n",
       "Q 25.25 74.21875 29.59375 74.21875 \n",
       "Q 40.828125 74.21875 47.359375 69.109375 \n",
       "Q 53.90625 64.015625 53.90625 55.328125 \n",
       "Q 53.90625 49.265625 50.4375 45.09375 \n",
       "Q 46.96875 40.921875 40.578125 39.3125 \n",
       "\" id=\"BitstreamVeraSans-Roman-33\"/>\n",
       "      </defs>\n",
       "      <g transform=\"translate(21.878124999999997 63.6132211538461)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_6\">\n",
       "     <g id=\"line2d_32\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 39.392308 \n",
       "L 243.44375 39.392308 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_33\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_34\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"39.39230769230767\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_12\">\n",
       "      <!-- 0.32 -->\n",
       "      <g transform=\"translate(21.878124999999997 42.151682692307645)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-32\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"ytick_7\">\n",
       "     <g id=\"line2d_35\">\n",
       "      <path clip-path=\"url(#p0dc0bac988)\" d=\"M 48.14375 17.930769 \n",
       "L 243.44375 17.930769 \n",
       "\" style=\"fill:none;stroke:#000000;stroke-dasharray:1.000000,3.000000;stroke-dashoffset:0.0;stroke-width:0.5;\"/>\n",
       "     </g>\n",
       "     <g id=\"line2d_36\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"48.14375\" xlink:href=\"#m2ca2c9356c\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"line2d_37\">\n",
       "      <g>\n",
       "       <use style=\"stroke:#000000;stroke-width:0.5;\" x=\"243.44375000000002\" xlink:href=\"#m3b4377a1fb\" y=\"17.930769230769158\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "     <g id=\"text_13\">\n",
       "      <!-- 0.34 -->\n",
       "      <g transform=\"translate(21.878124999999997 20.690144230769135)scale(0.1 -0.1)\">\n",
       "       <use xlink:href=\"#BitstreamVeraSans-Roman-30\"/>\n",
       "       <use x=\"63.623046875\" xlink:href=\"#BitstreamVeraSans-Roman-2e\"/>\n",
       "       <use x=\"95.41015625\" xlink:href=\"#BitstreamVeraSans-Roman-33\"/>\n",
       "       <use x=\"159.033203125\" xlink:href=\"#BitstreamVeraSans-Roman-34\"/>\n",
       "      </g>\n",
       "     </g>\n",
       "    </g>\n",
       "    <g id=\"text_14\">\n",
       "     <!-- loss -->\n",
       "     <defs>\n",
       "      <path d=\"M 9.421875 75.984375 \n",
       "L 18.40625 75.984375 \n",
       "L 18.40625 0 \n",
       "L 9.421875 0 \n",
       "z\n",
       "\" id=\"BitstreamVeraSans-Roman-6c\"/>\n",
       "      <path d=\"M 44.28125 53.078125 \n",
       "L 44.28125 44.578125 \n",
       "Q 40.484375 46.53125 36.375 47.5 \n",
       "Q 32.28125 48.484375 27.875 48.484375 \n",
       "Q 21.1875 48.484375 17.84375 46.4375 \n",
       "Q 14.5 44.390625 14.5 40.28125 \n",
       "Q 14.5 37.15625 16.890625 35.375 \n",
       "Q 19.28125 33.59375 26.515625 31.984375 \n",
       "L 29.59375 31.296875 \n",
       "Q 39.15625 29.25 43.1875 25.515625 \n",
       "Q 47.21875 21.78125 47.21875 15.09375 \n",
       "Q 47.21875 7.46875 41.1875 3.015625 \n",
       "Q 35.15625 -1.421875 24.609375 -1.421875 \n",
       "Q 20.21875 -1.421875 15.453125 -0.5625 \n",
       "Q 10.6875 0.296875 5.421875 2 \n",
       "L 5.421875 11.28125 \n",
       "Q 10.40625 8.6875 15.234375 7.390625 \n",
       "Q 20.0625 6.109375 24.8125 6.109375 \n",
       "Q 31.15625 6.109375 34.5625 8.28125 \n",
       "Q 37.984375 10.453125 37.984375 14.40625 \n",
       "Q 37.984375 18.0625 35.515625 20.015625 \n",
       "Q 33.0625 21.96875 24.703125 23.78125 \n",
       "L 21.578125 24.515625 \n",
       "Q 13.234375 26.265625 9.515625 29.90625 \n",
       "Q 5.8125 33.546875 5.8125 39.890625 \n",
       "Q 5.8125 47.609375 11.28125 51.796875 \n",
       "Q 16.75 56 26.8125 56 \n",
       "Q 31.78125 56 36.171875 55.265625 \n",
       "Q 40.578125 54.546875 44.28125 53.078125 \n",
       "\" id=\"BitstreamVeraSans-Roman-73\"/>\n",
       "     </defs>\n",
       "     <g transform=\"translate(14.798437499999997 86.60781250000002)rotate(-90.0)scale(0.1 -0.1)\">\n",
       "      <use xlink:href=\"#BitstreamVeraSans-Roman-6c\"/>\n",
       "      <use x=\"27.783203125\" xlink:href=\"#BitstreamVeraSans-Roman-6f\"/>\n",
       "      <use x=\"88.96484375\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "      <use x=\"141.064453125\" xlink:href=\"#BitstreamVeraSans-Roman-73\"/>\n",
       "     </g>\n",
       "    </g>\n",
       "   </g>\n",
       "  </g>\n",
       " </g>\n",
       " <defs>\n",
       "  <clipPath id=\"p0dc0bac988\">\n",
       "   <rect height=\"139.5\" width=\"195.3\" x=\"48.14375\" y=\"7.200000000000017\"/>\n",
       "  </clipPath>\n",
       " </defs>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f9a1f241518>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "train_momentum(0.004, 0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "- The momentum method uses the EWMA concept. It takes the weighted average of past time steps, with weights that decay exponentially by the time step.\n",
    "- Momentum makes independent variable updates for adjacent time steps more consistent in direction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Use other combinations of momentum hyperparameters and learning rates and observe and analyze the different experimental results.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}