examples/AIServingMnistOnnxModel.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inferencing MNIST ONNX Model using AI-Serving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST handwritten digit classification problem is a classic dataset used in computer vision and deep learning, and Convolutional Neural Network (CNN) is the current state-of-art architecture for image classification task.\n",
    "\n",
    "In this tutorial, we will use the Open Neural Network eXchange (ONNX) format to show how to deploy a pre-trained MNIST CNN model using AI-Serving."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id=\"contents\"></a>Contents\n",
    "This notebook contains the following parts:\n",
    "\n",
    "**[Setup](#setup)**<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Prerequisites to run the notebook](#prerequisites)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Additional information about `ai_serving_pb2.py`](#additional)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Install dependencies](#dependencies)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Import dependent modules](#import)<br />\n",
    "**[Validate server](#validate)**<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Define the base HTTP URL](#httpurl)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Test the server availability](#testserver)<br />\n",
    "**[Deploy the ONNX model](#deploy)**<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Deploy the MNIST model into AI-Serving](#deploy-onnx)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Retrieve metadata of the deployed model](#metadata)<br />\n",
    "**[Make predictions](#predictions)**<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Load the testing images](#test-data)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[HTTP request formats](#request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[Binary requests](#binary-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Construct binary requests](#construct-binary-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Make the HTTP requests with binary data](#make-binary-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Consume the HTTP responses with binary data](#consume-binary-response)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Parse the response message from the `records` request](#parse-binary-response-records)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Parse the response message from the `split` request](#parse-binary-response-split)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;[JSON requests](#json-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Construct JSON requests](#construct-json-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Make the HTTP requests with JSON data](#make-json-request)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Consume the HTTP response with JSON data](#consume-json-response)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Parse the json response message from the `records` request](#parse-json-response-records)<br />\n",
    "&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;[Parse the json response message from the `split` request](#parse-json-response-split)<br />\n",
    "**[Next steps](#next-steps)**<br />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id=\"setup\"></a>Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"prerequisites\"></a>Prerequisites to run the notebook\n",
    "\n",
    "Run a docker container of AI-Serving. The port `9090` is the port of HTTP endpoint while `9091` is for gRPC, you could see an error likes `Bind for 0.0.0.0:9090 failed: port is already allocated`, then use another new port instead of the first part as follows `-p $(NEW_PORT):9090` to run the command again, and remember the port is always needed in the URL of HTTP endpoint. It will aslo pull the latest docker image of AI-Serving with ONNX that leverages the CPU version of [ONNX Runtime](https://github.com/microsoft/onnxruntime) from [docker hub](https://hub.docker.com/r/autodeployai/ai-serving) if it hvaen't been downloaded yet. Please, refer to [Docker Containers for AI-Serving](https://github.com/autodeployai/ai-serving/tree/master/dockerfiles) about more docker images.\n",
    "\n",
    "```bash\n",
    "docker run --rm -it -v $(PWD):/opt/ai-serving -p 9090:9090 -p 9091:9091 autodeployai/ai-serving\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"additional\"></a>Additional information about `ai_serving_pb2.py`\n",
    "In the current directory, there is a python file `ai_serving_pb2.py`, which is generated from compiling the [ai-serving.proto](https://github.com/autodeployai/ai-serving/tree/master/src/main/protobuf/ai-serving.proto) using [protoc](https://developers.google.com/protocol-buffers/docs/pythontutorial), for example, the command as follows:\n",
    "\n",
    "```bash\n",
    "protoc -I=$SRC_DIR --python_out=. ai-serving.proto\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"dependencies\"></a>Install dependencies\n",
    "We will install python libraries for HTTP request, data manipulation, image manipulation and display:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install requests\n",
    "!pip install numpy\n",
    "!pip install onnx\n",
    "!pip install pillow\n",
    "!pip install matplotlib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"import\"></a>Import dependent modules\n",
    "Import some dependent modules that we are going to need to run the MNIST ONNX model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import requests\n",
    "from matplotlib import pyplot as plt\n",
    "from PIL import Image\n",
    "from pprint import pprint\n",
    "\n",
    "import onnx\n",
    "import ai_serving_pb2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id=\"validate\"></a>Validate server"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"httpurl\"></a>Define the base HTTP URL\n",
    "Change the port number `9090` to the appropriate port number if you had changed it during AI-Serving docker instantiation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "port = 9090\n",
    "base_url = 'https://localhost:' + str(port)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"testserver\"><a>Test the server availability\n",
    "Use the specific endpoint `https://host:port/up` to test whether the server has been initialized and is ready to accept requests. The `OK` message indicates it's already available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The status of the server:  OK\n"
     ]
    }
   ],
   "source": [
    "test_url = base_url + '/up'\n",
    "response = requests.get(test_url)\n",
    "print('The status of the server: ', response.text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id=\"deploy\"></a>Deploy the ONNX model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"deploy-onnx\"></a>Deploy the MNIST model into AI-Serving\n",
    "First, we need to deploy the MNIST ONNX model `models/mnist-8.onnx` that is from [here](https://github.com/onnx/models/raw/refs/heads/main/validated/vision/classification/mnist/model/mnist-8.onnx) into AI-Serving, which can serve multiple models or multiple versions for a named model at once.\n",
    "\n",
    "You must specify a correct content type for ONNX models when constructing an HTTP request to deploy an ONNX model, the candidates are:\n",
    " * application/octet-stream\n",
    " * application/vnd.google.protobuf\n",
    " * application/x-protobuf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The depoyment response:  {'name': 'mnist', 'version': 1}\n"
     ]
    }
   ],
   "source": [
    "# The specified servable name\n",
    "model_name = 'mnist'\n",
    "deployment_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# The specified content type for the model:\n",
    "headers = {'Content-Type': 'application/x-protobuf'}\n",
    "\n",
    "model_path = os.path.join('models', 'mnist-8.onnx')\n",
    "with open(model_path, 'rb') as file:\n",
    "    deployment_response = requests.put(deployment_url, headers=headers, data=file)\n",
    "\n",
    "# The response is a JSON object contains the sepcified servable name and the model version deployed\n",
    "deployment_response_info = deployment_response.json()\n",
    "print('The depoyment response: ', deployment_response_info)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"metadata\"></a>Retrieve metadata of the deployed model\n",
    "The metadata will contain model inputs and outputs, which are needed when constructing an input request and consume an output response."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The model metadata info:\n",
      "{'createdAt': '2024-10-09T21:25:27',\n",
      " 'id': '756bd65a-8805-4ea9-a087-1494f8a9c24e',\n",
      " 'latestVersion': 1,\n",
      " 'name': 'mnist',\n",
      " 'updateAt': '2024-10-09T21:25:27',\n",
      " 'versions': [{'createdAt': '2024-10-09T21:25:27',\n",
      "               'hash': 'd7cd24a0a76cd492f31065301d468c3d',\n",
      "               'inputs': [{'name': 'Input3',\n",
      "                           'shape': [1, 1, 28, 28],\n",
      "                           'type': 'tensor(float)'}],\n",
      "               'outputs': [{'name': 'Plus214_Output_0',\n",
      "                            'shape': [1, 10],\n",
      "                            'type': 'tensor(float)'}],\n",
      "               'runtime': 'ONNX Runtime',\n",
      "               'serialization': 'onnx',\n",
      "               'size': 26454,\n",
      "               'type': 'ONNX',\n",
      "               'version': 1}]}\n"
     ]
    }
   ],
   "source": [
    "model_version = deployment_response_info['version']\n",
    "metadata_url = base_url + '/v1/models/' + model_name + '/versions/' + str(model_version)\n",
    "metadata_response = requests.get(metadata_url)\n",
    "metadata_response_json = metadata_response.json()\n",
    "\n",
    "# Model info of the specified version\n",
    "model_info = metadata_response_json['versions'][0]\n",
    "\n",
    "# Extra some key values: the input tensor name and shape\n",
    "version_metadata_input = model_info['inputs'][0]\n",
    "input_shape = version_metadata_input['shape']\n",
    "input_tensor_name = version_metadata_input['name']\n",
    "\n",
    "# Extra some key values: the output tensor name and shape\n",
    "version_metadata_output = model_info['outputs'][0]\n",
    "output_shape = version_metadata_output['shape']\n",
    "output_tensor_name = version_metadata_output['name']\n",
    "\n",
    "# Show the metadata result in json\n",
    "print('The model metadata info:')\n",
    "pprint(metadata_response_json)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <a id=\"predictions\"></a>Make predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"test-data\"></a>Load the testing images\n",
    "We will use those test images in the directory `images`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The testing image list:\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmoAAACaCAYAAAAZ32RPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAkcElEQVR4nO3de3QU9fk/8GdDAgESbiFACAJCBFG0Bi9AARE4GjCKXASFgKDCUVBqi1YFCyigQnu81HoJYhqsoBZqW7CFo1ajQAvKXUUQI0bEgELEELlD5veHP/LN857JfnZz/Uz2/TqHP57NXmZ3np0ddt77TMBxHEeIiIiIyDpRNb0AREREROSNO2pEREREluKOGhEREZGluKNGREREZCnuqBERERFZijtqRERERJbijhoRERGRpbijRkRERGQp7qgRERERWarCO2qLFi2SQCAgGzdurIzl8aVPP/1URowYIYmJiVKvXj1p3769TJ482Xi7s69dXl5e1S9kLRbJPfjwww9LIBAo89/rr78e9PbswcoTyX0oIpKbmytjx46Vtm3bSv369aVjx44ydepUKSgoMN6WfVg5Ir0Hd+3aJcOHD5emTZtKgwYNpHv37rJixYqQbmtzD0bX9AL4XU5OjqSnp0ufPn0kMzNTmjdvLnv27JEtW7YYb5ueni7r1q2TpKSkalhSqo0mTJggAwcOdF0+ceJE+fLLLz3/Vhp7kCrDgQMHpEePHtKoUSOZM2eOtG3bVrZs2SKzZs2SnJwc2bRpk0RFlf29APuQKiovL0969uwpSUlJkpmZKXFxcfLCCy/IkCFDZNmyZTJ8+PCgt7e5B7mjVgFHjx6VjIwM6d+/v7z55psSCARK/jZ27Fjj7RMTEyUxMbEqF5FquTZt2kibNm3UZXl5ebJ9+3bJyMiQJk2aBL09e5Aqw/Lly6WgoED++te/yoABA0REpF+/fnLixAmZPn26bNu2TVJTU8u8PfuQKmrevHly9OhReeuttyQ5OVlERAYOHCgXXXSR/OY3v5GhQ4cG/c+CzT1YJRm18ePHS1xcnOzcuVPS0tKkYcOGkpSUJPPmzRMRkfXr10vv3r2lYcOG0qlTJ3n55ZfV7Q8cOCCTJ0+WCy64QOLi4qRFixbSv39/WbNmjeux9u7dKzfeeKPEx8dLkyZNJCMjQzZs2CCBQEAWLVqkrrtx40YZPHiwNGvWTGJjYyU1NVWWLl1a7ue5bNky2bdvn/z2t79VO2mh8vqq9aqrrpKuXbvKmjVrpEePHlK/fn1JTk6WGTNmyJkzZ9Ttw3nukSZSetDLn//8Z3EcRyZMmGC8LnuwakVKH8bExIiISOPGjdXlZ/+jEBsbG/T27MOqEyk9+N///ld+8YtflOykiYjUqVNHBg0aJN9884189NFHQW9vcw9W2Y8JTp06JcOGDZP09HRZvny5DBo0SKZNmybTp0+XcePGyW233Sb/+Mc/pHPnzjJ+/HjZtGlTyW1/+OEHERGZNWuW/Pvf/5bs7Gzp0KGDXHXVVfL++++XXO/IkSPSr18/ycnJkfnz58vSpUulZcuWctNNN7mWJycnR3r16iU//vijZGZmyvLly+WSSy6Rm266yfUitm/fXtq3b298jqtXrxYRkTNnzkjv3r2lbt260rRpUxk1apTk5+eH/6L9f/v375ebb75ZMjIyZPny5XLjjTfK3Llz5Z577inXc49UkdCDqLi4WBYtWiQpKSnSt2/fsG9/Fnuw8kRCHw4ZMkTatm0r9957r2zfvl1++uknWb16tcybN0+uv/566dKlS7leO/Zh5YiEHjx58qTUq1fPdfnZyz7++OMQXik3K3rQqaDs7GxHRJwNGzaUXDZu3DhHRJw33nij5LJTp045iYmJjog4mzdvLrm8oKDAqVOnjjN16tQyH+P06dPOqVOnnAEDBjhDhw4tufy5555zRMRZtWqVuv4dd9zhiIiTnZ1dctn555/vpKamOqdOnVLXve6665ykpCTnzJkzJZd17NjR6dixo/G5p6WlOSLiNGnSxLn//vud9957z8nMzHQSEhKclJQU58iRI0Fvf/a1++qrr0ou69u3ryMizvLly9V1J06c6ERFRTlff/112M+9tovkHkSrVq1yRMR5/PHHQ7o+e7DyRHof5ufnOz179nREpOTfiBEjnOPHjxtvyz6sHJHcg0OGDHGaNGniFBUVqcv79OnjiIjz2GOPBb29zT1YZd+oBQIBufbaa0vq6OhoSUlJkaSkJJVVaNasmbRo0UK+/vprdfvMzEzp1q2bxMbGSnR0tMTExMi7774rO3bsKLnOBx98IPHx8a7A9KhRo1Sdm5srO3fulIyMDBEROX36dMm/a6+9Vvbt2yeff/65un5ubq7xORYXF4uIyE033STz58+Xfv36yR133CFZWVmSm5srr776qvE+vMTHx8vgwYPVZaNHj5bi4uKSb/FCfe6RLBJ6EGVlZUl0dLSMHz8+7NuWxh6sPJHQh4cOHZIbbrhBDh8+LEuWLJHVq1fL888/L2vXrpXBgwfL6dOnQ3il3NiHlSMSevDuu++WwsJCueWWW2T37t3y3XffyYwZM+R///ufiEjQfFowNvRgle2oNWjQwJVLqFu3rjRr1sx13bp168rx48dL6ieffFImTZok3bt3lzfeeEPWr18vGzZskIEDB8qxY8dKrldQUCAtW7Z03R9e9t1334mIyH333ScxMTHq39kxGgcPHgz7OSYkJIiISFpamro8LS1NAoGAbN68Oez7FHEvv4hIq1atRERKfuoe6nOPZJHQg6UdPHhQVqxYIenp6SX9Ul7swcoTCX04f/582bp1q7zzzjsyevRo6dOnj0yaNEmWLFkib7/9tixZsiTs+/RafhH2YXlEQg8OGDBAsrOzZfXq1dKxY0dp1aqV/P3vf5c5c+aIiKjsWjhs6EErf/W5ePFiueqqq+SFF15QlxcVFak6ISHBMyC4f/9+VTdv3lxERKZNmybDhg3zfMzOnTuHvZwXX3xx0DlV5d2DP9vIpZ19Tmd3DkN97lQ+funB0l555RU5efJkSD8iMGEP2sEvfbh161ZJTk52jTa4/PLLReTnWZPlwT6seX7pQRGRcePGSUZGhnzxxRcSExMjKSkp8vjjj0sgEJA+ffqU6z5t6EErz0wQCARcocCPP/5Y1q1bpy7r27evFBUVyapVq9TluPPUuXNnOe+882Tbtm1y2WWXef6Lj48PezmHDh0qgUDA9firVq0Sx3GkR48eYd+nyM9vABzS9+qrr0pUVJRceeWVIhL6c6fy8UsPlpaVlSWtW7eWQYMGVeh+RNiDtvBLH7Zu3Vr27t0r3377rbr87HLiCJlQsQ9rnl968Kzo6Gjp0qWLpKSkSGFhobz44otyww03SLt27cp1fzb0oJXfqF133XUyZ84cmTVrlvTt21c+//xzmT17tpx77rkq6zBu3Dh56qmnZMyYMTJ37lxJSUmRVatWyVtvvSUi+hutBQsWyKBBgyQtLU3Gjx8vycnJ8sMPP8iOHTtk8+bNsmzZspLrpqSkiIgYj4uff/75ctddd8nzzz8v8fHxMmjQINm1a5f87ne/k9TUVBk5cmTJdWfPni2zZ8+Wd9991/hrvISEBJk0aZLs2bNHOnXqJCtXrpSFCxfKpEmTpG3btmE/dwqfX3rwrA8//FC2b98u06dPlzp16nhehz3oP37pw7vuukuWLFkiV199tTz44INyzjnnyKeffipz586Vli1bluSRRNiHfuOXHvz+++/liSeekF69ekl8fLzs3LlTfv/730tUVJQ899xz6rq+68EK/RTBKftXJg0bNnRdt2/fvs6FF17ourxdu3ZOenp6SX3ixAnnvvvuc5KTk53Y2FinW7duzj//+U9n3LhxTrt27dRt9+zZ4wwbNsyJi4tz4uPjneHDhzsrV670/KXGtm3bnJEjRzotWrRwYmJinFatWjn9+/d3MjMzXcuDj1OW06dPO/PmzXNSUlKcmJgYJykpyZk0aZJz6NAhdb1Zs2Y5IuLk5OSUXHb2tcvLy3O9Ru+//75z2WWXOfXq1XOSkpKc6dOnu34hE85zr80ivQcd5+dfIQUCAefLL78s8zrswaoV6X24efNmZ+jQoU6bNm2cevXqOR06dHAmTJjg7NmzR12PfVh1IrkHCwoKnGuuucZJTEx0YmJinLZt2zpTpkxxDhw44Lqu33qwwjtqNnr00UedQCDgfPPNNzW9KEE9/fTTjoionxOX9eYJlV+ee23nl/XAHqzd/LIu2Ie1l1/Wg809aOWhz3A8++yzIvLzYchTp07Je++9J88884yMGTOm3LmIqlZYWCjr1q2TRYsWSdeuXSUuLq5c9+PH514b+XE9sAdrHz+uC/Zh7eLH9eCLHqzQbp4FsrKynK5duzpxcXFOTEyM07FjR2fGjBnOiRMnanrRypSTk+PExsY6V1xxhbNp0yb1t3D24P343GsjP64H9mDt48d1wT6sXfy4HvzQgwHHcZyK7eoRERERUVXgz2GIiIiILMUdNSIiIiJLhfRjguLiYsnPz5f4+HgJBAJVvUxUhRzHkaKiImndurWv5guxB2sPv/agCPuwtmAPkg1C7cOQdtTy8/PlnHPOqbSFo5r3zTffWPsrHC/swdrHbz0owj6sbdiDZANTH4a0o1bRU9tUlpiYGFWfOnUq6PVxQvuZM2dUjf8b8frfSXFxsaqjo6OD1qVPZiviXmb87Ubpyc7VyZZ1GqqqWl78Xwyu73CVdVaAsu4/kn/L47ceFLFnmevWravqkydPhnV73G6Fsh2q6Lasfv36qsZtJY5FwHNJVgVb1mc4bFnmcLedeH3sn1C2hXhieewhPNUVOnHihPExaoJpnYa0o2bL16vhLodpRyyUHbVw79N0/XDvv6o+yG1Zp6HyWl7TcwjltTOtz3Bf/3D7IZJ31PzWgyL2LHNFl6M8fV/Vj1kTr60t6zMctixzRT/bUHm21xVdJluYlttfB+eJiIiIIoivzkxg+no/3K9WsfY6bIV7uni4tWHDhkGXEQ8H4NfDTZs2VfWhQ4dcy0DecP2V539TeDgcmQ6fo4oeyvbqQTxMZevX91R9sAdw24eHUvAwoik2godWRcLf/uK2raCgIOjtDx8+rGo8jIWHvUR+nipPNSPcbSdux7BfcNvp1aPHjh1TNfYIvi9M39LhZwYuI35em55zVeE3akRERESW4o4aERERkaW4o0ZERERkKV9l1BAe48bjzeEeT/a6Pj5GcnKyqvfu3atqPMaNy4THvDGThsfcvTJPNXWc3BZnX1NT5hBzNjheQMSdu8H1gzmLcH+5FG7ttW4jfX2Tm2m7Yspu4XshlKGvOF4DexdHJZgyaaYcHW77mEezC25fsQdx/Zm2Y9jTXnldvA+sTblz7HvsWVN2s6bwGzUiIiIiS3FHjYiIiMhS3FEjIiIispSvM2rhZtLCPb2P120wk2Y6pQXCnAcuM2dkBRcdHV2y3jEDgfkEzJ+Fe5odkcrPh5l60GvuT0VPa0W1D/ZJQkKCqk35MNxuYY8dOXIk7GVq0KCBqo8ePapqzO9iJg23jbbmhehnpu2pKXNmykWG8nlsmlsZ7plg8PrYk16PV57PlXDxGzUiIiIiS3FHjYiIiMhS3FEjIiIispSvM2qm/BAeAzdlKLyOX5tyEnibSy+9VNVPPPGEql977TVVL1iwIOj9e83+iuTsRumMAM7xwXxBefJ+OMcO4WNgZsGUmcCeNc0CFAn/fKNU+2HfhJtJ++mnn1SN2zGv98Fll12m6oEDB6r66quvVvWPP/6o6jVr1qgat30HDx50PSbZC3sq3KxWZWS78PMR+zjc7TPeHvcRagq/USMiIiKyFHfUiIiIiCzFHTUiIiIiS3FHjYiIiMhSvv4xQbghaxxGG0oou2nTpqru3bu3qqdMmaLq7t27q7pRo0aqbtOmjaoXL16sahw0Gck/HDAJN4yKPz4QcQ/hrOrwqGnII38oQKHA7QoG9xFuKzE0jdu1adOmue4Df0zQvHlzVZt6u3///qru2rWrqidPnqzqQ4cOBb0/qlmm4e4m2C/Yk17ba/wBQ2FhYdDbxMXFqRp/TIDvC/y8xdprmapjSD2/USMiIiKyFHfUiIiIiCzFHTUiIiIiS/kqo4bDQDFfhPkePKmr6eTWHTt2dF02evRoVc+cOVPVmCnD7MgHH3ygajzmjbdv3LixqvEYfKSrX79+SR+EmyfzyrSZcm44+BN7Dm9vyhRiD5pyPUReMJOG2R3sK9zO9O3bV9ULFy5U9Xnnned6zK1bt6p63rx5qt6yZYuqU1NTVY3Dv2+++WZVZ2dnq/qdd95RdU3lgyg0uH7w89iUv8Ue9tqW4uchnjQdt6842Bnh7U2fB6aBuVWFnxJEREREluKOGhEREZGluKNGREREZClfZdRwzoopc4ZwZgrOSBs5cqTrNrNnzw56n3gi4QcffFDVf/nLX1R97rnnBr2/oqIiVXudpNvr5PGR4tixY2X+DU/QizkdrzwLZs6GDx+u6ltuuUXVuL53796t6k2bNqn6k08+UXV+fr6qQ5mTx5OyE8Lexj7CHvnlL3+p6pdfflnV7dq1U/Ujjzziesw//OEPqjblM029jX/H7TFu55hHsxuuL9N2Cmf3/frXv1Z1enq66zaYY0Ovvfaaqh977DFV4/YXZ/nh33Nzc1WN22KR6tke8xs1IiIiIktxR42IiIjIUtxRIyIiIrKUrzJqOPME80qY28AM2znnnKPqOXPmqHrMmDGux8QcBR4Dv/vuu1WN84pwGT777DNV4+wZvH5NzW3xA1zfmBXA184r74fXefTRR1WNs/Ww50w9iXbt2qVqzCR6rW/T7Lb4+HhVY882a9ZM1Rs3blQ1zsP64osvgj4e1TxTPhd7/amnnlJ1QkKCqmfMmKHqJ5980nWfprmFnTt3VvVLL72kasyY7dy5U9UrVqxQNb6/8X0gwt60CW53sMcuvPBCVS9ZskTVrVq1UrXX+sa5aLj9HTVqlKqxP5KSklQ9YMAAVU+cOFHVmFGrqXw4v1EjIiIishR31IiIiIgsxR01IiIiIkv5KqNmyv/geRnj4uJU/corr6gaZwt5wcwZHsPGY+A4y6tJkyaq/uGHH1RtOh8a3l7EfZ6/SBIdHV2Sv8FMBM64CXfOnog7o4Y9tW/fPlV36NBB1Zdccomqe/bsqepOnTqpGvsHz/VaHnv37lU19iRmMffv36/qadOmqdrrfKTleW2p6mBWB33//feqvuKKK1S9Y8cOVXvl0Ro2bKjq22+/XdVPP/20qjFvidutIUOGqBq376ZsJtUsUz4X873/+c9/VH38+HFV43YI82Yi7s9DnGuJ29tDhw6pGmcBYpYT87mIGTUiIiIiUrijRkRERGQp7qgRERERWcpXGTU8Po2ZBzxG/vDDD6sazy2GPvroI9dlaWlpqsaZVQUFBarG3BTOfcHcHP4dRXIezUtUVJTnPDQRd6YF836hnKcNz4GImQRTRqFBgwaqxvMXJiYmqhrPLYez/kTcWRDM/uDz3rp1q6oxV4cw04b3jz0rYu5bql6mmZJeM6lKw+xscnKy6zqYbbzvvvtUje/LLVu2qPrWW29VNfYdzpTETBq+t0TMs92o6uB24oILLlD1v/71L1VjVhbnOU6ZMkXVeN5kL82bN1c1zj3LysoKen3MVa5du1bV+D6qqbmm/EaNiIiIyFLcUSMiIiKyFHfUiIiIiCzlq4wa5rUwL9auXTtV33///UFvj5mH66+/3viYJjh3C89vZ5oFhNmQb7/9NqzHr+1OnjxZZkbNlB/zmv2FuTVTrg3vAx8TMzO4/g8fPqxqPNcnzu3zWgZ8TLyPoUOHqhp7EnMYeE5GfF8wj2YfXEeYpcF1hvOksGcWLFigaq9cIp4z1ivzWRpuO++66y5Vz5w5M+j1EWf32QXz2A899JCqsYdWrVql6jvvvFPVe/bsUTXO7RNxZzFxjhrORh08eLDrPkrLzs4O+vfY2FhV11Qmkt+oEREREVmKO2pEREREluKOGhEREZGlfJVRw2wS5iyGDx+uajyejXPY+vXrp2o83i3izvdghqlRo0aqxnlEOEcLsyKYLWEmzexsRgvzA3juuLJuVxquT4TrHzMKpmXAHA/2LPLKqJlgluOFF15QNc4vevHFF4M+JmZL8DUQcWfvqHqFm5XBeY+4Ts8777ywH3PNmjWqbt++vaq3b9+u6gcffFDVmLPD92K472+qXkuXLlX1iBEjVI1z8nCuKWbSkNfsP1NO8bHHHgv6d5zttmvXLlXj9hp73muZqmO2Gr9RIyIiIrIUd9SIiIiILMUdNSIiIiJL+SqjhhmjCy+8UNV4DByvj8eSe/bsaXzMXr16qRrPJYZz0d555x1VYyYNc3aYH8LsD2bcvO4zUpU1T62sv+O5BEXMWStTFgjvE3M0prl5mBXyykDgvCK8T8z+tGzZUtWYT3r//fdVjVkgzk2zH243sEfQypUrVT127FhVY/bHKw+GvYm9e/XVV6s6Ly8v6DJhH+NzwmXwev+a3l9UdRo3bqxqXBf4ebtjxw5Vm+ZDFhYWuh4TZ/k98sgjqu7Tp0+QJRaZN2+eqsPNPfJcn0RERESkcEeNiIiIyFLcUSMiIiKylK8yajh3BzNImMXBTAOeG3T27Nmq9soH3XrrrapeuHBh0PvEZcDZMjk5OarGjBQ+R+bRyoZz8hBmHkyZNhF3z5gyMOFm2HBWFK5/r6wR5od69+6t6gceeEDVmKMYMmSIqvft2xf0+jzfrP1M8/+w19etW6fqjz76SNWYL8Pcooj7vZCenq5q04zAVq1aqRq3lfh+5Rw1uzVv3lzVuK3D+Y64HTP18AUXXOC6bNSoUaq+4447VJ2fn69qzNFt2bJF1TgzEj9TcJ+gpuZH8hs1IiIiIktxR42IiIjIUtxRIyIiIrIUd9SIiIiILOWrHxNgcHvkyJGqTkhIUDWGU99++21VY9Bw9+7drsf0CtWWtnbtWlVj0Ds7O1vVV155parxxLT4HL0C8F4nF48UderUKXlNMASP4VU82XgoYWR8bTFMioFY0wBEDGCblhGHQHot07XXXht0mTZs2KBq7FH8wQo+B/zxQP369V3LZPohB9Wshx56SNW/+tWvVI2DQz/55BNV4w9ORESuueYaVW/btk3V/fr1UzUGu/fv3x9kid2iovg9gs3whObdunVTNW433n33XVXj5y8G9fGHeiIiHTp0UDVuG9u0aaPqBQsWqBq3W+F+lnptn00/iqgMfCcQERERWYo7akRERESW4o4aERERkaV8lVHDY9rjx49X9YEDB1Q9depUVS9evFjVeAzdK8OEOYlXX31V1TjkccmSJarGAX2mY+Ic8hhcsDwA5r3Kw3Rya6+MQmmYQcP1Z1pGr8fHTFlaWpqqsUfxJO0olBykaZno/14303sah2qW56T3eMJyNGHCBFVPmjRJ1YmJiarGgbfjxo1Ttdfwbxyai3mgmTNnqvr2228PssQiLVq0UPX333+vasx3Mq9rl9tuu03VuD7xBOmXX365qjEbiwNzDx486HpMzKXffPPNqh42bJiqX3rpJVXjthLzvdhPuL03DUCvKvxGjYiIiMhS3FEjIiIishR31IiIiIgs5auMWpcuXVTdunVrVeMctL/97W9B7w9nqnjNTMOM0eHDh4Pe5/bt21WNmSrM2eEy4HH5evXquR6jpk4MS+Y8F2bQGjVqpGrsH+yHwsJC133OmDFD1ampqapetmyZqrEHkSlnhz0bysnsI9HZPAvmxzDTF0omDbdlmNfCdfLcc8+pevTo0arGvsM5agsXLlR1KPMBMR/05ptvBv37E088oeodO3aoGp8jwswSLmNZy0nVAz+7BgwYoGqcKdqjRw9V47rbuXOnqlevXu16zDlz5qga55Lm5uaqGmf34fsIt4X49+qYkRYKfqNGREREZCnuqBERERFZijtqRERERJbyVUZt69atqv7ss89UjXkfzJfhucMwT+SVgTDBWWwTJ05UtSmfgpk0vD+eU9EumKswZdCwxh7E9XvppZe6HvOBBx5QNfY1zgr68ccfXfdRGmbOcLYQ5jLK876IJJhJ88qVluaVMcUsDb7mF198sarHjBmjapzVhjMk8ZyH5ZkHtX79elVj7+Iy4PlEcTYbvm74d+xT5tHs0rJlS1V/9913qsbzx2L/4PrEc3XjvEcRd9YSewbnnBYUFLjuI9jtcY6aLflcfqNGREREZCnuqBERERFZijtqRERERJbyVUYN55zhvCivfE9peF5OVJ7cRufOnVWNmSV8TFN+CHMfXuf447kXaw7O3THN1cNzLOL5aDHn8fbbb7vuA/NKWVlZql67dm3QZUCYDcHnhLkMW2YJ2aRhw4YlrxPmUEM5RyXCddy8eXNV49w0PKcszih79tlnVY3bDFO+CLOyXrfBTBrOsczPzw+6DAj7jJk0u2HPIK+ZkMFgnuz88893XQczZdhjr732mqpNn+nYk6Y8bk19HvMbNSIiIiJLcUeNiIiIyFLcUSMiIiKylK8yapjvwnOBDRs2TNXjx49X9dKlS1XtNacFderUSdU4G2j+/Pmqbtq0qaqfeuopVeNzwJlLOGOJGTW7YEYCczW4/jGThlauXKlqnLMmIpKXl6fqzMxMVWOu0dRT+BwwC4TnWPTqN5w3FGlOnjxZZvYMM3+hZK2aNGmiaszn9OzZU9X42Pfee6+qMc+Ls/JMmTScBykiMnPmTNdlpS1atCjoYyBTHyLM5Ym4Z2FSzcGew3mPuP5wG4Kfx3feeafrMfA2mOn98MMPg14fmTJp+PeaminJb9SIiIiILMUdNSIiIiJLcUeNiIiIyFK+yqghPD69c+dOVf/xj39Udffu3VW9a9cuVePMKxGRKVOmqBrzPnh+ss8//1zVzz//vKpN5/7ELEkoOTqqPqbMw6FDh1SN55/t0aOHqjGL5JVJnDx5sqqxz5HXuSRLwwwbZoNw9hBmrkQ4Wy2cnChmc7ze0ziPD7MwpllsmNXCPrryyitVPXLkSFVfdNFFqu7QoYPrMTA/+cwzz6gazzmLGaVw4XNmNtcu+FmFsAdNecJu3bqp+pZbbnFdB3ti06ZNqq5odjbc8yBXF36jRkRERGQp7qgRERERWYo7akRERESW8lVGDee0fPnll6qeO3euql9++WVVe81lKc1rjg/mc/A4+9SpU1X9pz/9SdU4KwiZZl4xH2QXzG9h/gj7A+djPf7446rG/pg1a5brMTGLaeopk3DPoVhTs4NsFhsbW5JnwTl2+PpizsUrR4OX4WuO5zTEvvvggw9UjdsRXAbMD2Ge7NNPP3Ut4+LFi1X9+uuvq9o0Nw2XAZ8jbutwO1eeczFT1cH1hT2E6xs/v3FeI+YkvWZKYr5z48aNqsZ5gPjeRKbr2zIvkt+oEREREVmKO2pEREREluKOGhEREZGlfJVRw2PgcXFxql62bJmqcaZZ+/btVT1kyBBV4zFyEXfO7dlnn1W1KVeBeRU8FyTO3TLdH9UszF1gJg17cuzYsapOTU0Nev/Lly83LoNpphZmO4qKilSN+SVTzs6WnIZNSm+LGjRooP6GOZryzP9av369qm+44QZVYyYN1zH2QG5urqrxHMWYSduyZYtrmUzz+Ux5H+xb3LaZ+tprbhfzkzUH+xpzj/h3/PzGGs+j7ZXFxRmSGzZsCLoMyDQnDZlylCLV8xnNb9SIiIiILMUdNSIiIiJLcUeNiIiIyFLcUSMiIiKylK9+TIBMJzj/+OOPg9YrVqyo8DKEGyTEHw9U9P6oemHAFYdw9unTR9X33HOPqk0Djb3CsKahyAiD5SamkyVTcF4nWS+tMn6MsXnzZlV7DQOtaabhoqZtm+l14g8H7II/QjINJDb9yCkjI8P4mFlZWarGngq3x8Ld9vGk7ERERESkcEeNiIiIyFLcUSMiIiKylK8zakTVzXRSdhya3LhxY1VjBu2rr74yPqYpk0ZEVN0w3+U1oLY0zKTh8Nm8vDxVX3rppa77wLxupOA3akRERESW4o4aERERkaW4o0ZERERkKWbUiCoAcxp4YmrMpG3dulXVQ4cOVTXmNEREWrVqper9+/eHuZRERFUrLi5O1ThXD7O2DRo0UPXrr7+uaq882qZNmyqyiL7Fb9SIiIiILMUdNSIiIiJLcUeNiIiIyFIBJ4QT0R0+fNg1D4r8rbCwUBo1alTTixEyW3oQcxWm8zyGKzY21nXZ8ePHK/UxbOG3HhSxpw+pcrAHyw/noFXGOW1LS0lJcV2Wm5tbqY9hC1Mf8hs1IiIiIktxR42IiIjIUiGN56jsrzSp5vltndqyvFW9HLY8z+rgx+fqx2WmsvlxfdqyzFW9HMXFxVV6/zYxvZYh7ajhObrI/4qKiqzIOYTKlh7E2UCVLZLO6+m3HhSxpw+pcrAH7bV79+6aXoRqY+rDkH5MUFxcLPn5+RIfH+8KEJK/OI4jRUVF0rp1a4mK8s+Rb/Zg7eHXHhRhH9YW7EGyQah9GNKOGhERERFVP3/9V4KIiIgognBHjYiIiMhS3FEjIiIishR31IiIiIgsxR01IiIiIktxR42IiIjIUtxRIyIiIrLU/wOAIXzpt6t2fgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "input_tensors = []\n",
    "input_arrays = []\n",
    "\n",
    "image_size = input_shape[-2:]\n",
    "def preprocess_image_toarray(img):\n",
    "    \"\"\"Resize input images and convert them to grayscale.\"\"\"\n",
    "    gray_img = img.convert('L')\n",
    "    processed_img = gray_img.resize(image_size)\n",
    "    return np.asarray(processed_img)\n",
    "\n",
    "def postprocess(predictions):\n",
    "    \"\"\"postprocess the raw predictions\"\"\"\n",
    "    return int(np.argmax(np.array(predictions).squeeze(), axis=0))\n",
    "\n",
    "# Collect the testing images\n",
    "images_dir = 'images'\n",
    "name_list = []\n",
    "for filename in os.listdir(images_dir):\n",
    "    if filename.split(\".\")[-1] in ('png', 'jpg'):\n",
    "        name_list.append(filename)\n",
    "name_list.sort()\n",
    "\n",
    "# Read the testing images, and preprocess them into the desired size.\n",
    "images = []\n",
    "for filename in name_list:\n",
    "    filepath = os.path.join(images_dir, filename)\n",
    "    img = Image.open(filepath)\n",
    "    img_array = preprocess_image_toarray(img)\n",
    "    images.append((filename, img_array))\n",
    "\n",
    "# Create the input tensor from the testing image.\n",
    "i = 0\n",
    "for x in images:\n",
    "    filename = x[0]\n",
    "    img_processed = x[1]\n",
    "\n",
    "    # Create onnx.TensorProto. In general, there are two ways\n",
    "    # 1. Save the data by the specified data type\n",
    "    # 2. Save the data by the raw way that contains the serialized content of the tensor.\n",
    "    onnx_tensor = onnx.helper.make_tensor(name='', \n",
    "                                          data_type=onnx.TensorProto.FLOAT, \n",
    "                                          dims=input_shape,\n",
    "                                          vals=img_processed.astype(np.float32))\n",
    "\n",
    "    # Convert to ai_serving_pb2.TensorProto from onnx.TensorProto\n",
    "    # Both messages are compatiable with each other\n",
    "    input_tensor = ai_serving_pb2.TensorProto()\n",
    "    input_tensor.ParseFromString(onnx_tensor.SerializeToString())\n",
    "    input_tensors.append(input_tensor)\n",
    "    input_arrays.append(img_processed)        \n",
    "    \n",
    "    # Add a subplot for the current digit.\n",
    "    plt.subplot(1, len(images), i+1)\n",
    "    plt.tight_layout()\n",
    "    plt.imshow(img_processed.reshape([28, 28]), cmap='gray', interpolation='none')\n",
    "    plt.title('Image: {}'.format(filename))\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    \n",
    "    i += 1\n",
    "print('The testing image list:')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"request\"></a>HTTP request formats\n",
    "The request for AI-Serving could have two formats: JSON and binary, the HTTP header Content-Type tells the server which format to handle and thus it is required for all requests. The binary payload has better latency, especially for the big tensor value for ONNX models, while the JSON format is easy for human readability.\n",
    "\n",
    "- Content-Type: application/octet-stream, application/vnd.google.protobuf or application/x-protobuf. The request body must be the protobuf message PredictRequest, besides of those common scalar values, it can use the standard onnx.TensorProto value directly.\n",
    "\n",
    "\n",
    "- Content-Type: application/json. The request body must be a JSON object formatted as described [here](https://github.com/autodeployai/ai-serving#4-predict-api)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"binary-request\"></a>Binary requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"construct-binary-request\"></a>Construct binary requests\n",
    "We will create both instances of PredictRequest, one is using the `Records` format that has the first 2 cases, the other is using the `Split` format that contains others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ai_serving_pb2 import RecordSpec, Record, PredictRequest, ListValue, Value\n",
    "\n",
    "# Create an instance of RecordSpec using `records` that contains the first 2 tensors.\n",
    "request_message_records = PredictRequest(X=RecordSpec(\n",
    "    records=[ Record(fields={input_tensor_name: Value(tensor_value=x)}) for x in input_tensors[0:2] ]\n",
    "))\n",
    "\n",
    "# Create an instance of RecordSpec using `split` that contains the left tensors.\n",
    "request_message_split = PredictRequest(X=RecordSpec(\n",
    "    columns = [input_tensor_name],\n",
    "    data = [ ListValue(values=[Value(tensor_value=x)]) for x in input_tensors[2:] ]\n",
    "))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"make-binary-request\"></a>Make the HTTP requests with binary data\n",
    "Make predictions using the AI-Serving, the content type of requests with binary data must be one of those three candidates above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {'Content-Type': 'application/x-protobuf'}\n",
    "\n",
    "# When version is omitted, the latest version is used.\n",
    "prediction_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# Make prediction for the `records` request message.\n",
    "prediction_response_records = requests.post(prediction_url, \n",
    "                                           headers=headers, \n",
    "                                           data=request_message_records.SerializeToString())\n",
    "\n",
    "# Make prediciton for the `split` request message.\n",
    "prediction_response_split = requests.post(prediction_url, \n",
    "                                           headers=headers, \n",
    "                                           data=request_message_split.SerializeToString())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"consume-binary-response\"></a>Consume the HTTP responses with binary data\n",
    "Having received the results from the server, we are going to parse the \"serialized\" message that we just received for us to make sense of the results. And compare the actual results to the desired ones. **NOTE: The data format of the output response is always the same as the input request.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id=\"parse-binary-response-records\"></a>Parse the response message from the `records` request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw predictions of \"6.jpg\": [  714.41437 -2176.6016  -1575.0326   -880.8216  -1648.9232   2636.4424\n",
      "  4059.8008  -3973.2715   1262.9858  -1492.2789 ]\n",
      "Final recognized digit: 6\n",
      "------------------------------------------------------------------------------------------------------------------------\n",
      "Raw predictions of \"6.jpg\": [ -514.49316   138.14052  2192.4036   2244.1516  -3340.5215  -1775.897\n",
      " -6222.5186   7784.408   -1814.3502    934.81805]\n",
      "Final recognized digit: 7\n",
      "------------------------------------------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAADUCAYAAADEO8eJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAdb0lEQVR4nO3da1BU5xkH8AcWFlZYEIhCRAQ0UUzqrRqvKGhqjVFblUBtawQnNbGXtI11EtKZaDQ2xlpTMzZV0yjRqBnFKl5qjCb1OjpTraMxYqImMcYgXoKRBQSUffrBYSvwPgfOcoB98f+b8YPP7p7z7u55eTjLf9/jx8xMAAAAmvBv6QEAAACYgcYFAABaQeMCAACtoHEBAIBW0LgAAEAraFwAAKAVNC4AANAKGhcAAGgFjQsAALRiqnG988475Ofn5/kXHBxMMTExNHz4cJo/fz5duXLF64Hk5+fTyy+/TOfPn/d6G43dT1ZWFiUkJDTp/iUtuW9fwMyUk5ND/fv3p5CQEAoLC6Pvf//7tGXLlnofm5qaSqmpqU0/yGaEudZ07uW5lpCQUOO4qn2M1cdn5hqbkJOTw0TEOTk5fPjwYd6/fz9v3LiRf//733N4eDhHRkby7t27zWzSIzc3l4mI9+zZ49XjrdjPuXPn+NixY026f0lL7tsXPPPMMxwUFMTZ2dn84Ycf8s6dO3nhwoW8bt26eh976tQpPnXqVDOMsvlgrjWde3muHTt2jA8fPlzj3/r165mIeNKkSfU+3lfmWoA3ze573/se9evXz/P/tLQ0eu655yg5OZkmTpxIZ8+epejo6MZ31WbWpUuXe3LfLS0vL4+WL19O69evp4yMDE991KhRDXr8Qw891FRDa3GYa61r3y2tT58+dWoffPABERH94he/qPfxPjPXzHS56t8Cjxw5orx9w4YNTEQ8Z86cGvUjR47wuHHjOCIigoOCgrh37968fv36Otut/S8nJ8dzn927d/OIESPY6XSyw+HgwYMH84cfflhnDKdPn+ZJkyZx+/bt2W63c1xcHD/55JNcXl5e734yMzM5Pj6+xvaIiH/961/z6tWrOSkpiR0OB/fs2ZO3bdtWZ995eXnco0cPttvtnJiYyIsXL+bZs2dzQ15mo32vXLmSu3btysHBwdy3b18+fPgwu91u/vOf/8wJCQkcEhLCw4cP57Nnz9Z4/K5du/hHP/oRx8bGclBQEHfp0oWffvppvnr1qtdjd7vd/Oabb3KvXr04ODiY27Zty2lpafz555/X+xwlqampnJCQ4PXjU1JSOCUlxfP/L7/8komIFyxYwPPmzeO4uDgOCgrivn37Ko+ZxrxvTQVzDXOtKeZabW63m+Pj47lz587sdrvrvb+vzDVLG1dJSQnbbDZ+9NFHPbV///vfbLfbeejQobx+/XreuXMnZ2Vl1TiIr1y5wq+++ioTEb/55pueU9grV64wM/O7777Lfn5+PH78eN60aRNv27aNx44dyzabrcaLc/z4cQ4NDeWEhARetmwZf/TRR7xmzRrOyMjg4uLievcjHdAJCQncv39/3rBhA+/YsYNTU1M5ICCgxgH0/vvvs7+/P6empvLmzZs5NzeXBwwYwAkJCY2aTPHx8Tx48GDetGkTb968mbt27cqRkZH83HPP8Y9//GPevn07r127lqOjo7lnz541Dr6lS5fy/PnzeevWrbxv3z5etWoV9+rVi7t168aVlZVejX3atGkcGBjIf/jDH3jnzp28bt06TkpK4ujoaC4sLPTcb8+ePUxEPHv2bMPnfevWLQ4KCuIJEybwokWLuFOnTuzv78+JiYm8cOHCRk2muLg4Tk5O5n/+85+cm5vLjzzyCAcGBvKhQ4e8eu7NCXMNc83quaaya9cuJiKeN29eg+7vK3PN0sbFzBwdHc3du3f3/D8pKYn79OnDt27dqnG/sWPH8v33389VVVXMLH8eXlpaypGRkTxu3Lga9aqqKu7Vqxf379/fUxsxYgS3bdvWMzlUjD53lw7o6OhoLi4u9tQKCwvZ39+f58+f76k98sgjHBcXxxUVFZ6ay+XiqKioRk2mmJgYLikp8dTy8vKYiLh37941Js7ixYuZiPjjjz9Wbt/tdvOtW7f4q6++YiLiLVu2mB774cOHmYh40aJFNbb99ddfs8Ph4Oeff95T27t3L9tstjpnBLVdunSJiYjDwsK4Y8eOvGrVKv7oo494+vTpTET8xz/+0fDxzPJk6tChA9+8edNTLy4u5sjISP7BD35g+rk3N8y1OzDXrJtrKj/5yU/YZrPxxYsXG3R/X5lrlsfh+a7Le507d44+/fRT+vnPf05ERLdv3/b8e/zxx+nSpUv02WefGW7v0KFDVFRURJmZmTUe73a76bHHHqMjR45QaWkplZWV0b59+ygjI4PatWtn6XMaPnw4OZ1Oz/+jo6Opffv29NVXXxERUWlpKR09epTGjx9Pdrvdc7/Q0FAaN25co/cdEhLi+X/37t2JiGj06NHk5+dXp149JiKiK1eu0PTp0ykuLo4CAgIoMDCQ4uPjiYjo9OnTpse+fft28vPzo8mTJ9d4L2JiYqhXr160d+9ez31TUlLo9u3bNGvWLMPn53a7iYiouLiYcnNzacqUKTRixAhaunQpjR8/nl5//XUqKSlp8Ot1t4kTJ9ZISjmdTho3bhzt37+fqqqqmvR9aw6Ya5hrRA2fa7UVFRVRXl4ePfbYYxQbG2vqsbU191zzKpwhKS0tpW+//ZZ69OhBRESXL18mIqKZM2fSzJkzlY+5du2a4Tart/HEE0+I9ykqKiJ/f3+qqqqijh07ejN0Q1FRUXVqQUFBdPPmTSIiun79OjGz8o/kjf3DeWRkZI3/V7/pUr28vJyI7jSEH/7wh1RQUEAvvfQS9ejRg0JCQsjtdtPAgQO9Gvvly5fF+xIRde7c2fTzi4iIID8/P3I6nTRw4MAat40ePZry8vIoPz+f+vfvb3rbMTExylplZSWVlJSQy+VqsvetqWGuYa411po1a6iioqJBoYz6NPdcs7Rx/etf/6KqqipPzv++++4jIqIXX3yRJk6cqHxMt27dDLdZvY0lS5bU+cFWLTo6mqqqqshms9HFixe9HL33qn/4Vk/8uxUWFjb7eIiIPvnkEzpx4gS98847lJmZ6amfO3euxv3MjP2+++4jPz8/OnDgAAUFBdW5v6pWH4fDQQ8++KDydao+o/D39+6DAdU2CwsLyW63U2hoKAUEBPjc+9ZQmGu+857pMtdqW7FiBUVHR9PYsWMbva3mnmuWfVR44cIFmjlzJoWHh9MzzzxDRHcmyoMPPkgnTpygfv36Kf9VfyxQ/UZU/3ZSbciQIdS2bVvKz88Xt2G328nhcFBKSgrl5uYa/mYp7acxQkJCqF+/fpSXl0eVlZWeeklJCW3fvt2y/ZhR/dFG7QN8+fLlNf5vZuxjx44lZqZvvvlG+T5U//ZvVlpaGhUXF9OhQ4dq1Hfs2EGhoaH08MMPe7XdTZs2eX4rJiJyuVy0bds2Gjp0KNlsNp983xoCc8233jOd5lq1o0eP0scff0yZmZkUEND485fmnmtejfiTTz7xfOZ65coVOnDgAOXk5JDNZqPNmzfX+Nx7+fLlNHr0aBo1ahRlZWVRbGwsFRUV0enTp+nYsWOUm5tLRHe+r0JE9NZbb5HT6aTg4GBKTEykqKgoWrJkCWVmZlJRURE98cQT1L59e7p69SqdOHGCrl69SkuXLiUiotdff52Sk5NpwIABlJ2dTQ888ABdvnyZtm7dSsuXLyen02m4n8aYO3cujRkzhkaNGkW/+93vqKqqihYuXEihoaFUVFTUqG17Iykpibp06ULZ2dnEzBQZGUnbtm2j3bt3ez32IUOG0NNPP01Tp06lo0eP0rBhwygkJIQuXbpEBw8epB49etAvf/lLIiLat28fPfroozRr1qx6P3ufOXMmrV27ltLT0+mVV16hjh070saNG2nr1q30l7/8hRwOh+e+DzzwABHV/W1WxWaz0ciRI2nGjBnkdrtpwYIFVFxcTHPmzDH93FsK5lpdvvae6TTXqq1YsYKIiJ566inxPj4918wkOWp/N8Nut3P79u05JSWFX331VTFhdOLECc7IyOD27dtzYGAgx8TE8IgRI3jZsmU17rd48WJOTExkm81W57sl+/bt4zFjxnBkZCQHBgZybGwsjxkzhnNzc2tsIz8/n9PT0zkqKortdjt36tSJs7KyuLy8vN79GH2/o7b4+HjOzMysUdu8ebPnOwqdOnXi1157jX/7299yREREPa9sw/ddneJZuHBhjXp1JPbu1yM/P59HjhzJTqeTIyIiOD09nS9cuKCMzpoZ+8qVK3nAgAEcEhLCDoeDu3TpwlOmTOGjR4/WGU9DI7oXLlzgSZMmcUREBNvtdu7ZsyevXLmyzv3i4+PrvE4pKSmcmppa5zVasGABz5kzhzt27Mh2u5379OnDH3zwQZ1tNuZ9ayqYa/+HuWbtXCsrK+Pw8HAeNmyY4f18ea61XN73HlBZWckPPfQQjxw5sqWHYppOY+/duzenpaV5/i/9wGkonZ473KHze6bT2H1lrlkazrjXPfXUUzRy5Ei6//77qbCwkJYtW0anT5+mN954o6WHVi8dx37mzBk6cOAAnTx5kiZPnuz1dnR87vc6nd8zHcfuc3PNqzYJSunp6RwbG8t2u51DQkJ46NCh/P7777f0sBpEx7FnZWVxVFQUT506lcvKyjx1s78F6vjc73U6v2c6jt3X5pof813fYgQAAPBxuJAkAABoBY0LAAC0gsYFAABasTRV6Ha7qaCggJxOZ41FKQF8ETOTy+WiDh06eL2sVEvBXAOdWD3XLG1cBQUFFBcXZ+UmAZrc119/3SQLxjYlzDXQkVVzzdLGdfflCHQWGBgo3nbr1i1T27LZbMp6VVWVsi799mz0W3X1pUFqk9Ygk+p3rzV2N6PXQwql3r59W3yMr9HxuG3pMUu/NUvHopWkOSWRxoRAdfOz6ri1tHG1lo8srHweZhuRN43Lqn2b3Y43pG215A8RHY9bozF783zMvv5mjy0r31+rjl80ruZn1VzT64N9AAC456FxAQCAVrBWocLd14tpKOkzf+njCLN1o8/1pdNv6e9xd1+e/G7S8zb6e5X094OIiAhl/fr16+K2wBrSMWTlR6LS32glZv/Wa6Sp/35qNNekvw9XVFQ01XBAAWdcAACgFTQuAADQChoXAABoBY0LAAC0gsYFAABaQarQJCk9KCW2vElNmd2ONKbY2Fhl/eLFi8q6lJjyZtUOKT0YFBSkrEtJMatev9as9vtjNrFKRGS325V1adUUKYEqHQ9SUs/KL0tbVTc65nA8+gaccQEAgFbQuAAAQCtoXAAAoBU0LgAA0AoaFwAAaAWNCwAAtII4vElWxd6tuhie0bak2HtwcLCyLl1I0ojD4VDWpdcDi5FaKyAgoM4xKX21wCgOL8XbvVlwWqUlY+Rm55rR69QcF8qE+uGMCwAAtILGBQAAWkHjAgAAraBxAQCAVtC4AABAK0gVmmQ2HSUtgNumTRtlvaysTFk3SjrdunXL1JikbfXt21dZX7Rokbit9957T1lfvny5qTFJi7mafW73GlWCUFow12hBW6vSntIiyhKjMUnpSKkukeas2QWzieSEIhbfbV444wIAAK2gcQEAgFbQuAAAQCtoXAAAoBU0LgAA0ApShSZZlSqS1gX0Jp0UERGhrCcnJyvrzz77rLI+YMAAZT0sLEzcd8eOHZX1NWvWKOulpaXKOtKD1rFqfUEiOaEYEKD+0SGlYluSlB6UICHo+3DGBQAAWkHjAgAAraBxAQCAVtC4AABAK2hcAACgFaQKFYzWKpPSVFISSbpiqtkrqXbp0kW87Wc/+5myPmvWLGVdSvZJ6cF9+/aJ+5bSgNI+wsPDlfUbN26I+wCZw+Goc7xameyz6srI0hqG0nwy2ofZBKo018ymDcF34J0DAACtoHEBAIBW0LgAAEAraFwAAKAVNC4AANAKUoUKRlcbNpsGlEhrHkrrDmZkZIjbmjt3rql9X7t2TVnPzs5W1levXi1uKzEx0dS+XS6Xsi4lOY3eCyC6efNmg+8rXWWaSE7YSVdGltKAaWlpyvqUKVOUdelYJCL64osvlPX//ve/yvrJkyeV9YKCAmXdm/UxcQVk34AzLgAA0AoaFwAAaAWNCwAAtILGBQAAWkHjAgAArfixhbGt4uJicS06nTgcDvE2KcUlpbKkFGJCQoKy/sorryjrkydPFsckpaPee+89Zf03v/mNsi6tL2iUpJTSgFKCTdrW7du3xX00tRs3bhhe5dkXGc01b9bgM/tjQLr/uXPnlHVprU2jVKQ0D80kKYmIzpw5o6xLCVejY1FKU0rrKjqdTmVdmrORkZHivo8ePaqsv/baa8r62bNnlXUrr5BtllVzDWdcAACgFTQuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtYJFdBbNxWyL50uShoaHK+rvvvqusDx482PS+pRj7tGnTlHUpDitF2Nu2bSvuu6ioSFmXFh2V6tI+vvvuO3HfcCeeXfsrCVLUWloglsi6xaP/9Kc/KevS/Lh06ZK4rc6dOyvrvXv3VtYHDRqkrHft2lVZl+ZBc3yl5+LFi8q60ULI0ldiCgsLlfUXX3xRWTf71R1fhDMuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtIFWoYJSik1JuUhLx5ZdfVtaTk5NNjek///mPeNuoUaOUdWmBz2+//VZZl9JoJSUl4r6l1KTRY1SQHvSOv7+/uNBxbdICsURy2tPspepXrVqlrEuL8nqzxnebNm2U9YiICGW9Xbt2ynpBQYGyHhcXJ+5bWvhXWphXes2PHz+urBulLCVSQlEak1VztiXhjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtIFSoYJdykpF58fLyy/vzzz5vah5SYGjdunDgmqxJ50npyFRUV4mPMXgY8NjZWWf/mm29MbQfuqKysbHCq0JsEn7R+ndm0oXR/o/XxpPGWlZUp69JxWlxcrKy7XC5lXVr7k0h+HtJYpX1MmDBBWZfmIBHRwYMHlfW3335bWZd+luiUHpTgjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtIFSoYpbSklFBaWpqyLq1hKK2HOHz4cGX92rVr4pikJJKU8AoLC1PWpasZS2vAERFdv35dWZeusor0oPVqJ9qCg4OV9ysvL2/0tqtJx5ZEOkalhCCR+echJf6kOSsxShWaFRISoqwvXbpUWTe6AvJbb72lrEvjldYk9CY97GtwxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBWkChWM1nR7+OGHlXXpSsfStqSrkw4aNMh4cApDhgxR1s+dO6esS+sL7t69W1mXkoNEcgJTSkdJySUpuWi0b1Br6NqFDXmM3W5X1s0m0IzSgxJp31Kq0Oy6mVK6zuhK0dJVwqV9Z2dnK+vR0dHKunR1ciKivXv3KutS+rI1rEkowRkXAABoBY0LAAC0gsYFAABaQeMCAACtoHEBAIBWkCpUkK4cSiSnr6Q0kJSMkq6kPHfuXGXdKOk0depUZf0f//iHqX1LzyE9PV3c9549e5R1KXUmvbZID1pHWh/TiJR+NZtQlI53s4k/IvNJRGnf0rqK0jEqJQeJ5Cs2JycnK+svvPCCsi6lisePHy/u+9KlS6a21ZqvNo4zLgAA0AoaFwAAaAWNCwAAtILGBQAAWkHjAgAAraBxAQCAVhCHVzCK4WZkZCjrUVFRyroUM961a5eyHh4erqx/8cUX4pikRTYlBw8eVNalSG9OTo64rWHDhinrFy5cUNal11aKXRsteAx3Lldf+7WT4tHSZeSJ5Mu/SwvaSqT3S/o6hxQvJ5Kfh0SK3EvPW3rONptN3If0/B5//HFlXXp+R44cUdaluUkkf5VEep2k2LvD4VDWvfkaRUvBGRcAAGgFjQsAALSCxgUAAFpB4wIAAK2gcQEAgFaQKlSQkn1ERFlZWcr61atXlfUZM2Yo62vWrFHWpcSPUbrL31/9+8e6deuUdZfLpayvXbtWWf/pT38q7tts6k9KQJpNr8Ed0gKyKlKKzkpGC9SqGCX4JFJKUDqGzD5vo+cgJftGjRqlrEtzMzs729SYiMwnciVm3yNfhDMuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtIFWo0L17d/G2Dh06KOvSWoIbN240tW9pvTCj9QilNFVxcbGpfZ86dUpZN0quSQlM6Xlcu3ZNWQ8KClLWpcurQ+tgNhFHJKcEw8LClHVpHkjH7o0bN8R9v/TSS8p6nz59lPXc3FxlXZprRswmMKV5681r7mtwxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBWkChWOHz8u3pafn6+sSwklKfHndDqVdSkxZXSlWLOk9RCnTZumrJeUlJjeh5QebA1XXwXrGF3l2GxKUKpLc0065vr27SuO6YUXXlDWpXn+9ttvK+vfffeduA+JlAaU1kOUUoVW/ixpKTjjAgAAraBxAQCAVtC4AABAK2hcAACgFTQuAADQClKFCkbrAkprjBklkVSkqxBLKisrTd3fSLdu3ZR1KcVlNFaz6SgpyRUYGKist4artYLMaP09s2tttmvXTlmXrk4eHR2trO/atUvch5TIW7FihbJ+8OBBcVtmSQlM6TWUUohmrprtq3DGBQAAWkHjAgAAraBxAQCAVtC4AABAK2hcAACgFTQuAADQCuLwCkYR7/379yvrEydOVNazsrKU9Q0bNijrZWVlhmNT6dq1q7IeGRmprC9YsEBZj4iIUNb/+te/ivuWXqugoCBlvaKiQllHHP7eFBAg/wiSYtvScSrF3iU7duxQ1qVFeYmIzp8/r6wvW7ZMWZe+/mF2fhDJr5UUk7fb7cq6NKeYWdy3r8EZFwAAaAWNCwAAtILGBQAAWkHjAgAAraBxAQCAVpAqNElagPPTTz9V1t944w1lfcCAAcr6mTNnlHVpAVEiomeffVZZlxJKUVFRyvpnn32mrP/9738X911SUiLepiJdZtybNCXoz5sk2/Xr15X18PBwZX3gwIHKelJSkrIuJVyJiH71q18p69L8lxilByVSElFKFUoLc0uL8uq0+C7OuAAAQCtoXAAAoBU0LgAA0AoaFwAAaAWNCwAAtIJUoUJwcLB42+eff66sz5s3T1lftWqVsj59+nRTY5KSQ0RySqi0tFRZnzFjhrK+ZMkSZd1oPTmJ2bXYWkPSCcyTkm9ERCEhIcq6dFy3bdtWWZ8/f76yLh3Xs2fPFsckpYq9mSNmGf0MMMPtdluynZaEMy4AANAKGhcAAGgFjQsAALSCxgUAAFpB4wIAAK0gVahQXl4u3hYaGqqs5+bmKuvS+n8JCQnK+vjx45X1Hj16iGOSkot/+9vflHWzCT6jNJN0NVppPTmkB+Fu0tqVRHJ6UJqDTz75pLLep08fU2PasmWLqfsTEfn5+Zm6v3SVZZfLJT5GSuSaTV/qdKVjCc64AABAK2hcAACgFTQuAADQChoXAABoBY0LAAC04scWRkyKi4vFq5AC+KobN25QWFhYSw/DlNYy1+x2u3ibtI7h6NGjlfXVq1cr61KCT0q4Dho0SBzTyZMnlXVvrmh8L7JqruGMCwAAtILGBQAAWkHjAgAAraBxAQCAVtC4AABAK1irEABajDdXQJbW7ZRSloGBgcr6l19+Wc/o6kJ60DfgjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBXE4QHAJ0mXnnc4HMq6FHs/fvy4sj5hwgRl/fz58+KYYmJilPXCwkLxMWA9nHEBAIBW0LgAAEAraFwAAKAVNC4AANAKGhcAAGjFj5nZqo21lsuJw73FqsuJN6fWMtfatGkj3lZWVtaMI/m/4OBg8bby8vJmHEnrY9VcwxkXAABoBY0LAAC0gsYFAABaQeMCAACtWLrkk4U5D4Bmo+Nxq+OYVXzxefjimFoLq15bSxuXy+WycnMAzcLlcmmX0Gstc+3mzZstPYQ6KioqWnoIrZZVc83SOLzb7aaCggJyOp3k5+dn1WYBmgQzk8vlog4dOpC/v16fmmOugU6snmuWNi4AAICmptevmQAAcM9D4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBU0LgAA0AoaFwAAaOV/22GpFpMR1i4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response_message = ai_serving_pb2.PredictResponse()\n",
    "response_message.ParseFromString(prediction_response_records.content)\n",
    "\n",
    "i = 0\n",
    "for x in response_message.result.records:\n",
    "    output_tensor_value = x.fields[output_tensor_name].tensor_value\n",
    "    output_tensor_array = np.asarray(output_tensor_value.float_data, dtype=np.dtype('float32'))\n",
    "    \n",
    "    # Print the prediction for the testing image\n",
    "    print('Raw predictions of \"{}\": {}'.format(images[0][0], output_tensor_array))\n",
    "    print('Final recognized digit: {}'.format(postprocess(output_tensor_array)))\n",
    "    print('-'*120)\n",
    "    \n",
    "    # Add a subplot for the current digit.\n",
    "    plt.subplot(1, len(images) - 1, i + 1)\n",
    "    plt.tight_layout()\n",
    "    plt.imshow(images[i][1].reshape([28, 28]), cmap='gray', interpolation='none')\n",
    "    plt.title('Detecting image: {}'.format(images[i][0]))\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    i += 1\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id=\"parse-binary-response-split\"></a>Parse the response message from the `split` request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw predictions of \"8.jpg\": [  397.46228 -2270.2764   1143.2241    850.3334   -468.74387  -592.69666\n",
      "   327.9959  -4857.1685   4933.365    -390.23157]\n",
      "Final recognized digit: 8\n",
      "------------------------------------------------------------------------------------------------------------------------\n",
      "Raw predictions of \"9.jpg\": [-2689.0283  -3374.9338  -1715.207    -871.94696  3004.0815   -926.28235\n",
      " -3160.6943   1612.2079    463.7397   5574.235  ]\n",
      "Final recognized digit: 9\n",
      "------------------------------------------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAADUCAYAAADEO8eJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAehUlEQVR4nO3deXBUVfYH8JM06SR0QpJOzALEhE0CyCarbAlgRAcQCiSDsiQIKGK5jFJTOJYoMxbIUDKIUpBhCbKpZFgGGEDRkcUizBBAFAhqxmGbEAIEskECdJ/fH1b6R8g9D17TSfqG76eKP/Lt7vdud7+b0685uc+HmZkAAAA04VvXAwAAADADhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0AoKFwAAaMVU4VqxYgX5+Pi4/gUEBFB0dDT179+fZs+eTQUFBW4P5Pjx4/Tuu+/SyZMn3d7Gve4nLS2N4uPja3T/krrctzdYv3499e7dm+x2O4WGhlL37t1p1apVd/XYpKQkSkpKqtkB1jLMtZpzv8+1NWvWUOfOnSkgIIAiIiLo2WefpTNnztzVY71mrrEJGRkZTESckZHBWVlZvGfPHv7b3/7Gr732GoeEhLDdbuedO3ea2aRLZmYmExF/8803bj3eE/vJzc3lQ4cO1ej+JXW577q2bNkyJiIeOXIkb9u2jbdv386jR49mIuJ58+bd8fHHjh3jY8eO1cJIaw/mWs25n+faggULmIh40qRJvGPHDl66dCnHxMRwXFwcFxYW3vHx3jLX3CpcBw4cqHbbqVOnODY2loODgzk/P9/0QLxhMkHd6N27N8fFxbHD4XBlTqeTExISuEOHDnU4srqDuQaeVl5eziEhITx06NAq+b59+5iI+A9/+EMdjcw8jxUuZuZ169YxEfHMmTOr5AcOHOChQ4dyWFgY+/v7c6dOnfjzzz+vtt3b/2VkZLjus3PnTh4wYAAHBwdzYGAg9+rVi7/66qtqY8jJyeHRo0dzZGQkW61Wjo2N5XHjxnF5efkd95OamspxcXFVXyAifumll3jlypWckJDAgYGB3KFDB96yZUu1fW/atInbt2/PVquVmzVrxvPnz+d33nmH7+bE1mjfy5cv54ceeogDAgK4S5cunJWVxU6nk//85z9zfHw822w27t+/P//8889VHv/ll1/yU089xU2aNGF/f39u0aIFP//883zhwgW3x+50OnnhwoXcsWNHDggI4NDQUB45ciT/5z//ueNzlCQlJXG7du2q5d27d+fu3bvf8fGJiYmcmJjo+vm///0vExHPmTOH33vvPY6NjWV/f3/u0qWL8pi5l/etpmCuYa55eq5lZ2czEfGiRYuq3Wa327l169Z33Ia3zDWPFq7S0lK2WCw8cOBAV/bPf/6TrVYr9+3blz///HPesWMHp6WlVTmICwoKeNasWUxEvHDhQs7KyuKsrCwuKChgZuZVq1axj48PDx8+nDds2MBbtmzhIUOGsMViqfLifPfddxwUFMTx8fG8ePFi/vrrr3n16tWckpLCxcXFd9yPdEDHx8dz9+7ded26dbxt2zZOSkriBg0aVDmAtm/fzr6+vpyUlMQbN27kzMxM7tGjB8fHx9/TZIqLi+NevXrxhg0beOPGjfzQQw+x3W7n3/3udzxs2DDeunUrr1mzhqOiorhDhw7sdDpdj1+0aBHPnj2bN2/ezLt37+ZPPvmEO3bsyK1bt+br16+7NfbJkyezn58fv/HGG7xjxw5eu3YtJyQkcFRUVJVP/9988w0TEb/zzjt3fO7r169nX19ffu+997igoIAvXLjAc+fOZYvFwuvWrbvj46XJFBsby3369OH169dzZmYmd+vWjf38/Hjfvn1uPffahLmGuebpuVZ5ZrV8+fJqt8XExLCvry9fu3bNcBveMtc8WriYmaOiorhNmzaunxMSErhz585848aNKvcbMmQIx8TEuL4ekr5WKCsrY7vdXu301uFwcMeOHat8Ih8wYACHhoa6JoeK0dcX0gEdFRXFxcXFriw/P599fX159uzZrqxbt24cGxvLFRUVrqykpITDw8PvaTJFR0dzaWmpK9u0aRMTEXfq1KnKxJk/fz4TEX///ffK7TudTr5x4wafOnWKiYj//ve/mx57VlYWExF/8MEHVbZ95swZDgwM5N///veubNeuXWyxWKqdEUg2bdrEISEhrk/mgYGBvHr16rt6rDSZGjduXGUiFhcXs91u58cee8z0c69tmGu/wlzz3Fy7dOkS+/r68sSJE6vkubm5rnmXl5dnuA1vmWseb4fnWy7vlZubSydOnKAxY8YQEdHNmzdd/37zm9/QuXPn6McffzTc3r59+6iwsJBSU1OrPN7pdNITTzxBBw4coLKyMrp69Srt3r2bUlJS6IEHHvDoc+rfvz8FBwe7fo6KiqLIyEg6deoUERGVlZVRdnY2DR8+nKxWq+t+QUFBNHTo0Hvet81mc/3cpk0bIiJ68sknycfHp1peOSYiooKCApoyZQrFxsZSgwYNyM/Pj+Li4oiIKCcnx/TYt27dSj4+PjR27Ngq70V0dDR17NiRdu3a5bpvYmIi3bx5k2bMmHHH57hjxw4aO3YsjRgxgrZv3047d+6kSZMmUVpaGmVkZNztS1XNiBEjKCAgwPVzcHAwDR06lPbs2UMOh6NG37fagLmGuUZ093PNbrfTmDFjaOXKlZSenk6FhYX0/fff05gxY8hisRARka+veyWhtudaA7dGKSgrK6NLly5R+/btiYjo/PnzREQ0bdo0mjZtmvIxFy9eNNxm5Taefvpp8T6FhYXk6+tLDoeDmjZt6s7QDYWHh1fL/P396dq1a0REdPnyZWJmioqKqnY/VWaG3W6v8nPlmy7l5eXlRETkdDrp8ccfp7y8PHr77bepffv2ZLPZyOl0Us+ePd0a+/nz58X7EhE1b97c9PNjZnruueeoX79+tHz5clf+2GOPUVFREb388suUkpJS5RfK3YqOjlZm169fp9LSUiopKamx962mYa5hrrlj0aJFxMw0depUmjJlCvn6+tK4ceMoKiqKvvjiC+Xrfzdqe655tHD94x//IIfD4erzj4iIICKiN998k0aMGKF8TOvWrQ23WbmNjz76iHr27Km8T1RUFDkcDrJYLHT27Fk3R+++sLAw8vHxcU38W+Xn59f6eIiIjh49SkeOHKEVK1ZQamqqK8/Nza1yPzNjj4iIIB8fH9q7dy/5+/tXu78qu5Pz58/TuXPn6IUXXqh2W7du3WjlypV08uRJateuneltq177/Px8slqtFBQURA0aNPC69+1uYa55z3umy1wjIrLZbLRq1SpasGABnTlzhho3bkwRERGUkJBAvXr1ogYN3CsJtT3XPPZV4enTp2natGkUEhLi+iXUunVratWqFR05coS6du2q/Ff5tUDlG1H56aRS7969KTQ0lI4fPy5uw2q1UmBgICUmJlJmZqbhJ0tpP/fCZrNR165dadOmTXT9+nVXXlpaSlu3bvXYfsyo/Grj9gM8PT29ys9mxj5kyBBiZvrf//6nfB8qP/2bERYWRgEBAbR///5qt2VlZZGvry/FxMSY3i4R0YYNG1yfiomISkpKaMuWLdS3b1+yWCxe+b7dDcw173rPdJlrtwoLC6MOHTpQREQEbd68mX788Ud69dVX3d5ebc81t8rr0aNHXd+5FhQU0N69eykjI4MsFgtt3Lixyvfe6enp9OSTT9KgQYMoLS2NmjRpQoWFhZSTk0OHDh2izMxMIiJ6+OGHiYjor3/9KwUHB1NAQAA1a9aMwsPD6aOPPqLU1FQqLCykp59+miIjI+nChQt05MgRunDhAi1atIiIiObNm0d9+vShHj160PTp06lly5Z0/vx52rx5M6Wnp1NwcLDhfu7FH//4Rxo8eDANGjSIXn31VXI4HDR37lwKCgqiwsLCe9q2OxISEqhFixY0ffp0Ymay2+20ZcsW2rlzp9tj7927Nz3//PM0YcIEys7Opn79+pHNZqNz587Rt99+S+3bt6cXX3yRiIh2795NAwcOpBkzZhh+9+7v709Tp06lefPm0fjx4+m3v/0tWSwW2rRpE61du5YmTpxY5auali1bElH1T7MqFouFkpOT6fXXXyen00lz5syh4uJimjlzpunnXlcw16rztvdMl7lG9OsKNXl5edSmTRsqLy+nXbt20YcffkhTpkyhYcOGVbmvV881M50ct/9thtVq5cjISE5MTORZs2aJHUZHjhzhlJQUjoyMZD8/P46OjuYBAwbw4sWLq9xv/vz53KxZM7ZYLNX+tmT37t08ePBgttvt7Ofnx02aNOHBgwdzZmZmlW0cP36cR40axeHh4Wy1WvnBBx/ktLQ0Li8vv+N+jP6+43ZxcXGcmppaJdu4caPrbxQefPBBfv/99/mVV17hsLCwO7yyd7/vyi6euXPnVskrW2JvfT2OHz/OycnJHBwczGFhYTxq1Cg+ffq0snXWzNiXL1/OPXr0YJvNxoGBgdyiRQseP348Z2dnVxvP3bTDOxwOXrJkCXft2pVDQ0O5UaNG3LlzZ/7444+rtBIz//q63/46JSYmclJSUrXXaM6cOTxz5kxu2rQpW61W7ty5M3/xxRfV9n8v71tNwVz7f5hrnptrGzdu5E6dOrm217VrV162bFmVrslK3jzX6q7f9z5w/fp1btu2LScnJ9f1UEzTaeydOnXikSNHun6WfuHcLZ2eO/xK5/dMp7F7y1zzaHPG/W7ixImUnJxMMTExlJ+fT4sXL6acnBz68MMP63pod6Tj2H/66Sfau3cv/fDDDzR27Fi3t6Pjc7/f6fye6Th2r5trbpVJUBo1ahQ3adKErVYr22w27tu3L2/fvr2uh3VXdBx7Wloah4eH84QJE/jq1auu3OynQB2f+/1O5/dMx7F721zzYb7lrxgBAAC8HC4kCQAAWkHhAgAAraBwAQCAVjzaVeh0OikvL4+Cg4OrLEoJ4I2YmUpKSqhx48ZuLy5aVzDXQCeenmseLVx5eXkUGxvryU0C1LgzZ87UyIKxNQlzDXTkqbnm0cJ16+UIdHbrsvu3u3WdrXshLWZ58+ZN09vy8/NT5lLDqNl9BAYGirfduj7ZrYKCgpR5SUmJqX3XBh2PWx3HrGL06dvpdHpkW9I8cKeh+tZLd9xKmgdmF8OtqKgwPSadeOq49Wjhqi9fWdTG85D2IeVGk6ymx2u0fbPPwxvpNNZKOo5ZxZPPw+y23ClcZvdRX94nT/HU66HXF/sAAHDfQ+ECAACtYK1CBaPvmaXv0aXvbqX/07lx44apMXny/92k5xAWFqbML126ZGr7RETFxcXKXPrOX/q/g6KiItP7Bn04HA7Tj6m8zPztpP83lo536f96jeamdG0x6biWfpd48mtK6XlL/0fozmvubXDGBQAAWkHhAgAAraBwAQCAVlC4AABAKyhcAACgFXQVKhj9kZzUqWO2+01a7cKddbyklS2kziXpr/zd6R40200pdXKhe/D+ZNQtK8016Rgy2y0nzXOpa9FoH1IuzUFpH9LvBSJ53prtUK4PcMYFAABaQeECAACtoHABAIBWULgAAEArKFwAAKAVFC4AANAK2uEVjBbADA8PV+ZmW8mlRWWlFuCysjJT2zfSsGFDZX716lVlLi3iSSS3vUst+vdj6y7I3Lkwq9k2drN/YmJ0AUtpH2YvzurOdffMbkuag9JYPXWR3NqAMy4AANAKChcAAGgFhQsAALSCwgUAAFpB4QIAAK2gq1DBqAvJU92DpaWlylzqKpIuDU5E1LVrV2X+xBNPKPPk5GRlfuXKFWW+d+9ecd/p6enK/OLFi+JjACpJ84PIc11utdEtJy2OK81nqbPPbHei0T6kLuH6AGdcAACgFRQuAADQCgoXAABoBYULAAC0gsIFAABaQVehQqNGjcTbpM47ibS2mdQJ1KdPH2X+5ptvivuQugojIiKUudm12wYMGCDe9vDDDyvzqVOnKvPLly+b2jfUb9Ll6D1JOt6lOWi1WsVtSV2QRUVFprYVFBSkzI26CqXfJdL6n1IujamiokLct7fBGRcAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZ82J1LbgqKi4spJCTEU5vzSlJXkdS5JK0XlpiYqMyXLFmizFu1aiWO6bvvvlPmK1euVOaHDx9W5p07d1bmH3zwgbhv6eqrgwYNUuY7d+5U5nXZ6VRUVGTYSeqN7oe5Jh0TDofDVC6R5rLRVbqlfUhXG5aupuzOcS3t49q1a6a2I3Unmn393OGpuYYzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAArWCtQgWjtfykjiOpI6dXr17K/JNPPlHmcXFxynzmzJnimObOnavMpSu/ml3bzIj0mLCwMGUuNbHqtE4a1A7pWDHb/Sat//naa68p88GDB4vbMrpis8qnn36qzGfNmqXM8/LyxG1J64JKj8nNzVXmddlV6Ck44wIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0Aq6ChWk9cWMSGv2/eUvf1Hm4eHhyvztt99W5vPmzRP3La2HKGndurUyX7p0qTI36vg7ceKEMt+8ebMylzo2GzRQH4pSZyTUf1LHqjR32rVrp8zXrFmjzKOjo5W5dCwSEZWWlipzaR3BZ555RplLx3VMTIy474EDByrzyZMnK3Opq9CDy9PWGZxxAQCAVlC4AABAKyhcAACgFRQuAADQCgoXAABoBYULAAC0gnZ4k6S2V0lBQYEy7969uzLPyclR5kYt7zabTZlPnDhRmc+fP1+Z37x5U5lfuXJF3Pfw4cOVuXQ5cbS9w62M5pN0DLVo0UKZf/XVV8q8vLxcmefn5ytzqYWdiCg0NFSZX7x4UZk/+uijyvzy5cvKXFowm0j+k5uff/5ZfIwK2uEBAABqGQoXAABoBYULAAC0gsIFAABaQeECAACtoKvQJKnTyezisZLCwkJl3qRJE/ExY8eOVebTpk1T5lJ30uHDh5X5hAkTxH2fPXtWmVutVmUudQ82bNhQmZtdQBj0InWyEhG1bdtWmW/dulWZ+/n5KfPs7Gxl/vLLLyvzgwcPimOSREREKHNpodtly5aZ2g6R3A387bffKnPpd5LRa64LnHEBAIBWULgAAEArKFwAAKAVFC4AANAKChcAAGgFXYUKUocbkdypI13SW1qTrKSkRJmnp6cr86CgIHFMdrtdmVssFvExKtKahC+99JL4mBkzZpjalsTpdJq6P9QPN27cEG976623lLk0F7Zv367Mp0yZosxPnz6tzKW1P4nkrmJprcJevXop86eeekrchyQjI8PU/QMCApR5fejUxRkXAABoBYULAAC0gsIFAABaQeECAACtoHABAIBW0FWo4Mmum0uXLilzf39/Zd6qVSvT+5DGu3fvXmUeHx+vzI8dO6bMp0+fLu5b6sB0OBzKXOp0kq5SC/XbunXrxNtGjRqlzKX1Md99911lLnUPSozWFzXb/Tpr1ixT95fWYSQi+umnn5S51D0s/V6Qnp9OaxjijAsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCvoKlSQrqRKZLy2msq2bduU+bhx45S51LVk1HUndQlJnYvJycnK/OTJk+I+JNIVjaXXUHoeZq+YDPVDSEiIeJv03kvdbzk5Ocpc6rpjZmVeVFQkjklaF3TmzJnKvG/fvuK2VN5//33xNk913urUPSjBGRcAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFbQVaggrbNnxMfHR5lnZWUp83//+9/KXOr4k9b4I5K7rwYPHqzMpasvS6Kjo8XbpLUYpY4trFUIt4qIiBBvkzpNpSsUSx25Zudz27ZtxdueeeYZZf7CCy8o87y8PGUudVMePnxY3Ld05WfpqsxSt3FFRYW4D13gjAsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCsoXAAAoBW0w3vIW2+9pcxfeeUVZS4t1vnDDz8o83Pnzon7fvzxx5X5kSNHlHn//v2VudS6m5+fL+7bLF9ffFaC/yddjp6I6JFHHlHmgYGByvzrr79W5lLrudQWHhwcLI6pefPmylz684+mTZsq8/T0dGUutbYb7cMsadFhd/4MqK7gtwgAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZqrKvw9kVn3emIkRaVLC0tdWtMt5MuL29k0qRJyvzFF19U5g888IAylxbZTU1NVebSgplE8kK+UkfTjBkzlPnEiRPFfUgiIyOVeUFBgTKXFgSWFin2VCcVeKfnnntOvE06tvr27avMu3XrpsylS9VLi/hevHhRHFNKSooyHz16tDIfMWKEMl+6dKkyN+q6lRYRluaI1D0ozUGd4IwLAAC0gsIFAABaQeECAACtoHABAIBWULgAAEArPuzBtq3i4mJxXTCpg+/GjRue2j01btxYmUsdblKXzsKFC8V9PPvss8q8UaNGylxaq3DJkiWmxiR1RhERDRo0SJlv2bJFmUvrofXo0UOZ5+TkiPs2e/hInVzuPG9PKSoqEt8/b2U01+q7Pn36KPOePXsqc+kYOnHihDLfs2ePuO8//elPynz8+PHK/MqVK8pcWi/07Nmz4r7NrjHojWsSemqu4YwLAAC0gsIFAABaQeECAACtoHABAIBWULgAAEArtXYFZKl70N/f3/S2pCuXSlfqlTrWOnTooMzHjh0r7ltaP/H1119X5tKVTj25Xtj+/fuVudQ9KD0H6arMRuskSu+r9BhpTcLa6B4E7xMVFSXedv78eWUuXSVcmgfSsRUeHq7Mr169Ko5J6hKWjve1a9cq80uXLon7kEj7kDp7pblWH+CMCwAAtILCBQAAWkHhAgAAraBwAQCAVlC4AABAKzXSVWiz2ap1tEhXLTbqrjPbFSN1D0ZERChzaU1Cm80m7kNat+/jjz9W5lLXndRNJXVSBQYGimOStiV1D/7yyy/KPC8vT5m7s56ktB4augfhVtLxbqSoqMgj+5Y6+xISEsTHSJ190tz59NNPlbk7XcXSPJR+70lqY93YmoYzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAAraBwAQCAVmqkHf769et33couXV6ayHzrdGhoqDKX2lsfffRRZW409jfeeEOZBwQEKHNfX/VnA7Nt75MnTxbHNGPGDPE2lRUrVpgakxGpPdjseyf9CUJZWZnpMUH9IM2p8vJyZS4dQ9IitNJiulOmTBHHJG3ryy+/VOb/+te/TG3HiNm2d+n+ZrfjjXDGBQAAWkHhAgAAraBwAQCAVlC4AABAKyhcAACglRrpKvTUYo1Sl5DUDVRcXKzMpS4ady5tLXW5SQtX9uvXT5mnpKQo8/bt2yvz5s2bi2MKDg5W5gsWLFDmS5cuVeZSt5YnSa+5Tgt8gudIXbdGpLlmtgP1kUceUebjx48XHyMdvwcPHlTm7nQPmiWNSXptpQWwdYIzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAArdRIV2FAQEC1Tpdr164p72u0pp3UFSN16ki51FUoXW5b6mYkItq9e7cyr6ioUObSc5A6o6TOvqNHj4pjWr16tTL/7LPPlLnZNQmNOr+k11Zag1LqaHLnUuagP6O1SqW5IB2P0tqGVqtVmUsdvFKXLpHc0Zydna3MpbVHpd+HRsxuqzY6GusKzrgAAEArKFwAAKAVFC4AANAKChcAAGgFhQsAALRSI12Fqm6ghg0bKu8rdekQeW79uv379yvzYcOGKXOpc5CIqKSkRJlLnUi5ubnKfM6cOcpc6h48fPiwOCapo1FitjvJaE1HqUvQ7DqQUqdYfbhaK8iM5rjUeSs9RupClHK73a7Mpat6ExGdOHFCmR84cECZS89BYjRvzK7r6KmOX2+EMy4AANAKChcAAGgFhQsAALSCwgUAAFpB4QIAAK3USFehilH3oKSm19o6dOiQMjdaq6w+MLtOmjvdRmbfO3QP3p+M1gU1u36lNG+lTuAxY8aY2j4R0bJly5S5NEfMzh2jeWP2Cs8SnboHJTjjAgAAraBwAQCAVlC4AABAKyhcAACgFRQuAADQSq11FQIA3M6oU85ozUAVqXtQWv/v5MmTyrxLly7iPvz9/U2NCWoGzrgAAEArKFwAAKAVFC4AANAKChcAAGgFhQsAALSCwgUAAFpBOzwAeKWgoCBlLi0SXVFRocwbNmyozD/77DNlbtTyfvDgQfE2qD044wIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0IoPm73GuoHi4mIKCQnx1OYAakVRURE1atSorodhSn2Za9ICuETGl7GvSS1bthRvy83NrcWR1D+emms44wIAAK2gcAEAgFZQuAAAQCsoXAAAoBWPLvlUV/+ZCnAvdDxudRyzijc+D6fTWddDqLc89X57tHBJl84G8GYlJSXadehhrtWcX375pa6HUG95aq55tB3e6XRSXl4eBQcHG7a5AngDZqaSkhJq3Lgx+frq9a055hroxNNzzaOFCwAAoKbp9TETAADueyhcAACgFRQuAADQCgoXAABoBYULAAC0gsIFAABaQeECAACt/B8dGPqNlg7cwQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response_message = ai_serving_pb2.PredictResponse()\n",
    "response_message.ParseFromString(prediction_response_split.content)\n",
    "\n",
    "i = 2\n",
    "for x in response_message.result.data:\n",
    "    output_tensor_value = x.values[0].tensor_value\n",
    "    output_tensor_array = np.asarray(output_tensor_value.float_data, dtype=np.dtype('float32'))\n",
    "\n",
    "    # Print the prediction for the testing image\n",
    "    print('Raw predictions of \"{}\": {}'.format(images[i][0], output_tensor_array))\n",
    "    print('Final recognized digit: {}'.format(postprocess(output_tensor_array)))\n",
    "    print('-'*120)\n",
    "\n",
    "    # Add a subplot for the current digit.\n",
    "    plt.subplot(1, len(images) - 1, i - 1)\n",
    "    plt.tight_layout()\n",
    "    plt.imshow(images[i][1].reshape([28, 28]), cmap='gray', interpolation='none')\n",
    "    plt.title('Detecting image: {}'.format(images[i][0]))\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    i += 1\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"json-request\"></a>JSON requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"construct-json-request\"></a>Construct JSON requests\n",
    "Create both JSON objects, one is using the `Records` format that has one case, the other is using `Split` that contains other cases."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a JSON object with records that contains only the first tensor.\n",
    "request_json_recoreds = {\n",
    "    'X': [ {input_tensor_name: x.tolist()} for x in input_arrays[0:2]]\n",
    "}\n",
    "\n",
    "# Create a JSON object with columns and data that contains the last two tensors.\n",
    "request_json_split = {\n",
    "    'X': {\n",
    "        'columns': [input_tensor_name],\n",
    "        'data': [ [x.tolist()] for x in input_arrays[2:] ]\n",
    "    }\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"make-json-request\"></a>Make the HTTP requests with JSON data\n",
    "Make predictions using the AI-Serving, the content type of requests with JSON data must be `application/json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# When version is omitted, the latest version is used.\n",
    "prediction_url = base_url + '/v1/models/' + model_name\n",
    "\n",
    "# The Content-Type: application/json is specified implicitly when using json instead of data\n",
    "prediction_json_response_records = requests.post(prediction_url, json=request_json_recoreds)\n",
    "prediction_json_response_split = requests.post(prediction_url, json=request_json_split)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <a id=\"consume-json-response\"></a>Consume the HTTP response with JSON data\n",
    "Having received the results from the server, we are going to parse the JSON text that we just received for us to make sense of the results. And compare the actual results to the desired ones. **NOTE: The data format of the output response is always the same as the input request.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id=\"parse-json-response-records\"></a>Parse the json response message from the `records` request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw predictions of \"6.jpg\": [[  714.41437 -2176.6016  -1575.0326   -880.8216  -1648.9232   2636.4424\n",
      "   4059.8008  -3973.2715   1262.9858  -1492.2789 ]]\n",
      "Final recognized digit: 6\n",
      "------------------------------------------------------------------------------------------------------------------------\n",
      "Raw predictions of \"6.jpg\": [[ -514.49316   138.14052  2192.4036   2244.1516  -3340.5215  -1775.897\n",
      "  -6222.5186   7784.408   -1814.3502    934.81805]]\n",
      "Final recognized digit: 7\n",
      "------------------------------------------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAADUCAYAAADEO8eJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAdb0lEQVR4nO3da1BU5xkH8AcWFlZYEIhCRAQ0UUzqrRqvKGhqjVFblUBtawQnNbGXtI11EtKZaDQ2xlpTMzZV0yjRqBnFKl5qjCb1OjpTraMxYqImMcYgXoKRBQSUffrBYSvwPgfOcoB98f+b8YPP7p7z7u55eTjLf9/jx8xMAAAAmvBv6QEAAACYgcYFAABaQeMCAACtoHEBAIBW0LgAAEAraFwAAKAVNC4AANAKGhcAAGgFjQsAALRiqnG988475Ofn5/kXHBxMMTExNHz4cJo/fz5duXLF64Hk5+fTyy+/TOfPn/d6G43dT1ZWFiUkJDTp/iUtuW9fwMyUk5ND/fv3p5CQEAoLC6Pvf//7tGXLlnofm5qaSqmpqU0/yGaEudZ07uW5lpCQUOO4qn2M1cdn5hqbkJOTw0TEOTk5fPjwYd6/fz9v3LiRf//733N4eDhHRkby7t27zWzSIzc3l4mI9+zZ49XjrdjPuXPn+NixY026f0lL7tsXPPPMMxwUFMTZ2dn84Ycf8s6dO3nhwoW8bt26eh976tQpPnXqVDOMsvlgrjWde3muHTt2jA8fPlzj3/r165mIeNKkSfU+3lfmWoA3ze573/se9evXz/P/tLQ0eu655yg5OZkmTpxIZ8+epejo6MZ31WbWpUuXe3LfLS0vL4+WL19O69evp4yMDE991KhRDXr8Qw891FRDa3GYa61r3y2tT58+dWoffPABERH94he/qPfxPjPXzHS56t8Cjxw5orx9w4YNTEQ8Z86cGvUjR47wuHHjOCIigoOCgrh37968fv36Otut/S8nJ8dzn927d/OIESPY6XSyw+HgwYMH84cfflhnDKdPn+ZJkyZx+/bt2W63c1xcHD/55JNcXl5e734yMzM5Pj6+xvaIiH/961/z6tWrOSkpiR0OB/fs2ZO3bdtWZ995eXnco0cPttvtnJiYyIsXL+bZs2dzQ15mo32vXLmSu3btysHBwdy3b18+fPgwu91u/vOf/8wJCQkcEhLCw4cP57Nnz9Z4/K5du/hHP/oRx8bGclBQEHfp0oWffvppvnr1qtdjd7vd/Oabb3KvXr04ODiY27Zty2lpafz555/X+xwlqampnJCQ4PXjU1JSOCUlxfP/L7/8komIFyxYwPPmzeO4uDgOCgrivn37Ko+ZxrxvTQVzDXOtKeZabW63m+Pj47lz587sdrvrvb+vzDVLG1dJSQnbbDZ+9NFHPbV///vfbLfbeejQobx+/XreuXMnZ2Vl1TiIr1y5wq+++ioTEb/55pueU9grV64wM/O7777Lfn5+PH78eN60aRNv27aNx44dyzabrcaLc/z4cQ4NDeWEhARetmwZf/TRR7xmzRrOyMjg4uLievcjHdAJCQncv39/3rBhA+/YsYNTU1M5ICCgxgH0/vvvs7+/P6empvLmzZs5NzeXBwwYwAkJCY2aTPHx8Tx48GDetGkTb968mbt27cqRkZH83HPP8Y9//GPevn07r127lqOjo7lnz541Dr6lS5fy/PnzeevWrbxv3z5etWoV9+rVi7t168aVlZVejX3atGkcGBjIf/jDH3jnzp28bt06TkpK4ujoaC4sLPTcb8+ePUxEPHv2bMPnfevWLQ4KCuIJEybwokWLuFOnTuzv78+JiYm8cOHCRk2muLg4Tk5O5n/+85+cm5vLjzzyCAcGBvKhQ4e8eu7NCXMNc83quaaya9cuJiKeN29eg+7vK3PN0sbFzBwdHc3du3f3/D8pKYn79OnDt27dqnG/sWPH8v33389VVVXMLH8eXlpaypGRkTxu3Lga9aqqKu7Vqxf379/fUxsxYgS3bdvWMzlUjD53lw7o6OhoLi4u9tQKCwvZ39+f58+f76k98sgjHBcXxxUVFZ6ay+XiqKioRk2mmJgYLikp8dTy8vKYiLh37941Js7ixYuZiPjjjz9Wbt/tdvOtW7f4q6++YiLiLVu2mB774cOHmYh40aJFNbb99ddfs8Ph4Oeff95T27t3L9tstjpnBLVdunSJiYjDwsK4Y8eOvGrVKv7oo494+vTpTET8xz/+0fDxzPJk6tChA9+8edNTLy4u5sjISP7BD35g+rk3N8y1OzDXrJtrKj/5yU/YZrPxxYsXG3R/X5lrlsfh+a7Le507d44+/fRT+vnPf05ERLdv3/b8e/zxx+nSpUv02WefGW7v0KFDVFRURJmZmTUe73a76bHHHqMjR45QaWkplZWV0b59+ygjI4PatWtn6XMaPnw4OZ1Oz/+jo6Opffv29NVXXxERUWlpKR09epTGjx9Pdrvdc7/Q0FAaN25co/cdEhLi+X/37t2JiGj06NHk5+dXp149JiKiK1eu0PTp0ykuLo4CAgIoMDCQ4uPjiYjo9OnTpse+fft28vPzo8mTJ9d4L2JiYqhXr160d+9ez31TUlLo9u3bNGvWLMPn53a7iYiouLiYcnNzacqUKTRixAhaunQpjR8/nl5//XUqKSlp8Ot1t4kTJ9ZISjmdTho3bhzt37+fqqqqmvR9aw6Ya5hrRA2fa7UVFRVRXl4ePfbYYxQbG2vqsbU191zzKpwhKS0tpW+//ZZ69OhBRESXL18mIqKZM2fSzJkzlY+5du2a4Tart/HEE0+I9ykqKiJ/f3+qqqqijh07ejN0Q1FRUXVqQUFBdPPmTSIiun79OjGz8o/kjf3DeWRkZI3/V7/pUr28vJyI7jSEH/7wh1RQUEAvvfQS9ejRg0JCQsjtdtPAgQO9Gvvly5fF+xIRde7c2fTzi4iIID8/P3I6nTRw4MAat40ePZry8vIoPz+f+vfvb3rbMTExylplZSWVlJSQy+VqsvetqWGuYa411po1a6iioqJBoYz6NPdcs7Rx/etf/6KqqipPzv++++4jIqIXX3yRJk6cqHxMt27dDLdZvY0lS5bU+cFWLTo6mqqqqshms9HFixe9HL33qn/4Vk/8uxUWFjb7eIiIPvnkEzpx4gS98847lJmZ6amfO3euxv3MjP2+++4jPz8/OnDgAAUFBdW5v6pWH4fDQQ8++KDydao+o/D39+6DAdU2CwsLyW63U2hoKAUEBPjc+9ZQmGu+857pMtdqW7FiBUVHR9PYsWMbva3mnmuWfVR44cIFmjlzJoWHh9MzzzxDRHcmyoMPPkgnTpygfv36Kf9VfyxQ/UZU/3ZSbciQIdS2bVvKz88Xt2G328nhcFBKSgrl5uYa/mYp7acxQkJCqF+/fpSXl0eVlZWeeklJCW3fvt2y/ZhR/dFG7QN8+fLlNf5vZuxjx44lZqZvvvlG+T5U//ZvVlpaGhUXF9OhQ4dq1Hfs2EGhoaH08MMPe7XdTZs2eX4rJiJyuVy0bds2Gjp0KNlsNp983xoCc8233jOd5lq1o0eP0scff0yZmZkUEND485fmnmtejfiTTz7xfOZ65coVOnDgAOXk5JDNZqPNmzfX+Nx7+fLlNHr0aBo1ahRlZWVRbGwsFRUV0enTp+nYsWOUm5tLRHe+r0JE9NZbb5HT6aTg4GBKTEykqKgoWrJkCWVmZlJRURE98cQT1L59e7p69SqdOHGCrl69SkuXLiUiotdff52Sk5NpwIABlJ2dTQ888ABdvnyZtm7dSsuXLyen02m4n8aYO3cujRkzhkaNGkW/+93vqKqqihYuXEihoaFUVFTUqG17Iykpibp06ULZ2dnEzBQZGUnbtm2j3bt3ez32IUOG0NNPP01Tp06lo0eP0rBhwygkJIQuXbpEBw8epB49etAvf/lLIiLat28fPfroozRr1qx6P3ufOXMmrV27ltLT0+mVV16hjh070saNG2nr1q30l7/8hRwOh+e+DzzwABHV/W1WxWaz0ciRI2nGjBnkdrtpwYIFVFxcTHPmzDH93FsK5lpdvvae6TTXqq1YsYKIiJ566inxPj4918wkOWp/N8Nut3P79u05JSWFX331VTFhdOLECc7IyOD27dtzYGAgx8TE8IgRI3jZsmU17rd48WJOTExkm81W57sl+/bt4zFjxnBkZCQHBgZybGwsjxkzhnNzc2tsIz8/n9PT0zkqKortdjt36tSJs7KyuLy8vN79GH2/o7b4+HjOzMysUdu8ebPnOwqdOnXi1157jX/7299yREREPa9sw/ddneJZuHBhjXp1JPbu1yM/P59HjhzJTqeTIyIiOD09nS9cuKCMzpoZ+8qVK3nAgAEcEhLCDoeDu3TpwlOmTOGjR4/WGU9DI7oXLlzgSZMmcUREBNvtdu7ZsyevXLmyzv3i4+PrvE4pKSmcmppa5zVasGABz5kzhzt27Mh2u5379OnDH3zwQZ1tNuZ9ayqYa/+HuWbtXCsrK+Pw8HAeNmyY4f18ea61XN73HlBZWckPPfQQjxw5sqWHYppOY+/duzenpaV5/i/9wGkonZ473KHze6bT2H1lrlkazrjXPfXUUzRy5Ei6//77qbCwkJYtW0anT5+mN954o6WHVi8dx37mzBk6cOAAnTx5kiZPnuz1dnR87vc6nd8zHcfuc3PNqzYJSunp6RwbG8t2u51DQkJ46NCh/P7777f0sBpEx7FnZWVxVFQUT506lcvKyjx1s78F6vjc73U6v2c6jt3X5pof813fYgQAAPBxuJAkAABoBY0LAAC0gsYFAABasTRV6Ha7qaCggJxOZ41FKQF8ETOTy+WiDh06eL2sVEvBXAOdWD3XLG1cBQUFFBcXZ+UmAZrc119/3SQLxjYlzDXQkVVzzdLGdfflCHQWGBgo3nbr1i1T27LZbMp6VVWVsi799mz0W3X1pUFqk9Ygk+p3rzV2N6PXQwql3r59W3yMr9HxuG3pMUu/NUvHopWkOSWRxoRAdfOz6ri1tHG1lo8srHweZhuRN43Lqn2b3Y43pG215A8RHY9bozF783zMvv5mjy0r31+rjl80ruZn1VzT64N9AAC456FxAQCAVrBWocLd14tpKOkzf+njCLN1o8/1pdNv6e9xd1+e/G7S8zb6e5X094OIiAhl/fr16+K2wBrSMWTlR6LS32glZv/Wa6Sp/35qNNekvw9XVFQ01XBAAWdcAACgFTQuAADQChoXAABoBY0LAAC0gsYFAABaQarQJCk9KCW2vElNmd2ONKbY2Fhl/eLFi8q6lJjyZtUOKT0YFBSkrEtJMatev9as9vtjNrFKRGS325V1adUUKYEqHQ9SUs/KL0tbVTc65nA8+gaccQEAgFbQuAAAQCtoXAAAoBU0LgAA0AoaFwAAaAWNCwAAtII4vElWxd6tuhie0bak2HtwcLCyLl1I0ojD4VDWpdcDi5FaKyAgoM4xKX21wCgOL8XbvVlwWqUlY+Rm55rR69QcF8qE+uGMCwAAtILGBQAAWkHjAgAAraBxAQCAVtC4AABAK0gVmmQ2HSUtgNumTRtlvaysTFk3SjrdunXL1JikbfXt21dZX7Rokbit9957T1lfvny5qTFJi7mafW73GlWCUFow12hBW6vSntIiyhKjMUnpSKkukeas2QWzieSEIhbfbV444wIAAK2gcQEAgFbQuAAAQCtoXAAAoBU0LgAA0ApShSZZlSqS1gX0Jp0UERGhrCcnJyvrzz77rLI+YMAAZT0sLEzcd8eOHZX1NWvWKOulpaXKOtKD1rFqfUEiOaEYEKD+0SGlYluSlB6UICHo+3DGBQAAWkHjAgAAraBxAQCAVtC4AABAK2hcAACgFaQKFYzWKpPSVFISSbpiqtkrqXbp0kW87Wc/+5myPmvWLGVdSvZJ6cF9+/aJ+5bSgNI+wsPDlfUbN26I+wCZw+Goc7xameyz6srI0hqG0nwy2ofZBKo018ymDcF34J0DAACtoHEBAIBW0LgAAEAraFwAAKAVNC4AANAKUoUKRlcbNpsGlEhrHkrrDmZkZIjbmjt3rql9X7t2TVnPzs5W1levXi1uKzEx0dS+XS6Xsi4lOY3eCyC6efNmg+8rXWWaSE7YSVdGltKAaWlpyvqUKVOUdelYJCL64osvlPX//ve/yvrJkyeV9YKCAmXdm/UxcQVk34AzLgAA0AoaFwAAaAWNCwAAtILGBQAAWkHjAgAArfixhbGt4uJicS06nTgcDvE2KcUlpbKkFGJCQoKy/sorryjrkydPFsckpaPee+89Zf03v/mNsi6tL2iUpJTSgFKCTdrW7du3xX00tRs3bhhe5dkXGc01b9bgM/tjQLr/uXPnlHVprU2jVKQ0D80kKYmIzpw5o6xLCVejY1FKU0rrKjqdTmVdmrORkZHivo8ePaqsv/baa8r62bNnlXUrr5BtllVzDWdcAACgFTQuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtYJFdBbNxWyL50uShoaHK+rvvvqusDx482PS+pRj7tGnTlHUpDitF2Nu2bSvuu6ioSFmXFh2V6tI+vvvuO3HfcCeeXfsrCVLUWloglsi6xaP/9Kc/KevS/Lh06ZK4rc6dOyvrvXv3VtYHDRqkrHft2lVZl+ZBc3yl5+LFi8q60ULI0ldiCgsLlfUXX3xRWTf71R1fhDMuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtIFWoYJSik1JuUhLx5ZdfVtaTk5NNjek///mPeNuoUaOUdWmBz2+//VZZl9JoJSUl4r6l1KTRY1SQHvSOv7+/uNBxbdICsURy2tPspepXrVqlrEuL8nqzxnebNm2U9YiICGW9Xbt2ynpBQYGyHhcXJ+5bWvhXWphXes2PHz+urBulLCVSQlEak1VztiXhjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtIFSoYJdykpF58fLyy/vzzz5vah5SYGjdunDgmqxJ50npyFRUV4mPMXgY8NjZWWf/mm29MbQfuqKysbHCq0JsEn7R+ndm0oXR/o/XxpPGWlZUp69JxWlxcrKy7XC5lXVr7k0h+HtJYpX1MmDBBWZfmIBHRwYMHlfW3335bWZd+luiUHpTgjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtIFSoYpbSklFBaWpqyLq1hKK2HOHz4cGX92rVr4pikJJKU8AoLC1PWpasZS2vAERFdv35dWZeusor0oPVqJ9qCg4OV9ysvL2/0tqtJx5ZEOkalhCCR+echJf6kOSsxShWaFRISoqwvXbpUWTe6AvJbb72lrEvjldYk9CY97GtwxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBWkChWM1nR7+OGHlXXpSsfStqSrkw4aNMh4cApDhgxR1s+dO6esS+sL7t69W1mXkoNEcgJTSkdJySUpuWi0b1Br6NqFDXmM3W5X1s0m0IzSgxJp31Kq0Oy6mVK6zuhK0dJVwqV9Z2dnK+vR0dHKunR1ciKivXv3KutS+rI1rEkowRkXAABoBY0LAAC0gsYFAABaQeMCAACtoHEBAIBWkCpUkK4cSiSnr6Q0kJSMkq6kPHfuXGXdKOk0depUZf0f//iHqX1LzyE9PV3c9549e5R1KXUmvbZID1pHWh/TiJR+NZtQlI53s4k/IvNJRGnf0rqK0jEqJQeJ5Cs2JycnK+svvPCCsi6lisePHy/u+9KlS6a21ZqvNo4zLgAA0AoaFwAAaAWNCwAAtILGBQAAWkHjAgAAraBxAQCAVhCHVzCK4WZkZCjrUVFRyroUM961a5eyHh4erqx/8cUX4pikRTYlBw8eVNalSG9OTo64rWHDhinrFy5cUNal11aKXRsteAx3Lldf+7WT4tHSZeSJ5Mu/SwvaSqT3S/o6hxQvJ5Kfh0SK3EvPW3rONptN3If0/B5//HFlXXp+R44cUdaluUkkf5VEep2k2LvD4VDWvfkaRUvBGRcAAGgFjQsAALSCxgUAAFpB4wIAAK2gcQEAgFaQKlSQkn1ERFlZWcr61atXlfUZM2Yo62vWrFHWpcSPUbrL31/9+8e6deuUdZfLpayvXbtWWf/pT38q7tts6k9KQJpNr8Ed0gKyKlKKzkpGC9SqGCX4JFJKUDqGzD5vo+cgJftGjRqlrEtzMzs729SYiMwnciVm3yNfhDMuAADQChoXAABoBY0LAAC0gsYFAABaQeMCAACtIFWo0L17d/G2Dh06KOvSWoIbN240tW9pvTCj9QilNFVxcbGpfZ86dUpZN0quSQlM6Xlcu3ZNWQ8KClLWpcurQ+tgNhFHJKcEw8LClHVpHkjH7o0bN8R9v/TSS8p6nz59lPXc3FxlXZprRswmMKV5681r7mtwxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBWkChWOHz8u3pafn6+sSwklKfHndDqVdSkxZXSlWLOk9RCnTZumrJeUlJjeh5QebA1XXwXrGF3l2GxKUKpLc0065vr27SuO6YUXXlDWpXn+9ttvK+vfffeduA+JlAaU1kOUUoVW/ixpKTjjAgAAraBxAQCAVtC4AABAK2hcAACgFTQuAADQClKFCkbrAkprjBklkVSkqxBLKisrTd3fSLdu3ZR1KcVlNFaz6SgpyRUYGKist4artYLMaP09s2tttmvXTlmXrk4eHR2trO/atUvch5TIW7FihbJ+8OBBcVtmSQlM6TWUUohmrprtq3DGBQAAWkHjAgAAraBxAQCAVtC4AABAK2hcAACgFTQuAADQCuLwCkYR7/379yvrEydOVNazsrKU9Q0bNijrZWVlhmNT6dq1q7IeGRmprC9YsEBZj4iIUNb/+te/ivuWXqugoCBlvaKiQllHHP7eFBAg/wiSYtvScSrF3iU7duxQ1qVFeYmIzp8/r6wvW7ZMWZe+/mF2fhDJr5UUk7fb7cq6NKeYWdy3r8EZFwAAaAWNCwAAtILGBQAAWkHjAgAAraBxAQCAVpAqNElagPPTTz9V1t944w1lfcCAAcr6mTNnlHVpAVEiomeffVZZlxJKUVFRyvpnn32mrP/9738X911SUiLepiJdZtybNCXoz5sk2/Xr15X18PBwZX3gwIHKelJSkrIuJVyJiH71q18p69L8lxilByVSElFKFUoLc0uL8uq0+C7OuAAAQCtoXAAAoBU0LgAA0AoaFwAAaAWNCwAAtIJUoUJwcLB42+eff66sz5s3T1lftWqVsj59+nRTY5KSQ0RySqi0tFRZnzFjhrK+ZMkSZd1oPTmJ2bXYWkPSCcyTkm9ERCEhIcq6dFy3bdtWWZ8/f76yLh3Xs2fPFsckpYq9mSNmGf0MMMPtdluynZaEMy4AANAKGhcAAGgFjQsAALSCxgUAAFpB4wIAAK0gVahQXl4u3hYaGqqs5+bmKuvS+n8JCQnK+vjx45X1Hj16iGOSkot/+9vflHWzCT6jNJN0NVppPTmkB+Fu0tqVRHJ6UJqDTz75pLLep08fU2PasmWLqfsTEfn5+Zm6v3SVZZfLJT5GSuSaTV/qdKVjCc64AABAK2hcAACgFTQuAADQChoXAABoBY0LAAC04scWRkyKi4vFq5AC+KobN25QWFhYSw/DlNYy1+x2u3ibtI7h6NGjlfXVq1cr61KCT0q4Dho0SBzTyZMnlXVvrmh8L7JqruGMCwAAtILGBQAAWkHjAgAAraBxAQCAVtC4AABAK1irEABajDdXQJbW7ZRSloGBgcr6l19+Wc/o6kJ60DfgjAsAALSCxgUAAFpB4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBXE4QHAJ0mXnnc4HMq6FHs/fvy4sj5hwgRl/fz58+KYYmJilPXCwkLxMWA9nHEBAIBW0LgAAEAraFwAAKAVNC4AANAKGhcAAGjFj5nZqo21lsuJw73FqsuJN6fWMtfatGkj3lZWVtaMI/m/4OBg8bby8vJmHEnrY9VcwxkXAABoBY0LAAC0gsYFAABaQeMCAACtWLrkk4U5D4Bmo+Nxq+OYVXzxefjimFoLq15bSxuXy+WycnMAzcLlcmmX0Gstc+3mzZstPYQ6KioqWnoIrZZVc83SOLzb7aaCggJyOp3k5+dn1WYBmgQzk8vlog4dOpC/v16fmmOugU6snmuWNi4AAICmptevmQAAcM9D4wIAAK2gcQEAgFbQuAAAQCtoXAAAoBU0LgAA0AoaFwAAaOV/22GpFpMR1i4AAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response_json = prediction_json_response_records.json()\n",
    "\n",
    "i = 0\n",
    "for x in response_json['result']:\n",
    "    output_list = x[output_tensor_name]\n",
    "    output_tensor_array = np.asarray(output_list, dtype=np.dtype('float32'))\n",
    "    \n",
    "    # Print the result for the firest testing image\n",
    "    print('Raw predictions of \"{}\": {}'.format(images[0][0], output_tensor_array))\n",
    "    print('Final recognized digit: {}'.format(postprocess(output_tensor_array)))\n",
    "    print('-'*120)\n",
    "    \n",
    "    # Add a subplot for the current digit.\n",
    "    plt.subplot(1, len(images) - 1, i + 1)\n",
    "    plt.tight_layout()\n",
    "    plt.imshow(images[i][1].reshape([28, 28]), cmap='gray', interpolation='none')\n",
    "    plt.title('Detecting image: {}'.format(images[i][0]))\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    i += 1\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <a id=\"parse-json-response-split\"></a>Parse the json response message from the `split` request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw predictions of \"8.jpg\": [[[  397.46228 -2270.2764   1143.2241    850.3334   -468.74387\n",
      "    -592.69666   327.9959  -4857.1685   4933.365    -390.23157]]]\n",
      "Final recognized digit: 8\n",
      "------------------------------------------------------------------------------------------------------------------------\n",
      "Raw predictions of \"9.jpg\": [[[-2689.0283  -3374.9338  -1715.207    -871.94696  3004.0815\n",
      "    -926.28235 -3160.6943   1612.2079    463.7397   5574.235  ]]]\n",
      "Final recognized digit: 9\n",
      "------------------------------------------------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAa4AAADUCAYAAADEO8eJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAAehUlEQVR4nO3deXBUVfYH8JM06SR0QpJOzALEhE0CyCarbAlgRAcQCiSDsiQIKGK5jFJTOJYoMxbIUDKIUpBhCbKpZFgGGEDRkcUizBBAFAhqxmGbEAIEskECdJ/fH1b6R8g9D17TSfqG76eKP/Lt7vdud7+b0685uc+HmZkAAAA04VvXAwAAADADhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0AoKFwAAaMVU4VqxYgX5+Pi4/gUEBFB0dDT179+fZs+eTQUFBW4P5Pjx4/Tuu+/SyZMn3d7Gve4nLS2N4uPja3T/krrctzdYv3499e7dm+x2O4WGhlL37t1p1apVd/XYpKQkSkpKqtkB1jLMtZpzv8+1NWvWUOfOnSkgIIAiIiLo2WefpTNnztzVY71mrrEJGRkZTESckZHBWVlZvGfPHv7b3/7Gr732GoeEhLDdbuedO3ea2aRLZmYmExF/8803bj3eE/vJzc3lQ4cO1ej+JXW577q2bNkyJiIeOXIkb9u2jbdv386jR49mIuJ58+bd8fHHjh3jY8eO1cJIaw/mWs25n+faggULmIh40qRJvGPHDl66dCnHxMRwXFwcFxYW3vHx3jLX3CpcBw4cqHbbqVOnODY2loODgzk/P9/0QLxhMkHd6N27N8fFxbHD4XBlTqeTExISuEOHDnU4srqDuQaeVl5eziEhITx06NAq+b59+5iI+A9/+EMdjcw8jxUuZuZ169YxEfHMmTOr5AcOHOChQ4dyWFgY+/v7c6dOnfjzzz+vtt3b/2VkZLjus3PnTh4wYAAHBwdzYGAg9+rVi7/66qtqY8jJyeHRo0dzZGQkW61Wjo2N5XHjxnF5efkd95OamspxcXFVXyAifumll3jlypWckJDAgYGB3KFDB96yZUu1fW/atInbt2/PVquVmzVrxvPnz+d33nmH7+bE1mjfy5cv54ceeogDAgK4S5cunJWVxU6nk//85z9zfHw822w27t+/P//8889VHv/ll1/yU089xU2aNGF/f39u0aIFP//883zhwgW3x+50OnnhwoXcsWNHDggI4NDQUB45ciT/5z//ueNzlCQlJXG7du2q5d27d+fu3bvf8fGJiYmcmJjo+vm///0vExHPmTOH33vvPY6NjWV/f3/u0qWL8pi5l/etpmCuYa55eq5lZ2czEfGiRYuq3Wa327l169Z33Ia3zDWPFq7S0lK2WCw8cOBAV/bPf/6TrVYr9+3blz///HPesWMHp6WlVTmICwoKeNasWUxEvHDhQs7KyuKsrCwuKChgZuZVq1axj48PDx8+nDds2MBbtmzhIUOGsMViqfLifPfddxwUFMTx8fG8ePFi/vrrr3n16tWckpLCxcXFd9yPdEDHx8dz9+7ded26dbxt2zZOSkriBg0aVDmAtm/fzr6+vpyUlMQbN27kzMxM7tGjB8fHx9/TZIqLi+NevXrxhg0beOPGjfzQQw+x3W7n3/3udzxs2DDeunUrr1mzhqOiorhDhw7sdDpdj1+0aBHPnj2bN2/ezLt37+ZPPvmEO3bsyK1bt+br16+7NfbJkyezn58fv/HGG7xjxw5eu3YtJyQkcFRUVJVP/9988w0TEb/zzjt3fO7r169nX19ffu+997igoIAvXLjAc+fOZYvFwuvWrbvj46XJFBsby3369OH169dzZmYmd+vWjf38/Hjfvn1uPffahLmGuebpuVZ5ZrV8+fJqt8XExLCvry9fu3bNcBveMtc8WriYmaOiorhNmzaunxMSErhz585848aNKvcbMmQIx8TEuL4ekr5WKCsrY7vdXu301uFwcMeOHat8Ih8wYACHhoa6JoeK0dcX0gEdFRXFxcXFriw/P599fX159uzZrqxbt24cGxvLFRUVrqykpITDw8PvaTJFR0dzaWmpK9u0aRMTEXfq1KnKxJk/fz4TEX///ffK7TudTr5x4wafOnWKiYj//ve/mx57VlYWExF/8MEHVbZ95swZDgwM5N///veubNeuXWyxWKqdEUg2bdrEISEhrk/mgYGBvHr16rt6rDSZGjduXGUiFhcXs91u58cee8z0c69tmGu/wlzz3Fy7dOkS+/r68sSJE6vkubm5rnmXl5dnuA1vmWseb4fnWy7vlZubSydOnKAxY8YQEdHNmzdd/37zm9/QuXPn6McffzTc3r59+6iwsJBSU1OrPN7pdNITTzxBBw4coLKyMrp69Srt3r2bUlJS6IEHHvDoc+rfvz8FBwe7fo6KiqLIyEg6deoUERGVlZVRdnY2DR8+nKxWq+t+QUFBNHTo0Hvet81mc/3cpk0bIiJ68sknycfHp1peOSYiooKCApoyZQrFxsZSgwYNyM/Pj+Li4oiIKCcnx/TYt27dSj4+PjR27Ngq70V0dDR17NiRdu3a5bpvYmIi3bx5k2bMmHHH57hjxw4aO3YsjRgxgrZv3047d+6kSZMmUVpaGmVkZNztS1XNiBEjKCAgwPVzcHAwDR06lPbs2UMOh6NG37fagLmGuUZ093PNbrfTmDFjaOXKlZSenk6FhYX0/fff05gxY8hisRARka+veyWhtudaA7dGKSgrK6NLly5R+/btiYjo/PnzREQ0bdo0mjZtmvIxFy9eNNxm5Taefvpp8T6FhYXk6+tLDoeDmjZt6s7QDYWHh1fL/P396dq1a0REdPnyZWJmioqKqnY/VWaG3W6v8nPlmy7l5eXlRETkdDrp8ccfp7y8PHr77bepffv2ZLPZyOl0Us+ePd0a+/nz58X7EhE1b97c9PNjZnruueeoX79+tHz5clf+2GOPUVFREb388suUkpJS5RfK3YqOjlZm169fp9LSUiopKamx962mYa5hrrlj0aJFxMw0depUmjJlCvn6+tK4ceMoKiqKvvjiC+Xrfzdqe655tHD94x//IIfD4erzj4iIICKiN998k0aMGKF8TOvWrQ23WbmNjz76iHr27Km8T1RUFDkcDrJYLHT27Fk3R+++sLAw8vHxcU38W+Xn59f6eIiIjh49SkeOHKEVK1ZQamqqK8/Nza1yPzNjj4iIIB8fH9q7dy/5+/tXu78qu5Pz58/TuXPn6IUXXqh2W7du3WjlypV08uRJateuneltq177/Px8slqtFBQURA0aNPC69+1uYa55z3umy1wjIrLZbLRq1SpasGABnTlzhho3bkwRERGUkJBAvXr1ogYN3CsJtT3XPPZV4enTp2natGkUEhLi+iXUunVratWqFR05coS6du2q/Ff5tUDlG1H56aRS7969KTQ0lI4fPy5uw2q1UmBgICUmJlJmZqbhJ0tpP/fCZrNR165dadOmTXT9+nVXXlpaSlu3bvXYfsyo/Grj9gM8PT29ys9mxj5kyBBiZvrf//6nfB8qP/2bERYWRgEBAbR///5qt2VlZZGvry/FxMSY3i4R0YYNG1yfiomISkpKaMuWLdS3b1+yWCxe+b7dDcw173rPdJlrtwoLC6MOHTpQREQEbd68mX788Ud69dVX3d5ebc81t8rr0aNHXd+5FhQU0N69eykjI4MsFgtt3Lixyvfe6enp9OSTT9KgQYMoLS2NmjRpQoWFhZSTk0OHDh2izMxMIiJ6+OGHiYjor3/9KwUHB1NAQAA1a9aMwsPD6aOPPqLU1FQqLCykp59+miIjI+nChQt05MgRunDhAi1atIiIiObNm0d9+vShHj160PTp06lly5Z0/vx52rx5M6Wnp1NwcLDhfu7FH//4Rxo8eDANGjSIXn31VXI4HDR37lwKCgqiwsLCe9q2OxISEqhFixY0ffp0Ymay2+20ZcsW2rlzp9tj7927Nz3//PM0YcIEys7Opn79+pHNZqNz587Rt99+S+3bt6cXX3yRiIh2795NAwcOpBkzZhh+9+7v709Tp06lefPm0fjx4+m3v/0tWSwW2rRpE61du5YmTpxY5auali1bElH1T7MqFouFkpOT6fXXXyen00lz5syh4uJimjlzpunnXlcw16rztvdMl7lG9OsKNXl5edSmTRsqLy+nXbt20YcffkhTpkyhYcOGVbmvV881M50ct/9thtVq5cjISE5MTORZs2aJHUZHjhzhlJQUjoyMZD8/P46OjuYBAwbw4sWLq9xv/vz53KxZM7ZYLNX+tmT37t08ePBgttvt7Ofnx02aNOHBgwdzZmZmlW0cP36cR40axeHh4Wy1WvnBBx/ktLQ0Li8vv+N+jP6+43ZxcXGcmppaJdu4caPrbxQefPBBfv/99/mVV17hsLCwO7yyd7/vyi6euXPnVskrW2JvfT2OHz/OycnJHBwczGFhYTxq1Cg+ffq0snXWzNiXL1/OPXr0YJvNxoGBgdyiRQseP348Z2dnVxvP3bTDOxwOXrJkCXft2pVDQ0O5UaNG3LlzZ/7444+rtBIz//q63/46JSYmclJSUrXXaM6cOTxz5kxu2rQpW61W7ty5M3/xxRfV9n8v71tNwVz7f5hrnptrGzdu5E6dOrm217VrV162bFmVrslK3jzX6q7f9z5w/fp1btu2LScnJ9f1UEzTaeydOnXikSNHun6WfuHcLZ2eO/xK5/dMp7F7y1zzaHPG/W7ixImUnJxMMTExlJ+fT4sXL6acnBz68MMP63pod6Tj2H/66Sfau3cv/fDDDzR27Fi3t6Pjc7/f6fye6Th2r5trbpVJUBo1ahQ3adKErVYr22w27tu3L2/fvr2uh3VXdBx7Wloah4eH84QJE/jq1auu3OynQB2f+/1O5/dMx7F721zzYb7lrxgBAAC8HC4kCQAAWkHhAgAAraBwAQCAVjzaVeh0OikvL4+Cg4OrLEoJ4I2YmUpKSqhx48ZuLy5aVzDXQCeenmseLVx5eXkUGxvryU0C1LgzZ87UyIKxNQlzDXTkqbnm0cJ16+UIdHbrsvu3u3WdrXshLWZ58+ZN09vy8/NT5lLDqNl9BAYGirfduj7ZrYKCgpR5SUmJqX3XBh2PWx3HrGL06dvpdHpkW9I8cKeh+tZLd9xKmgdmF8OtqKgwPSadeOq49Wjhqi9fWdTG85D2IeVGk6ymx2u0fbPPwxvpNNZKOo5ZxZPPw+y23ClcZvdRX94nT/HU66HXF/sAAHDfQ+ECAACtYK1CBaPvmaXv0aXvbqX/07lx44apMXny/92k5xAWFqbML126ZGr7RETFxcXKXPrOX/q/g6KiItP7Bn04HA7Tj6m8zPztpP83lo536f96jeamdG0x6biWfpd48mtK6XlL/0fozmvubXDGBQAAWkHhAgAAraBwAQCAVlC4AABAKyhcAACgFXQVKhj9kZzUqWO2+01a7cKddbyklS2kziXpr/zd6R40200pdXKhe/D+ZNQtK8016Rgy2y0nzXOpa9FoH1IuzUFpH9LvBSJ53prtUK4PcMYFAABaQeECAACtoHABAIBWULgAAEArKFwAAKAVFC4AANAK2uEVjBbADA8PV+ZmW8mlRWWlFuCysjJT2zfSsGFDZX716lVlLi3iSSS3vUst+vdj6y7I3Lkwq9k2drN/YmJ0AUtpH2YvzurOdffMbkuag9JYPXWR3NqAMy4AANAKChcAAGgFhQsAALSCwgUAAFpB4QIAAK2gq1DBqAvJU92DpaWlylzqKpIuDU5E1LVrV2X+xBNPKPPk5GRlfuXKFWW+d+9ecd/p6enK/OLFi+JjACpJ84PIc11utdEtJy2OK81nqbPPbHei0T6kLuH6AGdcAACgFRQuAADQCgoXAABoBYULAAC0gsIFAABaQVehQqNGjcTbpM47ibS2mdQJ1KdPH2X+5ptvivuQugojIiKUudm12wYMGCDe9vDDDyvzqVOnKvPLly+b2jfUb9Ll6D1JOt6lOWi1WsVtSV2QRUVFprYVFBSkzI26CqXfJdL6n1IujamiokLct7fBGRcAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZ82J1LbgqKi4spJCTEU5vzSlJXkdS5JK0XlpiYqMyXLFmizFu1aiWO6bvvvlPmK1euVOaHDx9W5p07d1bmH3zwgbhv6eqrgwYNUuY7d+5U5nXZ6VRUVGTYSeqN7oe5Jh0TDofDVC6R5rLRVbqlfUhXG5aupuzOcS3t49q1a6a2I3Unmn393OGpuYYzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAArWCtQgWjtfykjiOpI6dXr17K/JNPPlHmcXFxynzmzJnimObOnavMpSu/ml3bzIj0mLCwMGUuNbHqtE4a1A7pWDHb/Sat//naa68p88GDB4vbMrpis8qnn36qzGfNmqXM8/LyxG1J64JKj8nNzVXmddlV6Ck44wIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0Aq6ChWk9cWMSGv2/eUvf1Hm4eHhyvztt99W5vPmzRP3La2HKGndurUyX7p0qTI36vg7ceKEMt+8ebMylzo2GzRQH4pSZyTUf1LHqjR32rVrp8zXrFmjzKOjo5W5dCwSEZWWlipzaR3BZ555RplLx3VMTIy474EDByrzyZMnK3Opq9CDy9PWGZxxAQCAVlC4AABAKyhcAACgFRQuAADQCgoXAABoBYULAAC0gnZ4k6S2V0lBQYEy7969uzLPyclR5kYt7zabTZlPnDhRmc+fP1+Z37x5U5lfuXJF3Pfw4cOVuXQ5cbS9w62M5pN0DLVo0UKZf/XVV8q8vLxcmefn5ytzqYWdiCg0NFSZX7x4UZk/+uijyvzy5cvKXFowm0j+k5uff/5ZfIwK2uEBAABqGQoXAABoBYULAAC0gsIFAABaQeECAACtoKvQJKnTyezisZLCwkJl3qRJE/ExY8eOVebTpk1T5lJ30uHDh5X5hAkTxH2fPXtWmVutVmUudQ82bNhQmZtdQBj0InWyEhG1bdtWmW/dulWZ+/n5KfPs7Gxl/vLLLyvzgwcPimOSREREKHNpodtly5aZ2g6R3A387bffKnPpd5LRa64LnHEBAIBWULgAAEArKFwAAKAVFC4AANAKChcAAGgFXYUKUocbkdypI13SW1qTrKSkRJmnp6cr86CgIHFMdrtdmVssFvExKtKahC+99JL4mBkzZpjalsTpdJq6P9QPN27cEG976623lLk0F7Zv367Mp0yZosxPnz6tzKW1P4nkrmJprcJevXop86eeekrchyQjI8PU/QMCApR5fejUxRkXAABoBYULAAC0gsIFAABaQeECAACtoHABAIBW0FWo4Mmum0uXLilzf39/Zd6qVSvT+5DGu3fvXmUeHx+vzI8dO6bMp0+fLu5b6sB0OBzKXOp0kq5SC/XbunXrxNtGjRqlzKX1Md99911lLnUPSozWFzXb/Tpr1ixT95fWYSQi+umnn5S51D0s/V6Qnp9OaxjijAsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCvoKlSQrqRKZLy2msq2bduU+bhx45S51LVk1HUndQlJnYvJycnK/OTJk+I+JNIVjaXXUHoeZq+YDPVDSEiIeJv03kvdbzk5Ocpc6rpjZmVeVFQkjklaF3TmzJnKvG/fvuK2VN5//33xNk913urUPSjBGRcAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFbQVaggrbNnxMfHR5lnZWUp83//+9/KXOr4k9b4I5K7rwYPHqzMpasvS6Kjo8XbpLUYpY4trFUIt4qIiBBvkzpNpSsUSx25Zudz27ZtxdueeeYZZf7CCy8o87y8PGUudVMePnxY3Ld05WfpqsxSt3FFRYW4D13gjAsAALSCwgUAAFpB4QIAAK2gcAEAgFZQuAAAQCsoXAAAoBW0w3vIW2+9pcxfeeUVZS4t1vnDDz8o83Pnzon7fvzxx5X5kSNHlHn//v2VudS6m5+fL+7bLF9ffFaC/yddjp6I6JFHHlHmgYGByvzrr79W5lLrudQWHhwcLI6pefPmylz684+mTZsq8/T0dGUutbYb7cMsadFhd/4MqK7gtwgAAGgFhQsAALSCwgUAAFpB4QIAAK2gcAEAgFZqrKvw9kVn3emIkRaVLC0tdWtMt5MuL29k0qRJyvzFF19U5g888IAylxbZTU1NVebSgplE8kK+UkfTjBkzlPnEiRPFfUgiIyOVeUFBgTKXFgSWFin2VCcVeKfnnntOvE06tvr27avMu3XrpsylS9VLi/hevHhRHFNKSooyHz16tDIfMWKEMl+6dKkyN+q6lRYRluaI1D0ozUGd4IwLAAC0gsIFAABaQeECAACtoHABAIBWULgAAEArPuzBtq3i4mJxXTCpg+/GjRue2j01btxYmUsdblKXzsKFC8V9PPvss8q8UaNGylxaq3DJkiWmxiR1RhERDRo0SJlv2bJFmUvrofXo0UOZ5+TkiPs2e/hInVzuPG9PKSoqEt8/b2U01+q7Pn36KPOePXsqc+kYOnHihDLfs2ePuO8//elPynz8+PHK/MqVK8pcWi/07Nmz4r7NrjHojWsSemqu4YwLAAC0gsIFAABaQeECAACtoHABAIBWULgAAEArtXYFZKl70N/f3/S2pCuXSlfqlTrWOnTooMzHjh0r7ltaP/H1119X5tKVTj25Xtj+/fuVudQ9KD0H6arMRuskSu+r9BhpTcLa6B4E7xMVFSXedv78eWUuXSVcmgfSsRUeHq7Mr169Ko5J6hKWjve1a9cq80uXLon7kEj7kDp7pblWH+CMCwAAtILCBQAAWkHhAgAAraBwAQCAVlC4AABAKzXSVWiz2ap1tEhXLTbqrjPbFSN1D0ZERChzaU1Cm80m7kNat+/jjz9W5lLXndRNJXVSBQYGimOStiV1D/7yyy/KPC8vT5m7s56ktB4augfhVtLxbqSoqMgj+5Y6+xISEsTHSJ190tz59NNPlbk7XcXSPJR+70lqY93YmoYzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAAraBwAQCAVmqkHf769et33couXV6ayHzrdGhoqDKX2lsfffRRZW409jfeeEOZBwQEKHNfX/VnA7Nt75MnTxbHNGPGDPE2lRUrVpgakxGpPdjseyf9CUJZWZnpMUH9IM2p8vJyZS4dQ9IitNJiulOmTBHHJG3ryy+/VOb/+te/TG3HiNm2d+n+ZrfjjXDGBQAAWkHhAgAAraBwAQCAVlC4AABAKyhcAACglRrpKvTUYo1Sl5DUDVRcXKzMpS4ady5tLXW5SQtX9uvXT5mnpKQo8/bt2yvz5s2bi2MKDg5W5gsWLFDmS5cuVeZSt5YnSa+5Tgt8gudIXbdGpLlmtgP1kUceUebjx48XHyMdvwcPHlTm7nQPmiWNSXptpQWwdYIzLgAA0AoKFwAAaAWFCwAAtILCBQAAWkHhAgAArdRIV2FAQEC1Tpdr164p72u0pp3UFSN16ki51FUoXW5b6mYkItq9e7cyr6ioUObSc5A6o6TOvqNHj4pjWr16tTL/7LPPlLnZNQmNOr+k11Zag1LqaHLnUuagP6O1SqW5IB2P0tqGVqtVmUsdvFKXLpHc0Zydna3MpbVHpd+HRsxuqzY6GusKzrgAAEArKFwAAKAVFC4AANAKChcAAGgFhQsAALRSI12Fqm6ghg0bKu8rdekQeW79uv379yvzYcOGKXOpc5CIqKSkRJlLnUi5ubnKfM6cOcpc6h48fPiwOCapo1FitjvJaE1HqUvQ7DqQUqdYfbhaK8iM5rjUeSs9RupClHK73a7Mpat6ExGdOHFCmR84cECZS89BYjRvzK7r6KmOX2+EMy4AANAKChcAAGgFhQsAALSCwgUAAFpB4QIAAK3USFehilH3oKSm19o6dOiQMjdaq6w+MLtOmjvdRmbfO3QP3p+M1gU1u36lNG+lTuAxY8aY2j4R0bJly5S5NEfMzh2jeWP2Cs8SnboHJTjjAgAAraBwAQCAVlC4AABAKyhcAACgFRQuAADQSq11FQIA3M6oU85ozUAVqXtQWv/v5MmTyrxLly7iPvz9/U2NCWoGzrgAAEArKFwAAKAVFC4AANAKChcAAGgFhQsAALSCwgUAAFpBOzwAeKWgoCBlLi0SXVFRocwbNmyozD/77DNlbtTyfvDgQfE2qD044wIAAK2gcAEAgFZQuAAAQCsoXAAAoBUULgAA0IoPm73GuoHi4mIKCQnx1OYAakVRURE1atSorodhSn2Za9ICuETGl7GvSS1bthRvy83NrcWR1D+emms44wIAAK2gcAEAgFZQuAAAQCsoXAAAoBWPLvlUV/+ZCnAvdDxudRyzijc+D6fTWddDqLc89X57tHBJl84G8GYlJSXadehhrtWcX375pa6HUG95aq55tB3e6XRSXl4eBQcHG7a5AngDZqaSkhJq3Lgx+frq9a055hroxNNzzaOFCwAAoKbp9TETAADueyhcAACgFRQuAADQCgoXAABoBYULAAC0gsIFAABaQeECAACt/B8dGPqNlg7cwQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "response_json = prediction_json_response_split.json()\n",
    "\n",
    "i = 2\n",
    "for x in response_json['result']['data']:\n",
    "    output_list = x\n",
    "    output_tensor_array = np.asarray(output_list, dtype=np.dtype('float32'))\n",
    "\n",
    "    # Print the prediction for the testing image\n",
    "    print('Raw predictions of \"{}\": {}'.format(images[i][0], output_tensor_array))\n",
    "    print('Final recognized digit: {}'.format(postprocess(output_tensor_array)))\n",
    "    print('-'*120)\n",
    "\n",
    "    # Add a subplot for the current digit.\n",
    "    plt.subplot(1, len(images) - 1, i - 1)\n",
    "    plt.tight_layout()\n",
    "    plt.imshow(images[i][1].reshape([28, 28]), cmap='gray', interpolation='none')\n",
    "    plt.title('Detecting image: {}'.format(images[i][0]))\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    i += 1\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a id=\"next-steps\"></a>Next steps\n",
    "\n",
    "I hope the tutoiral can help you to learn how to use the AI-Serving. If you have any questions, please open issues on this repository. Feedback and contributions to the project, no matter what kind, are always very welcome.\n",
    "\n",
    "**Star the [AI-Serving](https://github.com/autodeployai/ai-serving) project if it's helpful for you!!!**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}