notebooks/official/automl/sdk_automl_video_classification_batch.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "copyright"
      },
      "outputs": [],
      "source": [
        "# Copyright 2021 Google LLC\n",
        "#\n",
        "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "#     https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "title"
      },
      "source": [
        "# Vertex AI SDK: AutoML training video classification model for batch prediction\n",
        "\n",
        "<table align=\"left\">\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://colab.research.google.com/github/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_classification_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/colab-logo-32px.png\" alt=\"Colab logo\"><br> Open in Colab\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://console.cloud.google.com/vertex-ai/colab/import/https:%2F%2Fraw.githubusercontent.com%2FGoogleCloudPlatform%2Fvertex-ai-samples%2Fmain%2Fnotebooks%2Fofficial%2Fautoml%2Fsdk_automl_video_classification_batch.ipynb\"\">\n",
        "      <img width=\"32px\" src=\"https://cloud.google.com/ml-engine/images/colab-enterprise-logo-32px.png\" alt=\"Google Cloud Colab Enterprise logo\"><br> Open in Colab Enterprise\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "    <a href=\"https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/automl/sdk_automl_video_classification_batch.ipynb\">\n",
        "      <img src=\"https://cloud.google.com/ml-engine/images/github-logo-32px.png\" alt=\"GitHub logo\"><br>\n",
        "      View on GitHub\n",
        "    </a>\n",
        "  </td>\n",
        "  <td style=\"text-align: center\">\n",
        "<a href=\"https://console.cloud.google.com/vertex-ai/workbench/deploy-notebook?download_url=https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/main/notebooks/official/automl/sdk_automl_video_classification_batch.ipynb\" target='_blank'>            \n",
        "    <img src=\"https://lh3.googleusercontent.com/UiNooY4LUgW_oTvpsNhPpQzsstV5W8F7rYgxgGBD85cWJoLmrOzhVs_ksK_vgx40SHs7jCqkTkCk=e14-rj-sc0xffffff-h130-w32\" alt=\"Vertex AI logo\"><br> \n",
        "Open in Vertex AI Workbench \n",
        "    </a>\n",
        "  </td>\n",
        "</table>\n",
        "<br/><br/><br/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "overview:automl"
      },
      "source": [
        "## Overview\n",
        "\n",
        "\n",
        "This tutorial demonstrates how to use the Vertex AI SDK to create video classification models and do batch prediction using a Vertex AI [AutoML](https://cloud.google.com/vertex-ai/docs/start/automl-users) model.\n",
        "\n",
        "Learn more about [Classification for video data](https://cloud.google.com/vertex-ai/docs/training-overview#classification_for_videos)."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "objective:automl,training,batch_prediction"
      },
      "source": [
        "### Objective\n",
        "\n",
        "In this tutorial, you learn how to create an AutoML video classification model from a Python script, and then do a batch prediction using the Vertex AI SDK.\n",
        "Alternatively, you can create and deploy models using the `gcloud` command-line tool or online using the Google Cloud Console.\n",
        "\n",
        "This tutorial uses the following Google Cloud ML services and resources:\n",
        "\n",
        "- Vertex AI Training\n",
        "- Vertex AI models\n",
        "- Vertex AI batch prediction\n",
        "\n",
        "The steps performed include:\n",
        "\n",
        "- Create a Vertex dataset resource.\n",
        "- Train the model.\n",
        "- View the model evaluation.\n",
        "- Make a batch prediction.\n",
        "\n",
        "There is one key difference between using batch prediction and using online prediction:\n",
        "\n",
        "**Prediction service**: Does an on-demand prediction for the entire set of instances (that is, one or more data items) and returns the results in real-time.\n",
        "\n",
        "**Batch prediction service**: Does a queued (batch) prediction for the entire set of instances in the background and stores the results in a Cloud Storage bucket when ready."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dataset:hmdb,vcn"
      },
      "source": [
        "### Dataset\n",
        "\n",
        "The dataset used for this tutorial is the golf swing recognition portion of the [Human Motion dataset from MIT](http://cbcl.mit.edu/publications/ps/Kuehne_etal_iccv11.pdf). The version of the dataset used in this tutorial is stored in a public Cloud Storage bucket. The trained model predicts the start frame where a golf swing begins.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "costs"
      },
      "source": [
        "### Costs\n",
        "\n",
        "This tutorial uses billable components of Google Cloud:\n",
        "\n",
        "* Vertex AI\n",
        "* Cloud Storage\n",
        "\n",
        "Learn about [Vertex AI\n",
        "pricing](https://cloud.google.com/vertex-ai/pricing) and [Cloud Storage\n",
        "pricing](https://cloud.google.com/storage/pricing), and use the [Pricing\n",
        "Calculator](https://cloud.google.com/products/calculator/)\n",
        "to generate a cost estimate based on your projected usage."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f0316df526f8"
      },
      "source": [
        "## Get started"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a2c2cb2109a0"
      },
      "source": [
        "### Install Vertex AI SDK for Python and other required packages\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "514a03ed1a82"
      },
      "outputs": [],
      "source": [
        "! pip3 install --upgrade --quiet google-cloud-aiplatform google-cloud-storage"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ff555b32bab8"
      },
      "source": [
        "### Restart runtime (Colab only)\n",
        "\n",
        "To use the newly installed packages, you must restart the runtime on Google Colab."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "f09b4dff629a"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    import IPython\n",
        "\n",
        "    app = IPython.Application.instance()\n",
        "    app.kernel.do_shutdown(True)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ee775571c2b5"
      },
      "source": [
        "<div class=\"alert alert-block alert-warning\">\n",
        "<b>⚠️ The kernel is going to restart. Wait until it's finished before continuing to the next step. ⚠️</b>\n",
        "</div>\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "92e68cfc3a90"
      },
      "source": [
        "### Authenticate your notebook environment (Colab only)\n",
        "\n",
        "Authenticate your environment on Google Colab.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "46604f70e831"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4f872cd812d0"
      },
      "source": [
        "### Set Google Cloud project information and initialize Vertex AI SDK for Python\n",
        "\n",
        "To get started using Vertex AI, you must have an existing Google Cloud project and [enable the Vertex AI API](https://console.cloud.google.com/flows/enableapi?apiid=aiplatform.googleapis.com). Learn more about [setting up a project and a development environment](https://cloud.google.com/vertex-ai/docs/start/cloud-environment)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "294fe4e5a671"
      },
      "outputs": [],
      "source": [
        "PROJECT_ID = \"[your-project-id]\"  # @param {type:\"string\"}\n",
        "LOCATION = \"us-central1\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bucket:mbsdk"
      },
      "source": [
        "### Create a Cloud Storage bucket\n",
        "\n",
        "Create a storage bucket to store intermediate artifacts such as datasets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bucket"
      },
      "outputs": [],
      "source": [
        "BUCKET_URI = f\"gs://your-bucket-name-{PROJECT_ID}-unique\"  # @param {type:\"string\"}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_bucket"
      },
      "source": [
        "**If your bucket doesn't already exist**: Run the following cell to create your Cloud Storage bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_bucket"
      },
      "outputs": [],
      "source": [
        "! gsutil mb -l $LOCATION -p $PROJECT_ID $BUCKET_URI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "source": [
        "### Initialize Vertex AI SDK for Python\n",
        "\n",
        "Initialize the Vertex AI SDK for Python for your project and corresponding bucket."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "init_aip:mbsdk"
      },
      "outputs": [],
      "source": [
        "from google.cloud import aiplatform\n",
        "\n",
        "aiplatform.init(project=PROJECT_ID, staging_bucket=BUCKET_URI)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tutorial_start:automl"
      },
      "source": [
        "## Tutorial\n",
        "\n",
        "Now you are ready to start creating your own AutoML video classification model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "import_file:u_dataset,csv"
      },
      "source": [
        "#### Location of Cloud Storage training data.\n",
        "\n",
        "Now set the variable `IMPORT_FILE` to the location of the CSV index file in Cloud Storage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "import_file:hmdb,csv,vcn"
      },
      "outputs": [],
      "source": [
        "IMPORT_FILE = \"gs://automl-video-demo-data/hmdb_split1_5classes_train_inf.csv\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "quick_peek:csv"
      },
      "source": [
        "#### Quick peek at your data\n",
        "\n",
        "This tutorial uses a version of the MIT Human Motion dataset that is stored in a public Cloud Storage bucket, using a CSV index file.\n",
        "\n",
        "Start by doing a quick peek at the data. You count the number of examples by counting the number of rows in the CSV index file  (`wc -l`) and then peek at the first few rows."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "quick_peek:csv"
      },
      "outputs": [],
      "source": [
        "count = ! gsutil cat $IMPORT_FILE | wc -l\n",
        "print(\"Number of Examples\", int(count[0]))\n",
        "\n",
        "print(\"First 10 rows\")\n",
        "! gsutil cat $IMPORT_FILE | head"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_dataset:video,vcn"
      },
      "source": [
        "### Create the Dataset\n",
        "\n",
        "Next, create the dataset resource using the `create` method for the `VideoDataset` class, which takes the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the dataset resource.\n",
        "- `gcs_source`: A list of one or more dataset index files to import the data items into the `Dataset` resource.\n",
        "\n",
        "This operation may take several minutes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_dataset:video,vcn"
      },
      "outputs": [],
      "source": [
        "dataset = aiplatform.VideoDataset.create(\n",
        "    display_name=\"MIT Human Motion\",\n",
        "    gcs_source=[IMPORT_FILE],\n",
        "    import_schema_uri=aiplatform.schema.dataset.ioformat.video.classification,\n",
        ")\n",
        "\n",
        "print(dataset.resource_name)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "create_automl_pipeline:video,vcn"
      },
      "source": [
        "### Create and run training pipeline\n",
        "\n",
        "To train an AutoML model, you perform two steps: 1) create a training pipeline, and 2) run the pipeline.\n",
        "\n",
        "#### Create training pipeline\n",
        "\n",
        "An AutoML training pipeline is created with the `AutoMLVideoTrainingJob` class, with the following parameters:\n",
        "\n",
        "- `display_name`: The human readable name for the `TrainingJob` resource.\n",
        "- `prediction_type`: The type task to train the model for.\n",
        "  - `classification`: A video classification model.\n",
        "  - `object_tracking`: A video object tracking model.\n",
        "  - `action_recognition`: A video action recognition model.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "create_automl_pipeline:video,vcn"
      },
      "outputs": [],
      "source": [
        "job = aiplatform.AutoMLVideoTrainingJob(\n",
        "    display_name=\"hmdb\",\n",
        "    prediction_type=\"classification\",\n",
        ")\n",
        "\n",
        "print(job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "run_automl_pipeline:video"
      },
      "source": [
        "#### Run the training pipeline\n",
        "\n",
        "Next, you run the job to start the training job by invoking the method `run`, with the following parameters:\n",
        "\n",
        "- `dataset`: The dataset resource to train the model.\n",
        "- `model_display_name`: The human readable name for the trained model.\n",
        "- `training_fraction_split`: The percentage of the dataset to use for training.\n",
        "- `test_fraction_split`: The percentage of the dataset to use for test (holdout data).\n",
        "\n",
        "The `run` method when completed returns the model resource.\n",
        "\n",
        "The execution of the training pipeline can take over 3 hours to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "run_automl_pipeline:video"
      },
      "outputs": [],
      "source": [
        "model = job.run(\n",
        "    dataset=dataset,\n",
        "    model_display_name=\"hmdb\",\n",
        "    training_fraction_split=0.8,\n",
        "    test_fraction_split=0.2,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "source": [
        "## Review model evaluation scores\n",
        "After your model has finished training, you can review the evaluation scores for it.\n",
        "\n",
        "You can check the model's evaluation results using the `get_model_evaluation` method of the Vertex AI model resource.\n",
        "\n",
        "Just like Vertex AI datasets, you can either use the reference to the model variable you created when you deployed the model or you can filter from the list of all of the models in your project using the model's display name as given below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "evaluate_the_model:mbsdk"
      },
      "outputs": [],
      "source": [
        "# Get model resource ID using the display_name\n",
        "models = aiplatform.Model.list(filter=\"display_name=hmdb\")\n",
        "\n",
        "if len(models) != 0:\n",
        "\n",
        "    # Get the model object\n",
        "    model_rsc_name = models[0].resource_name\n",
        "    print(\"Model resource name:\", model_rsc_name)\n",
        "    model = aiplatform.Model(model_rsc_name)\n",
        "\n",
        "    # Print the model evaluation\n",
        "    model_eval = model.get_model_evaluation()\n",
        "    print(model_eval.to_dict())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_prediction"
      },
      "source": [
        "## Send a batch prediction request\n",
        "\n",
        "Send a batch prediction to your deployed model."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_test_items:batch_prediction"
      },
      "source": [
        "### Get test item(s)\n",
        "\n",
        "Now do a batch prediction to your Vertex AI model. Use arbitrary examples from the dataset as test items.\n",
        "Don't be concerned that the examples were likely used in training the model since this tutorial is just about how to make a batch prediction."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_test_items:automl,vcn,csv"
      },
      "outputs": [],
      "source": [
        "test_items = ! gsutil cat $IMPORT_FILE | head -n2\n",
        "\n",
        "if len(test_items[0]) == 5:\n",
        "    _, test_item_1, test_label_1, _, _ = str(test_items[0]).split(\",\")\n",
        "    _, test_item_2, test_label_2, _, _ = str(test_items[1]).split(\",\")\n",
        "else:\n",
        "    test_item_1, test_label_1, _, _ = str(test_items[0]).split(\",\")\n",
        "    test_item_2, test_label_2, _, _ = str(test_items[1]).split(\",\")\n",
        "\n",
        "\n",
        "print(test_item_1, test_label_1)\n",
        "print(test_item_2, test_label_2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "make_batch_file:automl,video"
      },
      "source": [
        "### Make a batch input file\n",
        "\n",
        "Now make a batch input file, which you store in your local Cloud Storage bucket. The batch input file can be either CSV or JSONL.\n",
        "You use JSONL in this tutorial. In the JSONL file, make one dictionary entry per line for each video. The dictionary contains the key/value pairs:\n",
        "\n",
        "- `content`: The Cloud Storage path to the video.\n",
        "- `mimeType`: The content type. In our example, it is a `avi` file.\n",
        "- `timeSegmentStart`: The start timestamp in the video to do prediction on. *Note*, the timestamp must be specified as a string and followed by s (second), m (minute) or h (hour).\n",
        "- `timeSegmentEnd`: The end timestamp in the video to do prediction on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "make_batch_file:automl,video"
      },
      "outputs": [],
      "source": [
        "import json\n",
        "\n",
        "from google.cloud import storage\n",
        "\n",
        "test_filename = \"test.jsonl\"\n",
        "gcs_input_uri = BUCKET_URI + \"/\" + test_filename\n",
        "\n",
        "data_1 = {\n",
        "    \"content\": test_item_1,\n",
        "    \"mimeType\": \"video/avi\",\n",
        "    \"timeSegmentStart\": \"0.0s\",\n",
        "    \"timeSegmentEnd\": \"5.0s\",\n",
        "}\n",
        "data_2 = {\n",
        "    \"content\": test_item_2,\n",
        "    \"mimeType\": \"video/avi\",\n",
        "    \"timeSegmentStart\": \"0.0s\",\n",
        "    \"timeSegmentEnd\": \"5.0s\",\n",
        "}\n",
        "\n",
        "\n",
        "bucket = storage.Client(project=PROJECT_ID).bucket(BUCKET_URI[5:])\n",
        "blob = bucket.blob(blob_name=test_filename)\n",
        "data = json.dumps(data_1) + \"\\n\" + json.dumps(data_2) + \"\\n\"\n",
        "blob.upload_from_string(data)\n",
        "print(gcs_input_uri)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d56366168ec5"
      },
      "source": [
        "### Check input content\n",
        "Check the contents of the `test.jsonl`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "378131e21a7e"
      },
      "outputs": [],
      "source": [
        "! gsutil cat $gcs_input_uri"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "source": [
        "### Make the batch prediction request\n",
        "\n",
        "Now that your Vertex AI model resource is trained, you can make a batch prediction by invoking the `batch_predict()` method, with the following parameters:\n",
        "\n",
        "- `job_display_name`: The human readable name for the batch prediction job.\n",
        "- `gcs_source`: A list of one or more batch request input files.\n",
        "- `gcs_destination_prefix`: The Cloud Storage location for storing the batch prediction resuls.\n",
        "- `sync`: If set to True, the call blocks while waiting for the asynchronous batch job to complete."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job = model.batch_predict(\n",
        "    job_display_name=\"hmdb\",\n",
        "    gcs_source=gcs_input_uri,\n",
        "    gcs_destination_prefix=BUCKET_URI,\n",
        "    sync=False,\n",
        ")\n",
        "\n",
        "print(batch_predict_job)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "source": [
        "### Wait for completion of batch prediction job\n",
        "\n",
        "Next, wait for the batch job to complete. Alternatively, one can set the parameter `sync` to `True` in the `batch_predict()` method to block until the batch prediction job is completed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "batch_request_wait:mbsdk"
      },
      "outputs": [],
      "source": [
        "batch_predict_job.wait()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "get_batch_prediction:mbsdk,vcn"
      },
      "source": [
        "### Get the predictions\n",
        "\n",
        "Next, get the results from the completed batch prediction job.\n",
        "\n",
        "The results are written to the Cloud Storage output bucket you specified in the batch prediction request.\n",
        "Call the method `iter_outputs()` to get a list of each Cloud Storage file generated with the results.\n",
        "Each file contains one or more prediction requests in a JSON format:\n",
        "\n",
        "- `content`: The prediction request.\n",
        "- `prediction`: The prediction response.\n",
        " - `ids`: The internal assigned unique identifiers for each prediction request.\n",
        " - `displayNames`: The class names for each class label.\n",
        " - `confidences`: The predicted confidence, between 0 and 1, per class label.\n",
        " - `timeSegmentStart`: The time offset in the video to the start of the video sequence.\n",
        " - `timeSegmentEnd`: The time offset in the video to the end of the video sequence."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "get_batch_prediction:mbsdk,vcn"
      },
      "outputs": [],
      "source": [
        "bp_iter_outputs = batch_predict_job.iter_outputs()\n",
        "\n",
        "prediction_results = list()\n",
        "for blob in bp_iter_outputs:\n",
        "    if blob.name.split(\"/\")[-1].startswith(\"prediction\"):\n",
        "        prediction_results.append(blob.name)\n",
        "\n",
        "for prediction_result in prediction_results:\n",
        "    gfile_name = f\"gs://{bp_iter_outputs.bucket.name}/{prediction_result}\".replace(\n",
        "        BUCKET_URI + \"/\", \"\"\n",
        "    )\n",
        "    data = bucket.get_blob(gfile_name).download_as_string()\n",
        "    data = json.loads(data)\n",
        "    print(data)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "source": [
        "## Clean up\n",
        "\n",
        "To clean up all Google Cloud resources used in this project, you can [delete the Google Cloud\n",
        "project](https://cloud.google.com/resource-manager/docs/creating-managing-projects#shutting_down_projects) you used for the tutorial.\n",
        "\n",
        "Otherwise, you can delete the individual resources you created in this tutorial:\n",
        "\n",
        "- dataset\n",
        "- model\n",
        "- AutoML training job\n",
        "- batch job\n",
        "- Cloud Storage bucket\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cleanup:mbsdk"
      },
      "outputs": [],
      "source": [
        "# If the bucket needs to be deleted too, please set \"delete_bucket\" to True\n",
        "delete_bucket = False\n",
        "\n",
        "# Delete the dataset using the Vertex dataset object\n",
        "dataset.delete()\n",
        "\n",
        "# Delete the model using the Vertex model object\n",
        "model.delete()\n",
        "\n",
        "# Delete the AutoML or Pipeline training job\n",
        "job.delete()\n",
        "\n",
        "# Delete the batch prediction job using the Vertex batch prediction object\n",
        "batch_predict_job.delete()\n",
        "\n",
        "# Delete the Cloud storage bucket\n",
        "if delete_bucket:\n",
        "    ! gsutil rm -r $BUCKET_URI"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "name": "sdk_automl_video_classification_batch.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}