sdk/python/foundation-models/system/inference/text-to-image/safe-text-to-image-batch-deployment.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create an Azure AI Content Safety (AACS) enabled text-to-image batch endpoint (Preview)\n",
    "### This notebook is under preview.\n",
    "\n",
    "### Steps to create an __AACS__ enabled __text-to-image__ batch endpoint\n",
    "1. Create a __AACS__ enabled text-to-image batch endpoint with a custom [score_batch.py](./aacs-scoring-files/score/score_batch.py) script. This will integrate the batch endpoint with the AACS resource to moderate the response from the __text-to-image__ model and the request from the user.\n",
    "2. Create a new __AACS__ enabled __text-to-image__ batch endpoint with a custom [score_batch.py](./aacs-scoring-files/score/score_batch.py) which will integrate with the __AACS__ resource to moderate the response from the __text-to-image__ model and the request from the user. To make the custom [score_batch.py](./aacs-scoring-files/score/score_batch.py) to successfully authenticated to the __AACS__ resource, use __Environment variable__ to pass the access key of the __AACS__ resource to the custom [score_batch.py](./aacs-scoring-files/score/score_batch.py) via environment variable. The custom [score_batch.py](./aacs-scoring-files/score/score_batch.py) can use the key directly to access the AACS resource. This option is less secure, if someone in your org has access to the endpoint, he/she can get the access key from the environment variable and use it to access the AACS resource.\n",
    "\n",
    "### Task\n",
    "\n",
    "`text-to-image` tasks generates image as output based on text prompt given in input.\n",
    " \n",
    "### Model\n",
    "Models that can perform the `text-to-image` task are tagged with `text-to-image`. We will use the `runwayml-stable-diffusion-v1-5` model in this notebook. If you opened this notebook from a specific model card, remember to replace the specific model name.\n",
    "\n",
    "### Outline\n",
    "1. Setup pre-requisties\n",
    "2. Create AACS resource\n",
    "3. Create AACS enabled text-to-image batch endpoint\n",
    "4. Prepare data for inference - using a folder of csv files with text prompts\n",
    "5. Test the endpoint - using csv files\n",
    "6. Clean up resources - delete the endpoint\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Setup pre-requisties\n",
    "\n",
    "* Check List\n",
    "* Install dependencies\n",
    "* Connect to AzureML Workspace. Learn more at [set up SDK authentication](https://learn.microsoft.com/en-us/azure/machine-learning/how-to-setup-authentication?tabs=sdk). Replace  `<WORKSPACE_NAME>`, `<RESOURCE_GROUP>` and `<SUBSCRIPTION_ID>` below.\n",
    "* Connect to `azureml` system registry\n",
    "\n",
    "> [x] The identity you are using to execute this notebook(yourself or your VM) need to have the __Contributor__ role on the resource group where the AML Workspace your specified is located, because this notebook will create an AACS resource using that identity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install the required packages\n",
    "%pip install azure-identity==1.13.0\n",
    "%pip install azure-mgmt-cognitiveservices==13.4.0\n",
    "%pip install azure-ai-ml==1.8.0\n",
    "%pip install azure-mgmt-msi==7.0.0\n",
    "%pip install azure-mgmt-authorization==3.0.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient\n",
    "from azure.mgmt.cognitiveservices.models import Account, Sku, AccountProperties\n",
    "from IPython.core.display import display, HTML\n",
    "from azure.ai.ml import Input\n",
    "from azure.ai.ml.entities import (\n",
    "    ModelBatchDeployment,\n",
    "    ModelBatchDeploymentSettings,\n",
    "    AmlCompute,\n",
    "    Data,\n",
    "    BuildContext,\n",
    "    BatchRetrySettings,\n",
    "    CodeConfiguration,\n",
    "    Environment,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "\n",
    "timestamp = str(time.time()).split(\".\")[0]\n",
    "\n",
    "# The public registry name contains text-to-image models\n",
    "registry_name = \"azureml\"\n",
    "\n",
    "endpoint_name = f\"text-to-image-test-ep-{timestamp}\"  # Replace with your endpoint name,\n",
    "deployment_name = \"default\"  # Replace with your deployment name, lower case only!!!\n",
    "sku_name = \"STANDARD_NC6S_V3\"  # Name of the sku(compute instance type)\n",
    "\n",
    "environment_name = \"text-to-image-model-env\"  # Replace with your environment name\n",
    "compute_name = \"gpu-compute\"  # Replace with your compute name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    credential = DefaultAzureCredential()\n",
    "    # Check if given credential can get token successfully.\n",
    "    credential.get_token(\"https://management.azure.com/.default\")\n",
    "except Exception as ex:\n",
    "    # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n",
    "    credential = InteractiveBrowserCredential()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    ml_client = MLClient.from_config(credential=credential)\n",
    "except Exception as ex:\n",
    "    # enter details of your AML workspace\n",
    "    subscription_id = \"subscription_id\"\n",
    "    resource_group = \"resource_group\"\n",
    "    workspace = \"workspace\"\n",
    "\n",
    "    # get a handle to the workspace\n",
    "    ml_client = MLClient(\n",
    "        credential,\n",
    "        subscription_id,\n",
    "        resource_group,\n",
    "        workspace,\n",
    "        logging_enable=True,\n",
    "    )\n",
    "\n",
    "\n",
    "subscription_id = ml_client.subscription_id\n",
    "resource_group = ml_client.resource_group_name\n",
    "workspace = ml_client.workspace_name\n",
    "\n",
    "reg_client = MLClient(\n",
    "    credential,\n",
    "    subscription_id=subscription_id,\n",
    "    resource_group_name=resource_group,\n",
    "    registry_name=registry_name,\n",
    ")\n",
    "\n",
    "\n",
    "print(f\"Connected to workspace {workspace}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Create AACS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1 Assign variables for Azure Content Safety\n",
    "Currently, AACS is available in a limited set of regions:\n",
    "\n",
    "__NOTE__: before you choose the region to deploy the AACS, please be aware that your data will be transferred to the region you choose and by selecting a region outside your current location, you may be allowing the transmission of your data to regions outside your jurisdiction. It is important to note that data protection and privacy laws may vary between jurisdictions. Before proceeding, we strongly advise you to familiarize yourself with the local laws and regulations governing data transfer and ensure that you are legally permitted to transmit your data to an overseas location for processing. By continuing with the selection of a different region, you acknowledge that you have understood and accepted any potential risks associated with such data transmission. Please proceed with caution."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acs_client = CognitiveServicesManagementClient(credential, subscription_id)\n",
    "\n",
    "\n",
    "# settings for the AACS resource\n",
    "# we will choose existing AACS resource if it exists, otherwise create a new one\n",
    "# name of AACS resource, has to be unique\n",
    "\n",
    "aacs_name = f\"{endpoint_name}-aacs\"\n",
    "available_aacs_locations = [\"east us\", \"west europe\"]\n",
    "\n",
    "# create a new Cognitive Services Account\n",
    "kind = \"ContentSafety\"\n",
    "aacs_sku_name = \"S0\"\n",
    "aacs_location = available_aacs_locations[0]\n",
    "\n",
    "\n",
    "print(\"Available SKUs:\")\n",
    "aacs_skus = acs_client.resource_skus.list()\n",
    "print(\"SKU Name\\tSKU Tier\\tLocations\")\n",
    "for sku in aacs_skus:\n",
    "    if sku.kind == \"ContentSafety\":\n",
    "        locations = \",\".join(sku.locations)\n",
    "        print(sku.name + \"\\t\" + sku.tier + \"\\t\" + locations)\n",
    "\n",
    "print(\n",
    "    f\"Choose a new Azure AI Content Safety (AACS) resource in {aacs_location} with SKU {aacs_sku_name}\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2 Create AACS resource"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = Account(\n",
    "    sku=Sku(name=aacs_sku_name),\n",
    "    kind=kind,\n",
    "    location=aacs_location,\n",
    "    properties=AccountProperties(\n",
    "        custom_sub_domain_name=aacs_name, public_network_access=\"Enabled\"\n",
    "    ),\n",
    ")\n",
    "\n",
    "\n",
    "def find_acs(accounts):\n",
    "    return next(\n",
    "        x\n",
    "        for x in accounts\n",
    "        if x.kind == \"ContentSafety\"\n",
    "        and x.location == aacs_location\n",
    "        and x.sku.name == aacs_sku_name\n",
    "    )\n",
    "\n",
    "\n",
    "try:\n",
    "    # check if AACS exists\n",
    "    aacs = acs_client.accounts.get(resource_group, aacs_name)\n",
    "    print(f\"Found existing Azure AI content safety (AACS) Account {aacs.name}.\")\n",
    "except:\n",
    "    try:\n",
    "        # check if there is an existing AACS resource within same resource group\n",
    "        aacs = find_acs(acs_client.accounts.list_by_resource_group(resource_group))\n",
    "        print(\n",
    "            f\"Found existing Azure AI content safety (AACS) Account {aacs.name} in resource group {resource_group}.\"\n",
    "        )\n",
    "    except:\n",
    "        print(f\"Creating Azure AI content safety (AACS) Account {aacs_name}.\")\n",
    "        acs_client.accounts.begin_create(resource_group, aacs_name, parameters).wait()\n",
    "        print(\"Resource created.\")\n",
    "        aacs = acs_client.accounts.get(resource_group, aacs_name)\n",
    "\n",
    "\n",
    "aacs_endpoint = aacs.properties.endpoint\n",
    "aacs_resource_id = aacs.id\n",
    "print(f\"AACS endpoint is {aacs_endpoint}\")\n",
    "print(f\"AACS ResourceId is {aacs_resource_id}\")\n",
    "\n",
    "aacs_access_key = acs_client.accounts.list_keys(\n",
    "    resource_group_name=resource_group, account_name=aacs.name\n",
    ").key1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Create AACS enabled text-to-image batch endpoint"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3.1 Check if text-to-image model is available in the AML registry\n",
    "\n",
    "Browse models in the Model Catalog in the AzureML Studio, filtering by the text-to-image task. In this example, we use the runwayml-stable-diffusion-v1-5 model. If you have opened this notebook for a different model, replace the model name accordingly. This is a pre-trained model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    # Name of the text-to-image model to be deployed\n",
    "    model_name = \"runwayml-stable-diffusion-v1-5\"\n",
    "    model = reg_client.models.get(model_name, label=\"latest\")\n",
    "    print(\n",
    "        f\"Using model name: {model.name}, version: {model.version}, id: {model.id} for inference.\"\n",
    "    )\n",
    "except:\n",
    "    raise Exception(\n",
    "        f\"No model named {model_name} found in registry. \"\n",
    "        \"Please check model name in Azure model catalog.\"\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.2 Create environment for text-to-image endpoint\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    env = ml_client.environments.get(environment_name, label=\"latest\")\n",
    "    print(\"---Environment already exists---\")\n",
    "except:\n",
    "    print(\"---Creating environment---\")\n",
    "    env = Environment(\n",
    "        name=environment_name,\n",
    "        build=BuildContext(path=\"./scoring-files/docker_env\"),\n",
    "    )\n",
    "    ml_client.environments.create_or_update(env)\n",
    "    env = ml_client.environments.get(environment_name, label=\"latest\")\n",
    "    print(\"---Please use link below to check build status---\")\n",
    "\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        f\"\"\"\n",
    "             <a href=\"https://ml.azure.com/environments/{environment_name}/version/{env.version}?wsid=/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}\">\n",
    "                Click here to check env build status in AML studio\n",
    "             </a>\n",
    "             \"\"\"\n",
    "    )\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.3 Create compute cluster to run batch job on\n",
    "\n",
    "Use the model card from the AzureML system registry to check the minimum required inferencing SKU, referenced as size below. If you already have a sufficient compute cluster that you wish to use, you can simply define the name in `compute_name` in the following code block. Otherwise, the below snippet will create a new compute cluster."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.constants import AssetTypes, BatchDeploymentOutputAction\n",
    "\n",
    "if not any(filter(lambda m: m.name == compute_name, ml_client.compute.list())):\n",
    "    compute_cluster = AmlCompute(\n",
    "        name=compute_name,\n",
    "        size=sku_name,\n",
    "        min_instances=0,\n",
    "        max_instances=2,\n",
    "    )\n",
    "    ml_client.compute.begin_create_or_update(compute_cluster).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 3.4 Deploy the model to a batch endpoint\n",
    "\n",
    "Batch endpoints are endpoints that are used to do batch inferencing on large volumes of data over a period of time. The endpoints receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters. Batch endpoints store outputs to a data store for further analysis. For more information on batch endpoints and deployments, see <a href=\"https://learn.microsoft.com/en-us/azure/machine-learning/concept-endpoints?view=azureml-api-2#what-are-batch-endpoints\" target=\"_blank\"> What are batch endpoints?</a> In this sub-section, we will cover the following items:\n",
    "\n",
    "* Create a batch endpoint.\n",
    "* Create a batch deployment.\n",
    "* Set the deployment as default. Doing so allows invoking the endpoint without specifying the deployment's name."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Create a batch endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml.entities import BatchEndpoint\n",
    "\n",
    "# Check if the endpoint already exists in the workspace\n",
    "try:\n",
    "    endpoint = ml_client.batch_endpoints.get(endpoint_name)\n",
    "    print(\"---Endpoint already exists---\")\n",
    "except:\n",
    "    # Create an batch endpoint if it doesn't exist\n",
    "\n",
    "    # Define the endpoint\n",
    "    endpoint = BatchEndpoint(name=endpoint_name, description=\"Test endpoint for model\")\n",
    "\n",
    "    # Trigger the endpoint creation\n",
    "    try:\n",
    "        ml_client.begin_create_or_update(endpoint).wait()\n",
    "        print(\"\\n---Endpoint created successfully---\\n\")\n",
    "    except Exception as err:\n",
    "        raise RuntimeError(\n",
    "            f\"Endpoint creation failed. Detailed Response:\\n{err}\"\n",
    "        ) from err"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Deploy text-to-image model\n",
    "This step may take a few minutes.\n",
    "\n",
    "__Note__: `mini_batch_size` is the number of CSV files processed by the model in a single mini_batch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "deployment = ModelBatchDeployment(\n",
    "    name=deployment_name,\n",
    "    endpoint_name=endpoint.name,\n",
    "    model=model,\n",
    "    environment=env,\n",
    "    code_configuration=CodeConfiguration(\n",
    "        code=\"scoring-files/score\",\n",
    "        scoring_script=\"score_batch.py\",\n",
    "    ),\n",
    "    compute=compute_name,\n",
    "    settings=ModelBatchDeploymentSettings(\n",
    "        instance_count=1,\n",
    "        max_concurrency_per_instance=1,\n",
    "        mini_batch_size=2,\n",
    "        output_action=BatchDeploymentOutputAction.APPEND_ROW,\n",
    "        output_file_name=\"predictions.csv\",\n",
    "        retry_settings=BatchRetrySettings(max_retries=3, timeout=3000),\n",
    "        logging_level=\"info\",\n",
    "        environment_variables={\n",
    "            \"CONTENT_SAFETY_ENDPOINT\": aacs_endpoint,\n",
    "            \"CONTENT_SAFETY_KEY\": aacs_access_key,\n",
    "        },\n",
    "    ),\n",
    ")\n",
    "# Trigger the deployment creation\n",
    "try:\n",
    "    ml_client.begin_create_or_update(deployment).wait()\n",
    "    print(\"\\n---Deployment created successfully---\\n\")\n",
    "except Exception as err:\n",
    "    raise RuntimeError(\n",
    "        f\"Deployment creation failed. Detailed Response:\\n{err}\"\n",
    "    ) from err"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Update Batch endpoint to set the default deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "endpoint = ml_client.batch_endpoints.get(endpoint_name)\n",
    "endpoint.defaults.deployment_name = deployment.name\n",
    "ml_client.batch_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Prepare data for inference - using a folder of csv files with text prompts\n",
    "\n",
    "We provide the text prompts in a csv file starting from the first row of a column named \"prompt\". The deployment in the Create batch deployment section below takes the argument mini_batch_size, which is the number of CSV files processed by the model in a single mini_batch. To limit the number of prompts processed in each mini_batch we split the dataset into multiple csv files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read all the csvs in the data folder into a pandas dataframe\n",
    "import glob\n",
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "# Specify the folder where your CSV files are located\n",
    "data_path = \"scoring-files/text-to-image-batch-data\"\n",
    "\n",
    "# Use glob to get a list of CSV files in the folder\n",
    "csv_files = glob.glob(os.path.join(data_path, \"*.csv\"))\n",
    "\n",
    "# Read all CSV files into a single DataFrame using pd.concat\n",
    "batch_df = pd.concat((pd.read_csv(file) for file in csv_files), ignore_index=True)\n",
    "\n",
    "# Now, 'batch_df' contains all the data from the CSV files in the folder\n",
    "print(batch_df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "\n",
    "# Specify the folder where your CSV files should be saved\n",
    "processed_dataset_parent_dir = \"scoring-files/processed-text-to-image-batch-data\"\n",
    "os.makedirs(processed_dataset_parent_dir, exist_ok=True)\n",
    "batch_input_file = \"batch_input.csv\"\n",
    "\n",
    "# Divide this into files of <x> rows each\n",
    "batch_size_per_predict = 2\n",
    "for i in range(0, len(batch_df), batch_size_per_predict):\n",
    "    j = i + batch_size_per_predict\n",
    "    batch_df[i:j].to_csv(\n",
    "        os.path.join(processed_dataset_parent_dir, str(i) + batch_input_file)\n",
    "    )\n",
    "\n",
    "# Check out the first and last file name created\n",
    "input_paths = sorted(Path(processed_dataset_parent_dir).iterdir(), key=os.path.getmtime)\n",
    "input_files = [os.path.basename(path) for path in input_paths]\n",
    "print(f\"{input_files[0]} to {str(i)}{batch_input_file}.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Register folder containing csv files in AML as data asset to use in batch job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_name = \"text-to-image-data\"\n",
    "input = Data(\n",
    "    name=dataset_name,\n",
    "    description=\"A sample of the dataset for image generation for batch deployment, in CSV file format\",\n",
    "    type=AssetTypes.URI_FOLDER,\n",
    "    path=processed_dataset_parent_dir,\n",
    ")\n",
    "ml_client.data.create_or_update(input)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Test the endpoint - using csv files\n",
    "\n",
    "Invoke the batch endpoint with the input parameter pointing to the directory containing one or more csv files containing the batch inference input. This creates a pipeline job using the default deployment in the endpoint. Wait for the job to complete.\n",
    "\n",
    "__Note__: If job failed with Out of Memory Error then please try splitting your input into smaller csv files or decreasing mini_batch_size for the deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_data = ml_client.data.get(dataset_name, label=\"latest\")\n",
    "input = Input(type=AssetTypes.URI_FOLDER, path=input_data.id)\n",
    "job = ml_client.batch_endpoints.invoke(endpoint_name=endpoint.name, input=input)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5.1 Get the details of the invoked job\n",
    "Let us get details and logs of the invoked job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.get(job.name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can wait for the job to finish using the following code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(job.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "__Note__: If the job failed with error Assertion Error (The actual length exceeded max length 100 MB) then please consider dividing input csv file into multiple csv files."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5.2 Download the results\n",
    "\n",
    "The deployment creates a child job that executes the scoring. We can get the details of it using the following code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "scoring_job = list(ml_client.jobs.list(parent_job_name=job.name))[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Job name:\", scoring_job.name)\n",
    "print(\"Job status:\", scoring_job.status)\n",
    "print(\n",
    "    \"Job duration:\",\n",
    "    scoring_job.creation_context.last_modified_at\n",
    "    - scoring_job.creation_context.created_at,\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The outputs generated by the deployment job will be placed in an output named `named-outputs/score`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.jobs.download(name=scoring_job.name, download_path=\".\", output_name=\"score\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the batch predictions file with no headers into a dataframe and set your column names\n",
    "import os\n",
    "import pandas as pd\n",
    "\n",
    "predictions_file = os.path.join(\"named-outputs\", \"score\", \"predictions.csv\")\n",
    "\n",
    "score_df = pd.read_csv(\n",
    "    predictions_file,\n",
    "    header=None,\n",
    "    names=[\n",
    "        \"row_number_per_file\",\n",
    "        \"prompt\",\n",
    "        \"generated_image\",\n",
    "        \"nsfw_content_detected\",\n",
    "        \"file_name\",\n",
    "    ],\n",
    ")\n",
    "score_df"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6 Clean up resources - delete the endpoint\n",
    "Batch endpoints use compute resources only when jobs are submitted. You can keep the batch endpoint for your reference without worrying about compute bills, or choose to delete the endpoint. If you created your compute cluster to have zero minimum instances and scale down soon after being idle, you won't be charged for an unused compute."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client.batch_endpoints.begin_delete(endpoint.name).result()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "language_info": {
   "name": "ipython"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}