Added keras model REST protocol test on triton for Modelmesh #1835
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| @@ -110,6 +112,45 @@ | |||
| ... AND | |||
| ... Delete Serving Runtime Template From CLI displayed_name=modelmesh-triton | |||
|
|
|||
| Test Keras Model Rest Inference Via UI (Triton on Modelmesh) | |||
Check warning
Code scanning / Robocop
Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test
| @@ -110,6 +112,45 @@ | |||
| ... AND | |||
| ... Delete Serving Runtime Template From CLI displayed_name=modelmesh-triton | |||
|
|
|||
| Test Keras Model Rest Inference Via UI (Triton on Modelmesh) | |||
Check warning
Code scanning / Robocop
Test case '{{ test_name }}' has too many keywords inside ({{ keyword_count }}/{{ max_allowed_count }}) Warning test
| Recreate S3 Data Connection project_title=${PRJ_TITLE} dc_name=model-serving-connection | ||
| ... aws_access_key=${S3.AWS_ACCESS_KEY_ID} aws_secret_access=${S3.AWS_SECRET_ACCESS_KEY} | ||
| ... aws_bucket_name=ods-ci-s3 | ||
| Create Model Server token=${TRUE} runtime=${ONNX_RUNTIME_NAME} server_name=${ONNX_RUNTIME_NAME} existing_server=${TRUE} |
Check warning
Code scanning / Robocop
Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test
| ... Open Dashboard Settings settings_page=Serving runtime | ||
| ... AND | ||
| ... Delete Serving Runtime Template displayed_name=modelmesh-triton | ||
|
|
Check warning
Code scanning / Robocop
Invalid number of empty lines between sections ({{ empty_lines }}/{{ allowed_empty_lines }}) Warning test
Robot Results
|
| @@ -0,0 +1,73 @@ | |||
| apiVersion: serving.kserve.io/v1alpha1 | |||
There was a problem hiding this comment.
this is the same as https://github.com/red-hat-data-services/ods-ci/pull/1840/files
Can you avoid duplication and use one single runtime definition for multiple models?
| @@ -179,11 +179,11 @@ Verify Model Status | |||
| [Arguments] ${model_name} ${expected_status}=success | |||
| IF "${expected_status}"=="success" | |||
| SeleniumLibrary.Wait Until Page Contains Element | |||
| ... //div[.="${model_name} "]${MS_TABLE_STATUS}${MS_TABLE_STATUS_SUCCESS} timeout=2m | |||
| ... //div[.="${model_name} "]${MS_TABLE_STATUS}${MS_TABLE_STATUS_SUCCESS} timeout=4m | |||
There was a problem hiding this comment.
maybe better to parametrize it
| @@ -131,6 +172,7 @@ Triton On Kserve Suite Teardown | |||
| # Failure will be shown in the logs of the run nonetheless | |||
| IF ${MODEL_CREATED} | |||
| Clean All Models Of Current User | |||
| Log Skip | |||
| @@ -121,7 +121,7 @@ | |||
| ... Selects the given one | |||
| [Arguments] ${model_server} | |||
| ${selectable}= Run Keyword And Return Status | |||
| ... Wait Until Element Is Visible xpath://span[.="Model servers"]/../../..//button[@aria-label="Options menu"] timeout=1 | |||
| ... Wait Until Element Is Visible xpath://span[.="Model servers"]/../../..//button[@aria-label="Options menu"] timeout=3 | |||
Check warning
Code scanning / Robocop
Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test
| @@ -13,7 +13,7 @@ | |||
| Resource ../../../Resources/CLI/ModelServing/modelmesh.resource | |||
| Resource ../../../Resources/Common.robot | |||
| Suite Setup Triton On Kserve Suite Setup | |||
| Suite Teardown Triton On Kserve Suite Teardown | |||
| #Suite Teardown Triton On Kserve Suite Teardown | |||
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| @@ -130,12 +167,13 @@ | |||
| # Even if kw fails, deleting the whole project will also delete the model | |||
| # Failure will be shown in the logs of the run nonetheless | |||
| IF ${MODEL_CREATED} | |||
| Clean All Models Of Current User | |||
| #Clean All Models Of Current User | |||
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| ELSE | ||
| Log Model not deployed, skipping deletion step during teardown console=true | ||
| END | ||
| ${projects}= Create List ${PRJ_TITLE} | ||
| Delete List Of Projects Via CLI ocp_projects=${projects} | ||
| #Delete List Of Projects Via CLI ocp_projects=${projects} |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| ... ${EXPECTED_INFERENCE_REST_OUTPUT_KERAS} | ||
| ... token_auth=${TRUE} | ||
| ... project_title=${PRJ_TITLE} | ||
| #[Teardown] Run Keywords Get Modelmesh Events And Logs model_name=${ONNX_MODEL_NAME} |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| ... token_auth=${TRUE} | ||
| ... project_title=${PRJ_TITLE} | ||
| #[Teardown] Run Keywords Get Modelmesh Events And Logs model_name=${ONNX_MODEL_NAME} | ||
| #... project_title=${PRJ_TITLE} |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| ... project_title=${PRJ_TITLE} | ||
| #[Teardown] Run Keywords Get Modelmesh Events And Logs model_name=${ONNX_MODEL_NAME} | ||
| #... project_title=${PRJ_TITLE} | ||
| #... AND |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| #[Teardown] Run Keywords Get Modelmesh Events And Logs model_name=${ONNX_MODEL_NAME} | ||
| #... project_title=${PRJ_TITLE} | ||
| #... AND | ||
| #... Clean All Models Of Current User |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| @@ -32,6 +32,9 @@ | |||
| ${PYTORCH_MODEL_NAME}= resnet50 | |||
| ${PYTORCH_RUNTIME_FILEPATH}= ${RESOURCES_DIRPATH}/triton_pytorch_modelmesh_runtime.yaml | |||
| ${EXPECTED_INFERENCE_REST_OUTPUT_FILE_PYTORCH}= tests/Resources/Files/triton/modelmesh-triton-pytorch-rest-output.json | |||
| ${INFERENCE_REST_INPUT_KERAS_FILE}= @${RESOURCES_DIRPATH}/modelmesh-triton-keras-rest-input.json | |||
| ${EXPECTED_INFERENCE_KERAS_OUTPUT_FILE}= ${RESOURCES_DIRPATH}/modelmesh-triton-keras-rest-output.json | |||
| ${KERAS_RUNTIME_FILEPATH}= ${RESOURCES_DIRPATH}/triton_keras_modelmesh_runtime.yaml | |||
Check notice
Code scanning / Robocop
Variable '{{ name }}' is assigned but not used Note test
| Create Data Science Project title=${PRJ_TITLE} description=${PRJ_DESCRIPTION} | ||
| ... existing_project=${FALSE} | ||
| Open Dashboard Settings settings_page=Serving runtimes | ||
| #Upload Serving Runtime Template runtime_filepath=${KERAS_RUNTIME_FILEPATH} |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
| ... existing_project=${FALSE} | ||
| Open Dashboard Settings settings_page=Serving runtimes | ||
| #Upload Serving Runtime Template runtime_filepath=${KERAS_RUNTIME_FILEPATH} | ||
| #... serving_platform=multi runtime_protocol=REST |
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
|
| @@ -122,20 +159,21 @@ | |||
| ... ${ODH_DASHBOARD_URL} ${BROWSER.NAME} ${BROWSER.OPTIONS} | |||
|
|
|||
| Fetch Knative CA Certificate filename=openshift_ca_istio_knative.crt | |||
| Clean All Models Of Current User | |||
| #Clean All Models Of Current User | |||
Check warning
Code scanning / Robocop
Missing blank space after comment character Warning test
Added Automation Test for RHOAIENG-11564