Support Managing AI APIs within the API Manager via APICTL

[ashera96]

Description

• Import AI API support
• Export AI API support
• Import AI-specific (token-based) policy
• Export AI-specific (token-based) policy

Affected Component

APICTL

Version

4.4.0

Related Issues

No response

Suggested Labels

No response

[ashera96]

[Progress Update]

• API import fails with the following error:
  [Command: apictl import api --file --environment [flags]]

➜  ~ apictl import api -f /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip -e prod --verbose
Executed ImportExportCLI (apictl) on Sun, 06 Oct 2024 22:51:16 +0530
[INFO]: Insecure: false
[INFO]: api called
[INFO]: connecting to https://localhost:9444/oauth2/token
[INFO]: Resolving for API path...
[INFO]: API Location: /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip
[INFO]: Creating workspace
[INFO]: Extracting /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip to /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/apim1768527064
[INFO]: Substituting environment variables in API files...
[INFO]: Creating the project artifact /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/project-artifact1318025712.zip
[INFO]: Creating: MistralAIAPI-0.0.2/
[INFO]: Creating: MistralAIAPI-0.0.2/Definitions/
[INFO]: Creating: MistralAIAPI-0.0.2/Definitions/swagger.yaml
[INFO]: Creating: MistralAIAPI-0.0.2/api.yaml
[INFO]: Creating: MistralAIAPI-0.0.2/api_meta.yaml
[INFO]: Import URL: https://localhost:9444/api/am/publisher/v4/apis/import?preserveProvider=true&rotateRevision=false
Response : {"code":900907,"message":"Error while reading meta information from the definition","description":"Error while reading meta information from the definition","moreInfo":"","error":[]}Error importing API.
Status: 400
Response: {"code":900907,"message":"Error while reading meta information from the definition","description":"Error while reading meta information from the definition","moreInfo":"","error":[]}
[INFO]: Deleting /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/project-artifact1318025712.zip
[INFO]: Deleting /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/apim1768527064/MistralAIAPI-0.0.2
apictl: Error importing API Reason: 400
Exit status 1

• Policy import succeeds and the admin portal shows the policy. However, the carbon server logs the following error. [Command: apictl import policy rate-limiting --file --environment [flags]]

[2024-10-07 00:35:10,510] ERROR - PolicyUtil Error in deploying execution plan
org.wso2.carbon.event.processor.core.exception.ExecutionPlanConfigurationException: Couldn't parse execution plan:
		@Plan:name('carbon.super_sub_AIRateLimitingPolicy')
		@Plan:description('ExecutionPlan for sub_AIRateLimitingPolicy')
		@Import('org.wso2.throttle.processed.request.stream:1.0.0')
		define stream RequestStream (messageID string, appKey string, appTier string, subscriptionKey string, apiKey string, apiTier string, subscriptionTier string, resourceKey string, resourceTier string, userId string,  apiContext string, apiVersion string, appTenant string, apiTenant string, appId string, apiName string, propertiesMap string);

		@Export('org.wso2.throttle.globalThrottle.stream:1.0.0')
		define stream GlobalThrottleStream (throttleKey string, isThrottled bool, expiryTimeStamp long);

		FROM RequestStream
		SELECT messageID, (apiTenant == 'carbon.super' and subscriptionTier == 'AIRateLimitingPolicy') AS isEligible, subscriptionKey AS throttleKey, propertiesMap
		INSERT INTO EligibilityStream;

				FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min, 0)
		select throttleKey,
				ifThenElse((count(messageID) >= 300), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'PROMPT_TOKENS'),'long')) >= 600L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'COMPLETION_TOKENS'),'long')) >= 400L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'TOTAL_TOKENS'),'long')) >= 1000L), true,
						false)
				)
						)
						)
						)
				as isThrottled, expiryTimeStamp
		group by throttleKey
		INSERT ALL EVENTS into ResultStream;

		from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
		select *
		insert into GlobalThrottleStream;

	at org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorService.deployExecutionPlan(CarbonEventProcessorService.java:118) ~[?:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.PolicyUtil.deployPolicy(PolicyUtil.java:134) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.ThrottlePolicyJMSMessageListener.lambda$handleNotificationMessage$0(ThrottlePolicyJMSMessageListener.java:151) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
[2024-10-07 00:36:26,752] ERROR - PolicyUtil Error in deploying execution plan
org.wso2.carbon.event.processor.core.exception.ExecutionPlanConfigurationException: Couldn't parse execution plan:
		@Plan:name('carbon.super_sub_AIRateLimitingPolicy')
		@Plan:description('ExecutionPlan for sub_AIRateLimitingPolicy')
		@Import('org.wso2.throttle.processed.request.stream:1.0.0')
		define stream RequestStream (messageID string, appKey string, appTier string, subscriptionKey string, apiKey string, apiTier string, subscriptionTier string, resourceKey string, resourceTier string, userId string,  apiContext string, apiVersion string, appTenant string, apiTenant string, appId string, apiName string, propertiesMap string);

		@Export('org.wso2.throttle.globalThrottle.stream:1.0.0')
		define stream GlobalThrottleStream (throttleKey string, isThrottled bool, expiryTimeStamp long);

		FROM RequestStream
		SELECT messageID, (apiTenant == 'carbon.super' and subscriptionTier == 'AIRateLimitingPolicy') AS isEligible, subscriptionKey AS throttleKey, propertiesMap
		INSERT INTO EligibilityStream;

				FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min, 0)
		select throttleKey,
				ifThenElse((count(messageID) >= 300), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'PROMPT_TOKENS'),'long')) >= 600L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'COMPLETION_TOKENS'),'long')) >= 400L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'TOTAL_TOKENS'),'long')) >= 1000L), true,
						false)
				)
						)
						)
						)
				as isThrottled, expiryTimeStamp
		group by throttleKey
		INSERT ALL EVENTS into ResultStream;

		from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
		select *
		insert into GlobalThrottleStream;

	at org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorService.deployExecutionPlan(CarbonEventProcessorService.java:118) ~[?:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.PolicyUtil.deployPolicy(PolicyUtil.java:134) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.ThrottlePolicyJMSMessageListener.lambda$handleNotificationMessage$0(ThrottlePolicyJMSMessageListener.java:151) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

Following CTL commands work as is:

• apictl export api (--name --version --provider --environment ) [flags]
• apictl export policy rate-limiting (--type --environment ) [flags]

[ashera96]

• Upon debugging the API import flow; the point of failure was identified as [1]. The reason why it's failing is because the value that reaches this point is already base64encoded. Hence, we have to skip this flow for API import.

• Upon debugging the Policy import flow; the point of failure was identified. The stackTrace is getting logged due to the below-provided exception. As a result of [2] returning this exception, [3] is executed. Thereby, logging the error [4].

[1] https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.rest.api.publisher.v1.common/src/main/java/org/wso2/carbon/apimgt/rest/api/publisher/v1/common/mappings/PublisherCommonUtils.java#L615
[2] https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.throttle.policy.deployer/src/main/java/org/wso2/carbon/apimgt/throttle/policy/deployer/utils/PolicyUtil.java#L131
[3] https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.throttle.policy.deployer/src/main/java/org/wso2/carbon/apimgt/throttle/policy/deployer/utils/PolicyUtil.java#L134
[4] https://github.com/wso2/carbon-apimgt/blob/master/components/apimgt/org.wso2.carbon.apimgt.throttle.policy.deployer/src/main/java/org/wso2/carbon/apimgt/throttle/policy/deployer/utils/PolicyUtil.java#L145

[ashera96]

[Progress Update]

• Policy import is fixed in the latest APIM 4.4.0 pack
• API import still fails and we need to identify whether to fix the export flow or import flow. Created the issue [1] to track the fix for this.

1. [AI API] API download and export/import related errors  #3200

[ashera96]

[Progress Update]

• API export and import is working as expected
• AI policy export and import is working as expected
• Integration tests were added for AI API import/export as well as AI policy import/export via Add integration tests for AI API related import export flows product-apim-tooling#1220