Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Managing AI APIs within the API Manager via APICTL #3162

Closed
4 tasks done
ashera96 opened this issue Oct 2, 2024 · 4 comments
Closed
4 tasks done

Support Managing AI APIs within the API Manager via APICTL #3162

ashera96 opened this issue Oct 2, 2024 · 4 comments

Comments

@ashera96
Copy link

ashera96 commented Oct 2, 2024

Description

  • Import AI API support
  • Export AI API support
  • Import AI-specific (token-based) policy
  • Export AI-specific (token-based) policy

Affected Component

APICTL

Version

4.4.0

Related Issues

No response

Suggested Labels

No response

@ashera96
Copy link
Author

ashera96 commented Oct 6, 2024

[Progress Update]

  • API import fails with the following error:
    [Command: apictl import api --file --environment [flags]]
➜  ~ apictl import api -f /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip -e prod --verbose
Executed ImportExportCLI (apictl) on Sun, 06 Oct 2024 22:51:16 +0530
[INFO]: Insecure: false
[INFO]: api called
[INFO]: connecting to https://localhost:9444/oauth2/token
[INFO]: Resolving for API path...
[INFO]: API Location: /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip
[INFO]: Creating workspace
[INFO]: Extracting /Users/ashera/.wso2apictl/exported/apis/dev/MistralAIAPI_0.0.2.zip to /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/apim1768527064
[INFO]: Substituting environment variables in API files...
[INFO]: Creating the project artifact /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/project-artifact1318025712.zip
[INFO]: Creating: MistralAIAPI-0.0.2/
[INFO]: Creating: MistralAIAPI-0.0.2/Definitions/
[INFO]: Creating: MistralAIAPI-0.0.2/Definitions/swagger.yaml
[INFO]: Creating: MistralAIAPI-0.0.2/api.yaml
[INFO]: Creating: MistralAIAPI-0.0.2/api_meta.yaml
[INFO]: Import URL: https://localhost:9444/api/am/publisher/v4/apis/import?preserveProvider=true&rotateRevision=false
Response : {"code":900907,"message":"Error while reading meta information from the definition","description":"Error while reading meta information from the definition","moreInfo":"","error":[]}Error importing API.
Status: 400
Response: {"code":900907,"message":"Error while reading meta information from the definition","description":"Error while reading meta information from the definition","moreInfo":"","error":[]}
[INFO]: Deleting /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/project-artifact1318025712.zip
[INFO]: Deleting /var/folders/b5/3ywkk2nn2y54l17zj0863twm0000gn/T/apim1768527064/MistralAIAPI-0.0.2
apictl: Error importing API Reason: 400
Exit status 1
  • Policy import succeeds and the admin portal shows the policy. However, the carbon server logs the following error. [Command: apictl import policy rate-limiting --file --environment [flags]]
[2024-10-07 00:35:10,510] ERROR - PolicyUtil Error in deploying execution plan
org.wso2.carbon.event.processor.core.exception.ExecutionPlanConfigurationException: Couldn't parse execution plan:
		@Plan:name('carbon.super_sub_AIRateLimitingPolicy')
		@Plan:description('ExecutionPlan for sub_AIRateLimitingPolicy')
		@Import('org.wso2.throttle.processed.request.stream:1.0.0')
		define stream RequestStream (messageID string, appKey string, appTier string, subscriptionKey string, apiKey string, apiTier string, subscriptionTier string, resourceKey string, resourceTier string, userId string,  apiContext string, apiVersion string, appTenant string, apiTenant string, appId string, apiName string, propertiesMap string);

		@Export('org.wso2.throttle.globalThrottle.stream:1.0.0')
		define stream GlobalThrottleStream (throttleKey string, isThrottled bool, expiryTimeStamp long);

		FROM RequestStream
		SELECT messageID, (apiTenant == 'carbon.super' and subscriptionTier == 'AIRateLimitingPolicy') AS isEligible, subscriptionKey AS throttleKey, propertiesMap
		INSERT INTO EligibilityStream;

				FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min, 0)
		select throttleKey,
				ifThenElse((count(messageID) >= 300), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'PROMPT_TOKENS'),'long')) >= 600L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'COMPLETION_TOKENS'),'long')) >= 400L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'TOTAL_TOKENS'),'long')) >= 1000L), true,
						false)
				)
						)
						)
						)
				as isThrottled, expiryTimeStamp
		group by throttleKey
		INSERT ALL EVENTS into ResultStream;

		from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
		select *
		insert into GlobalThrottleStream;

	at org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorService.deployExecutionPlan(CarbonEventProcessorService.java:118) ~[?:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.PolicyUtil.deployPolicy(PolicyUtil.java:134) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.ThrottlePolicyJMSMessageListener.lambda$handleNotificationMessage$0(ThrottlePolicyJMSMessageListener.java:151) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]
[2024-10-07 00:36:26,752] ERROR - PolicyUtil Error in deploying execution plan
org.wso2.carbon.event.processor.core.exception.ExecutionPlanConfigurationException: Couldn't parse execution plan:
		@Plan:name('carbon.super_sub_AIRateLimitingPolicy')
		@Plan:description('ExecutionPlan for sub_AIRateLimitingPolicy')
		@Import('org.wso2.throttle.processed.request.stream:1.0.0')
		define stream RequestStream (messageID string, appKey string, appTier string, subscriptionKey string, apiKey string, apiTier string, subscriptionTier string, resourceKey string, resourceTier string, userId string,  apiContext string, apiVersion string, appTenant string, apiTenant string, appId string, apiName string, propertiesMap string);

		@Export('org.wso2.throttle.globalThrottle.stream:1.0.0')
		define stream GlobalThrottleStream (throttleKey string, isThrottled bool, expiryTimeStamp long);

		FROM RequestStream
		SELECT messageID, (apiTenant == 'carbon.super' and subscriptionTier == 'AIRateLimitingPolicy') AS isEligible, subscriptionKey AS throttleKey, propertiesMap
		INSERT INTO EligibilityStream;

				FROM EligibilityStream[isEligible==true]#throttler:timeBatch(1 min, 0)
		select throttleKey,
				ifThenElse((count(messageID) >= 300), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'PROMPT_TOKENS'),'long')) >= 600L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'COMPLETION_TOKENS'),'long')) >= 400L), true,
						ifThenElse((sum(cast(map:get(propertiesMap,'TOTAL_TOKENS'),'long')) >= 1000L), true,
						false)
				)
						)
						)
						)
				as isThrottled, expiryTimeStamp
		group by throttleKey
		INSERT ALL EVENTS into ResultStream;

		from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
		select *
		insert into GlobalThrottleStream;

	at org.wso2.carbon.event.processor.core.internal.CarbonEventProcessorService.deployExecutionPlan(CarbonEventProcessorService.java:118) ~[?:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.PolicyUtil.deployPolicy(PolicyUtil.java:134) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at org.wso2.carbon.apimgt.throttle.policy.deployer.utils.ThrottlePolicyJMSMessageListener.lambda$handleNotificationMessage$0(ThrottlePolicyJMSMessageListener.java:151) ~[org.wso2.carbon.apimgt.throttle.policy.deployer_9.30.10.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:829) ~[?:?]

Following CTL commands work as is:

  • apictl export api (--name --version --provider --environment ) [flags]
  • apictl export policy rate-limiting (--type --environment ) [flags]

@ashera96
Copy link
Author

ashera96 commented Oct 6, 2024

@ashera96
Copy link
Author

ashera96 commented Oct 7, 2024

[Progress Update]

  • Policy import is fixed in the latest APIM 4.4.0 pack
  • API import still fails and we need to identify whether to fix the export flow or import flow. Created the issue [1] to track the fix for this.
  1. [AI API] API download and export/import related errors  #3200

@ashera96
Copy link
Author

[Progress Update]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants