From 189a2386efe76add3a4fe2b2ad48e264444a1f75 Mon Sep 17 00:00:00 2001 From: Zhongpin Wang Date: Wed, 2 Oct 2024 12:18:07 +0200 Subject: [PATCH] chore: Docs add model version and api version (#188) * docs: add model version for orchestration * docs: add api version and request config * docs: add custom request config to orchestration * fix: Changes from lint * docs: remove api version configuration * docs: change headers in foundation-models --------- Co-authored-by: cloud-sdk-js --- packages/foundation-models/README.md | 56 ++++++++++++++++---------- packages/orchestration/README.md | 59 +++++++++++++++++++++------- 2 files changed, 80 insertions(+), 35 deletions(-) diff --git a/packages/foundation-models/README.md b/packages/foundation-models/README.md index f94ab181..198313ce 100644 --- a/packages/foundation-models/README.md +++ b/packages/foundation-models/README.md @@ -4,15 +4,15 @@ This package incorporates generative AI foundation models into your AI activitie ## Table of Contents -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Usage](#usage) - - [Client Initialization](#client-initialization) - - [Azure OpenAI Client](#azure-openai-client) - - [Chat Client](#chat-client) - - [Embedding Client](#embedding-client) -4. [Support, Feedback, Contribution](#support-feedback-contribution) -5. [License](#license) +- [Table of Contents](#table-of-contents) +- [Installation](#installation) +- [Prerequisites](#prerequisites) +- [Usage](#usage) + - [Client Initialization](#client-initialization) + - [Chat Client](#chat-client) + - [Embedding Client](#embedding-client) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation @@ -60,26 +60,38 @@ const chatClient = new AzureOpenAiChatClient({ }); ``` -### Azure OpenAI Client +### Chat Client -The Azure OpenAI client can then be used to send chat completion or embedding requests to models deployed in the SAP generative AI hub. +Use the `AzureOpenAiChatClient` to send chat completion requests to an OpenAI model deployed in SAP generative AI hub. -#### Chat Client +The client sends request with Azure OpenAI API version `2024-06-01`. -Use the `AzureOpenAiChatClient` to send chat completion requests to an OpenAI model deployed in SAP generative AI hub. +Set request configuration in the `requestConfig` parameter. The following example shows a call with messages and optional request configuration. ```ts import { AzureOpenAiChatClient } from '@sap-ai-sdk/foundation-models'; const chatClient = new AzureOpenAiChatClient('gpt-4o'); -const response = await chatClient.run({ - messages: [ - { - role: 'user', - content: 'Where is the deepest place on earth located' +const response = await chatClient.run( + { + messages: [ + { + role: 'user', + content: 'Where is the deepest place on earth located' + } + ] + }, + { + headers: { + 'x-custom-header': 'custom-value' + // Add more headers here + }, + params: { + // Add more parameters here } - ] -}); + // Add more request configuration here + } +); const responseContent = response.getContent(); ``` @@ -124,7 +136,7 @@ logger.info( Refer to `AzureOpenAiChatCompletionParameters` interface for other parameters that can be passed to the chat completion request. -#### Embedding Client +### Embedding Client Use the `AzureOpenAiEmbeddingClient` to send embedding requests to an OpenAI model deployed in SAP generative AI hub. @@ -140,6 +152,8 @@ const response = await embeddingClient.run({ const embedding = response.getEmbedding(); ``` +Define optional request configuration when running the client, similar to the chat client. + ## Support, Feedback, Contribution This project is open to feature requests/suggestions, bug reports etc. via [GitHub issues](https://github.com/SAP/ai-sdk-js/issues). diff --git a/packages/orchestration/README.md b/packages/orchestration/README.md index 58f1b967..b9fb33f8 100644 --- a/packages/orchestration/README.md +++ b/packages/orchestration/README.md @@ -2,16 +2,18 @@ This package incorporates generative AI orchestration capabilities into your AI activities in SAP AI Core and SAP AI Launchpad. -## Table of Contents - -1. [Installation](#installation) -2. [Prerequisites](#prerequisites) -3. [Orchestration Service](#orchestration-service) -4. [Usage](#usage) - - [Templating](#templating) - - [Content Filtering](#content-filtering) -5. [Support, Feedback, Contribution](#support-feedback-contribution) -6. [License](#license) +### Table of Contents + +- [Installation](#installation) + - [Prerequisites](#prerequisites) +- [Orchestration Service](#orchestration-service) +- [Usage](#usage) + - [Templating](#templating) + - [Token Usage](#token-usage) + - [Content Filtering](#content-filtering) + - [Data Masking](#data-masking) +- [Support, Feedback, Contribution](#support-feedback-contribution) +- [License](#license) ## Installation @@ -39,12 +41,30 @@ Find more details about orchestration workflow [here](https://help.sap.com/docs/ ## Usage Leverage the orchestration service capabilities by using the orchestration client. -The client allows you to configure various modules, such as templating and content filtering, while sending chat completion requests to an orchestration-compatible generative AI model. +Configure the LLM module by setting the `model_name` and `model_params` properties. +Define the optional `model_version` property to choose an available model version. +By default, the version is set to `latest`. + +```ts +import { OrchestrationClient } from '@sap-ai-sdk/orchestration'; + +const orchestrationClient = new OrchestrationClient({ + llm: { + model_name: 'gpt-4-32k', + model_params: { max_tokens: 50, temperature: 0.1 } + model_version: 'latest' + }, + ... +}); +``` + +The client allows you to combine various modules, such as templating and content filtering, while sending chat completion requests to an orchestration-compatible generative AI model. ### Templating Use the orchestration client with templating to pass a prompt containing placeholders that will be replaced with input parameters during a chat completion request. This allows for variations in the prompt based on the input parameters. +Set custom request configuration when calling `chatCompletion` function. ```ts import { OrchestrationClient } from '@sap-ai-sdk/orchestration'; @@ -61,9 +81,20 @@ const orchestrationClient = new OrchestrationClient({ } }); -const response = await orchestrationClient.chatCompletion({ - inputParams: { country: 'France' } -}); +const response = await orchestrationClient.chatCompletion( + { + inputParams: { country: 'France' } + }, + { + headers: { + // Add more headers here + }, + params: { + // Add more parameters here + } + // Add more request configuration here + } +); const responseContent = response.getContent(); ```