Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improving the content methods of the DefaultChatClient #1359

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

ricken07
Copy link
Contributor

Improving the content methods of the DefaultChatClient

This pull request contains a proposal for updating the content methods of the DefaultChatClient.

Improving the content methods of the DefaultChatClient

This pull request contains a proposal for updating the content methods of the DefaultChatClient class.

Here's a use case to avoid making another request on the content method:

I want to retrieve the metadata from the chat response and then use content to retrieve the final response.

In this use case, as shown in the example, I use the call method, which returns a CallResponseSpec. This CallResponseSpec allows me to retrieve the metadata by calling the chatResponse method (step 1). I then use the content method directly to obtain the final response via the spec. The current implementation makes a second request to the API to retrieve a ChatResponse that has already been obtained in step (1), which means there are two calls to the spec for a single specific action, which is excessive.

This is why I suggest checking in the spec whether a call to the model has already been made before launching a new one.

`
var responseSpec = chatclient.prompt()
.user("This is a test")
.call();

var metadata = responseSpec.chatResponse().getMetadata();

var response = responseSpec.content();
`
@tzolov this use case makes sense if we want to retrieve the metadata for billing or other purposes and use the content method to retrieve the final response directly and avoid consuming an additional request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants