Improving the content methods of the DefaultChatClient #1359
+41
−15
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improving the content methods of the DefaultChatClient
This pull request contains a proposal for updating the content methods of the DefaultChatClient.
Improving the content methods of the DefaultChatClient
This pull request contains a proposal for updating the content methods of the DefaultChatClient class.
Here's a use case to avoid making another request on the content method:
I want to retrieve the metadata from the chat response and then use content to retrieve the final response.
In this use case, as shown in the example, I use the
call
method, which returns aCallResponseSpec
. ThisCallResponseSpec
allows me to retrieve the metadata by calling thechatResponse
method (step 1). I then use thecontent
method directly to obtain the final response via the spec. The current implementation makes a second request to the API to retrieve aChatResponse
that has already been obtained in step (1), which means there are two calls to the spec for a single specific action, which is excessive.This is why I suggest checking in the spec whether a call to the model has already been made before launching a new one.
`
var responseSpec = chatclient.prompt()
.user("This is a test")
.call();
var metadata = responseSpec.chatResponse().getMetadata();
var response = responseSpec.content();
`
@tzolov this use case makes sense if we want to retrieve the metadata for billing or other purposes and use the content method to retrieve the final response directly and avoid consuming an additional request.