How to add fuctions for Ollama models

[ogbozoyan]

I've saw in 1.0.0-M3 added possibility to add functions for model, but didn't find how to "to teach" model "hey buddy you you can use this functions"

Here's my config

@Configuration
class AiModelConfiguration(
    private val chatClientBuilder: ChatClient.Builder,
    @Value("\${app.prompt.system-message}") val systemMessage: String,
    private val ollamaApi: OllamaApi
) {

    @Bean
    fun chatMemory(): ChatMemory {
        return InMemoryChatMemory()
    }

    @Bean
    fun chatClient(): ChatClient {
        TODO("ADD TOOL(FUNCTION) CONTEXT TO MODEL")
        return chatClientBuilder
            .defaultSystem(systemMessage)
            .defaultAdvisors(
                MessageChatMemoryAdvisor(chatMemory(), DEFAULT_CHAT_MEMORY_CONVERSATION_ID, 5),
                SimpleLoggerAdvisor()
            )
            .build()
    }

    @Bean
    fun vectorStore(embeddingModel: EmbeddingModel): VectorStore {
        return SimpleVectorStore(embeddingModel)
    }
}    
--------------

@Service
class AiDataProvider @Autowired constructor(
    @Qualifier("vectorStore") private val vectorStore: VectorStore
) {

    private val log = LoggerFactory.getLogger(AiDataProvider::class.java)

    init {
        log.info("Connected vector store: ${vectorStore.name}")
    }
}
-------------

@Configuration
class AiFunctionConfiguration {

    // The @Description annotation helps the model understand when to call the function
    @Bean
    @Description("test function for testing purpose")
    fun testFunction(aiDataProvider: AiDataProvider): java.util.function.Function<Any, Any> {
        return java.util.function.Function<Any, Any> {

        }
    }
}
---------------

spring:
  ai:
    ollama:
      base-url: ${OLLAMA_BASE_URL:http://localhost:11434/}
      chat:
        options:
          model: ${OLLAMA_CHAT_MODEL:deepseek-coder-v2:16b}
          temperature: ${OLLAMA_CHAT_TEMPERATURE:0.5}
          functions: TODO()
app:
  prompt:
    system-message: ${APP_PROMPT_SYSTEM-MESSAGE:"TODO ADD PROMPT"}

[ThomasVitale]

You can either pass the function name via the ChatClient API (example: https://github.com/ThomasVitale/llm-apps-java-spring-ai/blob/main/05-function-calling/function-calling-ollama/src/main/java/com/thomasvitale/ai/spring/ChatService.java) or via configuration properties.

If you use properties, based on your example, you would configure the function name as follows, where "testFunction" is the name of the method representing the function you want to make available to the model.

spring:
  ai:
    ollama:
      chat:
        options:
          functions:
            - "testFunction"

[ogbozoyan]

testFunction name should be equal as real fun name ?

[ThomasVitale]

Yes, it should be the same

[ogbozoyan]

Thanks, how to make sure that ai will call or ignore function ?

[ogbozoyan]

I've added functions, but when i'm observing

chatClient: ChatClient
in function fields theres not defined function

chatClient = {DefaultChatClient@11297} 
 defaultChatClientRequest = {DefaultChatClient$DefaultChatClientRequestSpec@11305} 
  observationRegistry = {SimpleObservationRegistry@8903} 
  customObservationConvention = null
  chatModel = {OllamaChatModel@11308} 
   chatApi = {OllamaApi@11328} 
   defaultOptions = {OllamaOptions@11329} 
   observationRegistry = {SimpleObservationRegistry@8903} 
   observationConvention = {DefaultChatModelObservationConvention@11330} 
   functionCallbackRegister = {ConcurrentHashMap@11331}  size = 0
   functionCallbackContext = {FunctionCallbackContext@11332} 
  userText = ""
  systemText = "TODO ADD PROMPT"
  chatOptions = {OllamaOptions@11311} 
  media = {ArrayList@11312} "[]"
  functionNames = {ArrayList@11313} "[]"
  functionCallbacks = {ArrayList@11314} "[]"
  messages = {ArrayList@11315} "[]"
  userParams = {HashMap@11316}  size = 0
  systemParams = {HashMap@11317}  size = 0
  advisors = {ArrayList@11318} "[org.springframework.ai.chat.client.DefaultChatClient$DefaultChatClientRequestSpec$1@651a9bdd, org.springframework.ai.chat.client.DefaultChatClient$DefaultChatClientRequestSpec$2@775675af, org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor@7a3e5425, SimpleLoggerAdvisor]"
  advisorParams = {HashMap@11319}  size = 0
  aroundAdvisorChainBuilder = {DefaultAroundAdvisorChain$Builder@11320} 
  toolContext = {HashMap@11321}  size = 0

spring:
  datasource:
    url: ${DATASOURCE_URL}
    username: ${DATASOURCE_USERNAME}
    password: ${DATASOURCE_PASSWORD}
  ai:
    ollama:
      base-url: ${OLLAMA_BASE_URL:http://localhost:11434/}
      embedding:
        options:
          model: nomic-embed-text
      chat:
        options:
          model: ${OLLAMA_CHAT_MODEL:deepseek-coder-v2:16b}
          temperature: ${OLLAMA_CHAT_TEMPERATURE:0.5}
          functions:
            - "testFunction"
      # Not sure about this configuration https://docs.rapidapp.io/blog/integrating-spring-ai-with-vector-databases
      vectorstore:
        pgvector:
          index-type: hnsw
          distance-type: cosine_distance
          dimensions: 384

@Bean
    fun chatClient(): ChatClient {
        return chatClientBuilder
            .defaultSystem(systemMessage)
            .defaultAdvisors(
                MessageChatMemoryAdvisor(chatMemory(), DEFAULT_CHAT_MEMORY_CONVERSATION_ID, 5),
                SimpleLoggerAdvisor()
            )
            .build()
    }

[ogbozoyan]

@ThomasVitale Thank's, i've fixed, your advice helped)