Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

LLama-3.2-1B-Instruct Function Calling error #994

Open
madroidmaq opened this issue Oct 18, 2024 · 1 comment
Open

LLama-3.2-1B-Instruct Function Calling error #994

madroidmaq opened this issue Oct 18, 2024 · 1 comment
Labels
platform:mobile

Comments

@madroidmaq
Copy link

Describe the bug

When I use the FunctionCalling feature of the LLama-3.2-1B-Instruct model on an Android device, an error occurs.

The prompt is as follows:

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Environment: ipython
Cutting Knowledge Date: December 2023
Today Date: 18 Oct 2024

Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous.<|eot_id|><|start_header_id|>user<|end_header_id|>

Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.

Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}.Do not use variables.

{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The location to get the weather for, e.g. San Francisco, CA"
                },
                "format": {
                    "type": "string",
                    "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
                    "enum": [
                        "celsius",
                        "fahrenheit"
                    ]
                }
            },
            "required": [
                "location",
                "format"
            ]
        }
    }
}

What's the weather like today<|eot_id|><|start_header_id|>assistant<|end_header_id|>

The error message is as follows:

                                 What is the weather today<|eot_id|><|start_header_id|>assistant<|end_header_id|>
02:09:04.367 libc             A  Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xb400006cc0dd2040 in tid 29979 (DefaultDispatch), pid 29931 (ative.app:genai)
02:09:04.737 DEBUG            A  Cmdline: ai.allama.generative.app:genai
02:09:04.737 DEBUG            A  pid: 29931, tid: 29979, name: DefaultDispatch  >>> ai.allama.generative.app:genai <<<
02:09:04.737 DEBUG            A        #00 pc 0000000000b4531c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #01 pc 0000000000e3952c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #02 pc 0000000000e35b10  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #03 pc 0000000000e32cf0  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #04 pc 0000000000e328dc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #05 pc 0000000000b43944  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #06 pc 0000000000b40c48  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #07 pc 0000000000cd46e8  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #08 pc 0000000000cd37a4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #09 pc 0000000000cefb30  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #10 pc 0000000000cd609c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #11 pc 0000000000d0b788  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #12 pc 0000000000d0b2b4  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #13 pc 00000000004db874  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #14 pc 00000000004ddb78  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #15 pc 00000000004a7dfc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime.so (offset 0xe7c000) (BuildId: e7475f852a0942724c0c168300dba612acdd4e7a)
02:09:04.737 DEBUG            A        #16 pc 0000000000147ccc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::State::Run(OrtSession&, OrtRunOptions&, int)+508) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #17 pc 0000000000138e28  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::DecoderOnly_State::Run(int, Generators::RoamingArray<int>, Generators::RoamingArray<int>)+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #18 pc 000000000012c190  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (Generators::Generator::ComputeLogits()+140) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #19 pc 000000000016ae7c  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai.so (offset 0x178000) (OgaGenerator_ComputeLogits+28) (BuildId: 3e67de3f11079c80638c65d55c883b0654441892)
02:09:04.737 DEBUG            A        #20 pc 00000000000047bc  /data/app/~~trduqFLKkQml631Cpq3c9g==/ai.allama.generative.app-rbOD3z6mK6UgcYbZjQxvAw==/base.apk!libonnxruntime-genai-jni.so (offset 0x110000) (Java_ai_onnxruntime_genai_Generator_computeLogitsNative+20) (BuildId: a4099e8a78f869b68088fe2ce0e0af6aab84894a)
02:09:04.737 DEBUG            A        #31 pc 000000000000606c  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxModel.generateText+0)
02:09:04.737 DEBUG            A        #37 pc 0000000000004bf0  /data/data/ai.allama.generative.app/code_cache/.overlay/base.apk/classes6.dex (ai.allama.generative.service.onnx.OnnxChatCompletion$chat$1.invokeSuspend+0)
---------------------------- PROCESS ENDED (29931) for package ai.allama.generative.app ----------------------------

Screenshots

Image

Desktop (please complete the following information):

  • OS: macOS M3
  • Browser：chrome
  • Version： 14.6.1

Smartphone (please complete the following information):

  • Device: Android simulators
  • OS: Android
  • Browser： chrome
  • Version ：Android 15

Additional context

Everything works fine when I don't include FunctionCalling content in my normal use.
@saibhaskerraju
Copy link

I dont think ONNX runtime supports function calling. I searched their API docs but no reference

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
platform:mobile
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@madroidmaq @saibhaskerraju