fix: Ensure JSON response is properly parsed in generate function when format is set to 'json' #199
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When I set
format
as"json"
, my response looks like this:{'model': 'llama3', 'created_at': '2024-06-28T05:44:32.12715Z', 'response': '{"results": [\n{\n"entity_type": "IMEI",\n"text": "06-184755-866851-3"\n}\n]}\n\n \n\n ', 'done': True, 'done_reason': 'stop', 'context': [], 'total_duration': 4528553708, 'load_duration': 2316500, 'prompt_eval_count': 437, 'prompt_eval_duration': 2743245000, 'eval_count': 32, 'eval_duration': 1781380000}
As you can notice, although it indeed returns a JSON object, the previous code parses the
'response'
as a string (hence the presence of\n
characters and type checking result).After I added some code to correctly parse the JSON response, the response now looks like this:
{'model': 'llama3', 'created_at': '2024-06-28T09:16:56.417739Z', 'response': {'results': [{'entity_type': 'IMEI', 'text': '06-184755-866851-3'}]}, 'done': True, 'done_reason': 'stop', 'context':
It becomes cleaner and matches my expected format.