Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reranking eval failing #1117

Open
riyajatar37003 opened this issue Sep 23, 2024 · 0 comments
Open

reranking eval failing #1117

riyajatar37003 opened this issue Sep 23, 2024 · 0 comments

Comments

@riyajatar37003
Copy link

i am using mteb==1.1.0, for eval reranker model but its showing following errorlog

Reranking
- AskUbuntuDupQuestions, s2s
- MindSmallReranking, s2s
- SciDocsRR, s2s
- StackOverflowDupQuestions, s2s

test.jsonl.gz: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 135k/135k [00:00<00:00, 4.76MB/s]
Evaluating: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 361/361 [00:00<00:00, 214559.12it/s]
Evaluating: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 361/361 [00:00<00:00, 1437.65it/s]
README.md: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 201/201 [00:00<00:00, 1.62MB/s]
Repo card metadata block was not found. Setting CardData to empty.
train.jsonl.gz: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 304M/304M [00:07<00:00, 42.9MB/s]
test.jsonl.gz: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 118M/118M [00:02<00:00, 44.5MB/s]
Failed to load JSON from file 'gzip://train.jsonl::/tmp/.cache/huggingface/hub/datasets--mteb--mind_small/snapshots/3bdac13927fdc888b903db93b2ffdbd90b295a69/train.jsonl.gz' with error <class 'pyarrow.lib.ArrowInvalid'>: JSON parse error: Invalid value. in row 0
Error while evaluating MindSmallReranking: An error occurred while generating the dataset
Traceback (most recent call last):
File "/tmp/.local/lib/python3.10/site-packages/datasets/packaged_modules/json/json.py", line 160, in _generate_tables
df = pandas_read_json(f)
File "/tmp/.local/lib/python3.10/site-packages/datasets/packaged_modules/json/json.py", line 38, in pandas_read_json
return pd.read_json(path_or_buf, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/pandas/io/json/_json.py", line 804, in read_json
return json_reader.read()
File "/opt/conda/lib/python3.10/site-packages/pandas/io/json/_json.py", line 1014, in read
obj = self._get_object_parser(self.data)
File "/opt/conda/lib/python3.10/site-packages/pandas/io/json/_json.py", line 1040, in _get_object_parser
obj = FrameParser(json, **kwargs).parse()
File "/opt/conda/lib/python3.10/site-packages/pandas/io/json/_json.py", line 1173, in parse
self._parse()
File "/opt/conda/lib/python3.10/site-packages/pandas/io/json/_json.py", line 1366, in _parse
ujson_loads(json, precise_float=self.precise_float), dtype=None
ValueError: Unexpected character found when decoding 'true'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/tmp/.local/lib/python3.10/site-packages/datasets/builder.py", line 1853, in _prepare_split_single
for _, table in generator:
File "/tmp/.local/lib/python3.10/site-packages/datasets/packaged_modules/json/json.py", line 163, in _generate_tables
raise e
File "/tmp/.local/lib/python3.10/site-packages/datasets/packaged_modules/json/json.py", line 137, in _generate_tables
pa_table = paj.read_json(
File "pyarrow/_json.pyx", line 308, in pyarrow._json.read_json
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: JSON parse error: Invalid value. in row 0

The above exception was the direct caus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant