Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quick fix for multi-doc YAML #321

Merged
merged 1 commit into from
Dec 15, 2023
Merged

Conversation

HebaruSan
Copy link
Member

@HebaruSan HebaruSan commented Dec 15, 2023

Problem

After the merge of KSP-CKAN/NetKAN#9882, the scheduler is throwing this:

Uncaught exception:
Traceback (most recent call last):
  File ".local/bin/netkan", line 8, in <module>
    sys.exit(netkan())
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/netkan/.local/lib/python3.7/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/home/netkan/.local/lib/python3.7/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/cli/services.py", line 63, in scheduler
    sched.schedule_all_netkans()
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/scheduler.py", line 59, in schedule_all_netkans
    for batch in sqs_batch_entries(messages):
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/common.py", line 34, in sqs_batch_entries
    for msg in messages:
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/scheduler.py", line 57, in <genexpr>
    messages = (nk.sqs_message(self.ckm_repo.highest_version(nk.identifier))
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/repos.py", line 170, in <genexpr>
    return (Netkan(f, game_id=self.game_id) for f in self.all_nk_paths())
  File "/home/netkan/.local/lib/python3.7/site-packages/netkan/metadata.py", line 38, in __init__
    self._raw = yaml.load(self.contents.replace('\t', '    '))
  File "/home/netkan/.local/lib/python3.7/site-packages/ruamel/yaml/main.py", line 451, in load
    return constructor.get_single_data()
  File "/home/netkan/.local/lib/python3.7/site-packages/ruamel/yaml/constructor.py", line 112, in get_single_data
    node = self.composer.get_single_node()
  File "_ruamel_yaml.pyx", line 718, in ruamel.yaml.clib._ruamel_yaml.CParser.get_single_node
ruamel.yaml.composer.ComposerError: expected a single document in the stream
  in "<unicode string>", line 1, column 1
but found another document
  in "<unicode string>", line 11, column 1

Cause

rueaml.yaml.load apparently only supports single documents. load_all is needed for mulitple.

Changes

Now we use load_all.

The full file contents will still be assigned to Netkan.contents, but _raw will contain only the first document (the part before the first --- in YAML). This is a quick patch just to get the scheduler online again. In the future we can consider some kind of merge logic to handle both documents, but this isn't important at this point because only a few properties are needed.

@HebaruSan HebaruSan added Bug Something isn't working Scheduler Adds netkans to the queue to be inflated labels Dec 15, 2023
@HebaruSan HebaruSan merged commit 4792389 into KSP-CKAN:master Dec 15, 2023
3 checks passed
@HebaruSan HebaruSan deleted the fix/yaml-multidoc branch December 15, 2023 19:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Scheduler Adds netkans to the queue to be inflated
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants