Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

io.test does not fail if term doesn't exist #5399

Open
ceedubs opened this issue Oct 8, 2024 · 0 comments
Open

io.test does not fail if term doesn't exist #5399

ceedubs opened this issue Oct 8, 2024 · 0 comments

Comments

@ceedubs
Copy link
Contributor

ceedubs commented Oct 8, 2024

Describe and demonstrate the bug

In a transcript if you call io.test and a test fails, then it errors with a non-zero exit code. However, if you call io.test with a term that doesn't exist, it returns a success (0) exit code. This can mask errors (such as in CI) where you think that tests are passing but in fact they aren't running.

Input:

```ucm
fresh/main> io.test doesNotExist
```

Output:

``` ucm
fresh/main> io.test doesNotExist

    New test results:
  
  😶 No tests available.

```

You can see in the output that it detects that there are no tests. What you can't see from the transcript output is that the transcript command exits with 0.

I think that the following should all probably be considered errors:

  • Passing in a nonexistent term
  • Passing in a term with the wrong type
  • Zero tests running (even if the term exists and has the right type, if it returns an empty test result list then something is probably off). But I'm less sure about this one.

Environment (please complete the following information):

  • ucm --version 5729439
  • OS/Architecture: x86 Linux

Additional context
Add any other context about the problem here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants