Adapt IT1788 tests to use Base.Test and run them in parallel #235
Conversation
Note that in ITF1788_tests.jl does not yet include running the tests in ieee1788-constructors.jl; some of these tests do not pass yet, since some functionality is lacking.
|
This completes #205 |
|
Checking what went wrong... |
|
This is great work! What does addprocs() with no arguments do? |
|
The failing tests seem to be since a file is missing. |
|
Thanks! It adds all possible workers of your machine (worker 1 is master), so you use them all to do the tests. |
|
Some tests, not (yet) related to the ITF1788 suite are not passing in travis. |
|
Sorry, it is indeed related to the ITF1788 suite.... |
|
It looks like you forgot to commit the bool test file? |
|
But I see it there. Maybe a question of path in travis...? |
|
Not sure. |
|
Indeed, it is the path: It is looking for "/home/travis/.julia/v0.5/ValidatedNumerics/test/libieeep1788_tests_bool.jl" while it should be "/home/travis/.julia/v0.5/ValidatedNumerics/test/ITF1788_tests/libieeep1788_tests_bool.jl" |
|
Ah OK, so you can just add the directory name. Or. Maybe do a "cd". It is maybe worth looking in the Julia base tests to see how they do it to be compatible with Windows at some point in the future. Or just leave it unixy for now for simplicity. |
|
Ah you can just use joinpath |
|
I just pushed a new commit that should solve this; if you tell me how to use it, I do change it |
|
Another try... |
|
Tests are running.... finally! |
|
Great! Is it possible to parallelise the non itf tests (ours) too? |
|
The scheme we have now is pretty naive: each worker includes a test file independently. This can be extended for all included test files, but we must be sure about two things: the master (worker 1) has to be in charge of the very first test, since it has to precompile. The second thing is to have all workers do the ITF1788 tests, which take really long. Shall I implement this in this PR? |
Codecov Report
@@ Coverage Diff @@
## master #235 +/- ##
=========================================
- Coverage 91.85% 0% -91.86%
=========================================
Files 26 26
Lines 1044 1021 -23
=========================================
- Hits 959 0 -959
- Misses 85 1021 +936
Continue to review full report at Codecov.
|
|
I suggest you make a new pr in a new branch based in this branch. In that way we can easily see. If it has any effect on the total time to run the tests |
|
Travis is giving problems: while tests pass in 0.4, they take too long in 0.5; in 0.6, I think the problem is different... |
|
Maybe we shouldn't wait for this to be ready for 0.7... |
|
Locally, it seems something happens with the "minimal_pow_test"... they are many, but are taking really too long. |
|
0.6 will require StaticArrays.jl after we release 0.7 I don't think the problem on 0.5 was that it took too long. It seems not to have realised that the tests finished. |
|
Yes that one was always problematical. One solution is to split it into
several files. The problem is that that is not so easy to do automatically.
We could write a script to do it.
…On 5 Mar 2017 2:34 p.m., "Luis Benet" ***@***.***> wrote:
Locally, it seems something happens with the "minimal_pow_test"... they
are many, but are taking really too long.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#235 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AALtTno2GtD_gJdZcOprjXYTcuI0xPTJks5rixxvgaJpZM4MTg5i>
.
|
|
No. travis says that it doesn't receive any output for more than 10 min. Yet, the next item ("minimal_fma_test") is not displayed, so it was in process. It corresponds to 564 tests... |
|
Thanks! See also #236, with similar problems... |
|
The problem as usual seems to be the huge size of the |
|
Locally, they work. But they take an enormous amount of time to complete. May this be related to Base.Test? |
|
We may still use FactCheck.jl, but implement that in parallel... |
|
Why not breaking those tests in few files? |
|
Yes, breaking up the files would be the best solution. But it seems difficult to automate with ITF1788? |
|
I agree... |
|
I'm not sure what are you aiming... |
|
I'm trying to make travis wait longer and not kill the tests when there's
no output. It's not working yet though.
…On 5 Mar 2017 7:05 p.m., "Luis Benet" ***@***.***> wrote:
I'm not sure what are you aiming...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#235 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AALtTvlbavBTf4l2jOP5bhTSyMxBaDEfks5ri1v1gaJpZM4MTg5i>
.
|
|
What ever you did seems to have worked! You should push another commit to get the correct code-coverage |
|
No, it's not actually running the tests... |
|
Given the unbelievably long time that the ITF1788 tests require with I was pointed to https://github.com/JuliaStats/Distributions.jl/blob/master/test/runtests.jl |
|
I've just checked the way they parallelize and the sole difference is that they use |
|
What is the status of this? |
|
I am not quite sure, but I think this is a leftover when |
Note that in ITF1788_tests.jl does not yet include running the tests in
ieee1788-constructors.jl; some of these tests do not pass yet, since
some functionality is lacking.