Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running lstchain_mc_r0_to_dl1 with default gamma_test_large.simtel.gz fails #1255

Open
morcuended opened this issue May 16, 2024 · 12 comments
Open
Labels
bug Something isn't working

Comments

@morcuended
Copy link
Member

Running lstchain_mc_r0_to_dl1 with v0.10.11 and gave this error:

Downloading gamma_test_large.simtel.gz ...

Traceback (most recent call last):
  File "/fefs/aswg/software/conda/envs/lstchain-v0.10.11/bin/lstchain_mc_r0_to_dl1", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/fefs/aswg/software/conda/envs/lstchain-v0.10.11/lib/python3.11/site-packages/lstchain/scripts/lstchain_mc_r0_to_dl1.py", line 74, in main
    r0_to_dl1.r0_to_dl1(
  File "/fefs/aswg/software/conda/envs/lstchain-v0.10.11/lib/python3.11/site-packages/lstchain/reco/r0_to_dl1.py", line 352, in r0_to_dl1
    source = EventSource(input_url=input_filename,
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/lstchain-v0.10.11/lib/python3.11/site-packages/ctapipe/io/simteleventsource.py", line 512, in __init__
    self._subarray_info = self.prepare_subarray_info(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/fefs/aswg/software/conda/envs/lstchain-v0.10.11/lib/python3.11/site-packages/ctapipe/io/simteleventsource.py", line 643, in prepare_subarray_info
    raise RuntimeError(
RuntimeError: `SimTelEventSource.focal_length_choice` was set to 'EFFECTIVE', but the effective focal length was not present in the file. Set `focal_length_choice='EQUIVALENT'` or make sure input files contain the effective focal length
@morcuended morcuended changed the title Running lstchain_mc_r0_to_dl1 with default gamma_test_large.simtel.gz gives error Running lstchain_mc_r0_to_dl1 with default gamma_test_large.simtel.gz fails May 16, 2024
@morcuended morcuended added the bug Something isn't working label May 16, 2024
@maxnoe
Copy link
Member

maxnoe commented May 16, 2024

Could be fixed by either

  • Requiring an inputfile and not using the old prod3 file as default
  • Changing the default file to gamma_prod5.simtel.zst which does have the effective focal length

@morcuended
Copy link
Member Author

With this file gamma_prod5.simtel.zst I got No events in file

@maxnoe
Copy link
Member

maxnoe commented May 16, 2024

There might be no LST 1 events in there, I guess the default is aalowedtels=[1]?

@morcuended
Copy link
Member Author

morcuended commented May 16, 2024

There might be no LST 1 events in there, I guess the default is aalowedtels=[1]?

yes, then I suggest to require the user explicitly an input file

@vuillaut
Copy link
Member

There might be no LST 1 events in there, I guess the default is aalowedtels=[1]?

yes, then I suggest to require the user explicitly an input file

agreed

@vuillaut
Copy link
Member

Note that you could also add to the config:

"source_config": {
    "EventSource": {
        "focal_length_choice": "EQUIVALENT"
    }
}

@maxnoe
Copy link
Member

maxnoe commented Sep 10, 2024

@vuillaut no! we should always use EFFECTIVE. All our current simulations files should have the correct effective focal length specified.

@vuillaut
Copy link
Member

@vuillaut no! we should always use EFFECTIVE. All our current simulations files should have the correct effective focal length specified.

I agree, if one wants meaningful results and if one has correct simulations files easily accessible.
However, having the possibility for users to quickly run a command and see how it behaves is also important and does not have to be 100% accurate.

@maxnoe
Copy link
Member

maxnoe commented Sep 11, 2024

However, having the possibility for users to quickly run a command and see how it behaves is also important and does not have to be 100% accurate.

But teaching people to just set an option that will result in a reconstruction bias is quite dangerous. We should replace the gamma_test_large.simtel.gz everywhere we no use it as an example or default with a better file.

We produced one some time ago following the LST simtel config, but I guess we never started to use it.

It's in the non-public test data:

test_data/mc/simtel_theta_20_az_180_gdiffuse_10evts.simtel.gz

@vuillaut
Copy link
Member

It's in the non-public test data:

test_data/mc/simtel_theta_20_az_180_gdiffuse_10evts.simtel.gz

The non-public part for such a test file is really a drawback. What is blocking to make it public? Aren't the whole chain and settings to produce it already public?

@maxnoe
Copy link
Member

maxnoe commented Sep 11, 2024

Aren't the whole chain and settings to produce it already public?

yes

What is blocking to make it public?

Nothing in particular I guess, just that we don't have easy access to add new files to the dataserver that hosts the files that makes ctapipe.utils.get_dataset_path() work, it's a legacy system hosted by in2p3 and the ctao system that should replace it is not in place yet and I asked for a whole for an LST solution to hosting test files (both private and public) but that's also not yet setup.

@vuillaut
Copy link
Member

Aren't the whole chain and settings to produce it already public?

yes

What is blocking to make it public?

Nothing in particular I guess, just that we don't have easy access to add new files to the dataserver that hosts the files that makes ctapipe.utils.get_dataset_path() work, it's a legacy system hosted by in2p3 and the ctao system that should replace it is not in place yet and I asked for a whole for an LST solution to hosting test files (both private and public) but that's also not yet setup.

wouldn't a simple github repo works? to be deleted once we have a more elegant solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants