This package is based on the paper Asymptotic formulae for likelihood-based tests of new physics.
(your daily dose of meme)
Well, I'm currently doing my PhD on dark matter search with Imaging Air Cherenkov Telescopes (IACTs) and during my research, I looked into a good share of DM papers... Turns out none of the ones I've read really explained how they are calculating their upper limits and most of them lack a good explanation for upper limit vs dark matter mass plots.
So to understand what's going on I went back in time...on arxiv. I thought, for sure the CERN people will know this stuff since calculating upper limits is daily business for them. I quickly found the paper mentioned above and it only took me three months to understand it.
(╯°□°)╯︵ ┻━┻
What have I learned?
- Statistical tests are difficult to understand
- The decentralized
$\chi^2$ -distribution is the final boss - A lot of researchers use a test statistic that is physically not meaningfull (signal strength can be smaller than zero)
- A lot of researchers calculate the median upper limits and bands for the expected signal by using a bunch of toy MCs, which is not necessarily needed if asymptotics are valid
- Adding Asimov datasets to gammapy ✔️
- Adding test statistics to gammapy ✔️
- Validation of test statistics ✔️
- Calculation of ULs ✔️
- Calculation of median ULs and bands with asymptotic formulae and asimov datasets (faster than toy MCs) ✔️
This is not a finished version yet and can contain drastic changes