Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FR: Dunn-Šidák #179

Open
wants to merge 4 commits into
base: developer
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions src/FSharp.Stats/Testing/MultipleTesting.fs
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,16 @@ module MultipleTesting =
|> Seq.ofList


// adapted from: https://en.wikipedia.org/wiki/%C5%A0id%C3%A1k_correction & https://personal.utdallas.edu/~herve/Abdi-Bonferroni2007-pretty.pdf
// Old: /// Computes the Dunn-Šidák correction onto a collection of p-values with a given alpha. Returns a list of booleans which state if the p-value should be rejected or not, and the new critical value for this alpha.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove old comment

/// Computes the Dunn-Šidák correction onto a collection of p-values with a given alpha. Returns a critical value. p-values above the critical value should be rejected.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to not use special characters in fs or fsx files. Describe what this method correct for? (FWER?)

let dunnSidak alpha (pValues : seq<float>) =
let m = float (Seq.length pValues)
let criticalValue = 1. - (1. - alpha) ** (1. / m)
// List.map (fun p -> if p > criticalValue then true, p else false, p) (List.ofSeq pValues),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • remove this line
  • What is the default procedure if pvalues in the input sequence are nan? Are they excluded from determining m? Should there be a special case? I don't know, but i can think of reasons why to exclude nan tests and maybe it is commonly performed.

Copy link
Contributor Author

@omaus omaus Jan 28, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find any case like this for this test. So I suggest to just exclude them. Same goes for values > 1 and < 0.

criticalValue


// John D. Storey
// http://dldcc-web.brc.bcm.edu/lilab/liguow/CGI/R/library/qvalue/html/qvalue.html
// http://qvalue.princeton.edu/
Expand Down
17 changes: 17 additions & 0 deletions tests/FSharp.Stats.Tests/Testing.fs
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,23 @@ let benjaminiHochbergTests =
]


[<Tests>]
let dunnSidakTest =
// taken from: https://www.real-statistics.com/hypothesis-testing/familywise-error/bonferroni-and-dunn-sidak-tests/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The link is broken. Please provide references to papers, R, SPSS packages or similar sources

let testPvalues = [0.01208; 0.00356; 0.11542; 0.02155; 0.03329; 0.01042]
let testAlpha1 = 0.05
let testAlpha2 = 0.01
let expectedValue1 = 0.008512
let expectedValue2 = 0.001674
let observedValue1 = MultipleTesting.dunnSidak testAlpha1 testPvalues |> round 6
let observedValue2 = MultipleTesting.dunnSidak testAlpha2 testPvalues |> round 6
testList "Testing.MultipleTesting.dunnSidak" [
testCase "Alpha = 5 %" <| fun () ->
Expect.floatClose Accuracy.veryHigh observedValue1 expectedValue1 "corrected p values are about equal."
testCase "Alpha = 1 %" <| fun () ->
Expect.floatClose Accuracy.veryHigh observedValue2 expectedValue2 "corrected p values are about equal."
]


[<Tests>]
let qValuesTest =
Expand Down