-
Notifications
You must be signed in to change notification settings - Fork 0
/
stats.html
1815 lines (1695 loc) · 74.1 KB
/
stats.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Basics</title>
<meta name="generator" content="Org mode">
<meta name="author" content="George Kontsevich">
<meta name="description" content="Basics in Stats using Octave/MATLAB"
>
<link rel="stylesheet" type="text/css" href="../web/worg.css" />
<link rel="shortcut icon" href="../web/panda.svg" type="image/x-icon">
</head>
<body>
<div id="content">
<h1 class="title">Basics</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#org5ac9d02">Intro</a></li>
<li><a href="#orgf6deb6f">Continious random variables</a></li>
<li><a href="#org7adbf62">Probability density functions (PDFs)</a>
<ul>
<li><a href="#org32b487e">Example: Normal distribution</a></li>
<li><a href="#org1ecfa53">Probability densities</a></li>
<li><a href="#org46baf89">Estimating a PDF</a></li>
<li><a href="#orgea681fb">Example: Estimating the Normal</a></li>
</ul>
</li>
<li><a href="#org8ee8f3e">Descriptive statistics</a>
<ul>
<li>
<ul>
<li><a href="#orgdaeaac3">Expected Value/Mean</a></li>
<li><a href="#org69339b3">Variance</a></li>
<li><a href="#orgc4e8c40">Standard deviation</a></li>
</ul>
</li>
<li><a href="#org82c6112">Estimating the Normal's descriptive statistics</a>
<ul>
<li><a href="#org043132c">Mean of the Normal</a></li>
<li><a href="#org4c2d329">Standard deviation of the normal</a></li>
</ul>
</li>
<li><a href="#orgf6a0a23">SigFigs</a></li>
<li><a href="#org8d7ee41">Adding two random variables</a>
<ul>
<li><a href="#org81b1c18">Expected Value</a></li>
<li><a href="#orgaffa2e4">Variance</a></li>
<li><a href="#org211f5f3">Standard Deviation</a></li>
<li><a href="#org0d67699">SigFigs</a></li>
<li><a href="#orga1ba726">Example: Standard deviation of the mean (SDOM) for a normal distribution</a></li>
</ul>
</li>
<li><a href="#org4dc42f6">Multiplying two random variables</a>
<ul>
<li><a href="#org4647717">Expected Value</a></li>
<li><a href="#org2ae08f5">Variance</a></li>
<li><a href="#org0cd8e57">Standard Deviation</a></li>
<li><a href="#org87de359">SigFigs</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#orgf5e3c80">Other Topics</a>
<ul>
<li><a href="#org84d2205">Data Rejection</a>
<ul>
<li><a href="#org1d664dc">Chauvenet's Criterion</a></li>
</ul>
</li>
<li><a href="#org1c6fc4f">Weighted Average</a>
<ul>
<li><a href="#org233602a">Mean</a></li>
<li><a href="#org3752565">Standard Deviation</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#org83847d2">Multiple Variables</a>
<ul>
<li><a href="#org6e57917">Covariance</a>
<ul>
<li><a href="#orgec3a23a">Example</a></li>
<li><a href="#org1222066">Improved!</a></li>
</ul>
</li>
<li><a href="#orgede6e65">Correlation Coefficient</a>
<ul>
<li><a href="#org804f4f1">Example</a></li>
<li><a href="#org1be3ac5">Improved!</a></li>
<li><a href="#org3b47044">Confidence</a></li>
</ul>
</li>
<li><a href="#org3cb1135">Functions of two variables</a>
<ul>
<li><a href="#org6a14d9d">Mean</a></li>
<li><a href="#org82116f4">Variance</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#orgca534bf">Binomial Distribution</a>
<ul>
<li><a href="#orgc1ebdc7">Binomial Coefficient</a></li>
<li><a href="#org2f39609">Coin toss</a></li>
<li><a href="#org15cfecc">Gaussian approximation</a>
<ul>
<li><a href="#org56dadda">Descriptive Statistics</a></li>
<li><a href="#orgfa2c846">Mean</a></li>
<li><a href="#org91dbba5">Standard Deviation</a></li>
</ul>
</li>
<li><a href="#orga8b002a">Random errors</a>
<ul>
<li><a href="#org68a9613">Null hypothesis test</a></li>
</ul>
</li>
</ul>
</li>
<li><a href="#org888a630">Poisson Distribution</a>
<ul>
<li><a href="#org4685be1">Mean</a></li>
<li><a href="#org960de03">Standard Deviation</a></li>
<li><a href="#org087bcad">Gaussian</a></li>
<li><a href="#org4942943">Subtracting a background</a></li>
</ul>
</li>
<li><a href="#org7994465">Chi-Squared</a>
<ul>
<li><a href="#org73e4720">Degrees of Freedom</a></li>
<li><a href="#orgaa6e558">Reduced <b>χ<sup>2</sup></b></a></li>
<li><a href="#org8f9334e">General equation</a>
<ul>
<li><a href="#org2f7a2f5">Example</a></li>
</ul>
</li>
</ul>
</li>
</ul>
</div>
</div>
<div id="outline-container-org5ac9d02" class="outline-2">
<h2 id="org5ac9d02">Intro</h2>
<div class="outline-text-2" id="text-org5ac9d02">
<p>
A digest of basic statistics from reading John Taylor's <i>An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements</i>
</p>
<p>
The book generally works "backwards": Starting from useful principles of working with uncertain values and then working through the justification and proofs. Since this is a quick digest and personal referesher/reference, the order is opposite of that presented in the book. When it makes sense I had inserted some MATLAB/Octave code to demonstrate things
</p>
</div>
</div>
<div id="outline-container-orgf6deb6f" class="outline-2">
<h2 id="orgf6deb6f">Continious random variables</h2>
<div class="outline-text-2" id="text-orgf6deb6f">
<p>
Are variables that when measured give a value of infinite precision. So no number can be measured twice
</p>
</div>
</div>
<div id="outline-container-org7adbf62" class="outline-2">
<h2 id="org7adbf62">Probability density functions (PDFs)</h2>
<div class="outline-text-2" id="text-org7adbf62">
<p>
Probability density functions (PDFs) are the basic functions that describe a continious random variable. The basic property of a PDF is that it can be integrated between any two value. The resulting value is the probability that your measurement will appear between the two values. Since each measurements yields <i>some</i> value, integrate between -∞ and ∞ must be equal to 1 (ie. 100%). The condition places a constraint on which functions are valid PDFs.
</p>
<div class="org-src-container">
<pre class="src src-octave">plot_range <span style="color: #483d8b;">=</span> 5
sigma <span style="color: #483d8b;">=</span> 1
center <span style="color: #483d8b;">=</span> 3
tau <span style="color: #483d8b;">=</span> 3.5
t <span style="color: #483d8b;">=</span> [<span style="color: #483d8b;">-</span>10<span style="color: #483d8b;">*</span>plot_range<span style="color: #483d8b;">:</span>plot_range<span style="color: #483d8b;">/</span>100<span style="color: #483d8b;">:</span>plot_range]
plot(t<span style="color: #483d8b;">,</span>(1<span style="color: #483d8b;">/</span>tau)<span style="color: #483d8b;">*</span>exp(<span style="color: #483d8b;">-</span>t<span style="color: #483d8b;">/</span>tau))
<span style="color: #b22222;">%</span><span style="color: #b22222;">axis "off"</span>
hold on<span style="color: #483d8b;">;</span>
<span style="color: #b22222;">%</span><span style="color: #b22222;">integration_xs = [a:(plot_range/50):b]</span>
<span style="color: #b22222;">%</span><span style="color: #b22222;">area(integration_xs,normal_dist(integration_xs,center,sigma), "facecolor",[0.74 0.9 1.0])</span>
print <span style="color: #8b2252;">"-S720,160"</span> <span style="color: #8b2252;">"exponential.svg"</span>
hold off<span style="color: #483d8b;">;</span>
</pre>
</div>
<div class="figure">
<p><object type="image/svg+xml" data="./exponential.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
<div id="outline-container-org32b487e" class="outline-3">
<h3 id="org32b487e">Example: Normal distribution</h3>
<div class="outline-text-3" id="text-org32b487e">
<p>
The most common PDF is the <i>normal distribution</i> and it'll serve as a frequent example
</p>
<blockquote>
<p>
[1/(σ√2π)] * e<sup>([x-X<sub>center</sub>]<sup>2</sup>)/(2σ<sup>2</sup>)</sup>
</p>
</blockquote>
<p>
It has two parameters, <b>σ</b> and <b>X<sub>center</sub></b>. Other functions may have different parameters. As we will see later, in this case these two happen to correspond to the <i>mean</i> and <i>standard deviation</i>, but this won't be the case for all PDFs.
</p>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> [points] <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">normal_dist</span>(xs<span style="color: #483d8b;">,</span> center<span style="color: #483d8b;">,</span> sigma)
normalization_factor <span style="color: #483d8b;">=</span> 1<span style="color: #483d8b;">/</span>(sigma<span style="color: #483d8b;">*</span>sqrt(2<span style="color: #483d8b;">*</span>pi))
points <span style="color: #483d8b;">=</span> normalization_factor <span style="color: #483d8b;">*</span> e<span style="color: #483d8b;">.^</span>(<span style="color: #483d8b;">-</span>((xs<span style="color: #483d8b;">-</span>center)<span style="color: #483d8b;">.^</span>2)<span style="color: #483d8b;">/</span>(2<span style="color: #483d8b;">*</span>sigma<span style="color: #483d8b;">^</span>2))
<span style="color: #a020f0;">end</span>
</pre>
</div>
<p>
We can plot it to see how it looks
</p>
<div class="figure">
<p><object type="image/svg+xml" data="./normal.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
</div>
</div>
<div id="outline-container-org1ecfa53" class="outline-3">
<h3 id="org1ecfa53">Probability densities</h3>
<div class="outline-text-3" id="text-org1ecfa53">
<p>
Again, because the random variable is continious, no value can be measured twice. We can see this in the PDF as well. If we integrate around some point x<sub>0</sub>. The integral from <i>x<sub>0</sub></i> to <i>x<sub>0</sub>+δx</i> will go to zero as <i>δx</i> goes to zero. We know that for very small integrals we can approximate them with the area of the immediate rectangle
</p>
<div class="figure">
<p><object type="image/svg+xml" data="./prob-density.svg" class="org-svg">
Sorry, your browser does not support SVG.</object>
</p>
</div>
<p>
The size of the <i>δx</i> isn't really important, but we can see that given a few points <i>x<sub>0</sub></i>, <i>x<sub>1</sub></i>, <i>x<sub>2</sub></i> .. etc. we can use their equivalent <b>probability densities</b> <i>f(x<sub>0</sub>)</i>, <i>f(x<sub>1</sub>)</i>, <i>f(x<sub>2</sub>)</i>, etc. to say in effect that one was more likely than the other
</p>
</div>
</div>
<div id="outline-container-org46baf89" class="outline-3">
<h3 id="org46baf89">Estimating a PDF</h3>
<div class="outline-text-3" id="text-org46baf89">
<p>
Often in an experimental scenario we will get a series of measurements from which we woujld like to estimate a PDF. There is typically some meta information about the process that generated the values which tells you the general formula for the PDF.
</p>
<p>
We can guess parameters for the PDF and then generate probabilities that the values landed in the vicinity of each measurment. For each measurement, the probability of a measurement being in its vicinity is the area of its equivalent skinny rectangle: <i>f(x<sub>n</sub>)δx</i>. And the cumulative probability of all the measurements landing where they did is just the product of the individual probabilities.
</p>
<blockquote>
<p>
f(x<sub>0</sub>) δx × f(x<sub>1</sub>) δx × f(x<sub>2</sub>) δx × …
</p>
</blockquote>
<p>
This ofcourse is a tiny value, that gets smaller the smaller the <i>δx</i> and the more measurements you make.
</p>
<p>
However the goal now is to adjust the PDF parameters so that this cumulative probability is maximized. Maximizing this product clearly doesn't depend on the value of <i>δx</i> you've chosen, so we can remove those
</p>
<blockquote>
<p>
f(x<sub>0</sub>) × f(x<sub>1</sub>) × f(x<sub>2</sub>) × …
</p>
</blockquote>
<p>
As is most often the case, maximization is done by differentiation and setting the result to zero.
</p>
</div>
</div>
<div id="outline-container-orgea681fb" class="outline-3">
<h3 id="orgea681fb">Example: Estimating the Normal</h3>
<div class="outline-text-3" id="text-orgea681fb">
<p>
As an example, we can plug in the previously introduced <i>normal distribution</i> for the PDF <i>f(x)</i> and then assume some measurements <i>x<sub>0</sub></i> <i>x<sub>1</sub></i> <i>x<sub>2</sub></i> ..
We know the PDF equation:
</p>
<blockquote>
<p>
f(x) = [1/(σ√2π)] e<sup>([x-X<sub>center</sub>]<sup>2</sup>)/(2σ<sup>2</sup>)</sup><br>
</p>
</blockquote>
<p>
and so the cumulative probability will be:
</p>
<blockquote>
<p>
f(x<sub>0</sub>)×f(x<sub>1</sub>)×f(x<sub>2</sub>)× .. <br>
[1/(σ√2π)]<sup>n</sup> e<sup>∑ ([x<sub>i</sub>-X<sub>center</sub>]<sup>2</sup>)/(2σ<sup>2</sup>)</sup>
</p>
</blockquote>
<p>
<i>X<sub>center</sub></i> and <i>σ</i> are our two unknow parameters. We could now differentiate and solve to find the maximal value
</p>
</div>
</div>
</div>
<div id="outline-container-org8ee8f3e" class="outline-2">
<h2 id="org8ee8f3e">Descriptive statistics</h2>
<div class="outline-text-2" id="text-org8ee8f3e">
<p>
A very unusual effect of continious random variables is that we can often skip their probability distributions entirely and summerizing them with just the <i>mean</i> and <i>variance</i>
</p>
</div>
<div id="outline-container-orgdaeaac3" class="outline-4">
<h4 id="orgdaeaac3">Expected Value/Mean</h4>
<div class="outline-text-4" id="text-orgdaeaac3">
<p>
The expected value of a random variable is the "average" of the probability distribution. Usually written as <b>E[x]</b>. For continious functions this translates to the integral
</p>
<blockquote>
<p>
E[x] = ∫f(u)×u … <i>(from -∞ to ∞)</i>
</p>
</blockquote>
<p>
(the notation is a bit confusing b/c <b>x</b> is a random variable and <b>u</b> is a possible value for <b>x</b> and <b>f(u)</b> is a probability density for that value)
</p>
</div>
</div>
<div id="outline-container-org69339b3" class="outline-4">
<h4 id="org69339b3">Variance</h4>
<div class="outline-text-4" id="text-org69339b3">
<p>
The variance is a value to describe how much the random variable differs from its <i>expected value</i>
</p>
<blockquote>
<p>
Var[x]= E[(E[x]-x)<sup>2</sup>]
</p>
</blockquote>
<p>
The power-of-2 ensures that the you are integrating over positive values. If the expected value of the random variable is <b>0</b> then the variance is just <b>E[x<sup>2</sup>]</b> or <i>∫(f(u))<sup>2</sup></i>
</p>
<dl class="org-dl">
<dt><b>TODO</b></dt><dd>Why not the absolute value? It's not just a mathematical convenience… It has some numerical advantage (I just forget what it is)</dd>
</dl>
</div>
</div>
<div id="outline-container-orgc4e8c40" class="outline-4">
<h4 id="orgc4e8c40">Standard deviation</h4>
<div class="outline-text-4" id="text-orgc4e8c40">
<p>
The <i>standard deviation</i> is just the square root of the variance and denoted with <i>σ</i>:
</p>
<blockquote>
<p>
Var[x] = σ<sub>x</sub><sup>2</sup>
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org82c6112" class="outline-3">
<h3 id="org82c6112">Estimating the Normal's descriptive statistics</h3>
<div class="outline-text-3" id="text-org82c6112">
<p>
The Normal Distributions is a bit unusual in that the mean and standard deviation are parameters of the PDF. So when we differentiate and solve for the PDF that fits our measurements we automatically end up with the best estimates for the mean and variance
</p>
</div>
<div id="outline-container-org043132c" class="outline-4">
<h4 id="org043132c">Mean of the Normal</h4>
<div class="outline-text-4" id="text-org043132c">
<p>
Finishing what we started before, we want to pick a <b>X<sub>center</sub></b> to maximize the cumulative probability of our measurements, which was
</p>
<blockquote>
<p>
[1/(σ√2π)]<sup>n</sup> e<sup>∑ ([x<sub>i</sub>-X<sub>center</sub>]<sup>2</sup>)/(2σ<sup>2</sup>)</sup>
</p>
</blockquote>
<p>
Fortunately, even though the cumulative probability equation a bit complicated, we only need to maximize the exponent part b/c everything else is not a function of <b>X<sub>center</sub></b>
</p>
<blockquote>
<p>
∑ (x<sub>i</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sup>2</sup>)
</p>
</blockquote>
<p>
To find its maximum we differentiating and set equal to zero:
</p>
<blockquote>
<p>
∑ (x<sub>i</sub>-X<sub>center</sub>) = 0
</p>
</blockquote>
<p>
Which gives us a
</p>
<blockquote>
<p>
X<sub>center</sub> = ∑x<sub>i</sub>/N
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org4c2d329" class="outline-4">
<h4 id="org4c2d329">Standard deviation of the normal</h4>
<div class="outline-text-4" id="text-org4c2d329">
<p>
To estimate an optimal standard deviation <b>σ</b> we follow the procedure as with the mean <b>X<sub>center</sub></b>. We differentiate with respect to <b>σ</b> and then solve for when it's equal to zero. Unlike before, we can't just maximize the exponent part b/c in this case there is a <b>/sigma{}</b> in the factor in front as well as in the exponent. So we need to differentiate <i>by-parts</i>
</p>
<blockquote>
<p>
0 = d/dσ [1/(σ√2π)]<sup>n</sup> e<sup>∑ ([x<sub>i</sub>-X<sub>center</sub>]<sup>2</sup>)/(2σ<sup>2</sup>)</sup><br>
σ<sub>best-estimate</sub> = √ (1/N) ∑(x<sub>i</sub>-X<sub>center</sub>)<sup>2</sup>
</p>
</blockquote>
<p>
The next issue we see is that the solution for <b>σ</b> is a function of <b>X<sub>center</sub></b> which we also don't know. The best we can do here is to subsitute in the best estimate we'd just found in the previous section: <b>∑x<sub>i</sub>/N</b>
</p>
<p>
<b>This unfortunately introducing a <i>bias</i>!</b> <br>
</p>
<p>
The estimated mean is such that the measurments are situated "optimally" around it. The true mean, whatever it is, will be <i>guaranteed</i> to be worse relative to our measurements and so the measurements in actuality deviate more from the mean than from its estimate. The true <b>X<sub>center</sub></b> of the underlying random variable will not be perfectly centered and the true standard deviation will always be a bit higher. The correct estimate for <b>σ</b> will be:
</p>
<blockquote>
<p>
σ<sub>best-estimate</sub> = √ (1/[N-1]) ∑(x<sub>i</sub>-X<sub>center</sub>)<sup>2</sup>
</p>
</blockquote>
<p>
Proof of this is ommitted.. but do note that as N gets larger this bias becomes insignificant
</p>
</div>
</div>
</div>
<div id="outline-container-orgf6a0a23" class="outline-3">
<h3 id="orgf6a0a23">SigFigs</h3>
<div class="outline-text-3" id="text-orgf6a0a23">
<p>
We can then use these two descriptive statistics to write out any random variable (normally distributed or otherwise) as a <i>value</i> + <i>standard deviation</i> pair. This is often what you will see on an instrument or quoted in a scientific publication. The underlying probability distribution will often either be infered or, as we'll soon see, be irrelevant
</p>
<p>
Uncertainties/standard-deviation should typically be one digit. And they should be rounded to be on the order of the significant figure.
</p>
<blockquote>
<p>
2.45 ± 0.04
</p>
</blockquote>
<p>
When we later see how to estimate these values it'll become clear that it doesn't make much sense to have more digits of precision in your uncertainty than in your expected value. In some cases you may want to two sigfigs in the uncertainty - if for instance the value is small
</p>
<blockquote>
<p>
1.3 ± 0.4<br>
1.3 ± 0.37
</p>
</blockquote>
<p>
You can also express the descriptive statistics as a percentage. <br>
Starting with:
</p>
<blockquote>
<p>
x<sub>0</sub> ± δx<sub>0</sub>
</p>
</blockquote>
<p>
You could also rewrite it as:
</p>
<blockquote>
<p>
x<sub>0</sub> with a [100*δx<sub>0</sub>/x<sub>0</sub>] percent uncertainty<br>
or<br>
x<sub>0</sub> ± xx%
</p>
</blockquote>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> [value<span style="color: #483d8b;">,</span> fractional] <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">uncertainty2fractional</span>(x<span style="color: #483d8b;">,</span> dx)
value <span style="color: #483d8b;">=</span> x
fractional <span style="color: #483d8b;">=</span> (dx<span style="color: #483d8b;">/</span>x)
<span style="color: #a020f0;">end</span>
</pre>
</div>
<div class="org-src-container">
<pre class="src src-octave">[x fx] <span style="color: #483d8b;">=</span> uncertainty2fractional(50<span style="color: #483d8b;">,</span>25)
ans <span style="color: #483d8b;">=</span> [x fx]
</pre>
</div>
<div class="org-src-container">
<pre class="src src-org"><span style="color: #0000ff;">| 50 | 0.5 |</span>
</pre>
</div>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> [value<span style="color: #483d8b;">,</span> uncertainty] <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">fractional2uncertainty</span>(x<span style="color: #483d8b;">,</span> fx)
value <span style="color: #483d8b;">=</span> x
uncertainty <span style="color: #483d8b;">=</span> x<span style="color: #483d8b;">*</span>fx
<span style="color: #a020f0;">end</span>
</pre>
</div>
<div class="org-src-container">
<pre class="src src-octave">[x dx] <span style="color: #483d8b;">=</span> fractional2uncertainty(50<span style="color: #483d8b;">,</span>0.5)
ans <span style="color: #483d8b;">=</span> [x dx]
</pre>
</div>
<div class="org-src-container">
<pre class="src src-org"><span style="color: #0000ff;">| 50 | 25 |</span>
</pre>
</div>
</div>
</div>
<div id="outline-container-org8d7ee41" class="outline-3">
<h3 id="org8d7ee41">Adding two random variables</h3>
<div class="outline-text-3" id="text-org8d7ee41">
<p>
Here we begin to see the utility of these <i>descriptive statistics</i>. They don't just serve as a quick summary of the random variable, but are something that can be manipulated directly!
</p>
</div>
<div id="outline-container-org81b1c18" class="outline-4">
<h4 id="org81b1c18">Expected Value</h4>
<div class="outline-text-4" id="text-org81b1c18">
<p>
If we add two random variables <b>z=x+y</b> then, regardless of their underlying probability distributions, their estimated values will add directly.
</p>
<blockquote>
<p>
<b>E[z] = E[x+y] = E[x] + E[y]</b>
</p>
</blockquote>
<p>
This is a consequence of integrals of sums <i>∫( f(u)×u+g(u)×u ) = ∫f(u)×u+∫g(u)×u</i> - and is independent of the distributions <i>f()</i> and <i>g()</i> (<i>they don't need to be normal</i>!)
</p>
</div>
</div>
<div id="outline-container-orgaffa2e4" class="outline-4">
<h4 id="orgaffa2e4">Variance</h4>
<div class="outline-text-4" id="text-orgaffa2e4">
<p>
The variance will also add directly, irrespective of distribution, as long as <i>a</i> and <i>b</i> are not correlated. To see this, imagine for simplicity that you have two random variables <b>x</b> and <b>y</b> that have an expected value of <b>0</b>. We will say the random variable <b>z</b> is their sum <b>z=x+y</b>
</p>
<blockquote>
<p>
E[x] = 0 <br>
Var(x) = E[0 - x<sup>2</sup>] = E[x<sup>2</sup>] <br>
E[y] = 0 <br>
Var(y) = E[0 - y<sup>2</sup>] = E[y<sup>2</sup>] <br>
E[z] = E[x] + E[y] = 0 <br>
Var[z]= E[(E[z]-(z))<sup>2</sup>] = E[(0-(x+y))<sup>2</sup>] = E[(x+y)<sup>2</sup>] = E[x<sup>2</sup>+y<sup>2</sup>+xy] = E[x<sup>2</sup>] + E[y<sup>2</sup>] + E[xy]
</p>
</blockquote>
<p>
As long as <i>a</i> and <i>b</i> are uncorrelated, we can split the last term <b>E[xy]</b> into <b>E[x]*E[y]</b> to see that it is equal to <b>0</b> (you can again go back to the integral definition here). So we are left with the fact that the variance of the sum is the sum of the variances
</p>
<blockquote>
<p>
Var[z]= Var[x] + Var[y]
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org211f5f3" class="outline-4">
<h4 id="org211f5f3">Standard Deviation</h4>
<div class="outline-text-4" id="text-org211f5f3">
<p>
This makes the standard deviation the <b>quadrature sum</b> of the two random variables
</p>
<blockquote>
<p>
σ<sub>z</sub> = √Var[z] <br>
= √[E[x<sup>2</sup>] + E[y<sup>2</sup>]] <br>
= √(σ<sub>x</sub><sup>2</sup> + σ<sub>y</sub><sup>2</sup>)
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org0d67699" class="outline-4">
<h4 id="org0d67699">SigFigs</h4>
<div class="outline-text-4" id="text-org0d67699">
<p>
Combining the two, we can now write out the sum of two random variables as:
</p>
<blockquote>
<p>
(x±δx) + (y±δy) = q±δq<br>
q = x+y<br>
δq = √{δx<sup>2</sup>+δy<sup>2</sup>}
</p>
</blockquote>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> [z<span style="color: #483d8b;">,</span> dz] <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">uncertainSum</span>(x<span style="color: #483d8b;">,</span> dx<span style="color: #483d8b;">,</span> y<span style="color: #483d8b;">,</span> dy)
z <span style="color: #483d8b;">=</span> x <span style="color: #483d8b;">+</span> y
dz <span style="color: #483d8b;">=</span> sqrt(dx<span style="color: #483d8b;">^</span>2<span style="color: #483d8b;">+</span>dy<span style="color: #483d8b;">^</span>2)
<span style="color: #a020f0;">end</span>
</pre>
</div>
<div class="org-src-container">
<pre class="src src-octave">[z dz] <span style="color: #483d8b;">=</span> uncertainSum(10<span style="color: #483d8b;">,</span> 1<span style="color: #483d8b;">,</span> 20<span style="color: #483d8b;">,</span> 3)
ans <span style="color: #483d8b;">=</span> [z dz]
</pre>
</div>
<div class="org-src-container">
<pre class="src src-org"><span style="color: #0000ff;">| 30 | 3.16227766016838 |</span>
</pre>
</div>
</div>
</div>
<div id="outline-container-orga1ba726" class="outline-4">
<h4 id="orga1ba726">Example: Standard deviation of the mean (SDOM) for a normal distribution</h4>
<div class="outline-text-4" id="text-orga1ba726">
<p>
Using these new distribution-independent properties we can now calculate how accuracy of our previous estimate for the mean of a normally distributed random variable. Remember that the estimate turned out to just be <i>X<sub>center</sub> = ∑x<sub>i</sub>/N</i> . ie. add up the measurements and then divide by the total number of them. We can write out the sum explicitely:
</p>
<blockquote>
<p>
X<sub>center</sub> = ∑x<sub>i</sub>/N <br>
= x<sub>0</sub>/N + x<sub>1</sub>/N + x<sub>2</sub>/N + …
</p>
</blockquote>
<p>
We know each measurement <i>x<sub>0|1|..</sub></i> came from a random variable of mean <b>X<sub>center</sub></b> and standard deviation <b>σ<sub>x</sub></b> so we can write this sum using descriptive statistics:
</p>
<blockquote>
<p>
= (X<sub>center</sub>±σ<sub>X</sub>)/N + (X<sub>center</sub>±σ<sub>X</sub>)/N + (X<sub>center</sub>±σ<sub>X</sub>)/N + … <br>
= (X<sub>center</sub>/N±σ<sub>X</sub>/N) + (X<sub>center</sub>/N±σ<sub>X</sub>/N) + (X<sub>center</sub>/N±σ<sub>X</sub>/N) + …
</p>
</blockquote>
<p>
So this is simply a sum of random variables and we know that the variance of the sum of random variables is always a quadrature sum:
</p>
<blockquote>
<p>
σ<sub>X<sub>center</sub></sub> = √ ∑ [ (σ<sub>x</sub>/N)<sup>2</sup> + (σ<sub>x</sub>/N)<sup>2</sup> + (σ<sub>x</sub>/N)<sup>2</sup> + … ] <br>
= √[N × (σ<sub>x</sub>/N)<sup>2</sup> ] <br>
= √[σ<sub>x</sub><sup>2</sup>/N ] <br>
= σ<sub>x</sub>/√N
</p>
</blockquote>
<p>
In some situations this can provide a useful error. Notably if what you are measuring has very little error, but your measurements has a relatively large error. For instance if you're measuring a resistor. The resistor is of a constant value and doesn't fluctuate (ie. it basically have no variance itself), but your voltmeter/power source will have some errors that affect each measurement you make
</p>
<blockquote>
<p>
<b>Side note..</b> : the uncertainty in <b>σ<sub>x</sub></b> is: <b>1 /√[2(N-1)] <br>
</b>
<i>Proof omitted for now..</i>
</p>
</blockquote>
</div>
</div>
</div>
<div id="outline-container-org4dc42f6" class="outline-3">
<h3 id="org4dc42f6">Multiplying two random variables</h3>
<div class="outline-text-3" id="text-org4dc42f6">
<p>
Again, we work directly with the descriptive statistics without even looking at the variables' PDFs
</p>
</div>
<div id="outline-container-org4647717" class="outline-4">
<h4 id="org4647717">Expected Value</h4>
<div class="outline-text-4" id="text-org4647717">
<blockquote>
<p>
E[z] = E[x] × E[y]
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org2ae08f5" class="outline-4">
<h4 id="org2ae08f5">Variance</h4>
<div class="outline-text-4" id="text-org2ae08f5">
<p>
This proof is quite tricky..<br>
We start directly with directly applying the equation for the variance
</p>
<blockquote>
<p>
Var(xy) = E[(xy-XY)<sup>2</sup>] …
</p>
</blockquote>
<p>
<b>X</b> and <b>Y</b> are short hand for <b>E[x]</b> and <b>E[y]</b>
</p>
<p>
First we focus on massaging the inner term <b>xy-XY</b>
</p>
<blockquote>
<p>
xy-XY <br>
XY × [x/X × y/Y -1] <br>
XY × [ ([x-X]/X + 1) × ([y-Y]/Y + 1) - 1 ] <br>
XY × [ [x-X]/X + [y-Y]/Y + [x-X]×[y-Y]/XY ]
</p>
</blockquote>
<p>
if <b>δx = [x-X]/X</b> and <b>δy = [y - Y]/Y</b>, then we can rewrite this as
</p>
<blockquote>
<p>
XY × [δx+δy+ δxδy]
</p>
</blockquote>
<p>
So now we can rewrite the original equation
</p>
<blockquote>
<p>
Var(xy) = E[(xy-XY)<sup>2</sup>] <br>
= (XY)<sup>2</sup> × E[(δx + δy + δxδy)<sup>2</sup>] <br>
= (XY)<sup>2</sup> × E[δx<sup>2</sup> + δxδy + δx<sup>2</sup>δy + δyδx + δy<sup>2</sup> + δxδy<sup>2</sup> + δx<sup>2</sup>δy + δxδy<sup>2</sup> + δx<sup>2</sup>δy<sup>2</sup>] <br>
= (XY)<sup>2</sup> × (E[δx<sup>2</sup>] + E[δxδy] + E[δx<sup>2</sup>δy] + E[δyδx] + E[δy<sup>2</sup>] + E[δxδy<sup>2</sup>] + E[δx<sup>2</sup>δy] + E[δxδy<sup>2</sup>] + E[δx<sup>2</sup>δy<sup>2</sup>]) <br>
</p>
</blockquote>
<p>
Not mentioned explicitely till now, but when we have want to find the <i>expected value</i> of a equation of multiple random variables we need to integrate across all variables. So in this case we want to integrate across <b>x</b> and <b>y</b> from -∞ to ∞. As premised at the beginning, <b>x</b> and <b>y</b> are independent, so when we write out these expected values we can actually rearrange them as each integral is independent (the other terms is treated as a constant in each integral).
</p>
<blockquote>
<p>
E[x<sup>n</sup>y<sup>m</sup>] <br>
= ∫<sub>x</sub> ∫<sub>y</sub> x<sup>n</sup>y<sup>m</sup><br>
= ∫<sub>x</sub> x<sup>n</sup>∫<sub>y</sub> y<sup>m</sup>
</p>
</blockquote>
<p>
The final trick is to notice that <b>δx</b> and <b>δy</b> have an expected values are <i>zero</i> - so if either <b>m</b> or <b>n</b> are equal to <b>1</b> then it's integral goes to zero and so <b>E[x<sup>n</sup>y<sup>m</sup>]</b> comes out to <b>0</b>. The only nonzero terms that remain are
</p>
<blockquote>
<p>
= (XY)<sup>2</sup> × (E[δx<sup>2</sup>] + E[δy<sup>2</sup>] + E[(δxδy)<sup>2</sup>]) <br>
</p>
</blockquote>
<blockquote>
<p>
= (XY)<sup>2</sup> × (V[x]/X<sup>2</sup> + V[y]/Y<sup>2</sup> + V[x]V[y]/(XY)<sup>2</sup>) <br>
= Y<sup>2</sup>V[x] + X<sup>2</sup>V[y] + V[x]V[y] <br>
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org0cd8e57" class="outline-4">
<h4 id="org0cd8e57">Standard Deviation</h4>
<div class="outline-text-4" id="text-org0cd8e57">
<p>
The standard deviation is the square root of the variance..
</p>
<blockquote>
<p>
σ<sub>xy</sub> ≅ √ [Y<sup>2</sup>V[x] + X<sup>2</sup>V[y] + V[x]V[y]]
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org87de359" class="outline-4">
<h4 id="org87de359">SigFigs</h4>
<div class="outline-text-4" id="text-org87de359">
<p>
Again, summarizing in our standard notation we get
</p>
<blockquote>
<p>
(x±δx) × (y±δy) = q±δq<br>
q = x×y<br>
δq/q = √[x<sup>2</sup>δy<sup>2</sup> + y<sup>2</sup>δx<sup>2</sup> + δx<sup>2</sup>δy<sup>2</sup>]
</p>
</blockquote>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> [z<span style="color: #483d8b;">,</span> dz] <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">uncertainProduct</span>(x<span style="color: #483d8b;">,</span> dx<span style="color: #483d8b;">,</span> y<span style="color: #483d8b;">,</span> dy)
z <span style="color: #483d8b;">=</span> x<span style="color: #483d8b;">*</span>y
dz <span style="color: #483d8b;">=</span> sqrt(x<span style="color: #483d8b;">^</span>2<span style="color: #483d8b;">*</span>dy<span style="color: #483d8b;">^</span>2 <span style="color: #483d8b;">+</span> y<span style="color: #483d8b;">^</span>2<span style="color: #483d8b;">*</span>dx<span style="color: #483d8b;">^</span>2 <span style="color: #483d8b;">+</span> dx<span style="color: #483d8b;">^</span>2<span style="color: #483d8b;">*</span>dy<span style="color: #483d8b;">^</span>2)
<span style="color: #a020f0;">end</span>
</pre>
</div>
</div>
</div>
</div>
</div>
<div id="outline-container-orgf5e3c80" class="outline-2">
<h2 id="orgf5e3c80">Other Topics</h2>
<div class="outline-text-2" id="text-orgf5e3c80">
</div>
<div id="outline-container-org84d2205" class="outline-3">
<h3 id="org84d2205">Data Rejection</h3>
<div class="outline-text-3" id="text-org84d2205">
<p>
Sometimes things go wrong and data is just way out there
</p>
</div>
<div id="outline-container-org1d664dc" class="outline-4">
<h4 id="org1d664dc">Chauvenet's Criterion</h4>
<div class="outline-text-4" id="text-org1d664dc">
<p>
This is a scheme for rejecting data - but it only makes sense if your data is normally distributed
You first find how many standard deviation this measurement is away from your estimated mean
</p>
<blockquote>
<p>
t<sub>sus</sub> = |x<sub>sus</sub> - X<sub>center</sub>|/σ<sub>x</sub>
</p>
</blockquote>
<p>
Then you look up the probability a measurement is that far off in a table. This probability represents the fraction of points that will lie either at the point in question or further out. Multiply it times the amount of measurements you've done, <b>N</b>, and you get the amount of points you'd expect to lie at the point-or-further after <b>N</b> measurements.
</p>
<blockquote>
<p>
N × Prob(outside t<sub>sus</sub>σ)
</p>
</blockquote>
<p>
If the value is smaller than <b>1/2</b> then we expect less tan half a point out as far as the point in question - so you can reject the point and recalculate your mean/sigma.
</p>
</div>
</div>
</div>
<div id="outline-container-org1c6fc4f" class="outline-3">
<h3 id="org1c6fc4f">Weighted Average</h3>
<div class="outline-text-3" id="text-org1c6fc4f">
<p>
We often want to average different measurements into one. It's clear a measurement with a larger error should have a smaller effect on the outcome than a measurement with a smaller error.
</p>
</div>
<div id="outline-container-org233602a" class="outline-4">
<h4 id="org233602a">Mean</h4>
<div class="outline-text-4" id="text-org233602a">
<p>
To get the combined mean we proceed in a direct manner. We multiply the two PDFs and minimize the result
</p>
<blockquote>
<p>
pdf(x<sub>a</sub>) = (1/(σ<sub>a</sub>√2π)) e<sup>(x<sub>a</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>a</sub><sup>2</sup>)</sup><br>
pdf(x<sub>b</sub>) = (1/(σ<sub>b</sub>√2π)) e<sup>(x<sub>b</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>b</sub><sup>2</sup>)</sup><br>
</p>
</blockquote>
<p>
Here <b>x<sub>a</sub></b> and <b>x<sub>b</sub></b> are the two prior measurements/estimates we have, and the <b>X<sub>center</sub></b> will be our combined average mean. We need to choose an <b>X<sub>center</sub></b> that will maximize the probability of both functions. We combine probabilities by multiplication
</p>
<blockquote>
<p>
pdf(x<sub>a</sub>) × pdf(x<sub>b</sub>) <br>
(1/(σ<sub>a</sub>√2π)) e<sup>(x<sub>a</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>a</sub><sup>2</sup>)</sup> × (1/(σ<sub>b</sub>√2π)) e<sup>(x<sub>b</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>b</sub><sup>2</sup>)</sup> <br>
(1/(σ<sub>a</sub>σ<sub>b</sub>2π)) e<sup>(x<sub>a</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>a</sub><sup>2</sup>) + (x<sub>b</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>b</sub><sup>2</sup>)</sup> <br>
</p>
</blockquote>
<p>
Again here we just care to maximize the exponent (the only part that's a function of <b>X<sub>center</sub></b>). We take its derivative and set to zero
</p>
<blockquote>
<p>
0 = d/dX (x<sub>a</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>a</sub><sup>2</sup>) + (x<sub>b</sub>-X<sub>center</sub>)<sup>2</sup>/(2σ<sub>b</sub><sup>2</sup>)<br>
0 = (x<sub>a</sub>-X<sub>center</sub>)/σ<sub>a</sub><sup>2</sup> + (x<sub>b</sub>-X<sub>center</sub>)/σ<sub>b</sub><sup>2</sup> <br>
X<sub>center</sub> = [x<sub>a</sub>/σ<sub>a</sub><sup>2</sup> + x<sub>b</sub>/σ<sub>b</sub><sup>2</sup>] / [(1/σ<sub>a</sub><sup>2</sup> + 1/σ<sub>b</sub><sup>2</sup>]
</p>
</blockquote>
<p>
You can also rewrite this by introducing a new variable called a <b>weight</b>
</p>
<blockquote>
<p>
w = 1/σ<sup>2</sup>
</p>
</blockquote>
</div>
</div>
<div id="outline-container-org3752565" class="outline-4">
<h4 id="org3752565">Standard Deviation</h4>
<div class="outline-text-4" id="text-org3752565">
<p>
The standard deviation we derive from the equation for the mean by applying error propagation.
The simplifying trick is to first look at the variance of the sum: <b>Var[z]= Var[x] + Var[y]</b>.
If you immediately try to calculate standard deviation and use the quadrature rule then things get overly complicated
(as a general rule, it's easier to deal with variances b/c you avoid the quadrature mathematical mess)
</p>
<blockquote>
<p>
X<sub>center</sub> = [x<sub>a</sub>/σ<sub>a</sub><sup>2</sup> + x<sub>b</sub>/σ<sub>b</sub><sup>2</sup>] / [(1/σ<sub>a</sub><sup>2</sup> + 1/σ<sub>b</sub><sup>2</sup>] <br>
Var[center] = Var[x<sub>a</sub>]/[σ<sub>a</sub><sup>2</sup> + Var[x<sub>b</sub>]/[σ<sub>b</sub><sup>2</sup> / [(1/σ<sub>a</sub><sup>2</sup> + 1/σ<sub>b</sub><sup>2</sup>]
</p>
</blockquote>
<p>
Since <b>Var[w] = σ<sub>w</sub><sup>2</sup></b>, the top terms all cancel out and you're left with the bottom term only
</p>
<blockquote>
<p>
Var[center] = 1 / [(1/σ<sub>a</sub><sup>2</sup> + 1/σ<sub>b</sub><sup>2</sup>] <br>
σ<sub>center</sub> = 1 / √[(1/σ<sub>a</sub><sup>2</sup> + 1/σ<sub>b</sub><sup>2</sup>]
</p>
</blockquote>
</div>
</div>
</div>
</div>
<div id="outline-container-org83847d2" class="outline-2">
<h2 id="org83847d2">Multiple Variables</h2>
<div class="outline-text-2" id="text-org83847d2">
</div>
<div id="outline-container-org6e57917" class="outline-3">
<h3 id="org6e57917">Covariance</h3>
<div class="outline-text-3" id="text-org6e57917">
<p>
If we have two variables changing simulatenously we can quantify how much they change "together" with a term called <b>covariance</b>
</p>
<blockquote>
<p>
σ<sub>xy</sub> = 1/N ∑ (x<sub>i</sub> - x<sub>mean</sub>)(y<sub>i</sub> - y<sub>mean</sub>)
</p>
</blockquote>
<p>
If the variables are independent then the <b>covariance</b> will be around zero b/c both terms in the multiplication, <b>(x<sub>i</sub> - x<sub>mean</sub>)</b> and <b>(y<sub>i</sub> - y<sub>mean</sub>)</b> are bouncing around zero randomly. If the terms are not independent and when one variable varies away from the mean then the other does as well - then this term will be non zero
</p>
<dl class="org-dl">
<dt><b>Schwarz Inequality</b></dt><dd>If the two variables perfectly correlate (one goes "high" when the other goes "high") then the maximal value attainable will be <b>σ<sub>x</sub>σ<sub>y</sub></b>. This provides a useful limit to the covariance.</dd>
</dl>
<blockquote>
<p>
σ<sub>xy</sub> ≤ σ<sub>x</sub>σ<sub>y</sub>
</p>
</blockquote>
<p>
While notationally they look like standard deviations (ex: <b>σ<sub>x</sub></b>), dimensionally they're equivalent to variances (ie. dimension of <b>x</b> times dimension of <b>y</b>) - hence the name. This is because variances are typically written as <b>σ<sub>x</sub><sup>2</sup></b> - which I've been writing so far as <b>Var[x]</b> to differentiate it and for clarity.
</p>
<div class="org-src-container">
<pre class="src src-octave"><span style="color: #a020f0;">function</span> result <span style="color: #483d8b;">=</span> <span style="color: #0000ff;">covariance</span> (x<span style="color: #483d8b;">,</span> y)
xMean <span style="color: #483d8b;">=</span> mean(x)
yMean <span style="color: #483d8b;">=</span> mean(y)
result <span style="color: #483d8b;">=</span> sum(((x <span style="color: #483d8b;">-</span> xMean) <span style="color: #483d8b;">.*</span> (y <span style="color: #483d8b;">-</span> yMean)))<span style="color: #483d8b;">/</span>length(x)
<span style="color: #a020f0;">end</span>
</pre>
</div>
</div>
<div id="outline-container-orgec3a23a" class="outline-4">
<h4 id="orgec3a23a">Example</h4>
<div class="outline-text-4" id="text-orgec3a23a">
<p>
Problem 9.1
</p>
<div class="org-src-container">
<pre class="src src-octave">x <span style="color: #483d8b;">=</span> [ 20 23 23 22 ]
y <span style="color: #483d8b;">=</span> [ 30 32 35 31 ]
covariance(x<span style="color: #483d8b;">,</span>y)
</pre>
</div>
<div class="org-src-container">
<pre class="src src-org">1.75
</pre>
</div>
<p>
Problem 9.2
</p>
<div class="org-src-container">
<pre class="src src-octave">t <span style="color: #483d8b;">=</span> [ 14 12 13 15 16]