Lecture Notes 1: Platonic Convergence and the Central Limit Theorem

1) An erroneous notion of limit: Take the standard formulation of the Central Limit Theorem (Feller 1971, Vol. II;Grimmet & Stirzaker, 1982):

Let X1 ,X2 ,... be a sequence of independent identically distributed random variables with mean m & variance s 2 satisfying m< and 0 <s 2 <, then N=1 Xi - N m D i Gaussian H0, 1L as N !!!! s N

Where is converges "in distribution".

Taking convergence for granted provides a plain illustration of the severe disease of Platonicity --or working backwards from theory to practice. Effectively we are dealing with a double problem. 1) The first, as uncovered by Jaynes, comes from to the abuses of formalism & measure theory: Jaynes 2003 (p.44):"The danger is that the present measure theory notation presupposes the infinite limit already accomplished, but contains no symbol indicating which limiting process was used (...) Any attempt to go directly to the limit can result in nonsense". Granted Jaynes is still too Platonic in general and idealizes his convergence process (he also falls headlong for the Gaussian by mixing thermodynamics and information). But we accord with him on this point --along with the definition of probability as information incompleteness, about which in later sessions. 2) The second problem is that we do not have a "clean" limiting process --the process cannot be idealized. It is very rare to find permanent idealized conditions that allow for temporal aggregation. Now how should we look at the Central Limit Theorem? Let us see how we arrive to it assuming "independence".

2) The Problem of Convergence The CLT works in a specific way:It does not fill-in uniformily,but in a near-Gaussian way--indeed,disturbingly so. Simply,whatever your distribution (assuming one mode),your sample is going to be skewed to deliver more central observations,and fewer tail events.The consequence is that,under aggregation,the sum of these variables will converge "much" faster in the body of the distribution than in the tails. As N,the number of observations increases,the Gaussian zone should cover more grounds... but not in the "tails". You can see it very easily with two very broad uniform distributions,say with a lower bound a and an upper bound b, b-a very large. As you convolute, you will see the peakedness in the center, which means that more observations will fall there (see Appendix).

This quick note shows the intuition of the convergence and presents the difference between distributions. (See Appendix)

Take the sum of of random independent variables Xi with finite variance under distribution j(X). Assume 0 mean for simplicity (and symmetry, absence of skewness to simplify). A better formulation of the Central Limit Theorem (Kolmogorov et al,x)

This quick note shows the intuition of the convergence and presents the difference between distributions. (See Appendix) 2 Lectures on Probability.nb

Take the sum of of random independent variables Xi with finite variance under distribution j(X). Assume 0 mean for simplicity (and symmetry, absence of skewness to simplify). A better formulation of the Central Limit Theorem (Kolmogorov et al,x) u Z n=0 Xi 1 i 2 P - u Z = u = e- Z !!! !!!!!!! n s 2 p -u
2

So the distribution is going to be:

u
2

inside the "tunnel" [-u,u] --the odds of falling inside the tunnel itself and

Z i j1 - e- Zy for - u z u z 2 j z k { -u

outside the tunnel [-u,u]

Where j'[n] is the n-summed distribution of j.

j '@nD HZL z + j '@nD HZL z

How j'[n] behaves is a bit interesting here --it is distribution dependent. And it depends on the initial distribution! Bouchaud-Potters Treatment of Width of the Tunnel [-u,u] (in class derivation)

3) Using Log Cumulants & Observing Gaussian Convergence

The normalized cumulant of order n, C(n) is the derivative of the log of the characteristic function f which we convolute N times divided by the second cumulant (i,e., second moment). H-Ln n logHfN L CHn, NL = n-1 . z 0 H-2 logHfLN L

Since C(N+M)=C(N)+C(M), the additivity of the Log Characteristic function under convolution makes it easy to see the speed of the convergence to the Gaussian. Fat tails implies that higher moments implode --not just the 4th . Table of Normalized Cumulants -Speed of Convergence (Dividing by sn where n is the order of the cumulant).

Distribution PDF N-convoluted Log Char acteristic 2nd Cum 3 rd 4 th 5 th 6 th 7 th 8 th 9 th 10 th

Normal@m, sD
2 N logI z m- M 2p s
z2 s 2

!!!!!!!

Hx-mL2 - 2 2s

PoissonHlL N logHH-1+ 1
1 Nl 1 l2 N2 1 l3 N3 1 l4 N4 1 l5 N5 1 l6 N6 1 l7 N7 1 l8 N8 l x!
x -l z L l

ExponentialHlL L -x l l
l N logH L l- z 2 l N 3! l N2
2

GHa, bL
b x GHaL
a-1 -a - b
x

1 0 0 0 0 0 0 0 0

N logHH1 - b zL-a L 1
2 N ab

3! N2 a2 b 2 4! N3 a3 b 3 5! N4 a4 b 4 6! N5 a5 b 5 7! N6 a6 b 6 8! N7 a7 b 7 9! N8 a8 b 8

4! l N3 5! l N4
4

6! l N5 7! l N6
6

8! l N7
7

9! l N8
8

Distribution

PDF

Mixed Gaussians HStoch VolL p

2 s1 2 !!!!!!! 2 p s1
-
x2

StudentTH3L
!!!!

StudentTH 4L
1 12 H L x2 +4 52

6 3 2 2

N - convoluted Log Characteristic

H1 - pL !!!!!!!
x2 - 2 s2 2

2 p s2
2

p Hx +3L

2nd Cum 3 rd 4 th

H1 - pL 1 0

N logIp

z s - 1 2

z2 s 2 - 2 2

-I3 H-1 + pL I15 H-1 + pL p H-1 + 2 pL 0 IN 4 Hp s2 1

3

5 th 6 th

2 p Hs2 - s2 L M 1 2 IN 2 Hp s2 1 3 H-1 + pL s2 L M 2

N !!! IlogI 3 z + 1M !!! 3 zM 1 Ind Ind

N logH 2 z2 K2 H2 zLL 1 Ind

Ind Ind

Ind Ind

Hs2 - s2 L M 1 2

H-1 + pL s2 L M 2
5

Lectures on Probability.nb

Note: On "Infinite Kurtosis"- Discussion Note on Chebyshev's Inequality and upper bound on deviations under finite variance

A lot of idiots talk about finite variance not considering that it still does not mean much. Consider Chebyshev's inequality: s2 P @X > aD a2

Which effectively accommodate power laws but puts a bound on the probability distribution of large deviations --but still significant.

1 P @X > n s D n2

The Effect of Finiteness of Variance This table shows the probability of exceeding a certain s for the Gaussian and the lower on probability limit for any distribution with finite variance. Deviation 3 4 5 6 7 8 9 10 Gaussian 7. 102 3. 104 3. 106 1. 10 8. 10 2. 10 9. 10 1. 10
9 11 15 18 23

Chebyshev Upper Bound 9 16 25 36 49 64 81 100

Calculations

Calculations

APPENDIX to 2- How We Converge Mostly in the Center - A Tutorial

300 250 200 150 100 50 0.2 N=2 500 400 300 200 100 0.4 0.6 0.8 1

0.5 N=3

1.5

Lectures on Probability.nb

800

600

400

200

0.5
Out[417]=

1.5

2.5

Graphics

Example: Uniform Distribution f HxL = 1 f2 Hz2L =

-

0x1 H f Hz - xLL H f xL x =

By Convoluting 2, 3, 4 times iteratively: 2 - z2 z2 1 < z2 < 2 0 < z2 1 0 < z3 1 1 < z3 < 2 z3 2 2 < z3 < 3

2 z3 2 3 3 -Hz3 - 3L z3 - 2 f3 z3 = Hf2 Hz3 - x2LL H f x2L x2 = 1 - Hz3 - 3L Hz3 - 1L 2 0 1 2 Hz3 - 3L 2

4 f4 x = Hf3 Hz4 - xLL H f x3L x3 = 0

1 4 1 2

z4 3 z4 2 0 < z4 1 3 < z4 < 4 1 < z4 < 2 2 < z4 < 3

z4 4

1 H-z42 + 4 z4 - 2L 4 1 4

Hz4 - 4L

A simple Uniform Distribution

4 3.5 3 2.5 2 1.5 1 0.5 -0.5 0.5 1 1.5

We can see how quickly, after one single addition, the net probabilistic "weight" is going to be skewed to the center of the distribution, and the vector will weight future densities.. 2.5 2 1.5 1 0.5

-0.5 0.7 0.6 0.5 0.4 0.3 0.2 0.1

0.5

1.5

2.5

Lectures on Probability.nb

0.5 0.4 0.3 0.2 0.1

1 Discussion Gaussian Case Paretan Case

Dealing With the Distribution of the Summed distribution j