Education
Quantutorial – Standard Deviation
28

# Quantutorial – Standard Deviation

May 11, 2017

Standard deviation. You see it mentioned all the time but if that inquisitive little niece of yours would ask you what it is, could you actually explain it? No, I’m not talking about googling the formula on yourÂ mobile and then telling her to scramÂ and kick a ball or something. What I mean is explaining standard deviation (SD) in a way that actually makes sense and may lead to her to taking an interest in STEM sciences later in life. Yeah, I didn’t think so. And you call yourself a trader? Step aside and let uncle Mole handleÂ this.

AlrightÂ Lucy (that’s her name), imagine you take your ball into the yard outside and start kicking it. Every time you kicked it you measure the distance. Warning:Â Since Lucy is a spoiled littleÂ American brat we’ll be talking about yards – which will cause European metric based children to experience instantÂ cerebral hemorrhage. So you kicked it the first time and it went 10 yards. You write that number down and then kick it again. Unfortunately you tripped a little and so the ball only went 3 yards. It counts anyway so write it down. Next time you take a running start and the ball flies 20 yards. Excellent, that definitely gets recorded for posterity. Meanwhile your annoying little brother shows up and kicks the ball away from you. He’s half your size so it only went 6Â yards but he kicks it again and scores 8. You run after the little bugger and kick it back about 12 yards. Who instantly starts bawlingÂ and just out of spite kicks thatÂ ball as far as he possibly can – 9Â yards – through the kitchen window. Game over and of course you blame your brother for everything.

So let’s do the numbers – we have measured 7Â kicks in yards:

10, 3, 20, 6, 8, 12, 9

First we need to figure out the average or mean of the distances, which we get by adding all those numbers together and dividing it by 7. So that would be 68 divided by 7 which yields us 9.71. Let’s draw a line on our chart:

Alright at this point it’s becoming clear that Lucy must haveÂ been feeding her ADD meds to her dog as she ran out to play about five minutes ago. Mental note to drop her from this year’s Christmas shopping list. So let’s explain the rest to that good-for-nothing brother of yours:

All we do from here is to deduct the mean (i.e. 9.71) from all the numbers:

0.29, -6.71, 10.29, -3.71, -1.71, 2.29, -0.71

Oh-ooooh – negative numbers.Â I don’t remember anyone kicking the ball backwards! What to do? Simple we square those numbers which gets us a positive series:

0.08, 45.08, 105.80, 13.80, 2.94, 5.22, 0.51

Now that’s weird. Seems like the negative numbers turned into very large exponents. Those are actually the relative ‘variances’ based on the mean and if we take the mean of all those numbers we get to… drum rolls… 24.78. That number representsÂ the average variance from the mean.

So how about standard deviation? I got you covered. All you need to do is to square that number and you get to 4.987, which is the standard deviation. Finally!

Of course one really annoying thing about that imbecileÂ of a brother is that he never trusted you. So he whips out his mobile phone and goes to an online SDÂ calculator. ‘Ha-haaa!’ he shoutsÂ ‘you wereÂ wrong!! See here, the real standard deviation is 5.376. I knew you always sucked in math’.

Ooops that’s embarrassing, what happened? Well, tell your brother that he calculated the ‘sample SD’ while you gave him the ‘population SD’. What’s the difference? If you take only a sample from a larger population of numbers then you need to actually deduct 1 from the number samples when calculating your variance. Instead of dividing the sum of the squared values by 7 we divide them by 6.

Here’s a spreadsheet I put together so you don’t get confused. On the bottom left you see ourÂ ‘control number’ which Excel’sÂ built-in SDÂ function. So we are spot on!

All that’s left to do for us now is to actually draw the SDÂ value(s) to our chart, in this case a very simple bar graph. Just like with a Bollinger you take the mean and then add the SD of 5.37. For the negative SD you simply deduct the same SD. That gives us two lines shown above in green, one at 4.338 and another at 15.09 (for sample SD which is what everyone uses by default). If you count the numbers of bars (i.e. kicks) that fall within that standard deviation range you arrive at 5 out of 7 (the 2nd and 3rd are outside). That sounds about right as that is 71% within a very small sample of 7. Strangely when writing that little story for Lucy I just pulled those numbers out of my butt and it’s interesting that the numbers worked out so well. So there appears to be a natural and almost instinctive order to distribution patterns.

Now if Lucy raided her mom’sÂ cookie jar and due to an epic sugar rushÂ kept kicking that ball for hoursÂ on end whilst collecting the distances odds suggest that the final tallyÂ would settle around 68% – a little over 2/3 of the sample size. Another way of phrasing it is that 68% of all the kicks would most likely fall within a standard deviation range of 4.338′ to 15.09′. And that is called ‘normal’ distribution or ‘Gaussian’ data.

But wait there is more. We can actually keep adding SD intervals simply by adding the same distance (i.e. 5.37) once again and then again. Which is visualized in the graph shown above – I’m sure you’ve come acrossÂ it in the past. In this specific case the kicking range spanning 2 standard deviationsÂ would be between 0.87′ to 18.558′ and encompass over 95% of all the samples. Each standard deviation interval is also often referred to as ‘confidence intervals’ and that is what the nerds mean by ‘sigma’.

Here’s a handy table that shows you each confidence interval, assuming normal distribution (a topic we have covered in the past but will revisit in the near future). If you click on the table it’ll get you to the page where you can play with the numbers or add your own.

##### Six Sigma Event?

Remember last time when you read some article over on ZeroEdge suggesting the possibility of a ‘six sigma’ event? Well, if you count the rows inÂ that table above then you realize that this puts usÂ in the top 0.001% of all trading days, which is 1/1000th of one percent, or one out of 100,000. Wait a minute. 100,000 / 365Â comes out to 273Â years? Almost threeÂ centuries? But we had severalÂ major market crashes in the past 100 years alone, not even counting recent flash crashes:

1. Florida Real Estate Craze in 1926
2. The Great Depression starting in 1929
3. The Big Crash of 1987
4. The Asian Crisis starting in 1989
5. The Dotcom Crash in 2000
6. The Housing Bubble Crash in 2007.

That comes out to at least 6Â large ‘unforeseen’ market events in the past century. If you would average that out over 300 years that would be 24 out of 100,000 trading days, or 0.024%. Deduct that from 100 and you get to 99.976. For it to be a six sigma event it would have to be > 99.999 but it’s smaller. It’s not even > 99.99% which would have made it a five sigma event. It is however > 99.9% so perhaps they should call them Four Sigma events instead?

##### What Did We Learn Today?
1. Lucy loves kicking balls but apparently doesn’t like math very much. Then again she’s nineÂ years old, what’s your excuse?
2. Mole is a horrible uncle who tortures innocent children with math riddles (I charge per hour by the way).
3. Standard deviation is a measure of how spread out numbers are.
4. SD is the square root of the variance throughout all the samples.
6. Apparently financial risk is not being assessed correctly. Big surprise there!
7. You should definitely buy Nassim Taleb’s new book (no I don’t earn any commissionsÂ from that link).

The Mole
Mole created Evil Speculator amidst the chaos of the financial crisis in early August of 2008. His vision for Evil Speculator is a refuge of reason, hands-on trading knowledge, and inspiration for traders of all ages and stripes. You can follow him and his nefarious schemes at various social media waterholes below.
• http://www.captainboom.com/ captainboom

Zero needs a kickstart Boss.

• Trouzzer_Snake

What an awesome post, thank you! This obviously took some effort and appreciate it.

• http://evilspeculator.com Sir Mole III

Really? I just turned it on. On it!

• http://evilspeculator.com Sir Mole III

It’s in preparation for a very juicy guest post which requires a basic understanding of standard deviation which in turn leads into volatility models.

• http://evilspeculator.com Sir Mole III
• Trouzzer_Snake

And here I was thinking you were just inspired… ðŸ™‚

• Yoda

Now that was hilarious. XD

• http://evilspeculator.com Sir Mole III

Two people wrote me about Zero problems but it’s working fine for me (via the same page). Anyone else having problems?

• Trouzzer_Snake

Working for me.

time was working but chart was frozen at 2395 then it started working after email was sent…thx!!

Drive-by posting

Bitcoin > \$1800.

If your Sigma event is too big, your model is probably broken.
Excellent post. back to the pool!
-GG

• HD

FWIW- Near perfect elliot wave off the 4/19 low. 5=1 2406, sellers came in, broke the 2-4TL, quantifying the wave >90% probability of retrace. Previous 4th= first target. Done! Notable fibonacci relationships the entire rally. What I refer to as Fibcontrolâ„¢ If you were following along on TWTR actually had the 2404 SPX from the double bottom last week for first entry. 40 point round trip based purely on pattern and elliot wave. No indicators. IF volatility returns efficacy for EW will only increase. https://uploads.disquscdn.com/images/1059f31699cac2953407aad82cac0c7f08fd425a259501338db0950e30034759.png https://uploads.disquscdn.com/images/cfc6e91f1733873defc92275f90c889e1646d4032fa5f9de542c7e32779d94a6.png

• http://evilspeculator.com Sir Mole III

That’s a relief…

• http://www.captainboom.com/ captainboom

Odd. We were 4 minutes into the trading day, and I wasn’t seeing any Zero updates. Time was updating, but not the chart. Working for me now.

• BobbyLow

Well done! ðŸ™‚

And I think the lower the SD on System Results, the less chance of blowing up?

• Yoda

Oh look a sell-off.

• http://evilspeculator.com Sir Mole III

Oh, I just grasped what you said, sorry. Yes, a lower standard deviation usually means that your winners and losers are better distributed and thus will be psychologically easier to trade. Mean reversion systems usually have a lower SD than trend trading systems which have a low win/loss ratio and where the winners far exceed 1R or 2R on average.

• http://evilspeculator.com Sir Mole III

Must have been a TOS problem then. Unfortunately that is out of my control.

• Yoda

Nutty gas is looking nicely bullish today. Bobby, are you still in L UNG?

• BobbyLow

Nope. I was trading Nutty Gas to see if it would be a good fit for me. I had mixed results with it. I didn’t have enough patience to trade the daily and trading NG with the hourly kind of sucked with my system so I put that horse back into the barn.

• http://evilspeculator.com Sir Mole III

Zero signal not very convincing during the sell off.

• Mary

Mr. Mole. We just arrived at your inflection point again that you mentioned 2 or 3 weeks ago.

• http://evilspeculator.com Sir Mole III

Still not going anywhere though – grrr…

• CandleStickEmUpper

Great post Mole. You have an excellent, clear way of explaining math. Ever think of writing an advanced college math book?

• http://evilspeculator.com Sir Mole III

I’d love to but I’m really bad at math ðŸ˜‰

• Yoda

Seasonality is supportive of a bull move until mid-June.
http://charts.equityclock.com/natural-gas-futures-ng-seasonal-chart

• Yoda
• http://evilspeculator.com Sir Mole III

Zero indicator smelled another rat today. Don’t trade without it.