Education
Now Reading
Blast From The Past: Statistical Musings
50

Blast From The Past: Statistical Musings

by The MoleDecember 27, 2016

While I am taking some time off I thought I may as well repost some of popular educational posts which many of you may not have seen or completely forgotten about. The first one is from early 2012 and covers volatility and kurtosis:

If you run the math you will find that on a daily basis – annually – the skew is close to normal, and but  KURT (i.e. kurtosis) is  large. That is due to compounding error of the VIX distribution and overlapping seasonality data. But nothing is black and white – I believe in conditional statistics and offsetting tail distribution. Common sense tells us that when the VIX is > 25 the market will exhibit a different distribution than when below – especially when you run it seasonally.

Let’s cover some of this as there is a ton of really good information in there. By kurtosis Volar refers to thin tail vs. fat tail distribution. For instance – on a weekly basis what are the percentages of times you get a +/- 0.5% move. What are the percentage of times you get a more violent 2% move? A thin tail market gives you a higher concentration of moves within a small percentage range, let’s say between 0% – 0.5%. A fat tail market gives you a more spread out profile where you see a lifting of the shoulder ranges (i.e. 1% – 2%) and a drop in the middle range (i.e. 0% – 1%). So now that we covered the basics let’s talk about the weather.

Yes, I am not kidding. Every morning you subs get a TICK TrendDay alert – that is your daily weather report based on statistical information captured via the TICK within the first 45 minutes of the NYSE session. Quite often the report nails it on the head and sometimes (like Thursday and Friday) it is completely off the mark. But it’s an attempt to gauge the ‘type of the market’ to expect for the day. Is it going to be a volatile day? Is it going to be a down or up day? When watching the Zero or your own indicators – should you trade divergences or is the short term trend so strong that they don’t matter and VWAP touches are nothing but buying (or selling) opportunities?

The work Volar has been putting together is purely statistical by nature and let me tell you that over 60% of institutional traders use nothing but statistical analysis for their trading. I think that needs to be repeated – a majority of the prop desks run by big institutional players do not care about indicators, golden crosses (or showers), technical analysis in general. If they do then it’s only a minor aspect of what drives their trading algos. It’s almost purely statistical mixed with some AI, genetic algos, HFT stuff, you name it.

For those quant boys ‘conditional statistics’ is everything – I call it the weather. You can express the daily weather in the markets in various ways via statistical conditions – maybe it’s a small number of parameters and maybe it’s hundreds. In the end the system will produce a set of odds for you and perhaps a trading range, entry points, exits, etc. It’s like figuring out if you’re going to take a drive to the mall or take the kids to the park for soccer practice – it all depends on the weather. Only retail rats use the very same algos (i.e. indicators and charts) every single day. Institutional traders have long figured out that what really matters are the ‘road conditions’ and they adjust their trading accordingly.

Which is why I keep telling you guys to pick your markets. The Zero for instance is a very simple tool, obviously geared toward retail traders. It kind of took over where the old ticker tape left off – instead of listening to the racket of the ticker the Zero gives us a feeling of participation and momentum of the trend. That is very valuable I think and it has kept us out of a lot of trouble in the past.

When Volar talks about the VIX being over or under 25 then he refers to essential market conditions that affect the way we should trade. I myself have figured that out when I started writing black box trading systems like Geronimo and evil.rat. Geronimo for instance does very well when the VIX is below 30 but if it’s above then it starts having issues and I actually disable it. It’s the opposite with evil.rat (which I haven’t run for a while now for obvious reasons) – it kicks ass especially during extended high volatility periods as for instance during 2008.

This is what I mean by picking your markets and I am considering extending my work on that end in collaboration with Volar. My vision would be to produce daily and more sophisticated weather reports based on seasonality, volatility, and other essential parameters.

Above graph shows us non seasonally adjusted distributions. By offsetting distribution i mean the holiday’s narrow tails overlap the fall fat tails. (now the VIX complicates this even more as it compounds the data). So things may look “Gaussian” when they are actually not- they are two different distributions. If you break out the months – the aug-oct period is far far from normal and highly skewed to the downside with large fat tails. Where as the “holiday” period is the opposite – it is narrow tailed and not skewed. So the holidays are also not normal, but narrow. I will also add that the high correlation among stocks also breaks the rule of normal distribution and portfolio theory- moreover, non constant distribution of both beta and correlation both imply that gaussian statistics have limits on how one can use them. Or bottom line on gaussian: only use conditional statistics to measure an expected outcome given 2-3 conditions; moreover, never use portfolio theory or VAR because the variables are ever dynamic and not constant.

Basically there are essentially two market conditions that take turns during the year – do you recall Volar’s seasonality charts? There are ‘easy’ times (e.g. the Santa Rally) and then there are more turbulent times. BTW, when I refer to ‘easy times’ then I mean more contained and uni-directional markets. A tough time would be a large correction followed by a fast short squeeze – can be extremely profitable but it’s not easy to trade. If you try then you most likely will have to abandon habits/strategies that work very well during ‘easy times’. There are also months when there is little edge to be had and you should just stay the heck out (i.e. summer IMNSHO).

When Volar talks about the ‘holidays’ then he refers to November until the end of December. Vacation starts after May and that may be a good time to dip into low volatility strategies. The fireworks (i.e. fat tail moves) usually happen in the August to October time frame – I’m sure you are aware of the concentration of market crashes and large corrections that have all occurred in fall.

As for volatility being non-uniform that is my whole point the overlapping non-uniformness makes much look normal or gaussian, when in fact there is 2-8 actual separate, underlying distributions. The other thing i will say is when you run conditions – e.g. if sma > sma_2 or something, the outcome of that particular trade may have normal distribution – regardless of actual market distribution. Which is why I run stats on outcome of conditions – because there is a reason for a normal distribution, not overlapping randomness. I mean if the market is 7% >100ema with the VIX<16 versus the market 15%< 100ema and VIX > 40 those markets will have utterly different expected outcomes- and for good reason. I will say if you are going to test the golden cross or ma’s (on a daily basis) you need to test the slope of the MA (eg. up or down) and then you need to test whether or not the market has made a new high or not. The stats will be much more clear.

Gaussian – if you have used Photoshop you may remember the Gaussian noise filter. It’s basically if you have a more randomness and there’s no clear pattern. Gaussian distribution also measures how many and what concentration of data points fall within a particular measured range – you usually get a bell shaped curve with dropping shoulders toward the 2nd and 3rd standard deviation.

Now what Volar is saying here is key – and in plain old English it means that the outcome of running your statistics depends on the type of market conditions you are looking at. I give you an example – you may find that a simple moving average cross makes you money for months on end but suddenly it starts breaking. What happened? Most likely it’s the market conditions that changes. Perhaps volatility dropped below a threshold, maybe the number of stocks above/below their 200-day SMA reached a certain threshold, or perhaps it’s a combination of factors. The trick here (and that’s the holy grail for the quant crowd) is to come up with a number of test conditions that allow you to find an edge for a particular ‘type’ of market.

It’s very tempting to go overboard and keep adding a boat load of test conditions – after all the more better, right? Well, wrong! I actually do have some experience with that from my time of writing software for a living. Back then we wrote a piece of code and then wrote a few small tests for it to make sure we weed out the majority of bugs early on. You can try to test for everything you can think of but usually the 80/20 rule applies – you catch 80% of the bugs with just some basic and tried test conditions. The same type of thinking applies to writing trading strategies – the root of all evil is premature optimization. Instead of adding more rules or conditions I always try to first figure out why a strategy does not work well during a particular time period. And often it’s the (market) weather that has a bigger effect than shifting/optimizing the strategy’s test parameters. Which means a particular strategy has its time to be active and its time to be silent.

If it snows outside then you’re probably not thinking of heading to the beach for a skinny dip. And if it’s 90 degrees and 80% humidity then soccer practice may not be high on your list and shopping for clothes in that air conditioned mall with your girlfriend may be much less of a painless endeavor. You need to approach the market with a similar perspective – when looking at the tape try to give the weather a bit of consideration. Try to remember which types of trades worked best in what market conditions and you are already light years ahead of the average retail retard.

 


About The Author
The Mole
Mole created Evil Speculator amidst the chaos of the financial crisis in early August of 2008. His vision for Evil Speculator is a refuge of reason, hands-on trading knowledge, and inspiration for traders of all ages and stripes. You can follow him and his nefarious schemes at various social media waterholes below.
  • Mark Shinnick

    Excellent mussing there Mole, history / perspective so damm important….oh, reentered miners.

  • http://ibergamot.blogspot.com/ i Bergamot

    Some of the really small ones look really juicy.
    See that GPL go? Amazing price action. Company is a total POS , imho…

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    ..kinda slow this week.

    bored? How about some history for the younger rats regarding Quants.

    https://www.youtube.com/watch?v=OINqYdkhOAw

  • ridingwaves

    gets thru 1.70 and target looks to be 2.25
    PNF charts have 4.50

  • ridingwaves

    dollar on fire….emerging markets are going to get throttled if it finds its way up to target
    https://s24.postimg.org/4c44cttxx/USD_120_target.png

  • Ronebadger
  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    VIX worthy of paying attention on the daily as well.

    Year End at the Door.

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb
  • Mark Shinnick

    Yeah…so reversed miners to short for the moment.

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    S&P down 18 pts??

    maybe I Bergamot is a sly fox! he he.

    (comment link from 8 days ago)
    http://evilspeculator.com/no_escape_from_winter/#comment-3061962924

  • Tomcat

    20k by YE appears to be an illusion for now…

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    Never say ‘Never’.
    look at the May 10th candle, matched nearly the next day in the opposite direction.
    In this thinly traded Market, all things are possible.
    I’ve seen weird things at OpX, Month end, Quarter End, and Yes, Year End.
    even if you ‘get it right’, you always have to ask…..did I just get lucky?

  • Tomcat

    As usual, you are right on point GG. It would be foolish of me to ignore your regular comments.

  • http://evilspeculator.com Sir Mole III

    I’m gone a few days and you guys are making a mess of things already!

    https://uploads.disquscdn.com/images/1cb617d596d40f26f657cdd4ddd32a6c5e011afb7c8c31288396ea273229ed14.png

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb
  • ridingwaves

    hope your still in NG trade….I think $5-6 is coming fast….Jan is going to be damn cold ….

  • Tomcat

    I exited last night for a quick [chump change] profit. I am not interested in $NG at these levels but I hope you get your desired price

  • Mark Shinnick

    BTW…where’s the cat??…gerb for that matter??

  • Yoda

    Putin did it

  • Yoda

    seriously top quality post you wrote up there.

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    is it *so* hard to get my coffee order correctly?
    it’s three ingredients, that’s it!
    I don’t ask for much.

    https://s3-media3.fl.yelpcdn.com/bphoto/Mj3JKu7gyEakcSjRdc1q0A/348s.jpg

  • Ronebadger

    I have the same problem at my local Panera

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    best I just roll with it then.
    Tell me, did you have a good Boxing Day?

  • Ronebadger

    I’m not in Canada…I’m in NY…I live on an island…

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    Nantucket?

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    wait, that’s MA.
    NY has an island? :-)

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb
  • ridingwaves

    me too….

  • Mark Shinnick

    Yeah…I really had no idea.

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    it’s a counter trend up in a downhill bigger trend in my lens.
    if you can get to $22..go for it!
    😉

    http://stockcharts.com/h-sc/ui?s=GDX&p=D&yr=1&mn=0&dy=0&id=p40916981983

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    stockcharts permalinks don’t work anymore.
    switching to drop-in https://uploads.disquscdn.com/images/e40b14b51618b9fa0d0fd7617ac8c7b35d01a730c4af27c8d7fcc402ee0b61cf.png chart.

  • Mark Shinnick

    Bummer about that permalinks

  • Tomcat

    Anyone else long OIL here?

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    not me. oil is indecisive for me, and gasoline, I’m looking for a drop.

    https://uploads.disquscdn.com/images/15fed3dd56598fc93df00fc0098131a5623efa47334c98e66ecdbdda8d1b069d.png

  • Ronebadger

    I live on the biggest one

  • ridingwaves

    yep…..been long for a while…60+ I will be out….$70 in 2017 is going to happen…think it will a bit later….

  • Tomcat

    Sure but why support that with a gasoline chart???
    Little correlation in my lens

  • Tomcat

    Which contract?

  • ridingwaves

    no contract, long UCO at 8.24 and UWTI at 18, sold out of that position at 25.50

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    it’s a downstream/upstream relationship.

    http://i.investopedia.com/inv/tutorials/site/economics/economics5.gif
    -GG

  • Tomcat

    Stronger dollar shouldnt change that trend (of cheaper commodities).

  • AcoBrasil

    I’m starting to wonder if this leg of the USD rally has peaked. The one-month chart is showing some potential resistance (which corresponds with some potential support for EUR/USD).

    European bond sales and data has been relatively strong this week, which could fuel speculation regarding the easing of ECB stimulus. The Trump stock market honeymoon may also be drawing to a close as we get more detail regarding what policies he will actually be able to implement.

    In any case, I bought FXE this morning to hedge some European travel planned later this year. Also holding long a few Brazilian ADRs (despite the USD strength, the BRL has been on a tear for the last couple of weeks).

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    it’s a tough racket. anything above 110 is uncharted IMHO.

    at this point, it’s purgatory between some monthly horizontals.

    https://uploads.disquscdn.com/images/74db9c68fe077a034d8c91c7cb413b63604afddb53bd0a10ae16a1feac0c4354.png

  • ridingwaves

    Happy New Year to you folks, I’m out of here for the weekend and beyond….
    2017 should be a great year….time to find some waves…..

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb
  • Ronebadger

    LONG ISLAND, NY, has a population of about 7.7 million in its 118
    mile length from New York Harbor to its eastern terminus on the Atlantic
    Ocean at Montauk Point. It is about 28
    miles in width from Long Island Sound on the north to the Atlantic Ocean
    on the south. Its western end includes the New York City boroughs of
    Queens and Brooklyn. When Long Island is defined it generally excludes
    the Queens and Brooklyn sections of New York City. Long Island’s
    political divisions consist of Nassau and Suffolk Counties, 11 towns and
    109 villages — all with their own administrations. The area has had an
    historical past from the arrival of the first settlers. Here are
    interesting facts:

    Mastic was the home of William Floyd, one of the signers of the Declaration of Independence.

    Long Island’s

    Deep Hollow Ranch in Montauk is the site of the oldest cattle ranch in
    America, built in 1658, and birthplace of the American cowboy

    The Lighthouse at Montauk became NY’s first coastal beacon in 1796.

    The first radio transmission, by wireless inventor Guglielmo Marconi, was in 1901 on Fire Island Avenue in Babylon

    In 1927, Charles Lindbergh took off from Roosevelt Field, as he
    embarked on the first non-stop solo flight across the Atlantic Ocean.

    America’s first supermarket, King Kullen, started on Long Island in 1930.

    Levittown, the first suburbia in the U.S., was built on Long Island in 1947.

    The Lunar Module which landed men on the moon in 1969, was built on Long Island by the Grumman Corp.

    The only working water mill and windmill in the US are located in the Long Island community of Water Mill.

    Richard Nixon’s deceased dog Checkers is buried at Long Island’s Bide-a-Wee Pet Cemetery.

    When the Brooklyn Bridge was completed in 1883, it became the first
    land -transport route between Long Island and mainland USA. Before that,
    the only way to travel between the two was by boat.

    Long Island
    was sculpted by melted glaciers, which is why there is a clear
    difference between the hilly North Shore and the flat South Shore of
    Long Island.

    Long Island is the most populated island in the United States and the

    17th most populated island in the world. Interesting fact: Long Island
    is more populated than the entire country of Jamaica and of Ireland!

    If Long Island was its own state, it would rank 12th in terms of population.

    Robert Moses creation of a cars-only parkway system created the modern
    suburbs of Long Island and influenced a generation of engineers,
    architects and urban planners across the nation

    The North Shore
    is known for its large mansions, exclusive vacationing, and rich history
    of its affluent citizens and architecture. The South Shore hosts more
    entertainment and sports venues together with Atlantic Ocean beaches
    such as Jones Beach.

    The largest industrial park in the US is
    located on Long Island – The Haupauge Industrial Park which houses
    1,300 companies and employs more than 55,000 people.

    Together
    Nassau and Suffolk counties’ nearly 180 fire agencies possess more fire
    trucks than New York City and Los Angeles County combined.

    Suffolk County is the leading agricultural county in New York State based on the wholesale value of its farm products.

    More 7-Eleven coffee is purchased on Long Island than in all of California.

    George Washington commissioned the Montauk Lighthouse so that the ships
    coming in from Europe could identify the Atlantic Ocean from the Long
    Island Sound.

    Fire Island got its name in the 17th century when it built fires to identify the coastline to the ships in the ocean.

    After World War II, the Japanese were nationalized and able to fly to
    just 5 places in the country with Idewild Airport (JFK International)
    being one, which is why Canon, Nikon and Olympus located their corporate
    offices there.

    The Town of Hempstead with 800,000 residents is
    the largest township in the country, larger than Boston, San Francisco
    and Baltimore.

    The Nassau and Suffolk population is greater than 20 of 50 states.

    Hicksville was the original home of Fudpucker World Airlines, the Worlds only steam powered Airline.

    Long Island is the largest college “town” in the US with 175,000 students registered in 21 colleges.

    Long Island has the highest level of volunteerism in the world.

  • http://gerb-reloaded.blogspot.com/ Gold_Gerb
  • http://gerb-reloaded.blogspot.com/ Gold_Gerb

    Last chart of the year. Nothing epic.

    https://uploads.disquscdn.com/images/56bcfe8b887a716f18f74d46f7ab282046de66dd38d337c1b7ab6d7afc75c0cd.png

    2016 Sucked Wind for me, Personally.
    Frankly, I plan to stay up late tomorrow, and see it Die. No greater Joy.

    Time is short, spend it wisely.
    -GG

  • http://www.linkedin.com/in/sharondsessions/ Sharon

    Happy New Year to everyone.
    Still in Texas visiting my family until next week. Blessings to all

  • http://evilspeculator.com Sir Mole III

    T’was a shit year but we made the best of it. That’s what really matters.