Now Reading
Fooled By Randomness

Fooled By Randomness

by The MoleJune 8, 2017

It is Mario Draghi’s turn to torment market participants this morning, which means a market overview will have to wait until the wave of volatility has washed over us and hopefully left some of our open campaigns intact. In the interim I decided to channel my inner Nicholas Taleb and ruin your collective day by singlehandedly smashing what you hold most dear as traders, i.e. your perspective on how markets function and your ability to anticipate what may come next. And if you think I am joking then you are most likely doubly mistaken. Read on at your own peril:


Still here? Very well then. Take a look at the chart above and then grab a piece of paper and write down your thoughts about what you are seeing. Would you trade this chart? What are your general impressions about it? Do you see any entry opportunities? Are we perhaps counting waves or looking at specific cycles? When you’re done please follow me to exhibit B:


I tell you what I think. Not a bad chart at first sight I’d say but it has its periods of sideways volatility. But when it gets going it really ramps so a trend trading system here may just work fine. Playing the swings may also work if you slap a Bollinger on it. Do you agree? Disagree? Make not of all that and then follow me to exhibit C:


Ouch, not a chart I would want to be trading – that looks pretty nasty. That’s usually the type of tape I try to avoid. Although it has trending periods it seems to turn on a dime at a moment’s notice. Agree – disagree? Take note.

It’s All Just Noise

I’m sure your curiosity has peaked by now and you wonder what the purpose of today’s exercise may be. And the sad truth of the matter is that it’s all nothing but noise. All three charts above were produced purely by the power of a simple python script using a vanilla random function:

import numpy as np
import pandas as pd

import statsmodels
import statsmodels.api as sm
from statsmodels.tsa.stattools import coint
# just set the seed for the random number generator

import matplotlib.pyplot as plt

X_returns = np.random.normal(0, 1, 10000) # Generate the daily returns
# sum them and shift all the prices up into a reasonable range
X = pd.Series(np.cumsum(X_returns), name=’X’) + 50 # so the chart starts at 50

Order And Chaos

Trust me, I know how you feel – it’s like the floor just gave way underneath you and took with it all the technical trading knowledge you’ve accumulated over the years. The good news is that it’s not as bad as you think, if that makes you feel any better. Let me explain. Over the past few years I spent quite a bit of time investigating fractal patterns in financial data series. A major aspect of my work was the use of machine learning tools in combination with time series classification parsers to find recurrent patterns, also called ‘motifs’. Some may call them fractals although technically speaking fractals are self-recurring on larger intervals, so I usually prefer the term motif as we normally look for the same recurring pattern within the same time window.

Turns out that I actually wrote a multiple-dimension parser and parsed for the same motif on a series of time windows, so in the end a fractal it is. What I learned in months of testing is that there are in fact recurring fractals in financial time series. However, the type and frequency significantly differ from one symbol to the next, plus the number of recurring patterns/fractals/motifs only account for about 5% of the series. Which means that 95% of it is noise, or more correctly what is known as a ‘random walk’.

All Models Are Wrong, But Some Are Useful

So is everything we have learned about the markets complete horse wash? Are there in fact no technical patterns and are we fooling ourselves? Well, yes but no. In the words of George Box (one of the great statistical minds of the 20th century): “all models are wrong, but some are useful.” In reality there is most definitely a significant amount of randomness to all financial markets. But I would call it ‘guided randomness’ because in between the noise are the actions of human traders who look at a chart and believe that buying or selling at a certain threshold makes a lot of sense. And as such it often becomes a self fulfilling prophecy, because just like water it seems that a random walk simply follows the path of least resistance and then finds its next level. Which of course may explain why ‘following the herd’ works so well until it doesn’t 

Do Entries Matter?

But still two of those random charts I posted above look pretty tradeable, don’t they? Which makes one think of course whether or not the true key to profitability and success as a trader lies in picking entries. And of course we already know that it doesn’t because markets change all the time and so do the systems that operate successfully during any arbitrary trading period. Meaning you may be picking great entries like your nose one quarter and then lose it all back and then some doing the very same thing the next.

Van Tharp once stated that [successful trading is 40% risk control and 60% self-control. In turn, the risk control portion is one half money management and one half market analysis. Thus, market analysis is only about 20% of successful trading. Yet most traders emphasize market analysis while avoiding self-control and de-emphasizing risk control. To become successful, traders need to invert their priorities].

We’ve talked about self control many times here but let’s set that aside for now. Focusing on the remaining 40% only half (i.e. 20%) supposedly should be devoted to market analysis. I think that’s a vast over estimation and my own belief is that market analysis should account for not more than 5% of your trading. A lot more time should be devoted to campaign management, risk management, and capital commitment.  Which are activities that are by definition a lot more analytical than technical. Instead of reading charts to find entries we should be spending a lot more time analyzing how to extract maximum returns on entries we have been taking. Of course as a financial blogger that would most likely reduce my audience by a significant margin.

About The Author
The Mole
Mole created Evil Speculator amidst the chaos of the financial crisis in early August of 2008. His vision for Evil Speculator is a refuge of reason, hands-on trading knowledge, and inspiration for traders of all ages and stripes. You can follow him and his nefarious schemes at the usual social media waterholes.
Enjoyed this post? Consider a small donation to keep those evil deeds coming!

BTC: 1MwMJifeBU3YziDoLLu8S54Vg4cbnJxvpL
BCH: qqxflhnr0jcfj4nejw75klmpcsfsp68exukcr0a29e
ETH: 0x9D0824b9553346df7EFB6B76DBAd1E2763bE6Ef1
LTC: LUuoD6sDWgbqSgnpo5hceYPnTD9MAvxi6c

  • ridingwaves

    Excellent post Mole! the randomness factor eats me up quite a bit….

  • Mark Shinnick

    Yes Mole, exceptionally well done explanation.

  • Gold_Gerb

    Mole’s onto something big.

    Forests are random, or are they?

  • Mark Shinnick

    This is just too cool, thanks. Perhaps no matter the presumed randomness, natural laws still somehow impose on the outcome.

  • Kidd Cudi

    Have you discounted the possibility that you are pattern matching where none exists in this post? I.e. you made some random charts and also looked at some real market charts and (potentially) falsely concluded that they were the same? For while the random charts you made are (nearly) truly random, the market charts are not created through a random process, but rather an extremely complex process that consists of humans (and machines) making decisions. Thus, since the real charts and the fake charts are created in two different ways, their visual similarity may not be a “true” match, but may be an illusion.

    We see these charts and think, “oh, everything is just random,” but it is also possible that we are incorrectly grouping separate phenomena.

  • Kidd Cudi

    Although I will say that the Van Tharp quote is one of the most important trading lessons I’ve ever learned.

  • Mark Shinnick

    Mainly I think it boils down to tradability; its very hard not to accept this as a reasonable model to guide our strategies.

  • ridingwaves

    interesting counter point and it could even break down further in sector separate phenomena..

  • OJuice

    The widow maker living up to its reputation this morning. If it can get up over 3.05 and hold there might be some nice upside.

  • OzarkHillBilly

    And a hard lesson for many, including me.

  • OzarkHillBilly

    Great post. Since you spend a fair amount of time discussing the reality of trading, I hope you’re not too disappointed by the size of your audience. 🙂

    I’m reminded of an economics professor I had when I was finally wrapping up college. I was practically an old man compared to everyone else in my classes, and definitely the only student who had done any trading and investing, or even had any experience paying their own bills. The professor had previously worked for a central bank, and he certainly took a dim view of anyone trying to trade the markets with any prospect for long term success. Any questions from me (after class) regarding tradeable patterns or trading systems was always met with yet another mention of “Fooled by Randomness” or “A Random Walk Down Wall Street.”

    I think I got more out of reading Taleb than my professor did. I tended to take Taleb’s work as wisdom to be added to enhance own thinking, and he took it as more evidence that it’s rather futile to try and “get lucky” in the market. Of course, this was coming from a guy who had at least one econ post doctorate degree and several years of experience crunching numbers at a central bank. He ended up teaching because he really only saw three options for himself; continue at the central bank, become a professor with a chance to pursue some research, or go work for a big US bank and run the numbers which decide who gets credit, and how much to give them (primarily regarding credit card offers).

  • ridingwaves

    XLF daily, wham bam thank you maam….could lift everything higher if it breaks thru here..

  • Tomcat

    The main message I got out of reading Taleb’s “Fooled by Randomness” book, was reminiscing of the expression that “Whatever you do in life will be insignificant, but it’s very important that you do it”. In other words, Taleb was saying that sure all the preparations that you do and all the effort that you put in is important, but not the only factor that will dictate outcome, bc you are but a very small part of the big equation.

  • Mark Shinnick

    Yep, Butterfly Effect .

  • Sir Mole III

    it’s interesting how two people can interpret the very same book or paper in completely different ways. IMHO your professor had a more pessimistic view of being a trader because he never really had been one. Being a banker doesn’t make you a trader, in particular as your perspective is actually rather narrow (i.e. bonds, equities, interest rates, economic measures).

    If being a trader was impossible then there wouldn’t be people who actually pull it off. After all I trade right here in front of a large audiencec on a daily basis and if I wasn’t banking me or my subs any coin then I would have been out of business a long time ago.

  • Sir Mole III

    No, not really what he’s saying. I think the point of what Taleb expresses there is that we vastly overestimate the impact our actions have on the rest of the world. But in the context of your own reality doing your very best is extremely important. Plus just as with market moves there are outliers, e.g. people who do what they do but they happen to affect many others. Examples: Einstein, Hitler, Newton, J.S. Bach, etc.

    You can never really know how your actions will affect your life and that of others as well as how they will be valued by others. I mentioned J.S. Bach who is a great example of that. You may be surprised to learn that Bach was NOT known as a composer in his time but as a skilled organist. He had a 9 to 5 job of sorts and simply wrote music on contract as part of his duties. With new music needed for church services needed each week he threw himself into writing cantatas. He also created interpretations of the bible using choruses, arias, etc. Understand that many of those pieces were only performed once and Bach needed to produce new material the week after. Many are known to be master pieces today and many more have been lost in time.

    Bach would be dumbstruck to learn how his reputation as one of the best composers of all time. That didn’t happen during his lifetime and not even during that of his own sons. Only over 100 years later his reputation as a composer suddenly surged and of course today every single person on the planet who enjoys classical music has heard of good old Bach.

    So the point here is exactly what Taleb suggested. Bach didn’t aspire to be one of the most famous composers of all time. His passion was dedicated to the worship of God in all his glory and his deep and unwavering belief in the Lutheran church. That sentiment, that world view is uniquely expressed in his music and the feelings it invokes are closely tied to that devout world view of the Baroque. By simply doing what he was doing as best as possible, no matter how insignificant it may have appeared at the time, he managed to make a huge mark on human art and history.

  • Sir Mole III

    I think the essence of my write up is that we as humans, despite our advanced ability to recognize patterns (or due to it perhaps) are incapable of clearly separating noise from correlated patterns.

  • Tomcat

    Its rather remarkable how inherently dumb and sheeple mentality we humans are we don’t realize the value of something until it’s gone.

  • Mark Shinnick

    Yes, the productive force potential of enough human creativity seems to be life’s biggest wildcard; nobody seems to really know where it really comes from, but if it happens to hit some natural fractal stuff seems to be unstoppable.

  • Mark Shinnick

    Its a great point; as proven by those who are not so, most of us are left too clueless about so friggin much going on all around us. For example, examples abound that our psychic functioning capabilities have devolved over time.

  • Mark Shinnick

    That’s the thing; we use objective systems but ultimatly its up to more important of our individual capacities/awareness to execute and manage trading.

  • OzarkHillBilly

    Right. His idea of a trader was a multi-millionaire type on Wall Street, or rather some fund or trading desk at GS or JPM. He really couldn’t conceive that a relatively small trader could have success, outside of luck and simply buying and holding the right investments.

    Somewhat off topic ……. I had a couple of classes with a younger associate professor at the same school who was just getting started with his career. He was from a central American country and really had to work hard all of his life to find success, and he was passionate about what he wanted to achieve and then contribute to the world. We spoke quite a bit after hours; he had a tough time understanding the attitudes of his typical US students, which could be characterized as uninterested, uncaring, and unquestioning. I tried to help him understand how schooling had probably unfolded for the majority of his students, and they were there simply because they were “smart” enough to be there and that’s what they had been told to do.

    In one of these classes, I made friends with a guy from Nigeria who had spent most of his educational years in British boarding schools. We did a presentation on “Peak Oil” even though we both thought it was largely BS, and told the professor as much before we did the PowerPoint. But we made it convincing and a bit sensationalistic, with a fair amount of doom and gloom. We got an “A+” largely because we got the whole class to pay attention and we made the professor laugh.

  • Gold_Gerb

    I found out at lunch, that I influenced two very close co-workers to get their ass in gear and SAVE. Apparently I just impacted 12 children’s futures.

    go figure.

  • Mark Shinnick

    Yes, this exact problem is recognised and dealt with in the field of remote viewing, where the flow of raw information is typically interfered with / prejudiced / distorted by the person. Its been my very immature new study into this issue for a couple of months now… no way ready for Primetime.

  • Gold_Gerb
  • Sir Mole III

    Yup, he’s a mean lean algo solving machine.

  • Sir Mole III

    That’s awesome, GG.

  • Sir Mole III

    To what extent? Modern brains are capable of doing things medieval brains were incapable of. On the other hand we have lost many hunting and fighting skills. It’s a give and take I think.

  • Mark Shinnick

    As the remote viewing guys explain it, the evolutionary premium went to those who were more successful avoiding danger to themselves and their clan; they were deemed the leaders demonstrating something people sensed was different in a subjective way. Evidently, that’s how the US Army selected its candidates for the early remote viewing experiments; those who seemed to be unusually capable of mission success beyond their peers. So, yes, give-and-take but holistically recouping some of what our scientific method has given up is where I’m looking to hone edge.

  • Sir Mole III

    Iiiii think I’ll stick to my algos – but best of luck! 😉

  • Mark Shinnick never thought that would ever impress a teutonic:)

  • Scott Phillips

    You haven’t seen his paper about the “intellectual yet idiot”?

    He says “never trust a man who doesn’t deadlift”

  • Scott Phillips

    Taleb is a fascinating example of how we build these incredible theories to justify pre existing worldview.

    Taleb’s dad was finance minister in Lebanon, when it was a beautiful country, the Paris of the East. When he was a kid, inside 6 months their country collapsed and they were refugees.

    Is there any doubt he would readily believe that “black swans” could come and fuck your whole life up? His later books are really just building on his formative experiences.

    Compare and contrast with for example, Tim Knight. There’s a guy who used to work at AAPL in the early days, who has lived in Silicon Valley his whole life, and never went long any of the stocks from his home town, which would have made him rich as fuck. Because he is a glass-half-full pessimistic piker with low self esteem, he wants things to fall apart so he can tell the world how clever he is (to prop up his low self esteem). Even now, he’s shorting away, because his world view is that he is a smart guy who needs to show the world how smart he is.

    Seykota says “we all get what we want from the markets. some people like to lose money so they win by losing”

    Van Tharp has a lot of wisdom in trying to recognise and solve these type of fatal flaws before you get anywhere near an open trade.

  • Scott Phillips

    Excellent point

  • Scott Phillips

    To medieval man we would seem to be dull companions I think. A lot of social interaction stuff has been lost with radio/tv/internet/facebook

  • Scott Phillips

    If it breaks daily high, it’s a nice trade on my systems. I’m taking it if it triggers.

  • sutluc

    I’m not implying that you and your subs are not successful traders, I think you are,
    however the blogsphere is full of subscription services that are consistent money losers except for the person collecting the fees.
    The out of business argument is not valid.

    Edit: I should add that I enjoyed the post.

  • Sir Mole III
  • Sir Mole III

    By the way it’s glass half empty pessimistic piker.

  • Sir Mole III

    Look at these ‘services’ and look at what I do here. Tell the difference.

  • ridingwaves

    I just keep looking at TSLA and BABA 52 wk lows vs. today’s price….
    pure doubles on both of them….there are so many trades, maybe complexity just overwhelms a mundane task….

    kind of like the auto retail parts business short I mentioned here….I see a disruption or black swan for those symbols via Amazon…based off my experience…simple stuff still wins…

  • ridingwaves

    XLF is now in break out territory…

  • Darrell Osgood

    Question: Were those the first three random outputs or did you cherry pick those because they were illustrative? I suspect the latter. Either way, well played – the point came across loud and clear. This is why we always corroborate with other data.