System Trading
Now Reading
Exercises in System Building – Part 2

Exercises in System Building – Part 2

by ScottMarch 9, 2017

Let’s be honest. Most of us came into trading, expecting to be good at it, discovered we weren’t, and tried to fix that before we blew up our accounts. I desperately scoured the internet for people I thought knew what they were doing. When I could I reverse engineered, or learned, or stole, their “setups”. I did their courses and read their books. If we were lucky the gurus we decided to pilfer from had real stuff to steal. Sometimes I was lucky, sometimes not.

What I’m covering today is the bedrock of trading. How to know if your setup is an edge. The gurus of trading like to emphasise the mental game of trading, but without a real edge in the market you are wasting your time.

“90% of trading is mental – the other half is solid math” – Laurent Bernut

In the comment section of the previous post Zsoult was talking about Inside Periods, a topic close to my heart, which we mentioned in the previous post also.


The problem with just backtesting the Inside period, is that you are not just testing one thing, but the complicated interaction of many different variables. A fragile fucking thing indeud.

If the setup bar is small, the distance to target is small, so it is possible to get a small win not just based on some inherent edge, just because you were only aiming for a small win within the normal range of movement. So you are testing setup bar length, how quickly you move stop to breakeven, the target and stop logic, and anything else you want to build into your system. You literally have no idea if there is an inherent edge, or different factors of randomness conspiring to fool you. Humans are easily fooled, and I’m as gullible as the next fool.

Let’s settle the argument, once and for all. And for the record, I’ve personally risked my own money well over 1000 times on this setup, I have skin in the game here. I really, really, wouldn’t like to think I was that dumb fucking yutz who got conned by a guru scam.

Methodology: I tested EURUSD 60m bar inside periods for the month of March 2017 to date. Not many data points but you get the idea if you want to continue the work. You can check my work on this public spreadsheet here or even fill in more data yourself.

What I did is measure the change in close from the setup bar to the bar of entry, and then the subsequent bar. If we have an edge we should see price tend to move in the direction of the setup working on both bars, positive if its a long setup, and negative if it’s a short setup. You can see clearly it is a weak negative correlation. What that means is that if you get long, what you would like to see the next further bar is the trade to move in your favour. Instead you see the exact opposite. When you get short what you would like to see is EURUSD get weaker after you are short. Instead it has a statistical tendency to get stronger. Well, that is fucked, isn’t it? Not just an edge but the opposite of an edge.

inside period edge

Let’s be clear. Saying it’s not an edge on EURUSD 60m bars is not the same as saying it’s not an edge everywhere. And the sample size is too small, this is just an illustration.

On EURUSD 480m bars on 502 real world trades (not backtest) it did 18R from 502 trades to the long side. To the short side it did slightly better,  47R from 541 trades. A net total of .06 expectancy before commissions. Roughly breakeven after paying brokerage on 1043 trades. Could be worse, it could have been a terrible loser. But in a way this is a tragedy, a not-edge that doesn’t lose much money will go through periods where it behaves like an edge, and makes money. In the long run, it’s going to just waste your time, and drive you crazy. This is why this setup is not part of Mole’s Crazy Ivan BTW. It also kind of fits with a hypothesis that lower timeframes are marginally less edge due to more noise and less signal, which would make intuitive sense.

Now, as I was typing this, I had a horrible feeling, that my own production systems might be based on this same kind of spurious correlation and curve fitted non-edge bullshit (let’s call it the Ivan method for now). After all I’ve been trading my current systems since 2014, before I learned all this new stuff. Entirely possible that I’m the one looking stupid here, turning over a billion dollars a year (yeah I really do) on a non-edge.

So as much as I wanted to hide from it, I’m forced to eat my own cooking, and test my own setups, which are based on a hammer candle in an uptrend, which pulls back and touches the 8EMA. A hammer candle is defined objectively as a candle with a close in the upper half of the range and a lower wick equal to or greater than 2x the real body of the candle. The 8EMA means nothing by itself, it is just objective proxy for a “reasonable pullback”.

thor edge

Now, we can see that even on a small amount of data we clearly have a weak positive correlation. I’ll obviously fill this in completely over time, but 15 minutes of work and I have something illustrative and able to sketch out a quick test of hypothesis.

To those who are thinking this is too hard, 15 minutes on a scrap of paper in my laptop in my hotel room, transcribe to google docs spreadsheet, and I have a viable quick test of a trading hypothesis.

What that is telling us is that the trend is tending to continue, which is exactly what we want to see for a trend following setup (the opposite for a mean reverting setup obviously). The advantage of testing this way is that we aren’t testing the exit mechanism, the stop distance, how quickly we move to breakeven, or any other filter conditions we might have. We are plainly and simply testing the raw edge of the setup.

When we test ONE THING, we cannot get fooled by randomness.

If we don’t have an edge, all the rest of the system building process is garbage. Garbage in, garbage out. We can produce all the backtests we want, but even non-edges can produce impressive backtests over hundreds of trades samples.

What kinds of things could we test with this?

Almost any hypothesis. Does price have a mean reverting tendency at bollinger/keltner/envelope extremes. I don’t mean to pick on the Gerb but let’s look at an image he posted in the comment section today.


Now obviously pointing to two prior instances of something happening doesn’t mean a thing. For example the last two Tuesdays of the month both went up, does that mean Tuesdays have an inherent bullish bias? (no, it doesn’t). So how could we test this concept without involving too much bias?

Firstly, mean reversion tendency is HIGHLY MARKET SPECIFIC. You can’t do it like trend following by lumping a whole pile of markets together and hoping one of them will trend. So if you want to test XLE for mean reversion, you must *just* test XLE. What you would do in this instance is test a few days before the touch of the mid line, and plot that against a few days after the touch of the mid line. If the mid line in the chart above means anything at all, we should see a strong negative correlation when plotted.


We could test the tendency of markets to revert to mean at an extreme, we could test whether certain days of week or month or times of day have a statistical edge. We could test whether momentum effect (strong markets stay strong and weak ones stay weak) is real or imaginary. We could test intermarket correlations and their breakdown (the basis of Victor Neiderhoffer’s game).

So now we have an extremely powerful tool in our toolbox. We can use that tool to build ROBUST systems that don’t fall apart at the most inopportune times. My friend Daniel told me once “Scott, you build great systems, that suck sometimes.” And this is why, I never used this kind of analysis to build them. Without this, your systems are fragile, and you try and fix them by adding more conditions and rules, but every extra rule makes them more fragile, not less fragile.

So now then, how about our prospective systems? What I’m proposing is to test two different things, a momentum system and a mean reversion system. What timeframe should we be building for? Please vote in the comment section and we will build accordingly.

The two setups I’d like to test are

Ivan Krastins Fakeout Sell off a historical extreme like a donchian channel


And one of Ken Long’s (a brilliant trader whom I admire) setups. What we are looking for here is a situation where price has left the 1 standard deviation bollinger band AND a short and longer period linear regression line also have left the 1 standard deviation bollinger. The rationale here is that the 1 standard deviation is a “reasonably strong” market without being at an extreme that will be subject to mean reversion tendencies.


Let me know your thoughts on markets and timeframes in the comment section. Those who want to participate we will crowdsource the grunt work out and hopefull build something useable.

As for today’s markets, we have what at first glance looks like a textbook bullish setup. A hammer candle in an uptrend. A perfectly proportioned pullback in a historically strong bull move. I think it smells like a trap.

quick scalp

Look a little closer. The big bar a few days ago was late bulls who have been sitting on the sidelines for a few weeks watching the market go up, wishing they were long. That big bar is literally traders going “fuckit, I just need to be long this market, and I don’t care what it costs”. A trader in a new position is a weak hand. If you held your position since the lows you don’t really care about a little pullback, you are secure in your profit, but if you bought last week and are underwater now, you start to become a little nervous. Any weakness here sets off a cascade of further weakness as those weak hands give up. In my opinion the highest probability is an attempt at going up, success or failure uncertain, without breaking out to a new leg up. I don’t believe we have betting odds of getting to fresh highs at this point. If your horizon was very short you could get long for a quick scalp, but I would be out with a quick win here. In my opinion, there are easier setups than this.

One other thing (which we will cover in the coming week) is that this move is a “low volatility melt up” which has very specific characteristics. At the first sign of volatility in a low vol melt up, you want to be open to both long and short again.

Tomorrow we resume the tape reading again. I plan to alternate between tape reading and system building. For your information, this is all preliminary to me releasing very comprehensive and much more detailed online courses in both these things, as well as all my production systems, which Mole will have a whitelabel version of to sell to you.

Questions in the comment section, you have me for 12 more days until Mole gets back.




About The Author

  • Sir Mole III
  • Scott Phillips

    Nicely caught!

  • Gold_Gerb

    I like it! I was thinking about this yesterday!
    have to finish my West Coast day job first.

  • Scott Phillips

    Good stuff mate, if you need help just post it in a google sheet and I’ll fix it for you 🙂

    For you especially this is a very powerful tool

  • ZigZag

    Hmm, which to choose? I guess the Ken Long setup. I’d need to check my notes, but the Ivan system looks also like a Linda Raschke failed breakout setup? I prefer shorter time frames since that is primarily what I trade, but it would be super interesting to compare the results of whatever the tribe votes for on say, /CL between 5 minute and daily time frames.

  • Scott Phillips

    I’ll test one timeframe of the ken long and one of the fakeout. The fakeout is not dissimilar to Linda’s (who I think is amazing) turtle soup setup, the difference being that she has a better entry point at the old spike high. Ivan waits for a break of the low of the setup bar.

    For the record the fakeout is in my opinion the best setup from Ivan’s game. His RTV and RT setups are also good, but there are certain circumstances where reversal setups are a sucker play (in strong trends obviously)

    The advantage of the fakeout is that its almost by definition a trading range, so you don’t have to do too much clumsy and inaccurate (read FRAGILE) filtering of things.

    Remember more rules = more moving parts = complex interactions we don’t fully understand = more things to go wrong. If there are 5 things that can go wrong, Murphys law tells us that one day all 5 things will start to go wrong.

    If you are purely measuring trading edge, it is a simple matter to include OBJECTIVE THRESHOLD tests for rsquared values (the statistical measure of how well your regression line fits the scatter plot) and p values (more pointy head math speak that I will cover in more detail later)

    What I mean is that it’s easier to say “stop trading when my trend following setups stop trending” than “stop trading when I get 7 losers in a row” because statistically you are likely to get those 7 losers just before your system starts working again.

    The rationale is sound, that roughly 75% of attempted breakouts from trading ranges fail.

    Also the rationale on Ken’s system is strong too. He finds the strongest markets in a statistical sense from a small universe, and trades them at 3min timeframes intraday.

    I think what makes most sense is to test the fakeout on daily, and the Ken Long intraday on whatever timeframes you prefer.

    I’m open to ideas, really. Whoever wants to pitch in and do the work, will get the full system at the end.

    And If Mole and I go on to program it and backtest it, then sell it, we will probably charge $500 for it, so by getting in and doing the work, you get experience and get the chance to get a valuable thing for free.

  • Greenlander

    Have you done any systems on entering low volatility consolidation patterns within an uptrend?

  • Greenlander

    Not the best example as not a clear uptrend. Was possible that the move could’ve gone down rather than up.

  • Greenlander
  • Scott Phillips

    Why yes I have, very strong edge particularly in a low-medium volatility bull market in stocks.

    The problem here is selection bias. You could test negative edge entries in a bull market and they will probably work.

    I would be extremely hesitant to run something like this without a market regime filter or equity curve scale in, which I’ll be covering in the coming weeks.

  • Greenlander

    Cool. Would really like to see yr thoughts on market regime filter or equity curve scale in.

  • Greenlander

    Oh something else (and I figure this is obvious shit to people who have done a lot of systems), do you ever add additional criteria to increase yr expectancy/ win %?

    Say for example, I am trading a stock trading within a low volatility consolidation pattern within an uptrend but then I add on the following entry requirements

    1. sector must also be in uptrend (will refer to associated ETF and check the leading stocks), i.e. sector’s activity act as clear tailwinds
    2. hitting lower end of range on both daily Bollingers
    3. decreasing volume within consolidation range

  • Scott Phillips

    Market regime filter Mole and I have gone around the block a number of times with all kinds of complicated stuff.

    Bottom line, something simple works as well as something complicated most of the time.

    50EMA > 100EMA is a decent proxy for trending behaviour in daily chart stock systems. Sounds dumb but it’s nearly as good as the complicated quant things.

  • Greenlander

    I have one mean reversion VXX short that has a bunch of requirements but the thing almost never triggers which is ok w me. Last time was pre-election and I’m not even sure it will give me a setup this year.

  • Scott Phillips

    Yes and for most systems at least one of these things is necessary.

    Here’s the catch. Each extra thing you add makes it exponentially more likely to fuck itself at some point.

    It’s so tempting to add that extra rule. Extra condition. And keep adding.

    You can do a few, but more than that you are adding fragility in an exponential way, so normal drawdowns are going to be catastrophic.

    Some things are necessary. Trend filters in trend following systems. Mean reversion in the direction of prevailing trend works markedly better. Stocks don’t behave the same long and short (no stupid chartists, its not just the same chart upside down, greed and fear are completely different)

    Take a look at Mole’s original scalpius system. It literally traded the house down, did 60R in 2 months. Then BAM! Got its face ripped off. The system concept is robust, the system itself had too many moving parts. And we learned from that.

  • Scott Phillips

    That’s whats stopped me from developing vix systems in the past. Its a very very solid edge, but happens so infrequently

  • segaa

    The problem with adding more noisy signals is getting in the final combination more noise and less signal, moreover many indicators are highly correlated, so actually you are even not adding more signal, just noise…

  • ZigZag

    Well, I have no coding skills, but hope to follow along and learn. This is very interesting.

  • ZigZag

    BTW and speaking of volatility, I know that Mole is on vacay (and he is King of all things vix-related) but WTF is going on with the VIX? we’ve had a definite downtrend for days now, and the vix doesn’t seem to have participated much. Was wondering if hedging is being accomplished by selling calls.

  • Scott Phillips

    A few days of down movement after the longest consecutive run up since 2007 is hardly bearish. Not surprised that VIX isn’t participating.

  • Scott Phillips

    I don’t use indicators personally for this and many other reasons.

  • Scott Phillips

    You don’t need coding skills to build systems. Sure they help, but not absolutely necessary. And coding alone isn’t worth shit.

    There is definite advantage to doing your backtesting with pen and paper. Ask Bobby, he has years of backtesting done on his systems with pen and paper just like me.

    What this gives you is a down in the weeds intimate knowledge of your system that you simply cannot get from pressing run and loading a spreadsheet.

    Of course the real backtesting method produces a better result, allows you to iterate through changes many times faster. But really, that doesn’t mean a thing if you turn your system off or change it after the first 15R drawdown.

  • Scott Phillips

    Scaling in is for chumps on trending setups. It makes logical sense in a pairs trade or mean reversion sense.

    For example if you are expecting mean reversion on a 3 standard deviation move, and you get a 4 standard deviation move, you should be MORE certain about your idea, not less.

  • Scott Phillips

    Oh I just read “equity curve scale in” – and yes that is a superb way to trade, I’ll be covering it in detail later on in the next weeks. It really does make an incredible difference

  • ridingwaves

    most likely the vix complacency period has been thrust upon us….in 2007 it was similar….

  • BobbyLow

    Mornin Folks, the last of my DUST got swept away this AM and I only have one open position left and that’s some short Silver. Overall, it’s been an outstanding last few days that has almost totally made up for the previous 8 weeks of shoveling shit against the tide. 🙂

  • ridingwaves

    I nailed the 48.50 target for support on CL a couple days ago…almost to the .001
    we are both rolling….

  • Yoda

    weird tape today

  • ridingwaves

    traders positioning for next week, the jobs report screams rate hike….maybe the fed raises .50….that would tame animal spirits quickly

  • Yoda

    Are you planning to reverse the trade to long?

  • Yoda

    Tape “should” have been running with a gap up like a hare. The fact that it doesn’t is suspicious.

  • BobbyLow

    Not yet Yoda. The reason why I closed DUST this morning is that price has been struggling on my Hourly Chart over the past couple of days and I wanted to hold onto my gains for the week. I would have closed it out by the COB today anyway because I don’t carry Miner positions over weekends so this was an easy decision. I’ll take another look at this on Monday to see if price calls for another bite of DUST or if the MOMO on NUGT is strong enough to go Long. One possible fly in the ointment is that next week is Fed week so there would have to be a pretty clear lens on Monday because I would be out before the Fed Announcement on Weds.

  • Tomcat

    I would be careful with oil. I have been arguing for a while here, that Oil is headed toward 40-45 area. There is just too much of it and there will be more with all the rigs that are being added every month.

  • Yoda

    Sounds like a plan. Seeing how DX got wacked before the open, I decided to rebuilt a long position in GDXJ today.

  • ridingwaves

    it should be, FIRE sector will be the tell for sell off, commodities are ahead of down move via my lends…

  • Mark Shinnick

    Things have remained generally supportive of volatility. Nothing particularly unusual with this setup more often than not resolving in vol crush….but not always 🙂

  • Kidd Cudi

    though using scatter plots is a great tool for analysis, you should keep in mind its limitations. Scatter plots ONLY test FREQUENCY. (at least as described in your examples)
    The issue is basic: the two important factors are Frequency and Magnitude.
    How often does the setup “predict” the future?
    How big of a move does the setup “predict”

    This is the classic case of out-of-the-money options. If your “setup” predicts wrong 90% of the time, but when it predicts right you make 15x your risk, then you have a net edge b/c:
    (0.1 * 15) – (0.9 * 1) = 0.6

  • ridingwaves

    no stake in that fight right now..

  • ridingwaves

    might get interesting, lets see if some profit taking transpires before the fed meeting…XLF is fading…

  • Yoda

    maybe the market doesn’t want a hike and would want to twist Yellen’s hand against it. Just a thought.

  • Ronebadger

    After all, the economy is overheating and inflation is out of control…of course they should raise rates!

  • ridingwaves

    .50 raise makes a lot of sense, bankers will love it…Reits not so much

  • ridingwaves

    temper tantrums are bad, master yoda…
    its happened before..

  • Scott Phillips

    That’s where the next part of the system building comes in. But without a REPEATABLE hypothesis you are just doing random bullshit.

    It doesn’t really matter what the options payout odds are if your underlying hypothesis is a pure random walk. And that’s the dirty secret of markets, a significant percentage of time charts are just what the quants call “random brownian motion” – Ie random bullshit.

    In the broad strokes the process is

    1) Test that you have a real hypothesis, whether its mean reversion, momentum, volatility breakout, trend continuation, whatever.

    2) Use whatever technical measures you are comfortable with to identify a LOW RISK entry point

    3) Develop an exit and stop adjustment method that suits the system goals.

  • Scott Phillips

    See the post above. “It’s a trap” was pretty clear, no?

  • Scott Phillips

    Some things we know about VIX
    – VX futures and therefore the ETF’s are almost permanently in contango, so betting on a breakout is most times a sucker play
    – When VIX breaks out from a low base it generally shoots hard and fast
    – But it’s like VX blowing it’s load, when it stops the mean reversion effect is one of the strongest in markets.

    The best strategy with VX systems is to wait for the pop to fizzle out and short it. Infrequent once or twice a year setups, but 75% win rates and big wins. If you are shorting it the contango works for you and not against you

  • ridingwaves

    vix is the joker showing up in a 52 card poker game…

  • Billabong

    IBB going into closing session with Hangman on daily.

  • ridingwaves

    miners showing hop on dollar lower. something beautiful is about to happen on IBB…for bulls

  • Billabong

    The $ maybe, but short covering will feed on itself until exhausted … miner reversals have a tendency to be volcanic. I would have said the $ today if it wasn’t for the lack of movement in PMs. PMs are starting to show signs of bottoming…

    We’ll see on IBB…

  • Mark Shinnick

    Yeah, Fed helped to cause the Great Depression in similar essential way by making credit less available. Strictly speaking its not the same in this case…but never is.

  • ridingwaves

    if IBB stays above 291 pretty bullish 288 less bullish

  • Scott Phillips

    How do you calculate the real odds of that happening though?

    That’s the real kicker.

    I *used* to think I’d go through a hundred charts and work out the odds of that. But its since been explained to me how wrong that is. Results vary plus or minus 1.5 standard deviations most of the time. For a setup with a win rate of 50% and an expectancy of .2 (a good setup) that would be entirely normal to have 200 trades at 30% win rate and -.05 expectancy, or 200 trades at 62% win rate and .4 expectancy.

    For example my core mean reversion system did .38 expectancy last year, this year it’s running at 0.02 expectancy.

    It actually doesn’t mean there is something wrong with it. Something *might* go wrong with it, in which case I need a framework for recognising that. But win rate isn’t the right path.

  • Scott Phillips

    Yeah it is!

    The best vix strategies are about maintaining a core short position to take advantage of contango and very occasionally reversing that to a long position.

    Take a look at the weekly chart here.

  • ZigZag

    Alright question for you Scott.
    Say I want to plot a correlation for one of my short term setups, a double bottom at a price support but with a higher corresponding tick low (this is a 1 min /tf chart with TIKRL running below).

    It seems like deciding what to measure is the difficult part without including all the extraneous factors you talk about. This sequence is never exactly the same, so is it specific enough to use the following parameters?

    • Price makes a double bottom at a prior support level within a greater than 10 min and less than 50 min timeframe

    • During the same minute that /tf makes the second low, TIKRL makes a higher low and bounces up from below the -500 area.

    • Measure the highest price change (or max price change?) in the next 1-2 hours without going below the trigger price?

    It’s a short term setup so the question is how to be objective about what I’m measuring. It could be a great short term setup but might not mean anything 3 hours later.

  • Scott Phillips

    OK there is a little bit of logic needed here. It’s simplified because you are currently trading it.

    There are a number of different ways of trading this.

    I would make a best guestimate of your average winning trade hold time. and use that number of bars as your Y axis.

    Therefore you need 3 measurements at least.

    1) Closing price at the first low
    2) Closing price at the second low
    3) Closing price some distance away

    Your best choices for 3) are all arbitrary best guesses. No harm in doing more than one and plotting them on the same chart. The more data the better.

    On the X axis you plot (2-1) On the Y axis you plot (3-2) easily done in a spreadsheet, or if you post the raw data I’ll do it for you.

    There is no harm in doing multiple points for 3.

    We aren’t going for exact science, we are trying to guess if there is an exploitable tendency.

    Almost certainly this is an exploitable tendency, I’d bet anything on it 🙂 I’ve traded similar things on ES with TICK in the past, super setups indeed.

    Obviously this is merely the starting point for system building, but without it, we are just firing bullets in the dark.

  • Kidd Cudi

    Like I said initially, these scatterplots to test correlation on *just* *one* thing are a very good idea/tool which I am excited to look into. It is very true that having too complex of a system makes it fragile in ways that you cannot understand until they sneak up on you.


    “It doesn’t really matter what the options payout odds are if your underlying hypothesis is a pure random walk.”

    I would disagree with this. if stock prices were true brownian motion, then the payout magnitude would be ALL that mattered. Simplified, if you flipped a coin to decide win/loss then obviously the only thing that determines expectation is payout:loss ratio. anything but 1:1 and someone is losing money.

    If I could find someone dumb enough to offer non-even money odds on brownian motion, then I would make a killing.

  • Kidd Cudi

    “How do you calculate the real odds of that happening though?

    That’s the real kicker.”

    yes, yes it is. Correctly determining BOTH projected win rate and projected win size is the fundamental task of deciding if a system is legit.

    I agree that 1 year sample size is far too tiny. Fortunately, computers and data make testing much larger samples possible, even easy if you know what you are doing.

    However, if you know something about the nature of the distribution of your returns (normal, fat-tailed, logarithmic, etc) you can calculate confidence intervals so that you can say with 95% confidence my expectancy is between 0.1 and 0.3 or with 99% confidence my expectancy is between 0.01 and 0.5 or whatever.

    I know I’m sort of talking about systems as a whole, mentioning expectancy, but these concepts apply to the study of even core setups, because some setups obviously tend to produce larger moves than others.

  • ZigZag

    Thanks much. I’m good with Excel, I just need to go through my history and charts library and get the data.

  • Scott Phillips

    This is actually quite interesting. Most markets are statistically trending part of the time, mean reverting some of the time, and some of the time behaves with what is called “geometric brownian motion”. Some markets almost never mean revert, especially currency markets where only a few pairs at a time behave in a mean reverting fashion.

    Later on I’ll be covering the test for it, it’s not as difficult as it sounds and you can do it in excel.

    Take a look at the charts here

    You can see that even brownian motion looks a lot like a tradable chart. Imagine if you put all your indicators on this or whatever you could get fooled quite easily. But it’s nothing, literally random bullshit.

  • Scott Phillips

    I’ve been down the rabbit hole with confidence intervals.

    The reality doesn’t quite match the promise.

    Here, for example is a 95% confidence interval for a very large backtest on es futures I did a few months ago.

    Obviously as you get more data it gets more accurate.

    So it starts out not being worth a pinch of shit, and ends up being “expectancy should be between zero and .2” Which isn’t really a great help at all since the reality never matches up with what the model says should be happening.

  • Scott Phillips

    My original plan was to use confidence intervals as a lower and higher boundary for switching the system off. It would work as a switch off, ie when absolute expectancy is below confidence interval stop trading.

    But in practical a long term negative expectancy system would be switched off long before you got the several hundred trades to decide to turn it off. Any system which starts out with a 50R drawdown right out of the gate is getting switched off, confidence interval or no 😉

    As a backup plan I looked at switching it off when a rolling expectancy dropped below the confidence interval. Problem is, those rolling expectancies are badly badly lagging so you switch the system off at the worst time and it lowers, not raises expectancy. A dry hole, in other words.

    Take a look at the results here. This is the system with 10 period, 20 period, 30 period confidence interval switch off plotted against the original EQ.

    I don’t mind lowering total R when drawdowns are lowered, but here we have increased drawdowns and decreased R

    Eventually I reached out to a hedge fund quant I met on and he visited me in Bangkok. We ended up getting on really well, he moved out of his hotel and stayed at my place for a few days, and he shared the risk modelling his hedge fund uses(the highest CAGR in Japan hedge funds over the last 15 years). It’s quite brilliant and completely avoids lag.

    Mole programmed it over a week, and it’s in production right now on some of our new systems.

  • Scott Phillips


  • Sir Mole III

    High standard deviation systems are a bitch and a system like that is extremely difficult to trade psychologically. Plus a system like that could *easily* paint 20 losers in a row. On paper these systems work but in reality they are not for most of us. BTW, if nothing else you would have to trade extremely small R sizes, otherwise the risk of ruin is just too high.

  • Marty Eble

    Yes, I realize that you, Zippy, and Wood Woodrow are all the same person.

  • Kidd Cudi

    It would be nice if I were zippy… he’s rich. also, I think you replied to the wrong thread…