Education
Now Reading
Linear Regression For Chipmunks
91

Linear Regression For Chipmunks

by The MoleJanuary 25, 2019

I’m working on a coding project today and neither have the time nor the inspiration to talk about the endless gyrations in equities. It’s heads down for me today and most likely I’ll be plugging through the weekend as well. So here’s a much needed refresher course on linear regression which most of you guys probably know as scatter plots/charts. This wills serve as the basis for a future post on how to correlate your prospective alpha factors when developing a trading system. So it’s important to fully understand how this works and why. Enjoy!

Alright now before you run for cover let me assure you that I’ll keep it very light on the math and promise to provide you with plenty of visual aids to make the process less painful. I personally am not a math wiz by any definition.

Plus as your humble host of ten years plus I would know better than to bore you with dry theory that has little application to your daily reality as a trader. Instead what I would like to do here today is to instill a deeper understanding of linear regression and why it is essential to fully grasp the concept in order to actively pursue RED in the context of your system development. Let’s dive right in:

Linear Regression

2017-04-21_LR_bad_and_good_fit

Here we have two scatter plots which I populated with random data, after which I added a best fit line. If I asked you which one is a better fit you would most likely point at the one on the right. But if I asked why you think it is a better fit then you probably would think for a few seconds and then tell me that it’s because the dots are closer to the line. And you would be exactly right.

Just looking at the plot on the left it is pretty safe to assume that there is either very weak or no linear relationship between each respective data point, each of which is produced by two values: x, and y. Thus we have an x axis and a y axis, which are an integral part of our formula:

2017-04-21_formula2

This simple formula is our basic equation which defines our best fit line. At any point along our (horizontal) x axis we plug each x value into our formula above we need m and b in order to arrive at the y values of our best fit line. So how do we figure those out? Well, m is the slope of the best fit line (also often referred to as beta) and b is the y intercept. But how do we arrive at those?

2017-04-21_formula3

It’s mostly leg work and it looks a lot more intimidating then it really is. We figure out m by:

  1. Multiplying the mean of all the xs by the mean of all the ys (the x and y with the line above them)
  2. minus the mean of all the xs multiplied by all the ys.
  3. Then we take the mean of all our xs and square it
  4. minus the mean of all the xs squared
  5. We divide the product of 2 by the product of 4.

Okay we got the slope. All that’s left is to figure out the intercept b, which is even easier:

2017-04-21_formula4

All we do is to take the mean of all the ys and deduct it from the slope m multiplied by the mean of all the xs. And we are DONE! See I promised you it would be pretty painless.

Squared Error

Alright now we are ready to put things a bit more into context:

2017-04-21_LR_good_fit_squared_error

Here again is the ‘better fit’ scatter plot but this time I have added a few red lines which attempt the visualize the concept of ‘squared error’ or e². Squared error is calculated by measuring the distances between your best fit line and each respective point on your scatter plot.

But just measuring the distance is insufficient because you’ll often have outliers which may skew your dataset plus sometimes the distance may be positive and sometimes negative (i.e. below the best fit line). For that reason we simply square the distance which elegantly solves both problems for us.

You may ask why we do we square it? Why not quadruple it or use an even bigger exponent? Well, you could actually but the accepted norm is a squared value and it seems to work pretty well for most applications. There may however be datasets that would benefit from a higher exponent and you are of course free to use that if you’re a) mathematically inclined and b) are able to write your own linear regression algorithm.

Which is what I actually wound up doing in the context of my own educational endeavors. I felt that it would give me a more deeper understanding of the underlying concept plus I did it in python as an exercise which I enjoyed very much and later helped me move on to more complex topics.

 

Coefficient Of Determination (r²)

2017-04-21_formula1

Now it’s time to talk about r-squared. First let me explain to you those strange symbols to you first. SE stands for squared error. The y with the little roof over it stands for the best fit or the regression line on the scatter above. You know – the one we just calculated? 😉

2017-04-21_LR_mean_of_the_ys

Finally the y with the little line above it stands for ‘the mean of the ys’, which refers to the black line on the plot above (step 4 in our m equation above). And all we’re going to do is to compare the accuracy of that black line with the accuracy of the blue best fit line. Which of course gives us r-squared. See, that wasn’t so hard!

Examples Of R-Squared

Now that  you understand the basic concept let’s look at a few example values of r². What we already know is that the range we’re going to be dealing with is 0 to 1. If r² = 0.8 then the product of the division must be 0.2, right? Which in turn would mean for example that the SE of the best fit would be 2, and the SE of the mean of the ys would be 10.

Given that example the squared error of the (blue) best fit line is significantly better than that of the (black) mean line. And that’s a good thing of course and it means that our data is pretty linear. If the r² is 0.3 on the other hand then the product of the division in the formula above must be 0.7, which in turn may be produced by 7 / 10. So now the SE of the best fit line is only marginally better than the SE, and yes all other things being equal that is a bad thing. Of course the accuracy of your model always depends on the type of data you are analyzing.

Done!

And there you go, NOW you understand the basic concept of linear regression and the mythological r² value you probably have been hearing about every once in a while. In a future post we’ll dive in a little deeper in order to understand how we can actually leverage our newly gained knowledge.


About The Author
The Mole
Mole created Evil Speculator amidst the chaos of the financial crisis in early August of 2008. His vision for Evil Speculator is a refuge of reason, hands-on trading knowledge, and inspiration for traders of all ages and stripes. You can follow him and his nefarious schemes at the usual social media waterholes.
Enjoyed this post? Consider a small donation to keep those evil deeds coming!

BTC: 1MwMJifeBU3YziDoLLu8S54Vg4cbnJxvpL
BCH: qqxflhnr0jcfj4nejw75klmpcsfsp68exukcr0a29e
ETH: 0x9D0824b9553346df7EFB6B76DBAd1E2763bE6Ef1
LTC: LUuoD6sDWgbqSgnpo5hceYPnTD9MAvxi6c

  • Jason13

    Metals are straight up..

  • http://www.captainboom.com/ captainboom

    Oooooh… Math! I like math!

  • Jason13

    Closed longs from late yesterday….wanna see what happens to gap and metals action is freaking me out a bit

  • Jason13

    China talks causing moves in Yuan..

  • Sp00nman

    Nice post Mole, the math so far is pretty straightforward, and you’ve broken it down for us simpletons.

  • Sp00nman

    Ya not looking good for the bears. China had another liquidity injection and I thought I read something that the Fed may end the balance sheet drawdown or QT. It’s helping the metals.

  • Yoda

    With such a strong start, we should be good for another 50 handles up, to finish the month at the high

    https://uploads.disquscdn.com/images/9e89951f5eaa46b566da31937e2b39864b546dc1d13abf8eb32a5a78c9988328.png

  • Julie 35 – 25 – 36

    Guys I did a scalp at the open (UPRO) closing scalp as I type.Noe break even on existing short I saw the opportunity to do so as they gaped it up over a key level on the open above 2655 (SPX) Will show chart why closed scalp. Overbought 60 min and 10 min going up into resistance and an upper https://uploads.disquscdn.com/images/e9947086977990997e0746cc866a49edb21fd2d263c6e79a9d54f1011230f564.png trend line Shown UPRO 10 min chart JULIE

  • TomW4

    Did you close your shorts or still holding to it?

  • Julie 35 – 25 – 36

    Tom still have existing short small position.SH
    JULIE

  • Julie 35 – 25 – 36

    Guys I am watching $TNX very closely for clues 28 very important Shown $TNX weekly chart JULIE https://uploads.disquscdn.com/images/5b90ed9871d44f2c504d51955fc53a711fbe94858d175d5a91d5b1c2950d27ff.png

  • Ted

    Julie were headed for 2750 minimum. The quants are in complete control. They don’t care about trend lines or over bought conditions. Here it is plain and simple: Chinese liquidity injections which may turn into outright QE and our Fed may back off on QT. Risk on – period. The market has gotten here without Apple or Intel. Apple chart is looking bullish to boot. Apple will report and it will be considered less bad than thought. Meanwhile, we are missing out because we just can’t believe it.

  • Julie 35 – 25 – 36

    Ted Line in the sand Daily Chart confluence (1) 144 ema (2) 233 ema (3) 61.8% retracement (4) upper down trend line (5) Volume at price Weekly chart not shown but going up into the confluence of the weekly 21 and 55 ema’s Also non confirming volume SPY. JULIE https://uploads.disquscdn.com/images/4d926123f6046d2322609b987cf831106f8611d6608b5866e275627e37e73966.png

  • Ted

    Your best chance was me to shoot my mouth off. It’s working. Sorry I’m frustrated.

  • Edd

    ” Meanwhile, we are missing out because we just can’t believe it.”
    Amen Ted

  • Julie 35 – 25 – 36

    Ted The bond market is 7 times the equity market .. that is why I am watching $TNX SPX and equities are going up into a congestion zone where there are going to be a ton of sellers just to break even. The bulls will take profits. If anyone does not think that there will be a pullback is deluding themselves esp with SPX overall pattern lower highs and lower lows.
    JULIE

  • Julie 35 – 25 – 36

    FOMO Fear of missing out will nail ya Bulls late to the party get slaughtered. Wait for a pullback
    JULIE

  • BobbyLow

    From Jesse Livermore from about 80 or 90 years ago.

    ““BUY RISING STOCKS AND SELL FALLING STOCKS”

    “There are very successful contrarian traders and investors in the world. Although you might become successful at going against the grain on the stock market, it is so much easier to just go with the flow.”

    Here are 23 quotes from Jesse. Some of the language is dated but I’m amazed at how well what he said so many decades ago have survived the task of time.

    https://www.marcellagerwerf.com/jesse-livermore-quotes/

  • StockTalker

    Very strong move today, Close to opening up my short.

  • Ted

    Yup – it really is what your system and Mole’s are all about. Take the trade and hold it until price takes you out and forget about the BS.

  • BKXtoZERO

    Do I see a Pic i Nic basket? Unsure of the future but as a scalper today and everyday, we have the former high that everyone sees. On TVIX there is a gap fill coming at at 44 from tuesday, VXX gap at 37.2. VIX itself already took out those gaps so there is pent up hope in those other products at the moment https://uploads.disquscdn.com/images/2f1d49f97e873b0ee7131405beedb8c50adf7f947e3b3a0ef0da570f9c5c5ce3.png

  • BKXtoZERO

    There is still room on the underside of ES downtrend line but if that breaks up then VIX targets are 1) the floor 2) the 2.0 bollinger area around 14.2 now https://uploads.disquscdn.com/images/8794b55ec1c89d61090668abb3037704e167f4f643ac9d6dbeb4e5057d56e35b.jpg

  • Ted

    Don’t make a Boo Boo! LOL

  • BKXtoZERO

    L TVIX 42.7

  • BKXtoZERO

    ES 1 min chart, rally is on 3rd angle of ascent and backed against it rt now, need vertical move now https://uploads.disquscdn.com/images/a0171a202de8a0544be41dcab67c4fda10addfe0d440dabf2104edcf9f25c709.jpg

  • BKXtoZERO

    0ut half at 44.6, super tight

  • zzezzezz

    Nibbling at a short… Mid-day reversal possible?

  • Jason13

    I’m hearing that there may be an announcement of some kind regarding the shutdown in 15 mins….anyone confirm?

  • BKXtoZERO

    +1R again 44.46

  • Jason13
  • Jason13

    Watch your positions….i’m staying flat

  • BKXtoZERO
  • BKXtoZERO

    Looks like everyone else in the world can also draw basic ascending trend lines and everyone is up for a free ride but they seem to run when the line breaks. Grade school 101

  • Ted

    I really don’t think this has much to do with the market moves.

  • evilasevildoes
  • BKXtoZERO

    Not looking good. Mr Ranger Sir is going to take my pic a nic basket again more than likely.

  • Jason13

    Yes that was obviously the case. Just wanted to post it here in case volatility picked up and give everyone a heads up. Apparently rumours are more influential.

  • Julie 35 – 25 – 36

    Guys Like I said earlier Better be watching $TNX The bond market 7 times larger than the equity market $TNX not even close to the rally of $SPX 28 very important $TNX as stated earlier below with the weekly chart JULIE https://uploads.disquscdn.com/images/f038c968ac40c97e16c276189f571f4014549d04fa2f0ef989d3c38107b7812c.png

  • StockTalker

    Long through Weds of next week (FED day)

  • Julie 35 – 25 – 36

    A gap up unable to get above the January 18 high and no where near the volume January18 I read the tape and do not care about China liquidity injection and subsequent B.S. Going to the moon horse poop and what not
    JULIE

  • Julie 35 – 25 – 36

    EDIT January 18 high
    JULIE

  • Julie 35 – 25 – 36

    Guys ask yourself is this market going up because of a phenomenal amount of buying or lack of selling.? Just watch when profit taking and the selling when the people that are stuck in the congestion zone mentioned previously sell to get even.
    JULIE

  • BKXtoZERO

    You sound pretty entrenched. I am in scalp mode…. Seems anything can happen.

  • Yoda

    No need to get too excited before the month is over

  • Julie 35 – 25 – 36

    Hi Yoda not excited Just reading the tape Glad to hear from you and have a great weekend. The skiing is great in the Big Sky. Going tomorrow and The Cold Smoke at Bridger Bowl is phenomenal
    JULIE

  • Julie 35 – 25 – 36

    Hi BK I pulled off a nice scalp on the open per 10 min UPRO chart this a.m.
    JULIE

  • TomW4

    Hi @disqus_qkFlPoMLUZ:disqus , I’m also biased to the short side. But we’ve been saying all these things since SPX was in the high 2500s. It’s now close to 2700. We can keep saying this whilst the market keeps on going up. When do you throw the towel? This is pretty much what Mole has been saying…

    Just playing devil’s advocate.

  • BKXtoZERO

    One of the old mkt sayings is so goes January so goes the year. Seems that is what they are trying for

  • maxcherry
  • Julie 35 – 25 – 36

    Mole also stated a Bear market too …. Overbought in a Bear Market is dangerous. Remember SPX Lower highs and lower lows. People are saying the Quants and Bots are moving the market up per program trading. Well …. I would say the Quants and prorgram trading moved the market down too ! Hmmm ! The decline from October … ! Tom The Bond Market is 7 times larger than the equity market and they are not buying into this rally per daily and weekly charts $TNX
    JULIE

  • TomW4

    @disqus_qkFlPoMLUZ:disqus , can you please explain how you deciphered that the bond market is not buying into this rally via TNX? I’d love to learn.

    Thanks!

  • BKXtoZERO

    Mole is going easy on math…. I can’t wait to see advanced

  • BKXtoZERO

    Time check for the Gerbil dude….. Out .5R tvix from 44.6 at 43.9 holding.5R

  • Darkthirty

    leading diagonal or bull flag?

  • maxcherry

    bull flag

  • BKXtoZERO

    Find out Monday

  • maxcherry

    or cup and handle

  • http://iberianviews.blogspot.com/ catracho

    Bear market rallies suck in players until they think the bull is back and then …

  • BKXtoZERO

    Have a good weekend!

  • Julie 35 – 25 – 36

    Tom $TNX and $SPX should move in tandem i.e. both in the same direction. If SPX is going up so should TNX. OK Very simple TNX is not going up in the same manner as SPX. TNX is more of a corrective pattern than SPX. NOW important SPX looks impulsive off the December 26th low which is bullish Now an easy chart to follow is $TNX:$SPX I have drawn a support line and a resistance line.and you can see the deceleration waning and attemting to bottom. When it reverses and goes above the small channel upper trend lne the rally will be over As is rises above the relative resistance relative blue line the SPX decline will intensify JULIE https://uploads.disquscdn.com/images/b2bd380e1ead70b0420d266e8098c2efd5760037ebe9a6c5a48c841b9e295cdb.png

  • Julie 35 – 25 – 36

    You got it Catracho
    JULIE

  • Darkthirty

    GG get kicked around too much?

  • BobbyLow

    Have no idea. 🙂

    BTW, I’m down to a 1/4 Position Long on Crude. I still don’t have enough evidence to flip to short yet but I’m a little closer. A close about .30 lower would have got me short and a close about .30 higher would have kept me in at 1/2 Long. So it’s a tweener for me right now. I’m down to a 1/4 Long and if I get a down gap on Monday NBD.

  • BobbyLow

    You too BKX

  • Ted

    Julie – have a great day of skiing! Don’t mind me – I like to bitch. Mad at nobody but myself.

  • Ted

    I’m their so take the darn thing down already. LOL

  • Julie 35 – 25 – 36

    GUYS — Easy chart for resumption of the bull SPX Weekly Chart RSI (27) must get above 50 ; STOCHS 50,3,3 must get above 50 AND last but not least weekly 8 ema must get above weekly 55 ema Everyone Have A Great Weekend JULIE https://uploads.disquscdn.com/images/892b1e5a68bf09a25859ffaea12d55cc2f2f0216ec44981fcf1f97c998d17e30.png

  • zzezzezz

    Gold 1300+

  • Julie 35 – 25 – 36

    You Have A Great Weemd too Ted
    JULIE

  • Julie 35 – 25 – 36

    You Also BK
    JULIE

  • evilasevildoes

    iuse that one easy peasy

    bonds were great today

  • evilasevildoes

    Enjoy your sleep this weedend!

  • Julie 35 – 25 – 36

    EV Great I follow $TNX … Next week gonna be good Have A Great Weekend EV
    JULIE

  • evilasevildoes

    you too sweetheart!

  • evilasevildoes

    Anyone here use Murrey lines on esig they rock

  • Julie 35 – 25 – 36

    EV I know two people that use Murrey math .. I would like to learn it
    JULIE

  • Yoda

    I am keeping some dry powder for February

  • Julie 35 – 25 – 36

    EDIT FULL STOCHS 50,3,3 must get above 50
    JULIE

  • Jason13

    You have a chart to demo? never heard of it.

  • Jason13

    Put/Call ratio .65 is lowest since Dec 4th..

  • Julie 35 – 25 – 36

    Jason SPY volume today much less than January 18th on a retest. Also SPY volume today (Friday) more than the two preceding days with a small candle closing below it’s open mid range which means sellers meeting buyers. SPX is stalling. Thanks for your P/C ratio. I have total P/C ratio $CPC @ 0.75 which is bearish. Under 0.80 is bearish. Jason Have A Great Weekend
    JULIE

  • sutluc

    You and your maths and figuring. The sooner you get away from that nonsense the better off you’ll be.
    All you need is a calender (for the moon phases), tide tables (helps with the waves), a daily newspaper, (astrological forecasts and the exact time of sunrise and sunset) and the internet for charting and trading.
    Pull up some charts, draw some slanted lines, curve fit a moving average or two or three or a dozen, select some indicators (between six and two dozen and you’ll want at least some of them to be Japanese) check your horoscope against the cycles of your choice, and make money.

  • BKXtoZERO

    The VIX may be extra volatile this week due to VXX expiration Tuesday, pardon one link to zerohedge/year, interesting and perhaps an opportunity: the timing may be better after Tuesday, because with the VXX
    maturing next week, it remains the largest VIX ETP and has around $700mm
    of AUM. https://www.zerohedge.com/news/2019-01-26/vix-unusually-low-goldman-warns-its-big-week-vol-complex

  • Julie 35 – 25 – 36

    Guys I watch Active Investment Managers along with “Hedgies” This past week a slight down https://uploads.disquscdn.com/images/e61353198cf56d77c6946a36a08791b42c2eddafcbcd980dbc9656c44c5b62e4.png tick in their equity exposure JULIE

  • Julie 35 – 25 – 36

    Guys two charts showing negative divergences using 10 sma as a filter (1) NYSE advance – decline net issues (2) NYSE up volume – down volume net If anyone thinks that there will not be a pullback and it’s up up and away …. Well .. ! JULIE https://uploads.disquscdn.com/images/68d9a775827ebaefacc242d04d89592dbd0e5bc12051eb3700561cd2c73d1c6b.png https://uploads.disquscdn.com/images/9035dcbd84ab6bfa8d4e5fd43124e2abcf0fd9e07345b224c850b4da306023ad.png

  • Julie 35 – 25 – 36

    Guys $CPCE Equity only P/C Ratio 0.51 Yipes ! Very Bearish .. When the $CPCE moves up above it’s green dotted down trend line Rally over Guys ! It’s coming … JULIE

  • Julie 35 – 25 – 36
  • BKXtoZERO

    last time I chatted w/ you it was .7 or so… I said .5 to .6….. let’s see how it works this time. Sometimes the MKT seems willing and able to carry a lot of call buyers and sometimes it bucks them off like fleas. I guess it is central bank backing.

  • Julie 35 – 25 – 36

    Hi BK That was the Total P/C Ratio symbol $CPC .. This is the equity only P/C Ratio symbol $CPCE They are two different ratios $CPC bearish below 0.80 $CPCE bearish below 0.60
    JULIE

  • Julie 35 – 25 – 36

    Thanks BK
    JULIE

  • Sp00nman

    Interesting info regarding debt issuance is the US had $1T+ in 2018 and expected to be another $1T+ in 2019. Should be bullish for gold and hard assets. I have wondered and asked myself how debt/deficits matter these days with the existence of primary dealers at auctions. They exist to make sure all the debt gets purchased. On that note, seems like a great investment for them. Levered up, a guaranteed 2-3% coupon turns into big money.