Worden Discussion Forum

Welcome Guest, please sign in to participate in a discussion.

Worden Discussion Forum » Customer Training & Support » PCFs, EasyScan and Custom Indicators » MACD 50,21,10 regression and general optimization issues

MACD 50,21,10 regression and general optimization issues

Rate this Topic:

Previous Topic · Next Topic

Watch this topic · Print this topic ·

krck

Posted : Tuesday, March 28, 2006 2:17:41 PM

Registered User
Joined: 2/27/2006
Posts: 16

I have derived a formula for MACD 50,21,10 regression slope over 10 days.

It has 40 terms involving moving averages and their averages.
It was slow when charts were being loaded. Few seconds per new chart.
I guess the scanning of 7000 stocks would suffer if I attempted it.

I used a generic regression formula involving X,Y and I put
X = [1, 2, ..., 9, 10]
Y = [MACD 10 days ago,...,MACD today]
for my formula data.

Then I optimized the formula by having 10 days of Y data as before, but replacing
every second day with its immediate neighbour.

This reduced the formula to
(8/165)*
(
0-2*XAVGC21.9+
2*XAVGC50.9-
XAVGC21.7+
XAVGC50.7+
2*XAVGC21-
2*XAVGC50+
XAVGC21.3-
XAVGC50.3+
2*XAVG(XAVGC50, 10)+
2*XAVG(XAVGC21.9, 10)-
2*XAVG(XAVGC50.9, 10)+
XAVG(XAVGC21.7, 10)-
XAVG(XAVGC50.7, 10)-
XAVG(XAVGC21.3, 10)+
XAVG(XAVGC50.3, 10)-
2*XAVG(XAVGC21, 10))

The curve was almost identical and I must have speeded it up a bit
( hard to measure as a user ).

The above formula still has repeating terms.

Can further optimization can be done in this instance?
Is there a more optimal regression slope formula and approach?

This also brings me to a few related questions regarding optimizing telechart:
1. If I have indicators on a chart that I have made invisible are they being calculated in the background?
2. Does TC PCF engine cache repeating terms internally so that they are calculated once?
For example above, most terms repeat at least once; once on their own and once as part of further averaging.

Bruce_L

Posted : Tuesday, March 28, 2006 5:29:54 PM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

QUOTE (krck)

The curve was almost identical and I must have speeded it up a bit ( hard to measure as a user ).

Your formula seems like a reasonable approximation of the slope of the Linear Regression of the MACD Histogram 21,50,10 (it doesn't appear to be 50,21,10). I have no idea how the computation time of your formula would compare to the full formula (Both calculated too quickly for me to notice a difference when they were the only PCF updating on my system).

QUOTE (krck)

Can further optimization can be done in this instance? Is there a more optimal regression slope formula and approach?

I don't know the answer to either question.

QUOTE (krck)

If I have indicators on a chart that I have made invisible are they being calculated in the background?

Yes.

QUOTE (krck)

Does TC PCF engine cache repeating terms internally so that they are calculated once?

I don't have access to the code, but when calculating very long and complex Personal Criteria Formulas, TeleChart displays two messages about duplicate functions that would lead me to believe it does (I'm assuming they always show up when there are repeated PCF fragments, but flash by too quickly to see except when calculating very long formulas).

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

krck

Posted : Wednesday, March 29, 2006 12:53:45 PM

Registered User
Joined: 2/27/2006
Posts: 16

Bruce, the original formula was

(0
-9*XAVGC21.9+
9*XAVGC50.9-
7*XAVGC21.8+
7*XAVGC50.8-
5*XAVGC21.7+
5*XAVGC50.7-
3*XAVGC21.6+
3*XAVGC50.6-
XAVGC21.5+
XAVGC50.5+
9*XAVG(XAVGC21.9, 10)-
9*XAVG(XAVGC50.9, 10)+
3*XAVG(XAVGC50.3, 10)+
9*XAVGC21-
9*XAVGC50-
7*XAVG(XAVGC21.1, 10)+
7*XAVG(XAVGC50.1, 10)-
9*XAVG(XAVGC21, 10)+
7*XAVG(XAVGC21.8, 10)-
7*XAVG(XAVGC50.8, 10)-
5*XAVG(XAVGC21.2, 10)+
5*XAVG(XAVGC50.2, 10)+
5*XAVG(XAVGC21.7, 10)+
3*XAVG(XAVGC21.6, 10)-
3*XAVG(XAVGC50.6, 10)-
5*XAVG(XAVGC50.7, 10)+
XAVGC21.4-XAVGC50.4+
3*XAVGC21.3-
3*XAVGC50.3+
5*XAVGC21.2-
5*XAVGC50.2+
7*XAVGC21.1-
7*XAVGC50.1-
XAVG(XAVGC21.4, 10)+
XAVG(XAVGC50.4, 10)+
XAVG(XAVGC21.5, 10)-
XAVG(XAVGC50.5, 10)+
9*XAVG(XAVGC50, 10)-
3*XAVG(XAVGC21.3, 10))/165

I do not know if it is the correct one for 50,21,10 histogram, I used the delayed PCF formulas for MACD histogram as elements.

X[10]= [1,2,3,4,5,6,7,8,9,10]
Y[10]= [XAVGC21.9-XAVGC50.9-XAVG(XAVGC21.9,10)+XAVG(XAVGC50.9,10),...,XAVGC21-XAVGC50-XAVG(XAVGC21,10)+XAVG(XAVGC50,10)]

Then plugged X and Y into a formula for regression slope for 10 elements.
The above formula was the result.

I do hope there is a better approach that produces a fast curve.
When I plot the formula as a custom indicator and then switch charts there is a several seconds lag on chart switch.

Bruce_L

Posted : Wednesday, March 29, 2006 1:50:59 PM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

The form I was using for testing (again, MACD 21,50,10 not MACD 50,21,10) was pretty similar (just re-arranged slightly):

(4.5 * (XAVGC21 - XAVGC50 - XAVG(XAVGC21,10) + XAVG(XAVGC50,10)) + 3.5 * (XAVGC21.1 - XAVGC50.1 - XAVG(XAVGC21.1,10) + XAVG(XAVGC50.1,10)) + 2.5 * (XAVGC21.2 - XAVGC50.2 - XAVG(XAVGC21.2,10) + XAVG(XAVGC50.2,10)) + 1.5 * (XAVGC21.3 - XAVGC50.3 - XAVG(XAVGC21.3,10) + XAVG(XAVGC50.3,10)) + .5 * (XAVGC21.4 - XAVGC50.4 - XAVG(XAVGC21.4,10) + XAVG(XAVGC50.4,10)) - .5 * (XAVGC21.5 - XAVGC50.5 - XAVG(XAVGC21.5,10) + XAVG(XAVGC50.5,10)) - 1.5 * (XAVGC21.6 - XAVGC50.6 - XAVG(XAVGC21.6,10) + XAVG(XAVGC50.6,10)) - 2.5 * (XAVGC21.7 - XAVGC50.7 - XAVG(XAVGC21.7,10) + XAVG(XAVGC50.7,10)) - 3.5 * (XAVGC21.8 - XAVGC50.8 - XAVG(XAVGC21.8,10) + XAVG(XAVGC50.8,10)) - 4.5 * (XAVGC21.9 - XAVGC50.9 - XAVG(XAVGC21.9,10) + XAVG(XAVGC50.9,10))) / 82.5

While I'm not saying it doesn't exist, I do not personally know of a more efficient form than what you already presented (both forms take approximately 1 second to switch symbols when plotted as Custom Indicators on my system at Zoom 5).

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

krck

Posted : Wednesday, March 29, 2006 4:53:31 PM

Registered User
Joined: 2/27/2006
Posts: 16

It seems the formulas match.

On further investigation I have found that on its own, the formula
takes approximately a second to load when 200 exponential moving average
is not in the upper window with the prices.

However once the 200 EMA is plotted in the upper window and the formula
together with its MACD21,50,10 in the middle window, then at zoom five I had on average 5 seconds and more.
Whereas with 200MA on its own or with the formula on its own, the lag was not significant.

bustermu

Posted : Thursday, March 30, 2006 6:41:16 AM

Registered User
Joined: 1/1/2005
Posts: 2,645

krck,

As pointed out by Bruce, you are saying MACD(50,21,10) Histogram but are using MACD(21,50,10) Histogram. They are related by:

(MACD(21,50,10) Histogram) = -(MACD(50,21,10) Histogram)

Let us define Y to be the MACD(21,50,10) Histogram so that:

Y0 = XAVGC21.0-XAVGC50.0-(XAVG(XAVGC21.0,10)-XAVG(XAVGC50.0,10))

Your "reduced" version of Linear Regression Slope of Period 10 on MACD(21,50,10) Histogram is:

(8/165)*(2*Y0+1*Y3-1*Y7-2*Y9)

Notice that the Y's are not equally spaced and the coefficients do not form a straight line. First, let's replace Y7 by Y6 so that the samples will be equally spaced and then, make the coefficients a straight line. The result is:

(4.5*Y0+1.5*Y3-1.5*Y6-4.5*Y9)/45

You will find this a poor substitute for the LRSlope10 using all 10 Y's.

My next post will offer a different approach which may speed up the computation.

Thanks,
Jim Murphy

Bruce_L

Posted : Thursday, March 30, 2006 9:56:07 AM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

QUOTE (krck)

However once the 200 EMA is plotted in the upper window and the formula together with its MACD21,50,10 in the middle window, then at zoom five I had on average 5 seconds and more. Whereas with 200MA on its own or with the formula on its own, the lag was not significant.

I'm suspecting the reason for this is that the indicator requiring the most history determines how much history is used to calculate most of the other indicators on the chart. At Zoom 5, XAVGC200 requires a lot more historical data to chart than XAVG(XAVGC50.9,10) and this causes your Custom Indicator to be calculated using more history than would be used if the XAVGC200 was not present.

QUOTE (bustermu)

My next post will offer a different approach which may speed up the computation.

I'm looking forward to it.

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

bustermu

Posted : Thursday, March 30, 2006 10:52:02 AM

Registered User
Joined: 1/1/2005
Posts: 2,645

Our purpose is to consider different forms of the Linear Regression Slope of Period 10 on MACD(21,50,10) Histogram in hopes of speeding up the computation.

As above, let Y to be the MACD(21,50,10) Histogram so that:

Y0 = XAVGC21.0-XAVGC50.0-(XAVG(XAVGC21.0,10)-XAVG(XAVGC50.0,10))

The Linear Regression Slope of Period 10 on Y is:

(4.5*Y0+3.5*Y1+2.5*Y2+1.5*Y3+0.5*Y4-0.5*Y5-1.5*Y6-2.5*Y7-3.5*Y8-4.5*Y9)/82.5

The first thing we can do is "uncascade" the EMAs in Y. This is accomplished by using Theorem 1 in:

Cascades of Moving Averages

The result is:

Y0 = (261/440)*XAVGC10.0-(9/11)*XAVGC21.0+(9/40)*XAVGC50.0

We are now down to 3*10 = 30 computations of EMAs on Closing Prices. The computation time may be faster than for the original form.

I do not know that it will reduce computation time, but we can reduce the 30 computations of EMAs to 3 by using the difference equation:

XAVGCP.0 = a*XAVGCP.1+(1-a)*C0

where

a = (P-1)/(P+1)

That would be the next step.

Thanks,
Jim Murphy

Bruce_L

Posted : Thursday, March 30, 2006 11:51:06 AM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

Please try the following:

261 / 113135 * C + 25707 / 7601912 * C1 + 5342 / 1506649 * C2 + 10601 / 3473214 * C3 + 16339 / 7781021 * C4 + 4261 / 5055578 * C5 - 5717 / 9530500 * C6 - 20552 / 9615185 * C7 - 11503 / 3107572 * C8 + 77796 / 1498421 * XAVGC21.9 - 32281 / 4276947 * XAVGC50.9 - 514422 / 9676643 * XAVGC10.9

I think I've properly translated bustermu's suggested method. It matches the charts produced by the other other formulas and switching charts seems to be quite a bit quicker when it is substituted as the formula for the Custom Indicator.

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

bustermu

Posted : Friday, March 31, 2006 5:15:35 AM

Registered User
Joined: 1/1/2005
Posts: 2,645

Bruce,

You are obviously way over qualified for your job. How soon are you leaving? Amazing!

What is your Erdös number anyway?

Thanks,
Jim Murphy

Bruce_L

Posted : Friday, March 31, 2006 9:51:55 AM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

bustermu,
You are a flatterer. While I know a few people with finite Erdös numbers, physics undergraduates rarely get a chance to publish collaborative papers and that's as far as my education goes (I'm not looking, but I've never seen a Bachelor of Arts in Physics listed as a job requirement).

Besides, I think my mathematics career peeked during my sophomore year of high school when my science fair project (with the rather verbose and pompous title Computer Hypergraphics: Visualizing n-Dimensional Hyperfigures using Computer Vector Graphics on a Raster Display as Projected into 3-Space) achieved a few national awards (I think letting the judges put on the 3-D glasses and play the project based n-Dimensional Asteroids game I wrote and always played between presentations helped).

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

bustermu

Posted : Friday, March 31, 2006 11:33:31 AM

Registered User
Joined: 1/1/2005
Posts: 2,645

Bruce,

You missed your chance. The opportunity to get an Erdös Number of Five was being auctioned off on eBay. But, just knowing someone with an Erdös Number puts you right up there.

I don't know about those degrees. Garrett Birkhoff did not have a PhD. His reason being that there was no one capable of examining him. You may be in the same boat.

Thanks,
Jim Murphy

Bruce_L

Posted : Friday, March 31, 2006 12:12:19 PM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

bustermu,
It's almost impossible to have a social life and not know people with Erdös numbers when you live less than six miles from both Duke University and the University of North Carolina at Chapel Hill (then again, as a geek, my perspective might be a bit skewed). It's too bad weblog posts don't count or I could get an Erdös number of my very own fairly easily.

BTW, I'm not sure if the article was actually published, but I read that an Erdös number of two was auctioned for $127.40 on July 22, 2004 for a prospective paper submitted to Chance on skill in the World Series of Poker and the World Poker Tour.

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

krck

Posted : Saturday, April 1, 2006 11:15:06 AM

Registered User
Joined: 2/27/2006
Posts: 16

Thank you both guys.

To say that you went above and beyond the call of duty on this one is an understatement, to say the least.
Busteremus posts are educational. I'll be sure to look them up for other gems.

I read your posts yesterday and was swept off my feet.
Needless to say, most of the things you presented initially went over my head.

After I collected myself, and after having re-read the posts, I implemented busteremu's suggestions in Maple on my own to see what I come up with. Also because in the future I may want do this on my own for any MACD periods.

I managed to get the same formula after several hours of careful effort ( but then I am totally new to this ).

Now I finally find that Bruce's result coincides to my own.
Bruce, I am curious if you used Maple, Excel or some other method to derive this? You seem to have got there fast.

I am presenting excerpts from what I did in Maple in the hope that maybe you would be willing to share your method.
And for general benefit to anyone reading this.

Thanks once again.

-----------------------------
-----------------------------
explanation of variables below:
-------------------------------

position = position of macd, ema etc in a sequence say 1..10
range = for the purpose of implementing difference equations, equals 10 in this instance.
period = averaging period for ema
inner, outer = inner and outer periods in cascading EMA as in EMA(EMA(price,inner),outer)

-------------------------------
the code, copied line by line
------------------------------

with(LinearAlgebra):
with(VectorCalculus):
with(ArrayTools):

prices := vector([C, C1, C2, C3, C4, C5, C6, C7, C8, C9])

a := proc ( x ) options operator, arrow; ( x-1 )/( x+1 ) end proc

xavgc := proc (period, range, position) options operator, arrow; piecewise(range < position, 0, position = range, XAVGC(period, position-1), position < range, a(period)*xavgc(period, range, position+1)+(1-a(period))*prices[position]) end proc

xavgcxavgc := proc (inner, outer, range, position) options operator, arrow; (inner-1)*xavgc(inner, range, position)/(inner-outer)+(outer-1)*xavgc(outer, range, position)/(outer-inner) end proc

macdhistogram := proc (short, long, signal, position) options operator, arrow; xavgc(short, signal, position)-xavgc(long, signal, position)-xavgcxavgc(short, signal, signal, position)+xavgcxavgc(long, signal, signal, position) end proc

X := vector([seq(position, position = 1 .. 10)])
Y := vector([seq(macdhistogram(21, 50, 10, 11-position), position = 1 .. 10)])

regressionSlope := proc (range) options operator, arrow; ((sum(X[position]*Y[position], position = 1 .. range))-(sum(X[position], position = 1 .. range))*(sum(Y[position], position = 1 .. range))/range)/((sum(X[position]^2, position = 1 .. range))-(sum(X[position], position = 1 .. range))^2/range) end proc

regressionSlope(10)

evalf(regressionSlope(10))

---------
simplified pseudo code of above:
-----------

// prices
prices[10] = [C,...,C9]

// used in moving average difference eq
a( x ) = ( x-1 )/( x+1 )

// piecewise moving average difference equation
xavgc(period,range,position)=
0 , for position > range
XAVGC(period, position-1) , for position = range
a(period)*xavgc(period, range, position+1)+(1-a(period))*prices[position]) , for position < range

// linear cascade of moving average
xavgcxavgc(inner,outer,range,position) = (inner-1)*xavgc(inner, range, position)/(inner-outer)+(outer-1)*xavgc(outer, range, position)/(outer-inner)

// macd histogram formula using above elements
macdhistogram(short,long,signal,position)=xavgc(short, signal, position)-xavgc(long, signal, position)-xavgcxavgc(short, signal, signal, position)+xavgcxavgc(long, signal, signal, position)

// inputs to regression slope equation
X := vector([seq(position, position = 1 .. 10)])
Y := vector([seq(macdhistogram(21, 50, 10, 11-position), position = 1 .. 10)])

---------

Bruce_L

Posted : Saturday, April 1, 2006 1:13:19 PM

Worden Trainer

Joined: 10/7/2004
Posts: 65,138

krck,
I don't have Maple (I think I'd have to buy the $2000 Professional License). I used a spreadsheet because when you have a hammer, everything looks like a nail. Maybe I should look into one of the Open Source alternatives to Maple such as MathAction, Maxima or Eigenmath.

My method was pretty straightforward. Since they were already calculated, I started with the coefficients for the formula for Y9 (hint: see bustermu's Thursday, March 30, 2006 10:52:02 AM post, they are the same as for Y0 if that's where I decide to start). Then I made formulas to calculate the coefficients for Y8 using the difference equations and copied the cells to calculate Y7 through Y0. A quick SUM() of the cells for the coefficients for Y9 through Y0 formatted as fractions gave me everything necessary to produce the formula posted in my Thursday, March 30, 2006 11:51:06 AM post.

It was an ugly one-off, but the modifications required to pretty it up and get it to work for a variety of MACD and LR settings would be fairly insignificant.

-Bruce
Personal Criteria Formulas
TC2000 Support Articles

Users browsing this topic

Guest-1

Forum Jump

You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.