Registered User
Joined: 6/6/2005
Posts: 1,157
|
I was hoping someone could help me arrive at a statistical probability. I'm just not sure how to go about it.
I ran a backtest of the stocks in the Russell 1000 today to determine the frequency of gaps going back over the last 10 years. I looked at gaps of 5%, 5-10%, 10-15%, 15-20% and finally those greater than 20%. I wanted to see how many times gaps of these sizes had taken place with the hope of discerning a statistical probability of these events reoccurring going forward.
5% Gaps
1914 times
5-10% Gaps
3077 times
10-15% gaps
692 times
15-20% gaps
219 times
20% or greater
230 times
Now, I know we have 2500 days, and I also know there are 1000 stocks, and for each percentage group we have a number of occurrences. Could anyone give me the probability of reoccurrence?
I initially began this exercise because downside gaps of magnitude seem to be a legitimate concern of many traders, myself included. It would be nice to know exactly how rare or frequent they may be. I realize I may need to give more information, but at this time I don't know what that would be.
Thanks in advance,
David John Hall
|
|
Registered User
Joined: 1/28/2005
Posts: 6,049
|
I believe if you had 1000 stocks and 2500 days.
Then you had (1000*2500) 2500000 opportunities
for a gap. Taking all gaps:
1914+3077+692+219+230=6132
So you had 6132 gaps out of 2500000 try's.
Approx .245 percent of the time.
------------------------------------------------------------
So if you watched the SP500 for 1 year.
(500*250)=125000
125000*.00245=306.25
you would expect to see 306 gaps.
At least that the way I think it goes.
Thanks
diceman
|
|
Registered User
Joined: 4/18/2005
Posts: 4,090
|
QUOTE (diceman)
I believe if you had 1000 stocks and 2500 days.
Then you had (1000*2500) 2500000 opportunities
for a gap. Taking all gaps:
1914+3077+692+219+230=6132
So you had 6132 gaps out of 2500000 try's.
Approx .245 percent of the time.
------------------------------------------------------------
So if you watched the SP500 for 1 year.
(500*250)=125000
125000*.00245=306.25
you would expect to see 306 gaps.
At least that the way I think it goes.
Thanks
diceman
Where does the red above figure come from? I'm not a math inclined person so wont' be contributing but I liek the "puzzles" and to think about it and learn etc etc. i did love a statistical math class I took several years ago.. but forget most of it by now.
|
|
Registered User
Joined: 4/18/2005
Posts: 4,090
|
what about positive gaps? They are out there as well. There is the one CowGirl caught last year. I recently rode the ILMN gap up. I think it safe to say they aren't the statistical norm.. but they happen enough that it's a matter of time before you get one or the other.
conversely I was also a holder for TZOO's 4/25 gap down.
|
|
Registered User
Joined: 1/28/2005
Posts: 6,049
|
"Where does the red above figure come from?"
------------------------------------------------------------
The red comes from:
6132/2500000=.245%
It should also be noted no adjustment was made
for 2 gaps in a row. If you had a rule of no gap
after a gap you would have to reduce the amount
of attempts. (the number is probably too small
to matter)
by the way. This works out to 1 gap every approx.
408 try's.
(2500000/6132)=407.69
How you see the gaps would depend on what you
were following.
1 Stock for 408 days you should see 1 gap.
25 stocks (408/25)=16.32 you should see a
gap every approx 16 days.
100 stocks a gap every approx 4 days.
-----------------------------------------------------------------------
One thing to note. This assumes a linear relationship.
I'm guessing in the real world the gaps happened in
clusters. (there were dry periods, then periods of
more gaps)
Its possible "length of days without a gap" could be
important for trading but that would depend on how
evenly dispersed they are.
Thanks
diceman
|
|
Registered User
Joined: 6/6/2005
Posts: 1,157
|
Thanks Diceman for your help. That is exactly the information I was looking for.
David John Hall
|
|
Registered User
Joined: 1/28/2005
Posts: 6,049
|
One thing to note. The 1000 as listed in telechart
is actually 982 stocks.
Thanks
diceman
|
|
Registered User
Joined: 4/18/2005
Posts: 4,090
|
Thanks.. I see it now.
|
|
Registered User
Joined: 2/7/2006
Posts: 218
|
I will assume that it is only useful to pick a stock that gaps since we want to profit from them...lets take the 5% scenario...so we have 6132 gaps. Now that is out of 1000 stocks and thats over 10 years which is 364250 or so days assuming average years. Im assuming we want the probability of choosing a gap
so we want to choose a single stock on a single day that it gaps that gap has to be a minimum of 5 %.
that is 3 seperate things happening at the same time
so (A)=gap (B)=day (C)=stock
so if (A), (B), and (C) are mutually independant events (all pairs of events are independant), then the probability that (A), (B), and (C) all occur is
P((A)intersect (B) intersect (C)) = P P P
so [(1/6132) * (1/1000) *(1/364250) ] = 4.5e -13 or 1/2233581000000
the previous situation was (A) U (B) U (C)
|
|
Registered User
Joined: 1/28/2005
Posts: 6,049
|
"lets take the 5% scenario...so we have 6132 gaps. Now that is out of 1000 stocks and thats over 10 years which is 364250 or so days assuming average years."
---------------------------------------------------------------------------------------------
The 5% scenario was 1914 Gaps.
(6132 is all gaps)
There are approx. 250 trading days in a year.
(even though the calendar is 365 we cant trade
on weekends and holidays)
Thanks
diceman
|
|
Registered User
Joined: 1/1/2005
Posts: 2,645
|
davidjohnhall,
I find your results most interesting and a bit surprising (to me). I have two questions and some observations.
1) How did you measure the size of a downside gap? I assume it is:
-100*(H0-L1)/L1
2) Suppose we divide the magnitude of downside gaps into equal intervals on the Logarithmic Scale. Then, starting with 0 to 5%, we have:
0.00 to 5.00
5.00 to 9.75
9.75 to 14.26
14.26 to 18.55
18.55 to 22.62
etc.
What I would expect is that the number of gaps falling in each these intervals would be decreasing. Your results indicate that this is not the case. In your 5% category, did you include all gaps in the interval 0 to 5%?
3) Your results indicate that when downside gaps occur they tend to be significant in magnitude, i.e., the probability density function of their magnitude is not monotonic decreasing.
Thanks,
Jim Murphy
|
|
Registered User
Joined: 6/6/2005
Posts: 1,157
|
Hi Jim,
For the first category of gaps I looked at gaps of only 5%. I began by looking at gaps of 0-5% but there were over 65,000 of them. I then decided to start with gaps of 2-5%. Once again there were over 65,000 of them (my backtesting software creates an excel spreadsheet for each backtest and it maxes out at 65,000 lines of data). To remedy this, I decided that, for my little experiment, I would be happy starting with gaps of 5% where 0<l1*.955 and 0>l1*.95. So technically the first set is for gaps between 4.5% and 5%.
What I really was asking is, which mathematical factors would I have to include to get a model (even a crude one) for determining, based on historical occurences, the probability that a certain event would repeat in the future.
As far as I'm concerned, trading has a lot to do with psychology, and (for me and maybe others) the dreaded GAP down is like the proverbial monster under the bed. What I wanted to do was create a mathematical picture of what these GAPS really are and how often they might show up so that information can be digested by the psyche and understood on a more rational basis.
Any help you could offer would be greatly appreciated. Diceman offered a number in an earlier thread. Have you read that number? Would you agree with it? Or do you have an alternative?
David John Hall
|
|
Registered User
Joined: 1/1/2005
Posts: 2,645
|
davidjohnhall,
Thanks for the response.
I could hardly have been more wrong in my interpretation of your tests.
1) For your tests, a downside gap is a gap down from the previous price bar low to the present price bar opening. It is not a gap down from the previous price bar low to the present price bar high.
2) The 5% category is the interval 4.5 to 5.0%. It is not the interval 0 to 5%.
3) In light of my now corrected perception of your tests, I am no longer surprised by the results.
I do not disagree with the numbers diceman gave; although, I doubt he knew their meaning at the time he gave them. But, he does now.
Thanks,
Jim Murphy
|
|
Registered User
Joined: 2/7/2006
Posts: 218
|
please tell me if im thinking backwards, i got the days wrong but if we are looking for a 5% gap up then a 20 % gap up would work also, because 20 includes 5, that is why i took all gaps. so replace 365250 with 2500 and run it again. we have to change the first number also because we are choosing a gap out of a number of choices.
so probability of choosing any gap in any stock on a any day
6132/2500000 x 1/1000 x 1/2500
so i pick a day, pick a stock, and look for a gap, that is different than watching a specific stock every day, but you still could not see a gap, and different then just counting the number of gaps there was.
the difference is in rolling 3 white dice as opposed to 3 dice, one red, one green, and one blue and wanting specific numbers on each color.
google search probability addition and multiplication
"AND" or Intersections
Independent Events
Two events are independent if the occurrence of one does not change the probability of the other occurring.
An example would be rolling a 2 on a die and flipping a head on a coin. Rolling the 2 does not affect the probability of flipping the head.
If events are independent, then the probability of them both occurring is the product of the probabilities of each occurring.
Only valid for independent events
P(A and B) = P(A) * P(B)
"OR" or UnionsMutually Exclusive EventsTwo events are mutually exclusive if they cannot occur at the same time. Another word that means mutually exclusive is disjoint. If two events are disjoint, then the probability of them both occurring at the same time is 0. Disjoint: P(A and B) = 0
If two events are mutually exclusive, then the probability of either occurring is the sum of the probabilities of each occurring.
Only valid when the events are mutually exclusive.
P(A or B) = P(A) + P(B)
|
|
Registered User
Joined: 2/7/2006
Posts: 218
|
I was thinking that next question was coming
conditional probability is the P(A given B) = P(A and B) / P( B )
Google bayes' theorm or conditional probability
|
|
Registered User
Joined: 6/6/2005
Posts: 1,157
|
ak -- thanks -- I had a feeling some crazy math that I don't understand would come into play.
A couple things...
We're talking gap downs not gap ups.
Also, I'm looking at gaps of different magnitudes. So gaps in the 15-20% range would not include gaps in the 5-10% range. So the numbers in my test by that range are for that range only, not including the smaller gaps.
Thanks for all of your input. I am reading up on Bayes' theorm. That should keep me busy for quite some time.
|
|
|
Guest-1 |