How long does a backtest need to be in order for the results to be statistically significant, assuming every variance such as slippage and commission were accurately accounted for?  This would be for a test run on intraday tick data.  On a daily chart, I assume 7-10 yesrs is needed to run a test.  Thanks for your opinions.

If during the "length" of the backtest the markets ar predominantly in a bull market, then you wouldn't get a reliable picture on how your test would behave in a bear market. So I guess it would be difficult to come up with an exact "minimum" length.

If I remember well, elsewhere on this forum, it was suggested you would at all times compare your test against  a series of randomly generated trades.

it was mentioned here, and I think there was en example somewhere as well

 http://forums.worden.com/default.aspx?g=posts&t=42329

I always try to test against a bullish, bearish and consolidating market phase.  I was more talking about what a potential investor wants to see more so than for my use.  Thank you though.

see  Bband - with CCI break out scan. for code for generating random sample backtest. The idea being to pick a random stock and hold it for a fixed time based on the average hold time of the approach you a comparing.

The link using an 18 day hold and an ajusted percet of 0.55% to get the right amount of trades during the test period.

---------------------------------------

As to the length of test, an equally critical question is how many trades. For example a 5 year test against S&P500 resulting in 150 trades might be to thin a selection. If the criteria are tuned as you evolve the strategy some curve fitting might be ocurring, fitting the strategy to the data.

One approach is to split the data into to sets. One to develop/train the strategy and the other to validate it. There is an excellent book
       Evidence Based Technical Analysis by David Aronson  
that covers may asppect of validating strategies.

I prefer doing a fixed length trade random sampling over the same data to see what a monkey with darts would do. This approach captures the market bias that the market trend of the test period comtributes.

One would want performance that significanlty exceeded the random sample to feel comfortable with the strategy.

________________________

 Two more points:

1. Some times a strategy produces good results but the trades are bunched because of some market condition that matches the criteria. Then one could not take full advantage of all the trades as there wouldn't be enough capital to fully commit.

2. Long duration tests against market indices like S&P500 or Russel 1000 suffer from membership bias. A form of survivorship bias. The fact that a stock is listed on the S&P500 today indicates some level of success. If it was added during the backtest period, that is not in the list at the start of the backtest, with data for the whole backtest, one can assume that is will outperform the market duing the backtest.

Applying the strategy gouing forward does not have this advantage as one doesn't know what stock will be added and removed from the list in advance.

Thanks for your input, although I am working on an intraday strategy, which averages 3 trades per day per vehicle.  If we test on the 500 most widely traded stocks/ETFs which have been adjusted for ATR and Beta, the system should average around 1500 trades a day or 375,000 a year, ball park.  That said, the number of trades is not as much of an issue.