Welcome Guest, please sign in to participate in a discussion. Search | Active Topics |

How to maximize speed of scans? Rate this Topic:
Previous Topic · Next Topic Watch this topic · Print this topic ·
remobile1
Posted : Tuesday, July 6, 2010 1:50:50 PM
Registered User
Joined: 9/10/2009
Posts: 53
I'm scanning 1000 stocks with a 6 part combo condition on 30 minute charts. It takes nearly four minutes to complete.

I'm running a fairly high end machine -is there any way of speeding this up?

For instance, is it faster to put the more simple conditions at the start of the combination so SF only tests the complex conditions on fewer stocks? 

Also, will the program run faster under Windows 7 than under XP which I use? Also, if I upgraded to quad core -can SF benefit from this?
thevinman
Posted : Wednesday, July 7, 2010 6:30:56 AM
Registered User
Joined: 2/22/2010
Posts: 187
Dear remobile1,

I had the same issue and there are some things you can do:
1a. The most complex conditions at the end of the filter as you already filtered out most of the symbols
1b. Do a general scan first of the 1000 shares to filter out the ones you will never trade. Filter on min and max price, average daily volume and/or min daily ATR. Then copy that list to you personalized scan.
2. When I changed to windows 7 the speed went up dramatically, but this is my personal experience.
3. Go to settings/data manager and lower the number of bars you minimum need for your conditions to work.
4. Some computer stuff you can do: defrag your computer, update windows manually, clear caches of java and explorer, upgrade your internal memory. 

I hope this will help you. Kind regards,

thevinman

remobile1
Posted : Thursday, July 8, 2010 3:10:31 PM
Registered User
Joined: 9/10/2009
Posts: 53
Thanks vinman
jas0501
Posted : Thursday, July 8, 2010 3:51:35 PM
Registered User
Joined: 12/31/2005
Posts: 2,499

QUOTE (thevinman)
Dear remobile1,
...
...
1a. The most complex conditions at the end of the filter as you already filtered out most of the symbols
...
...



Along these lines and considering realcode issues:

if A and B and C then
  whatever
endif

can be improved by using andalso instead of and

This results in not having to evaluate B or C if condition A is not true
So

if A andalso B andalso C then
  whatever
endif


would be faster.


Also ordering the least likely items first can improve things. For example
A is True 90% of the time
B is True 50% of the time
C is true 2% of the time
 
Reordering  the code to be 

if C andalso B andalso A then
  whatever
endif


 would require the evalutation of A only about 1% of the time instead of 90% in the first example.

The payoff is dependent on the cpu complexity of A and B and C. It could be that C take 200 times more time to calculate. If that were the case then moving it to the end would be the prudent thing to do.


Another example

if IND.value > 80 then
 x += price.close
end if
if IND.value < 20 then
 x -= price.close
end if

can be made faster by using else

if IND.value > 80 then
 x += price.close
else if IND.value < 20 then
 x -= price.close
end if


thus avoiding looking at IND.value < 20 when it is > 80.

The most fruitful code enhancement is the elimination of loops in the code. For example counting the bars since x happened with 
 
dim count as integer
    for i = 1 to lookback
       if  Xhappened then
            count = 0
       endif
       count + = 1
    next i
will loop 100 times for every bar

Instead

static count as integer
if isfirstbar then
   count = 0
end if

if xHappened then
  count = 0
endif
count += 1
...
avoids the loop



Clearly less bars and fewer symbols is the first approach to apply with the best payback. Modifying the code is a 2nd level enhancement with loop elimination providing the best return.



That said, the KISS concept, "Keep It Simple Stupid", often can be the best approach. Stripping out some of the complexity selectively and thoughtfully can often provide the best speed improvement.

Kuf
Posted : Friday, July 9, 2010 10:26:26 AM


Administration

Joined: 9/18/2004
Posts: 3,522
These are all excellent points in speeding up your calculations. I love it when the community can help each other.

One final point I can add is make sure you are not using cumulative indicators (like exponential averages) as these will have to calculate for the entire data set.  Switching a 20 bar exponental average to a 20 bar simple will go from having to use 500 bars (if that is what your data manager is set to) to using only 20 bars for the calculation.  Multiply that times the number of symbols you're scanning and you can see a huge performance increase.

Ken Gilb (Kuf)
Chief Software Engineer - Worden Brothers Inc.
Try/Catch - My RealCode Blog
Users browsing this topic
Guest-1

Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.