Welcome Guest, please sign in to participate in a discussion. Search | Active Topics |

What computer power & memory needed to multi-task and use stockfinder 5? Rate this Topic:
Previous Topic · Next Topic Watch this topic · Print this topic ·
mc7e
Posted : Tuesday, October 5, 2010 11:05:25 AM
Registered User
Joined: 3/8/2005
Posts: 23
I'm looking for some input from fellow stockfinder 5 users who also like to keep a trading platform open for trading during market hours. If I go through 20-30 charts in one sitting stockfinder 5 starts freezing up. I mean it freezes for 6-15 minutes and becomes unusable. I've been going back and forth with Worden reps for 5 weeks now on what to do get stockfinder 5 to stop freezing up. I've sent them layouts which they concede freezes on them too, I've given them remote access, and followed every 'solution' they've thrown my way, but with zilch results.  But that was all weeks ago now and follow up calls have been, 'we'll get back to you when we know what the problem is' or 'so-and-so will call you back' which doesn't really happen. I've been told it already went up the chain to the developers a month ago, but with no answers. I've also been told that my computer specs are double what Worden recommends, but I have to assume that's just wrong. So, what have others found to be enough cpu speed and memory enough to scan for stocks while trading stocks from an open platform.
burls
Posted : Sunday, October 10, 2010 6:05:55 PM
Registered User
Joined: 10/5/2009
Posts: 66
Hi,

Similar issues with me....and that includes after hours sorting/filtering....these are taking 7-10 minutes per and with a few of them freezing...

I have sent them layouts...they have logged onto my laptop...same results as you.....

They told me there is a problem with SF5 vs SF4 in some cases/situations and they don't have a fix or it will be put in some future update....So I'm thinking of going back to SF4 if possible....

I have an Intel i3 processor with 4 GB of ram.....

Jeff
mc7e
Posted : Thursday, October 14, 2010 4:04:39 PM
Registered User
Joined: 3/8/2005
Posts: 23
burls:

Thanks for your input. I have 1/2 your memory and a much older processor.  I am finding out now that when I drop the color coding on up/down stocks, I get much better performance.  I hate the idea of going back to SF4 as I've gotten comfy with SF5, when it doesn't freeze.  After hours sorting/ filtering hasn't been a problem for me.

Greg
burls
Posted : Sunday, October 17, 2010 8:14:11 PM
Registered User
Joined: 10/5/2009
Posts: 66
I hate having to go back to SF 4 as well....but the sorts/filters are taking sooooooo long-every night it's taking me 2-3 hours vs 1 hour to be done....very frustrating.

I'm going to a training session next week-end will be talk with them and see what else can be done....

Burls....
machado
Posted : Tuesday, October 19, 2010 10:41:43 PM
Registered User
Joined: 3/13/2005
Posts: 107
Hi All

My two cents, I was using a pentium M (1.4Ghz) laptop with Win XP and 1.5G of ram (a bit on the old side) and was having the same issues as all you were having. It use to take 1hour or so do do my scans. I got an HP Envy 17 Laptop with the Q840 I7 Proceesor with 8G of ram, Win 7  about 6 moths ago and let me tell you it throws SF5 around  like a rage doll. Insteade of taking 1hour to do my scans , it does it in about 4 min. Hope this helps

Carlos
pthegreat
Posted : Thursday, October 28, 2010 4:14:34 PM

Registered User
Joined: 6/15/2008
Posts: 1,356
Has anyone used the "Nvidia Tesla" boards? transforms your PC into a supercomputer. The boards are very affordable somewhere between $1500 and $2500 I believe.
Can turn your regular quad core CPU into a 960 core ! runs under Win7 64bit.
inreases your computing power up to 250 times!
http://www.youtube.com/watch?v=l8FUmS1h-5U

Gonna investigate a bit more. just wondering if this would benefit Stockfinders performance. I always would love to have my scans run faster.


pthegreat
Posted : Thursday, October 28, 2010 4:40:11 PM

Registered User
Joined: 6/15/2008
Posts: 1,356
wow, sweeeeet !
http://www.youtube.com/watch?v=_87a6P0-Xjw&feature=related
jas0501
Posted : Friday, October 29, 2010 3:13:47 AM
Registered User
Joined: 12/31/2005
Posts: 2,499
QUOTE (pthegreat)
Has anyone used the "Nvidia Tesla" boards? transforms your PC into a supercomputer. The boards are very affordable somewhere between $1500 and $2500 I believe.
Can turn your regular quad core CPU into a 960 core ! runs under Win7 64bit.
inreases your computing power up to 250 times!
http://www.youtube.com/watch?v=l8FUmS1h-5U

Gonna investigate a bit more. just wondering if this would benefit Stockfinders performance. I always would love to have my scans run faster.




This is an "educated guess"

It is not clear that off the shelf software like StockFinder can take full advantage of the super computer in a PC approach. The code needs to be able to run the same logic in multiple cpu's in parrallel. This requires special programming and/or special compilation to have cpus preloaded with the logic and the data shuffled of the to cpu's. 

It is the case that the type of processing being done by StockFinder, backscanning in particular, could take advantage of the parrallel architecture, but it would require major code modifications.

Given that few users would be willing to invest in the extra hardware and seeing that the backscanner still reports incorrect Annual Return/Trade after forever, and that is a 15 minute fix, I don't expect this is a hardware upgrade approach that has legs.


pthegreat
Posted : Monday, November 1, 2010 9:40:01 PM

Registered User
Joined: 6/15/2008
Posts: 1,356
yes, to bad. only software based on their prorietary progamming language "Cuda" (similar to C) takes advantage of this hardware. But the specs are off the chain. for less then $10K you can build a wrkstation with the power of a supercomputer. 

I would love to have my scans run faster on sub 5minute charts. 10K would be well worth the investment. Maybe something for the SF programmers to consider. I'm not a programmer so I'm probably asking for something that's not visible.

I did however gain about 25% scan speed increase by overclocking my good old Intel I7 920 CPU from 2.6Ghz to about 3.8Ghz..

thisstockguy
Posted : Friday, November 5, 2010 11:46:35 AM
Registered User
Joined: 10/7/2010
Posts: 92
QUOTE (pthegreat)
Has anyone used the "Nvidia Tesla" boards? transforms your PC into a supercomputer. The boards are very affordable somewhere between $1500 and $2500 I believe.
Can turn your regular quad core CPU into a 960 core ! runs under Win7 64bit.
inreases your computing power up to 250 times!
http://www.youtube.com/watch?v=l8FUmS1h-5U

Gonna investigate a bit more. just wondering if this would benefit Stockfinders performance. I always would love to have my scans run faster.




My CPU doesn't exceed 50-60% when Stockfinder is running a filter which indicates it is disk bound. A thousand processors won't help if they all have to wait to get the data off of the disk. (BTW, SF5 is multithreaded so having more CPU's will help.)

When doing a filter it is opening hundreds of files. If you upgrade to SATA 2 or 3 from an old SATA 1 or IDE drive, you should notice a significant speed difference. You will of course need to upgrade your controller as well as the drive to SATA 2 or 3. These drives have a much faster throughput than an old IDE drive.

If you have a software firewall or AV program, you should have it exclude the SF data directory so it doesn't check the files when they are accessed.

This should help. :-)

TSG
Doug
Posted : Friday, November 5, 2010 2:07:50 PM

Worden Staff

Joined: 11/8/2004
Posts: 574
QUOTE (mc7e)
I've been going back and forth with Worden reps for 5 weeks now on what to do get stockfinder 5 to stop freezing up. I've sent them layouts which they concede freezes on them too, I've given them remote access, and followed every 'solution' they've thrown my way, but with zilch results.  But that was all weeks ago now and follow up calls have been, 'we'll get back to you when we know what the problem is' or 'so-and-so will call you back' which doesn't really happen. I've been told it already went up the chain to the developers a month ago, but with no answers.
Can you email one of your layouts to support, attention Doug? I don't have any bug reports from you on this. I'll also follow up with tech support to find out why I haven't gotten anything yet.
Doug
Posted : Friday, November 5, 2010 2:18:17 PM

Worden Staff

Joined: 11/8/2004
Posts: 574
QUOTE (burls)
Hi,

Similar issues with me....and that includes after hours sorting/filtering....these are taking 7-10 minutes per and with a few of them freezing...

I have sent them layouts...they have logged onto my laptop...same results as you.....

They told me there is a problem with SF5 vs SF4 in some cases/situations and they don't have a fix or it will be put in some future update....So I'm thinking of going back to SF4 if possible....

I have an Intel i3 processor with 4 GB of ram.....

Jeff
Can you also send a copy of your layout to support, attention Doug? I don't have a copy of it and I need to get it tested and documented for the developers. I'll also follow up with support to find out where the breakdown in communication is. I need the layouts from you and mc7e to get the problem documented (duplicated, if possible) and then sent on to the developers. I'll email each of you with a case number.
Doug
Posted : Friday, November 5, 2010 2:23:16 PM

Worden Staff

Joined: 11/8/2004
Posts: 574
QUOTE (jas0501)
and seeing that the backscanner still reports incorrect Annual Return/Trade after forever, and that is a 15 minute fix
The issues with BackScanner are actually complicated. If it were a 15 minute fix, they'd have been addressed long ago. The good news is that we'll soon be working on StockFinder 5.1, which will tackle all documented BackScanner issues. If you have time, shoot an email to me with a list of the BackScanner bugs you're aware of. I'll review the database and make sure that I've got a case for everything. If I'm missing anything, I'll get with you for good instructions to reproduce the problem and get it in there ASAP.
Doug
Posted : Friday, November 5, 2010 3:58:08 PM

Worden Staff

Joined: 11/8/2004
Posts: 574
QUOTE (Doug)
QUOTE (mc7e)
I've been going back and forth with Worden reps for 5 weeks now on what to do get stockfinder 5 to stop freezing up. I've sent them layouts which they concede freezes on them too, I've given them remote access, and followed every 'solution' they've thrown my way, but with zilch results.  But that was all weeks ago now and follow up calls have been, 'we'll get back to you when we know what the problem is' or 'so-and-so will call you back' which doesn't really happen. I've been told it already went up the chain to the developers a month ago, but with no answers.
Can you email one of your layouts to support, attention Doug? I don't have any bug reports from you on this. I'll also follow up with tech support to find out why I haven't gotten anything yet.
mc7e - I've got your information from support and will work on this Monday. I'll send you an email with the case number when I am done.
Doug
Posted : Friday, November 5, 2010 4:02:28 PM

Worden Staff

Joined: 11/8/2004
Posts: 574
QUOTE (Doug)
QUOTE (burls)
Hi,

Similar issues with me....and that includes after hours sorting/filtering....these are taking 7-10 minutes per and with a few of them freezing...

I have sent them layouts...they have logged onto my laptop...same results as you.....

They told me there is a problem with SF5 vs SF4 in some cases/situations and they don't have a fix or it will be put in some future update....So I'm thinking of going back to SF4 if possible....

I have an Intel i3 processor with 4 GB of ram.....

Jeff
Can you also send a copy of your layout to support, attention Doug? I don't have a copy of it and I need to get it tested and documented for the developers. I'll also follow up with support to find out where the breakdown in communication is.I need the layouts from you and mc7e to get the problem documented (duplicated, if possible) and then sent on to the developers. I'll email each of you with a case number.
I've also been able to get a copy of your information. I don't need anything, thanks. I will email you on Monday with a case number or any requests for further information.
pthegreat
Posted : Friday, November 5, 2010 4:19:54 PM

Registered User
Joined: 6/15/2008
Posts: 1,356
I use a Solid State drive : Crucial C300.  Huge improvement, and well worth the $$$.

check for best performance on CPU's, Harddrives, etc..   at http://www.cpubenchmark.net/

jas0501
Posted : Monday, November 8, 2010 11:15:03 AM
Registered User
Joined: 12/31/2005
Posts: 2,499
QUOTE (Doug)
QUOTE (jas0501)
and seeing that the backscanner still reports incorrect Annual Return/Trade after forever, and that is a 15 minute fix
The issues with BackScanner are actually complicated. If it were a 15 minute fix, they'd have been addressed long ago. The good news is that we'll soon be working on StockFinder 5.1, which will tackle all documented BackScanner issues. If you have time, shoot an email to me with a list of the BackScanner bugs you're aware of. I'll review the database and make sure that I've got a case for everything. If I'm missing anything, I'll get with you for good instructions to reproduce the problem and get it in there ASAP.


The Annual Return/Trade is using 6.5 hours per day for backscans with bar => 1 day, instead of 24 hours. As a result the results are overstated by 24/6.5 or 3.69. It would seem to me that is the bar interval is >= 1 day, don't use 6.5, use 24.

=============================
As to "we'll soon be working on StockFinder 5.1, which will tackle all documented BackScanner issues".   Can this list be published so that users may comment on omissions?

 

I know it is not policy to reveal features, except by release, however it would be good to know what the general game plan is for the backscanner. Is there any intention of supporting any of the following?:

1.     A way of saving the results that permits reconstruction or at least realization  of what the backscan rules were at the time it was run.

2.     Improved run to run results tracking

a.     Tabbed scan results, so that one can appreciate result improvements

b.     The ability to provide description with the scan results. Typically consecutive scans are run tweaking settings. There is no easy way to review multiples runs and appreciate what caused the improvements

3.     Money management with support for

a.     Tie breaker, for picking which trades to execute when multiple opportunities are presented

b.     Margin

c.     Minimum/maximum  setups per bar

d.     Minimum/Maximum trades per bar

e.     Maximum open trades

f.      Multiple Trade sizing models

4.     A standard template and support for user specified result templates. This would permit users to compute their own metrics as in  Sharpe, Kelly, RAR,  etc. etc

a.     For scan results

b.     For MM results

c.     For optimization run summaries

5.     Intermediate results display of backscan results, to permit early termination if the results don’t look promising. This is particularly helpful when running long scans.

6.     Equity curve display during backscan

7.     Interruption of backscan with partial results retained

8.      Entry and exit @Price

9.     Realcode access to Trade information

10.   Saving backscan trade sets permitting running vai realcode post scan

a.     Money management studies

b.     Entry studies

c.     Exit studies

d.     Correlation studies

11.   For saved backscan trade sets, permitting additional indicator values to be associated with each trade. For example @ trade entry what various ATR values were.

12.   bars in Trade variable in realcode

13.   realcode Loop support for both backscan and Money Management  runs to permit optimization

14.   For optimization

a.     Multiple formats for stating variable values

                                          i.     A set, as in  [5,7,13,23,42]

                                         ii.    A stepped range, x to y stepped by z, as in (5 to 30 by 5)

b.     Realcode access to variable assignments.

15.   Standard IO for reading and writing results. This would permit post processing of run results and adjustments of parameters for scans or MM runs.

16.   Realcode support for multiple positions with scaling into and out of positions

jschott
Posted : Tuesday, November 9, 2010 11:05:02 PM
Registered User
Joined: 10/27/2004
Posts: 821
Location: Philly area
Lotts of Nvidia Tesla information can be googled up at the Nvidia site.

Boards are not too costly as indicated, Trouble is, it won't do Windows and SF is a Windows product so far - so you 'd need to write your own code or convince Worden to redevelope SF onto a new host.  Since SF uses (or used) was originally based on Java Beans (I think), but they use a lot of MSFT tools (even Silverlite on FreeStochCharts) that's a lot of work ..............

Eventually someone will offer some super professional tool - but at a price that likely will make the Tesla engine board a mere drop in the bucket..........

So we can dream on......
jschott
Posted : Tuesday, November 9, 2010 11:13:13 PM
Registered User
Joined: 10/27/2004
Posts: 821
Location: Philly area
A more practial thought is that more memory and faster disk are often the answer.  But consider breaking the process into several parts (for example, do filtering by stages on things that do not change at a fast pace.).

Also, there are some language tools in SF that allow you to limit the number of bars on which calculations are done at least for your "today" data calculations.  (The first suggestion would be to learn to live with the first stage of filetering for backtesting.)

Of course filtering out "things you would never be interested in" rather than calculating on every potential security, etc.

(As a odd idea, I remember that you could run a lot of TC out of RAM memory - even putting some data into RAM.  I am not at all sure that moving SF5 things (data and/or code) into RAM is possible.  You might check for any notes in the forum archive on that and if that fails put the question to Bruce or the other support people.)
thisstockguy
Posted : Thursday, November 11, 2010 4:42:40 PM
Registered User
Joined: 10/7/2010
Posts: 92
Run the Passmark benchmark program to see what areas of your computer are deficient. You can also compare your computer with others to see what could be achived if you have the $$$.

You can get a free 30 day Passmark eval from http://www.cpubenchmark.net/high_end_cpus.html

TSG
wwrightjuly4
Posted : Friday, November 12, 2010 6:16:58 PM
Gold Customer Gold Customer

Joined: 4/10/2006
Posts: 954

http://en.wikipedia.org/wiki/CUDA

http://forums.nvidia.com/index.php?showtopic=79086
In .NET, you need to do a Platform Invoke (P/Invoke) on the CUDA dll (nvcuda). I don't know what the exact syntax is to do that in VB.NET, since I'm primarily a C# developer...
GPU.NET: Write your GPU code in 100% .NET (e.g., C#, F#, VB.NET) -- No CUDA / OpenCL knowledge required!
Learn more at http://www.tidepowerd.com. Follow us on Twitter @tidepowerd for updates!

11-Nov-2010:
GPU.NET Beta 1 now available for download!
wwrightjuly4
Posted : Friday, November 12, 2010 7:33:26 PM
Gold Customer Gold Customer

Joined: 4/10/2006
Posts: 954


I  believe SF would have to be organized to use a CUDA library.  I am not sure whether vb.net programs even take advantage of multi-core processors.  i.e. it would be nice if each indicator or realcode snippet could be pushed out to a "core". 

The effort would assume that the average stockfinder user or majority could/would benefit from and afford this type of hardware.

And assume that the increases in Standard CPU motherboards would not chip away at this effort before you even finished making the code changes.  

Obviously, SF already takes advantage of threads and perhaps these can easily be tied to the CUDA calls, such that if the the CUDA/hardware exists then it uses them else uses stander multicore CPU.

-----
Assuming the vb.net is able to take advantage of the multicore arch.  Not sure if it even
     http://www.codeproject.com/KB/threads/tricky_multicore.aspx

 -- The Task Parallel Library (TPL) is designed to make it much easier to write managed code that can automatically use multiple processors. --

This test uses Microsoft Parallel Extensions to .NET Framework 3.5. The advantages of using Parallel FX are:

  • No need in thread synchronization (meaning main thread and both sorting threads)
  • The smallest code








Minimum SF computer is probably your Mean audience, so why put the extra effort to program/develop for CUDA environment if it takes special attention in designing code to run on CUDA.

However, once the SF is stable and you investigated

wwrightjuly4
Posted : Friday, November 12, 2010 11:08:46 PM
Gold Customer Gold Customer

Joined: 4/10/2006
Posts: 954
Wow, I really should read the other posts more closely....   <wheres the delete my on comment button>
Users browsing this topic
Guest-1

Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.