Worden Discussion Forum

Welcome Guest, please sign in to participate in a discussion.

Worden Discussion Forum » General Discussions » RealCode for StockFinder 5.0 » Should RealCode allow Full access to .net namespaces

Poll Question: Should we allow full .net namespace access?
Choice	Votes	Statistics
Yes, but only for your own code	3	13%
Yes, but warn me when running someone elses code	19	82%
No, just add sandbox file I/O instead	1	4%
No, too big of a security risk	0	0%

Should RealCode allow Full access to .net namespaces

Rate this Topic:

Previous Topic · Next Topic

Watch this topic · Print this topic ·

Kuf

Posted : Friday, July 9, 2010 11:02:04 AM

Administration

Joined: 9/18/2004
Posts: 3,522

There have been a lot of requests for full .net namespace acesss (IE file IO, DLL access etc). We have been hesitant in the past becuase of security and data vendor contracts. With some effort we can fullfill our data vendor obligations but security will always be an issue unless we do one of the following:

Allow it for your own code. Only code you have authored will be allowed to access the .net namespaces

Allow for all but warn when user is requesting file io or some other namespace: This will allow you to see what they want to access, but really doesn't let you know what they're doing with it. It will allow you to deny the code.

Allow sandbox file IO: This method would give you some wrapper code to read/write to a specific location on your computer and only that location. Much like the data import options in realcode.

Please vote in the poll above.

Ken Gilb (Kuf)
Chief Software Engineer - Worden Brothers Inc.
Try/Catch - My RealCode Blog

Kuf

Posted : Tuesday, July 13, 2010 10:38:48 AM

Administration

Joined: 9/18/2004
Posts: 3,522

Bumping for more votes

Ken Gilb (Kuf)
Chief Software Engineer - Worden Brothers Inc.
Try/Catch - My RealCode Blog

bobre1

Posted : Tuesday, July 13, 2010 12:03:04 PM

Registered User
Joined: 10/7/2004
Posts: 886

I just voted for option #1, but meant to vote for #2... that is IF the warning with option #2 is a one-time warning only. I wouldn't want a warning coming up everytime the code is executed.

Winnie

Posted : Tuesday, July 13, 2010 2:06:34 PM

Registered User
Joined: 10/7/2004
Posts: 1,006

One Cautious Caveat,

I’m not a programmer or fully understand the ramifications of the proposed changes. StockFinder’s wide range of customization of Charts/Indicators/Conditions/WatchList/ etc. is wonderful but the fundamental efficiency/accuracy and data feed should not be handicapped by the proposed changes. If the change is for the better, do it, but very often there is a fine line by offering more bells and whistles at the expense to the program.

Kuf

Posted : Tuesday, July 13, 2010 4:26:14 PM

Administration

Joined: 9/18/2004
Posts: 3,522

The question is one of lifting a current restriction we have in place. For instance if this was a car, we would be lifting a speed regulator that keeps your car from going over 150 mph. You could now go faster than 150, but it would be at your own risk.

Basically it's for programmers that want more access to their own computer instead of the sandbox world that they live inside of StockFinder.

Ken Gilb (Kuf)
Chief Software Engineer - Worden Brothers Inc.
Try/Catch - My RealCode Blog

Flash99

Posted : Tuesday, July 13, 2010 8:30:59 PM

Platinum Customer

Joined: 7/16/2009
Posts: 411

Kuf,
It was very hard for me to vote only one time. Do not want to mess with your poll but I could stay here and vote till tomorrow.
Hope to have IO available soon.
Thanks

robconley

Posted : Tuesday, July 13, 2010 10:11:50 PM

Registered User
Joined: 3/10/2005
Posts: 28

While the second option sounds appealing it also sounds as though it could open all of our computers up to potentially huge security issues. I hope you think this through VERY carefully before implementing it. Is it worth pleasing a handful of "power" users while possibly alienating everyone else due to a malicious attack or data theft? And don't kid yourselves. It WILL eventually happen. If you do decide to implement option 2, however, please include a (default to on) program-wide disable option for those not wanting to - or more importantly, not knowing how to - deal with this risk.

progster

Posted : Wednesday, July 14, 2010 7:31:40 AM

Registered User
Joined: 6/14/2005
Posts: 628

Why not just have either a configuration setting or a registry setting?

Default it to the "safest" behavior, and then let those who wish to change it to enable the supposedly more "risky" behaviors.

BTW folks, every time you run an installer for any any program, you are basically making the decision to trust the author of that installer and the chain of custody between that author and your machine.   Running "enabled" RealCode is not fundamentally different in this respect.

Further, for all you know, your data vendor's client-side app is scouring your computer in the background for your identity and financial information and sending it to criminals for later exploitation. Unless you are educated in and operating with methods and tools far beyond those of the typical user, there is simply no way you would ever know.

We live on a spectrum ranging from reckless disregard, though basic prudence, and on past irrational paranoia. However, the only true "safety" online is to be offline (with the power disconnected).

One might pose the question:   Would you prefer to have a life, or a truly secure computer? (Choose only one)

BeechTop

Posted : Sunday, July 25, 2010 12:17:59 PM

Registered User
Joined: 2/2/2005
Posts: 129

Here is a very real problem with the current RealCode data reader approach -- I collect some manual data that I use on certain charts. On my different computers, the username is necessarily different. Therefore, since the username is part of the path to the only data folder the RealCode data reader can read, charts using the data reader are not transferable between those computers.

Interestingly enough, this is not a problem with the block diagram CSV to bar/line approach. There, you can access a file in about any path. Thus by having identical paths to the data folder on the two computers, the charts are transferable.

Another question about the RealCode data reader -- it appears that it can only read date plus five data columns (presumably open, high, low, close and volume). One of my data sets has more than five columns of data, and I can't get the data reader to read and output those additional columns of data.
Is this a truly a limitation of the data reader?

paul1307

Posted : Monday, August 23, 2010 4:50:52 PM

Registered User
Joined: 10/7/2004
Posts: 41

I'm not sure what's being (potentially) offered; would this open up SF in the manner that the TC SDK did, that is, access to the underlying data, or just allow programmer to use the entire .NET namespace within code developed for custom indicators? I'd assume that SF data would remain 'read-only,' negating concerns that maliscious code could mangle SF or its underlying data.

Flash99

Posted : Monday, August 23, 2010 7:18:14 PM

Platinum Customer

Joined: 7/16/2009
Posts: 411

QUOTE (paul1307)

Paul, this would give you the possibliity to do things outside of the SF5 realm without leaving the realm. What Kuf is offering is a door open to the outside world (of course outside world means your PC). Once you can get out, the sky is the limit.

For example I am planning to create a control centre to manage buy/sell signals triggered by SF5. I could generate the signals in SF5 and analyse & prioritize them outside of SF5 by my own application (instead of browsing dummy log files).
Or you can send an email or SMS when a certain condition occurs (you can play golf and SF5 will let you know right away if your stock does something you want to know about, maybe it just completed a double top pattern).
Or generate complex analysis reports directly from SF5.
Or access(read/write) XML files directly (or other file types)
Or implement your own pattern recognition
Or...... the only limit is your imagination

I wish Kuf didn't ask the question the way he did. It makes it sound like there is a security concern.
In fact this is not more dangerous than developing & executing your own application on your own PC.

Further down the road, this has the potential of a gold mine for Worden, it may attract programmers to develop apps wich will extend the SF5 functionality, hence the number of SF5 users. One analogy I can think of is the AppStore for iPhone. This crappy iPhone (I cannot run even a Flash movie on this thing) enjoys such a success because of the huge number of apps which extend the device's functionality far beyond any competition.

paul1307

Posted : Friday, September 3, 2010 7:00:24 PM

Registered User
Joined: 10/7/2004
Posts: 41

Flash,

Thanks. I understand the part about extending SF5 to work with .Net 4.0. The part I'm unclear about is the ability I might have to exploit data within SF. For example, the SDK for TC allows me to pull data from TC, then write my own routines within my own program (I work in C# mainly), so I get the flexibility of working in C#, and my program will run at its own speed and pace (depending on how good or bad I code). That actually allows me to do all the things you noted.

What I'm not clear about is how any of this would apply to SF. For instance, if I don't have the ability to export prices or volume to my external program (or DLL) I'm still stuck using SF for calculations. Granted, I could generate an email if a stock reaches some criterion, but the calculations to determine that criterion might still have to reside within SF.

Legitimately, the question might be raised, "Well, if all you want to do is export or extract data from Worden, to where you can use your own program to do the things Flash99 mentioned, you can already do that with TC which, btw, comes free with SF, so why would we bother with providing that same capability with SF?" Good question.

I suspect what Kuff is really asking is whether or not we would like to be able to use (for instance) the .net graphics library, or the system.windows.forms library so we could develop our own user interface, or the system.collections.objectmodel library so we could use dictionary objects, etc. Presumably, if we could use system.linq, we could also export data to a db of our choice; is that what he's offering?

When SF first came out, Worden pointed out that developers could write code and sell it to SF users. I checked with Worden on how a developer could restrict distribution to only authorized purchasers and was told "there is no way." I see now where there is code in the user area within SF for download that requires a password, but that's not real security of the type Worden has for their code. To really produce a marketable product you'd need to replace SF (or TC) with your own program and distribution and authentication system.

Interesting disucssion though, and I'm looking forward to Kuff's comments on the subject to see if it clears anything up. Judging by how slowly responses are coming from Worden/Kuff/et al, to posts in these forums though, I'm assuming that they're either snowed with SF updates, or ...

Keep your stops tight, and good trading,

Paul

Flash99

Posted : Friday, September 10, 2010 5:20:45 PM

Platinum Customer

Joined: 7/16/2009
Posts: 411

QUOTE (paul1307)

Flash,
Legitimately, the question might be raised, "Well, if all you want to do is export or extract data from Worden, to where you can use your own program to do the things Flash99 mentioned, you can already do that with TC which, btw, comes free with SF, so why would we bother with providing that same capability with SF?" Good question.

Good question indeed. Was told SF is the flagship product when I signed up. I should have, at least, give TC a try.

jas0501

Posted : Saturday, September 11, 2010 12:06:16 PM

Registered User
Joined: 12/31/2005
Posts: 2,499

QUOTE (Flash99)

QUOTE (paul1307)

Good question indeed. Was told SF is the flagship product when I signed up. I should have, at least, give TC a try.

If all you want is to export HLOC & V the TeleChart SDK is fine. But if you don't want to write your own:
o indicators
o charting
o threaded application space
o watchlist management
you will have to find the code for then elsewhere. or write it yourself.

TC Companion is an excellent example of what a motivated individual can accomplish using the TeleChart SDK. Its backscanning capabilities, strategy optimization, money management, and integration with Excel are all eqaul to or superior to Stockfinder's backscanner. And this was done by a single individual on a part time basis.

That said StockFuinder provides a solid platform with great set of indicators, easily extendable. Offerring access to .net namespaces will go a long way toward permitting feature additons to StockFinder, third party of individually developed.

One other key ingredient....

Loop control with parameter control
Looping varying a parameter set until a termination criteria is satisfied is a basic optimization construct. Adding this construct would finally provide a key feature for optimizing on many fronts...

over watchlists
If stockfinder permitted loop control over watchlists, similar to what Market indicators users, but multi-pas and optimization parameter maintenance user could build there own optimization approaches.

over backscanner
This same loop control wrapper around the backscanner would permit strategy optimization, money managment optimizations.

over saved trade set
With access to the saved trade sets of multiple strategies one could to portfolio optimizations. The key is a controlled looping method

Examples:
With .net namespace one could prepare a spreadhseet, do a solver, get back the next set of parameters, and continue the loop or terminate.

Loop and collect performance results and use externale charting to prepare a 3d barchart showing the performance distribution. Based on this adjust the parameters and improve the strategy.

Loop and do walk forward backtests using x bars of history to prepare parameters for the next y bars test, tracking the performance, Wrap this walk forward loop in another loop with parameter optimization.

Loop over a trade set adjusting the parameters for
   o scalling in and out of positions
   o adjusting trade size
   o tie breaker trade selection when multiple trades are available

With or without the addition of .net name space, a basic optimization loop construct with parameter control within StockFinder is key.

Posted : Friday, December 31, 2010 5:33:17 AM

Registered User
Joined: 10/9/2010
Posts: 18

I am interested in deeper integration with SF -- some way to call external code, or at least write to tmp files (which I could then scan from an external app). Checking the forums, I found this thread... so it seems to be in the works. It's been a while since this thread was updated -- was wondering if there was any news on this, or a timeline we'll see some of this?

paul1307

Posted : Friday, December 31, 2010 1:18:00 PM

Registered User
Joined: 10/7/2004
Posts: 41

Very good points jas0501 and a nice synopsis of what I was saying.

In case anyone has not heard, Worden is beta testing a 2011 version of TeleChart which will, presumably, ship in 2011. I have been using it for some time, and find that the charts are quite similar to TC 2007 - though not an exact match - and very easy on the eye and easy to adjust to after having used TC for as long as I have. There are still aspects of the product being developed, but it looks like it will be a nice upgrade from TC 2007. On the downside, however, and I'll assume that this may be one of those items still in discussion and development, EasyScans are as abysmally slow as they are in StockFinder. I suspect they're either using the SF database rather than having abandoned the TC database model.

As a developer, the problem with the TC database is that having been developed under the COM spec way back in the mid to late 90's, it's incompatible with 64-bit application development, and it's necessary to change ones development environment from "any processor" to the alternative (whatever that's called) or you'll never connect to the database. The worrying aspect for me is that if TC2011 continues on the way it is going, developers won't be able to access TC2011 data, and - so far - the limitations of PCF creation in TC2007 are inherent also in TC2011, meaning no looping, variables, conditionals, etc. This will hopefully change as the product continues in development, that is we'll either get more functionality in TC2001 PCFs - or we won't, and in-depth exploitation of the available data will have to be done in SF.

That would be fine, as far as it goes, but i've found through my own experiences that it's generally necessary for me to develop indicators as "classes" within SF, and this has been frustratingly problematic either because my code sucks, my understanding of the interface is faulty, or there are underlying bugs at the conceptual level within SF (it's that whole "Blocks" underpinning that i've never been comfortable with and I suspect that adherence to Blocks has percolated up into the rest of the program in a tail-wagging-the-dog way that causes many of the problems) - take your pick, but know that I'm a reasonably accomplished C# programmer and first started programming BASIC in 1979, had a nice career as a digital-design engineer, and have been around the block a few times. Not to say that I'm infallible by any means, only that some things were less than intuitive and I found myself creating work-arounds, like zeroing out my array elements to prevent errors, something that the compiler should do automatically. Add to that the problem with having to plot only one line at a time on a chart (how does one do dual-plots like MACD when the code won't accomodate more than one plot statement per?), and basically I stopped writing for SF and am developing my own complete program so that I can create indicators, scanning algorithms and eventually a robust back-tester. Currently, tedious as it may be, that seems to be my only option within the confines of the Worden products. Unfortunately "customer service" in the way of responses to questions about SF programming seem to have dried up entirely, so no help from that department.

I really like TC2007. It's extremely fast at processing EasyScans simply because all the PCF values are calculated in advance and stored in the Worden "database." Not so, apparently with SF where they appear to be being calculated on the fly - and much slower. Hopefully TC2011 will return to the TC2007 model before development is finished. TC2007 of course, has severe shortcomings in PCFs but that's probably precisely because the PCFs are pre-calculated and values stored, and opening them up to too much programability also means expanding the TC "database" to accomodate them; tough choice for Worden, and at some point, something's gottag give. I'm experimenting with moving everything to a SQL 2008 - lite DB to see how that works out; 2008 now accomodates 10 GB databases whereas 2005 only managed 2 GB, so the added overhead may resolve space problems, though I still haven't resolved the question of "store PCF results or calculate scans (and hence backtesting calculations) on the fly." More than any other single factor, programming (and design work in general) is all about what to compromise on, where, and why. In the meantime, if anyone would like some source code for accessing TC data in C#, drop me an email and I'll forward it to you; perhaps from there some collaboration might be possible. (paul@fostersutton.com)

Paul1307

Users browsing this topic

Guest-1

Forum Jump

You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.