Welcome Guest, please sign in to participate in a discussion. Search | Active Topics |

When is the alert system going to be fixed ? Rate this Topic:
Previous Topic · Next Topic Watch this topic · Print this topic ·
jason.rennie
Posted : Thursday, January 10, 2013 12:34:58 PM
Registered User
Joined: 12/28/2011
Posts: 120

Hi,

On the subject of alerts, I've been raising the same bugs for many months now. The alerts system is not functioning correctly.

Alerts go missing, roughly one per day.. Possibly/probably more.. Individual alerts also randomly stop working and the alert needs stopped and restarted.. The worst thing is that you have no idea when this has happened.

As you probably know I've got a 'heartbeat' alert. As an experiment I decide to try changing the heartbeat frequency to 5 minutes.. 

http://www.dowce.com/~EbC

It lost 4 emails in 2 hours. 

4:20

4:30

4:35

4:40

Its happening so often that I've stopped reporting it to your support department. There's no point in raising it again and again if the reports are just filed and ignored.. Every time I check my heartbeat emails I find at least one missing every single day.. And that was when I had the frequency set to 15 minutes.. Now that its set to 5 minutes I will see just what the loss rate changes to.

Could somebody please get these issues investigated ? I dont understand why its not been given a high priority. Its very easy to replicate the issues, its definitely broken and the alerts have got to be one TC2000's most important features.

-Jason

 

Bruce_L
Posted : Thursday, January 10, 2013 1:57:01 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I have both tested how the alerts work (albeit an incredibly short term test) and checked to determine how the alerts are intended to work.

Alerts only trigger when the alert changes from Not Passing to Passing (or is Passing when initially created). The do not continue trigger each time the period specified under notify a maximum of once per has transpired.

This is by design. The notify a maximum of once per setting is designed to prevent you from being notified more than once during the specified time if the alert changes from Not Passing to Passing more than once during the specified time. This setting was not designed as a method of creating "heartbeat" type alerts which continue to trigger at the specified interval as long as the alert continues to be Passing.

There is currently no way to set up "heartbeat" type alerts in TC2000 version 12.3. Thank you for the suggestion. It has been assigned case number 9804.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Thursday, January 10, 2013 3:14:33 PM
Registered User
Joined: 12/28/2011
Posts: 120

Hi Bruce, thanks for clarifying the behaviour of the "Passing" alerts. this is what I suspected was the intended behaviour but I wanted to verify it wilth you. 

I think you misunderstood my intention when I mentioned the heartbeat alert. Really, the issue I am raising here is that the alerts system is malfunctioning..

My heartbeat alert is purely for my own purposes, to verify that the worden alerts system is operating correctly. If I stop receiving heartbeat alert emails then I know there is a problem. The heartbeat is configured like so.

http://www.dowce.com/~Eb1

I request crossing alerts on the yellow horizontal line at 0 on the MACD 1 2 pane. The time interval of the chart is set as 1 minute which more or less guarantees a crossing every minute or so. Remember that chart here is just showing the value of the MACD at the close of each interval. During each interval (1 min) the MACD normally crossed the zero line. I originally had the chart interval set at tick - but for some reason this stopped working a few months ago. The problem was also present when I had the interval set as tick. (months ago)

Here is a closeup of the timespan in question from today

http://www.dowce.com/~Ebg

As you can see from the chart - even just looking at close values - there have been repeated crossings of the MACD signal line with the yellow horizontal zero line. There has not been a long interval without the MACD 1 2 crossing 0 this afternoon.. certainly nothing to explain 4 missing alerts.. And especially when you consider that the movements of the MACD line during each minute are more than are recorded on the closing values visible on the chart. In reality there are many many more crossings on the tick chart.

the other thing that makes me believe the alerts are missing is because I expect to see a Cross up, cross down, cross up, cross down pattern.. And this pattern is broken when an alert is missed.. either a "cross up" or a "cross down" is missing.. I have therefore concluded that the alerts have simply gone AWOL... An this is the result of a bug in the alerts system..

If you have a better idea for how I could create an alert which guarantees it is raised I would be thankful. I'm pretty confident that the existing heartbeat alert is good enough for its purpose. I run it on a currency pair to get test coverage outside of normal hours (in case I need to phone worden to tell somebody that the alert system has gone offline before the market has opened.)

I'm surprised that you are not aware of the issues with the alerts system - I've been reporting bugs about these issues for many months. Jeremy has stacks of them with my name on them..

I have also recorded several cases of alerts failing in practival use.. When set on stocks.. Where a chart has crossed a trigger and yet no email was received and no alert was recroded in the alert history.. Its not just the heartbeat that exhibits the bugs.

This is not the only problem - alerts also randomly go silent and stop triggering alerts at all... and the only way to 'fix' them is to cancel them and restart them. But how can one normally know if an alert has 'gone silent' - so there is no workaround.

I hope I've explained myself clearly enough to be understood.

Kind Regards,

-Jason

 

Bruce_L
Posted : Thursday, January 10, 2013 6:00:40 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

Since I am in the customer training department and my primary responsibilities are related to the forums, it is not surprising that I would not have heard of your reports. I do not deal with bugs and suggestions other than to add bugs (after minimal testing from me) and suggestions to the database which have been submitted on the forums.

While I used to be the senior callback technical support agent, I have not done any technical support since before StockFinder was released and am not in quality control or development (the two departments which are tasked with identifying and correcting bugs).

The following is not intended to address the alerts system going entirely down or a particular alert going silent to the point of needing to be cancelled and restarted. You should probably continue to deal with technical support on the issue. It only attempts to cover the tests I'm running based on your original post in this topic and proposes a possible cause that is not actual a bug (although I am proceeding under the assumption that it actually is a bug and am attempting to verify it).

I'm running tests overnight as without exact time stamps for all of your alerts (the time stamps on my alerts certainly do not fall at even multiples of 5-minutes), I cannot say for sure if the alerts were actually missed or if the alerts just shouldn't have triggered at those points.

Note that it is quite possible for there to be spans when the alerts wouldn't trigger and that some of the gaps in these triggers look like that could fall at the times you mention (or close to them).

Plot the following formula as a Custom PCF % True Indicator to point to bars where if the alert system runs absolutely constantly without any possible missed ticks your MACD crossup 0 alert would change from Not Passing to Passing during that minute on a 1-Minute Chart:

H > XAVGC2.1 AND C1 <= XAVGC2.1

And the following Custom PCF % True to point to the same bars with the same caveats for your MACD crossing down 0 alert:

L < XAVGC2.1 AND C1 >= XAVGC2.1

While it seems strange that both the current bar and the previous bar are being compared to the previous value of the 2-Period Exponential Moving Average of the Close, note that this is a result of the way exponential moving averages are calculated and the difference equation which results:

XAVGC2 = (2 * C + XAVGC2.1) / 3

So at the high of price for the bar, XAVGC2 actually equals (2 * H + XAVGC2.1) / 3, so:

H > XAVGC2

At the time of the high would actually be:

H > (2 * H + XAVGC2.1) / 3

If we multiply both sides by 3 we get:

3 * H > 2 * H + XAVGC2.1

And if we then subtract 2 * H from both sides we get the formula for the Exponential MACD Histogram 1,2 being above zero at the time of the high of the bar:

H > XAVGC2.1

The Condition Formula given above actually represent the most optimistic possible case for the minutes where your alerts would change from Not Passing to Passing. Even though the Alerts are constantly cycling, there most likely going to be individual ticks where a calculation is not done. If this is the case, a better crossup formula would be:

C > XAVGC2 AND C1 < XAVGC2.1

And a more realistic crossdown formula would be:

C < XAVGC2 AND C1 > XAVGC2.1

The actual bars where the Alerts could change from Not Passing to Passing probably fall someplace in between these the more optimistic formulas given earlier and the more pessimistic formulas given later.

If alerts cannot trigger if the previous bar was Passing as opposed to an intrabar Not Passing result allowing for a trigger, then only the first and/or last bars in a row of possible trigger bars being represented in the Custom PCF % True Indicator might represent an actual trigger. I am hoping this is not the case, but will require more data to know the actual behavior.

It should probably be noted that my results will probably not match your results exactly for a variety of reasons. For one, it would seem that the exact start time of the alert makes a difference as to when alerts can trigger. So until we both have a trigger at exactly the same time, our triggers would not be synchronized. In addition, my copy of TC2000 is Gold and not Platinum. While I can get short term access to Platinum data, I really can't run the sorts of long term tests required to test this particular aspect of alerts using Platinum data.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Thursday, January 10, 2013 8:42:24 PM
Registered User
Joined: 12/28/2011
Posts: 120

for your 2nd formula I think you meant 

L < XAVGC2.1 AND C1 >= XAVGC2.1

I think I follow what you're trying to do.. and the first 2 formulas seem reasonable. But what you do after that goes a little over my head.. Or rather I dont understand why it is necessary. I often find it hard to keep up with you so I might be wrong here.. I get the jist of what you're saying. There is an ideal and a realistic model in terms of when the calculations are actually made.. So the server may have busy times when calculations on individual ticks - or periods of ticks are not performed.. there could be some truth to this as I've noticed that most of the dropped emails are occurring at roughly the same times of day. (normaly when I 'm makign dinner !) But I dont know how the server processing is scheduled so I've no way of knowing if this is in fact possible..

My understanding of the alerts system is that if the last event was a cross up. Then, once the timeout has expired, (5 minutes in this case) If we havent already had a cross down then we would still be waiting for one.. (of course during the 5 minute timeout we could have a crossdown and a crossup and then exit the timeout period without emitting an alert)

So to get no events for 15 minutes we would need the 5 minute timeout and then a complete 10 minute period without a single cross down event.. when I look at the chart for the time period I dont see such a gap. The system should be waiting for a cross down..  At some point I should get another cross down  - but I often dont - I often get further gaps of 5/10/15 minutes and then another cross up (or vice versa) Which looks the same as if an alert (or alerts) had been dropped. Bear in mind that this is with the timeout set at 5 mins - Until today I ran this with the timeout set at 15 minutes and I was still 'losing' around 1 alert every day..

But there's more - and I should have checked through this earlier before writing to you.. This is important (my apologies for not checking this earlier)

Here are my emails 

http://www.dowce.com/~EL4           (in GMT  -which is EST + 5)

And here is my alert log                    (in EST)

http://www.dowce.com/~ELE

And here is the Alert log for this specific alert

http://www.dowce.com/~ELJ

They dont match up..It looks like I'm missing *some* emails and also some of the alerts..  I'm actually missing the first email - from 9:49:22 EST ~= 14:49 GMT

the first email I got was at 14:54pm (9:54 EST) and it was a cross down event. 

The alert at 9:49 EST was a cross up.. but the email never arrived..

There is a gap between 11:25 and 11:46 EST - no alerts and no emails (16:25 and 16:46 GMT)

The alert at 11:25 was a cross up.. so looking at the charts with your first 2 formulas added

http://www.dowce.com/~ELZ

it looks like there were cross up and cross down events in that time window..

There is an alert at 11:20 EST (4:20 GMT) but no corresponding alert email.

I think there are a combination of factors here - emails are not arriving (or sending) - and also alerts are going silent for prolonged periods of time (as if they are being discarded) when it apears from the charts as if they should be producing alerts.

I cant properly follow the maths of what you were saying but using your first 2 formulas over the appropriate period it looks like there were alerts.. And looking at the MACD 1 2 chart it looks like there were both cross up and cross down events..

Also, I feel I may owe you an apology - I didn't mean to speak out of turn with you. I have no understanding of the internal structure of worden - I expected that the near daily bug reports I was sending to support (at one time) would have been mentioned around the coffee pot at some point. Or in a planning meeting or some such..  and I wasnt sure if you all shared a common open plan office area. so it would have come up in conversation.. In fact at one point I wasnt sure if worden actually had a permanent development department at all - or if the development work was all performed by short term contract or an extrernal development team.. I have no idea if worden has 10 or 100 staff !

It been very frustrating to repeatedly log these issues and yet never see them addressed.. And I hoped that by raising the issues here that another opportunity to have them fixed might open up.

It has occurred to me that maybe the missing alert problem is with Google.. since it is a gmail account - and maybe it is possible that google are losing some emails.. I suppose nothing is out of the realms of possibility.. But it seemed a lot more plausable to me that the emails were never being sent.

Perhaps because a mail server is stacking up emails and then being rebooted.. or something.But the fact that the alert log seems to be missing items implies that its more than that..

OK its been another very long day for me - so I need to get some sleep

Kind Regards (as always) and humble apologies if I over stepped the mark.

-Jason

 

jason.rennie
Posted : Friday, January 11, 2013 4:30:32 AM
Registered User
Joined: 12/28/2011
Posts: 120

reading this again (after some sleep) I now follow what you were meaning in your last 2 formulas. whilst I dont understand exactly how they were derived, I do understand what you are saying in terms of the optimisitc and pessimistic evaluation of all ticks - or just close prices. And also what you are saying here 

"If alerts cannot trigger if the previous bar was Passing as opposed to an intrabar Not Passing result allowing for a trigger, then only the first and/or last bars in a row of possible trigger bars being represented in the Custom PCF % True Indicator might represent an actual trigger. I am hoping this is not the case, but will require more data to know the actual behavior."

I dont think that is the case - I have seen plenty examples of times when intrabar cross up and down events are received. At least at the 15 minute and 1 hour timframe.. Whether this is possible at shorter time frames like 1 minute - I simply dont know. Especially if the processing is performed in an environment where not all ticks are processed.. To be honest I never imagined that some ticks would be discarded.

Here is the chart again with the additional formulas

http://www.dowce.com/~ELe

Again, it seems as though we should have crossing events. Then only question is if the processing of entire bars is being discarded - perhaps because the server is busy..

Of course none of this covers the unexplained absense of some emails corresponding to some alerts.. That seems like a computer systems fault (if not down to a bug in the programming)

-Jason

Bruce_L
Posted : Friday, January 11, 2013 11:43:12 AM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I've just had a chance to start looking at this.

After little more than a quick look through the data I can say for sure that there many alerts in my alert log for which I did not receive emails. That said, the reason for this seems quite obvious in my case. My alert log is not respecting the notify a maximum of once per setting while the emails are. I may have three alerts over the course of a minute in the alert log for the same Alert, but I'm only getting emails for that Alert a maximum of once every 5-Minutes.

There are no long gaps in my emails (which have both the crossing up and crossing down emails in the same folder). There is one gap of 10-minutes, 1 gap of 9-minutes and several gaps of 8-minutes, but most of the gaps are much shorter.

I have not done any sort of in depth analysis yet. The more I think about it, the more I'm not sure I will be able to using just Gold data because any charts that are more than 15-minutes old may not look like what they did at the time of the alert. The most recent 15-minutes of my data is based entirely on the BATS exchange, but any bars that are more than 15-minutes old are filled in with all of the available data from the consolidated tape.

Even so, I'll try to look at more detailed comparisons between the chart during the rest of the day. As I received several hundred alerts overnight, this may take some time before I can come to any conclusions (assuming I can).



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
Bruce_L
Posted : Friday, January 11, 2013 12:14:57 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

My original post this morning was a bit pessimistic. The Forex data does not appear to be delayed for Gold subscribers. My testing was quite consistent in that my Alert log exactly matched the results of the:

H > XAVGC2.1 AND C1 <= XAVGC2.1

And:

L < XAVGC2.1 AND C1 >= XAVGC2.1

Custom PCF % True Indicators on a 1-Minute Chart overnight. Since this seemed unlikely if the Alerts were based on BATS data and my charts older than 15-Minutes were based on consolidated data, I asked management and found out that we don't have to delay the Forex data (so we don't). In any case, this means I should be able to come up with fairly definitive results when looking at the data from overnight.

Just a note, that I certainly haven't looked at every minute, but in my preliminary results so far, the Custom PCF % True Indicators and my Alert Log do in fact match exactly.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 11, 2013 12:24:13 PM
Registered User
Joined: 12/28/2011
Posts: 120

If your alert log has 3 alerts in the course of a minute for the same alert then that is very very different from what I am seeing.. No wonder you have hundreds of alerts... I get alert log entries at 5 minute (or longer) periods - never more frequently.. normally synchronised with the emails. the emails arrive within a few seconds. normally. I cannot understand why you would be seeing different behaviour. Unless this is caused by your gold subscription

To take a specific example from yesterday, the alert at 11:20 EST (16:20 GMT) went awol.. It is present in the alert log but it is not present in my emails. The alerts on either side of it were delivered. but not that one. 

So in the alert log I have cross up 11:14, cross down 11:20, cross up 11:25

http://www.dowce.com/~ELN

But in my emails I have cross up 16:14 (11:14 EST), cross up 16:25 (11:25 EST)

http://www.dowce.com/~ELv

How can this be ?

Jason

jason.rennie
Posted : Friday, January 11, 2013 12:40:17 PM
Registered User
Joined: 12/28/2011
Posts: 120

 

sorry, just read your 2nd post from this morning.. are you sure that your alert log is filling with entries at faster than 5 min period ? I've just checked and mine are separated by at least 5 minutes.

Bruce_L
Posted : Friday, January 11, 2013 12:50:36 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

Yes, I am quite sure. I would not be able to compare all of my individual Alerts to the Custom PCF % Indicators on the chart otherwise.

The Alerts in my Alert Log are way more frequent than once every five minutes even though I set the notify a maximum of once per setting to 5-Minutes (as evidenced by the fact that I don't get emails from the same Alert more frequently than once every 5-Minutes). I don't think I've seen more than 4 of the same Alert in more than a single minute, but it is not something for which I have been specifically checking.

Maybe we are referring to something different. I get to the Alert Log by selecting More | Alerts and then choosing Alert Log in the upper right hand corner, but when I select More | Alerts and then click on the Alert Log icon for an individual Alert, I also get more than one Alert listed per minute. Do you access the Alert Log in some other fashion?



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 11, 2013 1:36:04 PM
Registered User
Joined: 12/28/2011
Posts: 120

I followed the steps you gave and got these windows..

http://www.dowce.com/~ELw

5 minute separations.

Quite puzzled by this..

Bruce_L
Posted : Friday, January 11, 2013 1:37:14 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I've been going through my emails and haven't found any that have been missed yet, but it is a fairly slow and tedious process to check each email, look it the chart, skip five 1-minute bars and then check for the next bar which returns true.

I think I am just going to delete all of the emails and start over with a notify a maximum of once per setting of Minute. I will get a very large sample in a very short period of time and the process of verifying each email which be much quicker.

I'll send the alerts to both my work and home emails to see if there is a difference between what I get at work and what I get using Google's email servers (I don't have a GMail account, but my personal email is handled by Google Apps for Business).



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
Bruce_L
Posted : Friday, January 11, 2013 1:48:51 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I do not know why you have the 5-minute separations and I do not. That fact that I do not have them would appear to be a bug, but at the moment, it is quite a useful bug.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 11, 2013 1:50:11 PM
Registered User
Joined: 12/28/2011
Posts: 120

the difference could be coming from how you created your alert on the PCF ?

My alert was created by right clicking the horizontal Zero line on the MACD pane.

I'll try doing what I presume you did for your PCF, right clicking the formula name in the top left of the pane and choosing "Set alert"

and setting the alert as follows

http://www.dowce.com/~EL3

Is this similar to what you did ?

Bruce_L
Posted : Friday, January 11, 2013 1:52:52 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

Actually my Alerts were created by clicking on the name of an Exponential MACD 1,2 plotted on a 1-Minute chart and selecting Set Alert. I have a crossing up value 0 alert and a crossing down value 0 alert.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 11, 2013 1:59:47 PM
Registered User
Joined: 12/28/2011
Posts: 120

ah well thats different - I'll try that

Thats 2 alerts so both will have independent timers (but I guess you realise that !)

 

 

jason.rennie
Posted : Friday, January 11, 2013 2:03:03 PM
Registered User
Joined: 12/28/2011
Posts: 120

http://www.dowce.com/~ELf

http://www.dowce.com/~ELM

 

jason.rennie
Posted : Friday, January 11, 2013 2:05:34 PM
Registered User
Joined: 12/28/2011
Posts: 120

I get the same behaviour as you

http://www.dowce.com/~ELQ

 

jason.rennie
Posted : Friday, January 11, 2013 2:22:34 PM
Registered User
Joined: 12/28/2011
Posts: 120

 

ok - dinner time  :) back in an hour or so

 

jason.rennie
Posted : Friday, January 11, 2013 3:31:21 PM
Registered User
Joined: 12/28/2011
Posts: 120

I think whats unique here is that these are the rather special "Passing", "Not passing" alerts.

I think their behaviour changed at some point in the last few months.. I think they used to send alerts every 5 minutes (or whatever the "Notify a maximum of once per" setting is set to) This is actually why I asked about how they were supposed to behave. In fact I thought I had asked you about them before.. a few months ago.. Perhaps not.

Anyway, I think this is a side issue.. I think the behaviour is not correct.. But its not the biggest fish we have to fry.. All the alerts should follow the same rules. and this one seems to be 'unique'

I also remember seeing other special cases states appear at one point. "Equal" became an alert condition at one point. I'm not sure if its still possible. So we had cross up, cross down and 'equal' (or some word along those lines) - and it was only possible in the alert log but wouldnt cause emails to be sent. (or some strangeness like that)

Anyway - as I said I think this is all just a side issue. 

 

Bruce_L
Posted : Friday, January 11, 2013 3:56:07 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

It is not a sideshow in that it provides a direct window into the behavior of the Alert system.

It is a bug, but it is, as I indicated before, a useful bug. It shows exactly when the Alert system thinks the Alerts are true without any consideration as to the notify a maximum of once per setting.

This allows you to correlate the emails being sent with all of the Alerts which are actually being generated internally. You can see exactly why the gaps in the Alert emails occur. If you look at the Alerts which don't adhere to the notify a maximum of once per setting, you can check when there is an email which corresponds to an Alert and then forward ahead by the time specified and then identify the next Alert on the list. At that point, another email gets sent and the process starts all over again.

As to the issue where you are missing emails while having an Alert show up in the Alert system when setup in a way that it does follow the notify a maximum of once per setting restrictions, I could not yet say. Up until this point I am not missing any emails in my personal account using Google's servers as compared to my work account. Both currently have 75 emails, all of which seem to match up with correctly with the Alerts in my Alert Log. And the Alerts in my Alert Log seem to match up with the Custom PCF % True Indicators on the chart.

It is a much shorter test so far than the overnight test I had done, but I had checked through about 50 of those (with the 5-Minute setting instead of the 1-Minute setting) before deleting the emails without identifying any discrepencies.

Additional testing from me is probably going to have to wait until Tuesday. Mondays are usually too busy for me to devote the type of attention required to do any testing then.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 11, 2013 4:14:05 PM
Registered User
Joined: 12/28/2011
Posts: 120

Good point, Understood.  it could be that the mechanism (method, function, whatever programming term you prefer) that decides if an email should be sent has a bug in it which erroneously stops some emails from being dispatched.. - And therefore these "passing", "not passing" alerts will not exhibit that bug.. since the method is not being called..

They maybe inherit from a different base class..

So its a bug that the method is not called - but It is a bug in that method which is causing the bug I was originally raising..

Which might explain why you cannot replicate the issue with the Passing, Not passing alerts...

OK, until Tuesday !

-Jason

Donk
Posted : Friday, January 11, 2013 8:51:19 PM
Gold Customer Gold Customer

Joined: 11/17/2007
Posts: 28

Dito. I have problem getting alerts to stop after I delete them.  Alerts are great IF & WHEN they work right.. Sure hope someone at Worden gets the bugs out for all of us.

Bruce_L
Posted : Tuesday, January 15, 2013 10:17:31 AM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I was getting ready to try and duplicate your Alert by creating it in exactly the same way you did, but when I click on the Horizontal Line plotted at 0 in pane of my Exponential MACD 1,2 indicator, I do not get an option to create an Alert for the MACD crossing up or down through zero. I do get an option to create a Reminder Alert or Price Alert, but that will not allow me to create the desired Alert.

I obviously can create the Alert using other methods, but I was attempting to duplicate the exact circumstances of your Alert.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Tuesday, January 15, 2013 10:39:37 AM
Registered User
Joined: 12/28/2011
Posts: 120

Thats odd ! If I understand you correctly then this is not possible for you

http://www.screencast.com/t/ngwnVKLzHz

 

jason.rennie
Posted : Tuesday, January 15, 2013 10:42:42 AM
Registered User
Joined: 12/28/2011
Posts: 120

My about box for TC2000

http://www.dowce.com/~EcN

jason.rennie
Posted : Tuesday, January 15, 2013 10:46:18 AM
Registered User
Joined: 12/28/2011
Posts: 120

there is no beta

http://www.dowce.com/~Ecv

 

Bruce_L
Posted : Tuesday, January 15, 2013 10:50:03 AM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I had not added a Horizontal Line Drawing Tool to the chart. I was trying to use the Horizontal Line that is built into the MACD indicator. Thank you for the clarification.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Tuesday, January 15, 2013 10:59:46 AM
Registered User
Joined: 12/28/2011
Posts: 120

ah ok - no problem

jason.rennie
Posted : Tuesday, January 15, 2013 4:19:27 PM
Registered User
Joined: 12/28/2011
Posts: 120

I've just had a quick look and I didnt notice any missing alerts today.

Bruce_L
Posted : Tuesday, January 15, 2013 4:21:41 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I haven't noticed any missing alerts either. That said, I think I may have had an alert or two when I did not expect to get an alert. I need figure out why. My Custom PCF % True Indicators have to be wrong, I'm misunderstanding something about how the Alerts are getting triggered or possibly both.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Thursday, January 17, 2013 9:44:28 AM
Registered User
Joined: 12/28/2011
Posts: 120

no missing alerts since I last posted

 

 

I've found that by flagging all messages containing  "crossed up" I do a quick visual scan - (for one of the kinds of missed alerts at least.)

And on my gmail window there's an even number of emails per page.. so that helps as well.. CTRL with the mouse wheel to reduce the font size so they fit on a page and then page forward and back to look for missing stars..

In addition I also manually checked every alert time stamp to check that a pair of alerts wasnt missing (or very late)

I'll continue checking

 

Bruce_L
Posted : Thursday, January 17, 2013 9:55:34 AM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

I have had ten minute gaps, but it was always fairly obvious as to why. The reason the old formulas worked for my old tests but not necessarily for your Alert is that the old formulas were specifically for crossing up through zero or crossing down through zero.

Your Alert is really two Alerts in one with a shared cooldown instead of being a crossover Alert. One of the Alerts is for just being above and the other Alert is just for being below. They act like a crossover Alert only in that they do not trigger if the most recent status is the same as the current check.

I am currently using the following two Custom PCF % True Indicators:

H > XAVGC2.1

And:

L < XAVGC2.1

They just check for the MACD being above (the first formula) and below (the second formula) zero. Anyway, the formulas do seem to correlate correctly as long it is taken into account how the opposite formula needs to return true outside of the cooldown before the next Alert can happen.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Thursday, January 17, 2013 10:49:18 AM
Registered User
Joined: 12/28/2011
Posts: 120

I really apperceiate that you're taking the time to investgate.

the problem I see is that its not the same test as I've been running.. There could be different code behind the "Custom PCF % True" alerts versus Trend Line "Crossing" alerts. We've already found that some alerts seem to use different code paths to others.. I dont know the architecture of the code so I dont know which alerts are related to each other.. and how they are related.

I've never run a long term test of the nature that you are running.. so when I fiind more missing alerts, you still might not believe me.. Or think that somehow I've made a mistake.. and your test might never show anything up. 

I need someone at worden to recreate the heartbeat alert so that the next time it fails then it cant be swept under the carpet.. 

In fact, even if you set up your test to be identical to mine, if I find more missing alerts, it is quite possible that you won't have any on your side. I dont know whats causing the alerts to go missing but theres no reason to believe that all alerts simultaneously go missing to all users.. as I'm sure you understand.

The MACD 1 2 crossing zero alert was the test that I thought would produce the most consistent stream of alerts that I could conceive. Its purpose was just to check that things were working.. But I found that it showed up many unexplained missing alerts. 

I'm doing my best to document in as clear and definite terms as I can the issues that I have witnessed. I've sent screenshots showing my email lists, and the alert lists. I dont know what more I can do to prove that there is a problem.

Really the solution is for a set of tests to be constructied to monitor the system and alert worden when any of the failures I've discovered are encountered.. It shouldnt be left to me to phone worden and tell them that the alert system has gone offline.. as I have done several times in the past.. But such a system takes time to devleop. and so far I've not been able to convince worden that the problems even exist.. Or perhaps I've just failed to convince worden that its worth spending developer time to fix the issues..

It could be that the email server is randomly dropping emails.. It might not even be an issue with the code.. 

<sigh>

How about if I create a small application that reads from a gmail account - and scans for missing emails.. or heavily delayed emails - and sends an email to bring it to your attention.. That will save you goign through all the emails/.. 

I'll see what I can come up with...

 

Bruce_L
Posted : Thursday, January 17, 2013 12:19:07 PM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

As far as I know, my current test is identical to your test, but I could be wrong.

I added a Horizontal Line Drawing Tool at 0 to an Exponential MACD 1,2 on a 1-Minute chart of GBP/USD. I then clicked on the Horizontal Line Drawing Tool and selecting Set Alert for when the MACD crosses this line. Notify a maximum of once per is set to 5-Minutes. Emails are being sent to both my work account and my personal account (which is Google Apps for My Domain based).

I fully understand if you do not wish to continue doing any tests yourself. Your observations as to my ability to reproduce the issue seem valid and you have already submitted your reports and documentation through multiple channels and our quality control and development departments should have what they need to attempt to duplicate and resolve the issue. It may be that my additional testing is not adding anything useful beyond what you have already provided.

I have submitted all of the information and screen shots from this topic to development so they can evaluate it in conjunction with the other data you have submitted previously.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Thursday, January 17, 2013 1:54:16 PM
Registered User
Joined: 12/28/2011
Posts: 120

I thought that your test was based on Custom PCF % True indicators. I think I have misunderstood. If you are running an identical test to mine then I'm pretty certain that you will find the same results as I have.. Given time

I've started putting together a little app to scan emails. I'll see what I can get completed and you can use it to check the emails - I'll be able to use it as well. In fact I've been meaning to add most of its functionality to my trade manager..

I'll supply it in the form of a Visual Studio 2010 solution. You dont need to run it on an office machine in order to get it to do the work - all it needs access to is the internet in order to read the emails from gmail..

Making good progress with it. so hopefully I'll have something in a few hours.

 

jason.rennie
Posted : Friday, January 18, 2013 8:29:03 AM
Registered User
Joined: 12/28/2011
Posts: 120

 

Hi Bruce, 
 
I've uploaded a zip file to dropbox 
 
https://www.dropbox.com/s/4aas0bkfkb5plxj/GmailAlertScanner.zip
 
It contains a Visual Studio 2010 solution entirely written in C#. Its a console application and I've included the prebuilt executable in the bin/debug folder. The source files are there in case you want to see how its written or make any changes.
 
Its purpose is to save you having to manually scan through hundreds of alert emails.
 
I couldnt send the zip file with gmail because it includes an exe.. And gmail wont allow it. (I originally planned to send it to your support department for them to pass it to you)
 
When you run the program it asks for the username of the gmail account that you want to use. It then asks for the password for the gmail account.
It then asks you for a number which is the number of minutes to consider "late". I suggest you enter "10" here.. meaning 10 minutes. 
 
It then reads all the unread messages using POP3 and starts scanning them for missing alerts. 
 
If you use the gmail.com web interface to read emails then this will not mark them as read in POP3. Its seems POP3 keeps a separate record of what's been read... Basically when you run this program it will read all the emails since it was last run. It compares sequential pairs of emails. It checks if more than 10 minutes elapsed between emails. It also checks if emails break the "crossed up, crossed down, crossed up, crossed down" sequence that we expect. 
 
I tested it out with a brand new gmail address and it worked fine
 
If your IT security department are nervous about receiving programs, solutions executables etc from strangers then maybe run this in a sandboxed environment. You dont need to do this - (Obviously I dont do this when I run it !)
But if it keeps them happy then why not.. I've tested it with sandboxie and it worked fine.
 
 
It doesnt need to be on your company network. It just needs access to the internet. All it does is open an SSL connection to pop.gmail.com on port 995 (secure SSL POP3 access)
 
Its written in C# using Visual Studio 2010. I used a library called OpenPOP to read the emails
 
 
I've been using OpenPOP for a few months and it seems robust enough.
The interesting part of the program is contained in GmailAlertScanner/GmailAlertScanner/Program.cs
 
Also, It talks !
When it finds something interesting it uses the microsoft .net speech synthesis module to say what it found. 
 
It can say the following:
"Email without Cross up or Cross down".. This is if an email arrives that doesnt contain either "crossed up" or "crossed down".. Maybe spam !
"Late alert. Time since last alert was XXX minutes and YYY seconds." 
"Repeat crossed up emails. The last 2 alerts received contained the text crossed up"
"Repeat crossed down emails. The last 2 alerts received contained the text crossed down"
 
Whenever it speaks it also writes a text file to the same folder as the exe. The filename has the date and time that the alert was sent and the body of the file is the body of the email. Something like "Late alert_18_1_2013_9_15_23.txt".. That equates to 18th Jnuary 2013 at 09:15:23 am.. This means that a permanent record of the warning is stored so you can check it in the morning. Note that it is the "sent time" - not the arrival time. I think this is preferable.
 
I recommend creating a completely fresh gmail account, deleting all the welcome emails. Emptying the trash folder to permanently delete them.. Then in TC2000, creating a new "heartbeat" alert and sending it to the new email address. (You may have to enable POP3 in the gmail account settings.)
then run GmailAlertScanner.exe and leave it running
 
Because the program checks PAIRS of emails it doesnt properly check the first email, since it doesnt know when the previous one arrived and it doesnt know if it was a "crossed up" or "crossed down" alert.. Its just a simple program.. So just leave it running and it will check the emails as they arrive. If you do kill it and restart it then be aware that it wont properly check the first new email. you can do that manually using the normal gmail.com web interface. Reading emails in gmail wont affect the POP3 read status. This program will still see them. Deleting emails using the web interface will stop it seeing them. I always empty the trash folder as well.
 
I ran it overnight and it processed the first group of 30+ emails that had built up - and then parsed all the subsequent emails as they arrived.. It checks for emails every 10 seconds.
 
Its quite simple.
 
If your IT department have a problem with any of this then I will create a server on Rackspace.com and give you the access details. You can then run anything on that since it is a perfect sandbox environment.. and it is a machine completely outside your company network.,
(I use rackspace to run my trade manager program with the Interactive Brokers workstation - guaranteed uptime and guaranteed internet connectivity !)
 
Please let me know if you have any problems. I've tried to think of everything and make it as simple as possible for you but it could be that I've overlooked something completely obvious !!
 
My house phone number is +44 131 225 9399. I'm normally home so feel free to call if you have any questions..
 
Jeremy and Jerry in support know my email address. (I dont want to post it to a public forum)
You can probably guess it !
 
Kind Regards,
 
-Jason
 
Bruce_L
Posted : Friday, January 18, 2013 10:19:15 AM


Worden Trainer

Joined: 10/7/2004
Posts: 65,138

Since it should make things a lot easier, I have restarted my tests using a new Gmail account setup just for this purpose. I have also brought your post and program to the attention of management.



-Bruce
Personal Criteria Formulas
TC2000 Support Articles
jason.rennie
Posted : Friday, January 18, 2013 10:33:34 AM
Registered User
Joined: 12/28/2011
Posts: 120

ok great, thanks Bruce :)

 

 

Users browsing this topic
Guest-1

Forum Jump
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.