| Welcome Guest, please sign in to participate in a discussion. | Search | Active Topics | |
Registered User
Joined: 12/28/2004
Posts: 15
|
If I am trying to compare a 200 day moving average from one day to the next, how many decimal places are used in the calculation?
ie. AVGC200.10 > AVGC200.11
if AVGC200.10 = 24.2511 and AVGC200.11 = 24.2512 would the above formula return a value of True or is it comparing 24.25 and 24.25?
|
|
Worden Trainer
Joined: 10/1/2004
Posts: 18,819
|
AVGC10<(AVGC10+.00000000000001)
This returns a true
AVGC10<(AVGC10+.000000000000001)
This returns a false
It appears it calculates to 14 decimal places.
- Craig
Here to Help!
|
|
Registered User
Joined: 12/28/2004
Posts: 15
|
just to verify, the formula AVGC200.10 > AVGC200.11 will calculate to 14 decimals, so where AVGC200.10 = 24.2512 and AVGC200.11 = 24.2511, the above formula would return a value of true?
|
|
Worden Trainer
Joined: 10/1/2004
Posts: 18,819
|
Correct, based on my testing. It shows only 2 decimal places in the test screen but my test shows that it calculates out 14 places.
- Craig
Here to Help!
|
|
|
Guest-1 |