Hi experts! We have a calculation and we must use all digits (i.e. 0.066666663) to get the right value for process control, but if we use a CALC block or standard math blocks the result doesn't give us all digits. Is tehre any way to do it? Does anybody have an idea?
Today the CALC block has the following script:
OUT1 := ((0.066666663 * (IN1 * IN1 * IN1)) - (19.799999 * (IN1 * IN1)) + (1960.6827 * IN1) - 64635.16
And if we compare with a standard calculator, assuming IN1 = 97.75:
CALC = 98.2227
Calculator = 98.49984704
Thanks for your help.
In reply to Maarten van der Waal:
I know this is old, but I think your formula has an extra closing parenthesis. The formula that worked for me is:
OUT1 := IN1 * (IN1 * (0.066666663 * IN1 - 19.799999) + 1960.6827) - 64635.16
In reply to jgrunschel:
This is a crude solution but I have had to do it occasionally to transfer serial data on occasion. Multiply the decimals to make them whole numbers, do the calculation, then divide it back down to return to floating decimal. Not ideal so don't flame me, but it will work in a pinch.
In reply to Jason.Brumfield:
Andre Dicaire