Weird Result from sub()

[wstock]

I am working on a flow that is giving me some weird results and I'm not sure why...

I initialize 3 variables, all of type "float" then load into the first two some numbers, and finally use the sub() function to populate the value of the third variable. 

FlightTimeRemaining:

sub(variables('NextMaintDue'),variables('CurrentFltTime'))

This appears to work, but the results are wierd.

NextMaintDue value is: 13621

CurrentFltTime value is: 13620.1

FlightTimeRemaining value is:  0.899999999996362 

How is this not just 0.9?  

The values for NextMainDue and CurrentFltTime both start life as strings, but I convert them to float and store in their respective variables using this:  

float(items('For_each_2')?['TACH1_NEXT_EVENT'])

and

float(sub(variables('NextMaintDue'),variables('CurrentFltTime')))

Finally, this is what the output in FLOW looks like: 

I can't seem to sort this one out...any suggestions? 

[efialttes]

[RobElliott]

@wstock yes that is the normal result. I'm sure I had a solution to this a while back, I'll have a look and post if I find it.

Rob
Los Gallardos

[abm]

Hi @wstock ,

Excellent question.

@efialttes  nice reply LOL

This is what you need to do.

You need to use the below.  Below under the Format click Dynamic and enter 0.00

This will ensure that you need precision of two digits only.  So this will round the value accordingly.

My test result

Maths is fun sometimes 🙂

Thanks

[edgonzales · Accepted solution]

@abm 

When I try your solution, it yells at me because it's counting the output of the Format Number as a string and the Sub() doesn't like that.  Weird, right?

@wstock 

I think what is causing the issue is how Floats actually figure decimals using binary fractions.  I found this article which seems to cover what you're seeing.

https://docs.python.org/3.4/tutorial/floatingpoint.html

Another solution might be to multiply your MaintDue and Flight times both by 10 or 100 (depending on how many decimals you expect), then subtract, and then divide the result by 10 or 100 (depending on which you used to begin with).

Keep us posted.

-Ed

If you liked this reply, please give it a thumbs up! If this reply has answered your question or resolved your challenge, please consider marking it as a Solution. This helps other users find it more easily via search.

[wstock]

@edgonzales They multiply by 10  then divide by 10 worked.  Thanks!

Who knew that 2 - 1 = 0.8999

[efialttes]

@abm @edgonzales @RobElliott  This community is great thanx to guys like you three!

[wstock]

@edgonzales  Thanks for this link.  It clears it all up for me!

---

I think what is causing the issue is how Floats actually figure decimals using binary fractions.  I found this article which seems to cover what you're seeing.

https://docs.python.org/3.4/tutorial/floatingpoint.html

---