0 votes

Anyone know of issues where stepify doesn't round correctly?


Raw float: using var Cost = float(get_node("../nodepath/Cost").get_text())

Stepified float: using stepify(Cost, 0.02)

I tried both Stepify 0.01 and 0.02 and get different rounding errors:
0.01 then 1.555 rounds to 1.55 but 1.333 round to 1.33
0.02 then 1.555 rounds to 1.56 but 1.333 rounds to 1.34

asked Aug 13 in Engine by GodotUser (91 points)
edited Aug 13 by GodotUser

1 Answer

0 votes

What is the problem you are having? If any, problems you would have with rounding errors happen with any sort of calculation you do with floats, that's just how it is. Past some thresholds, precision diminishes. It doesnt mean everything breaks, it just means you should not rely on them being exact (i.e don't use ==, at least have some error margin). You'd start having precision issues with stepify with very large numbers just like you would with floor or sin.
If you want to use large numbers and you absolutely want a stepped value, prefer using integers.

answered Aug 13 by Zylann (18,349 points)

I spoke with someone on Discord who gave me a similar response regarding the inaccuracies of using stepify on a float.

The initial thought was to handle the user input beyond 2 decimal places and round it.

I tested the rounding by making another simple script that didn't use get_text() and the values were correct.

My errors appeared when using get_text().

Upon further research of how trading platforms figure the cost basis, it appears most traders do not enter half a cent (2 decimal accuracy, ie. 0.01) on trading platforms unless they are trading penny stocks (which open a whole other can of worms) I believe that is 4 decimal accuracy (ie. 0.0001).

I have may just limit the user input to 2 decimals for now and add an option for how many decimal accuracy is desired later; however I was curios about the problem.

Welcome to Godot Engine Q&A, where you can ask questions and receive answers from other members of the community.

Please make sure to read How to use this Q&A? before posting your first questions.