0 votes

Anyone know of issues where stepify doesn't round correctly?


Raw float: using var Cost = float(get_node("../nodepath/Cost").get_text())

Stepified float: using stepify(Cost, 0.02)

I tried both Stepify 0.01 and 0.02 and get different rounding errors:
0.01 then 1.555 rounds to 1.55 (INCORRECT) but 1.333 round to 1.33 (CORRECT)
0.02 then 1.555 rounds to 1.56 (CORRECT )but 1.333 rounds to 1.34 (INCORRECT)

in Engine by (93 points)
edited by

1 Answer

0 votes

What is the problem you are having? If any, problems you would have with rounding errors happen with any sort of calculation you do with floats, that's just how it is. Past some thresholds, precision diminishes. It doesnt mean everything breaks, it just means you should not rely on them being exact (i.e don't use ==, at least have some error margin). You'd start having precision issues with stepify with very large numbers just like you would with floor or sin.
If you want to use large numbers and you absolutely want a stepped value, prefer using integers.

by (28,735 points)

I spoke with someone on Discord who gave me a similar response regarding the inaccuracies of using stepify on a float.

The initial thought was to handle the user input beyond 2 decimal places and round it.

I tested the rounding by making another simple script that didn't use get_text() and the values were correct.

My errors appeared when using get_text().

Upon further research of how trading platforms figure the cost basis, it appears most traders do not enter half a cent (2 decimal accuracy, ie. 0.01) on trading platforms unless they are trading penny stocks (which open a whole other can of worms) I believe that is 4 decimal accuracy (ie. 0.0001).

I have may just limit the user input to 2 decimals for now and add an option for how many decimal accuracy is desired later; however I was curios about the problem.

Could you please provide an example?
Welcome to Godot Engine Q&A, where you can ask questions and receive answers from other members of the community.

Please make sure to read How to use this Q&A? before posting your first questions.
Social login is currently unavailable. If you've previously logged in with a Facebook or GitHub account, use the I forgot my password link in the login box to set a password for your account. If you still can't access your account, send an email to webmaster@godotengine.org with your username.