Hi David,
sort of...

We know that smallInt asFloat may be inexact.
We could test whether it is exact or not with smallInt asFloat asInteger = smallInt like what is done in Squeak SmallInteger>>#isAnExactFloat check, but that's not what we do here.
What we do is to check if ever that rounding error could change the result of comparison. If it could not, then the rounding error is innocuous and we can proceed with float comparison.

So more exactly, the algorithm is:

• check if there is a possible ambiguity
• if yes, use exact comparison (the usual image side smallInt = smallFloat asTrueFraction)
• if no, use inexact but innocuous comparison (the simple smallInt asFloat = smallFloat)
  (strictly speaking, the comparison is exact, only the operands may not)

If smallInt asFloat > smallFloat we know for sure that smallInt > smallFloat.
If smallInt asFloat < smallFloat we know for sure that smallInt < smallFloat.
The only case where we could have ambiguity is when smallInt asFloat = smallFloat.

But in this case, we know that smallFloat value is integer. Indeed, either asFloat is exact, and smallInt = smallFloat, or inexact, but that means that smallInt > (2 raisedTo: Float precision) and then Float has not enough precision to have a fraction part. Thus smallFloat asTrueFraction = smallFloat asInteger in this ambiguous case, a nice thing for the VM to not deal with Fraction!

A potential remaining problem could be that smallFloat asInteger may be a LargeInteger, for example SmallInteger maxVal asInteger > SmallInteger maxVal in 64 bits image.
But we know that:
SmallInteger minVal <= smallFloat and: [smallFloat <= SmallInteger maxVal nextPowerOfTwo]
Thus in the VM, we are safe, SmallInteger span only 61 bits and we have 64 bits registers.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.