[squeak-dev] Float storage as Double?

Lawson English lenglish5 at cox.net
Mon Apr 26 10:05:34 UTC 2010


Levente Uzonyi wrote:
> On Mon, 26 Apr 2010, Lawson English wrote:
>
>> I've been playing around with the Mandelbrot set and I noticed that 
>> the implementation using Float looked somewhat vague. After 
>> researching a while, it turns out (corrections welcome) that while 
>> Float performs 64-bit FP calculations, only a limited number of 
>> decimal places are stored for subsequent calculations. Is there a way 
>> around this?
>
> That's not true, every Float stores 64 bits. Just because some digits 
> are not shown in the printString, it doesn't mean they aren't there:
>
> 1.01 printString. "==> '1.01'"
> 1.01 printShowingDecimalPlaces: 60. "==> 
> '1.010000000000000008881784197001252323389053344726562500000000'"
>

I'm not sure what this means:

1.4142135623730950488016887242096980785696718753769480 
printShowingDecimalPlaces: 60
'1.414213562373095145474621858738828450441360473632812500000000'


>
>>
>> is there a faster way to truncate a ScaledDecimal than the above?
>
> Sure, don't convert it to strings. :) I think you shouldn't use 
> ScaledDecimal at all, since it's just a combination of a fraction and 
> scale. You only need the fraction. Use #truncateTo: to truncate a number.
Still confused, sorry.

Even with 15 decimal places, a Float-based M set generator is going to 
be inaccurate after a few levels of zoom.  Fractions don't truncate the 
way I need them to.

Lawson






More information about the Squeak-dev mailing list