[squeak-dev] Float storage as Double?
Lawson English
lenglish5 at cox.net
Mon Apr 26 10:05:34 UTC 2010
Levente Uzonyi wrote:
> On Mon, 26 Apr 2010, Lawson English wrote:
>
>> I've been playing around with the Mandelbrot set and I noticed that
>> the implementation using Float looked somewhat vague. After
>> researching a while, it turns out (corrections welcome) that while
>> Float performs 64-bit FP calculations, only a limited number of
>> decimal places are stored for subsequent calculations. Is there a way
>> around this?
>
> That's not true, every Float stores 64 bits. Just because some digits
> are not shown in the printString, it doesn't mean they aren't there:
>
> 1.01 printString. "==> '1.01'"
> 1.01 printShowingDecimalPlaces: 60. "==>
> '1.010000000000000008881784197001252323389053344726562500000000'"
>
I'm not sure what this means:
1.4142135623730950488016887242096980785696718753769480
printShowingDecimalPlaces: 60
'1.414213562373095145474621858738828450441360473632812500000000'
>
>>
>> is there a faster way to truncate a ScaledDecimal than the above?
>
> Sure, don't convert it to strings. :) I think you shouldn't use
> ScaledDecimal at all, since it's just a combination of a fraction and
> scale. You only need the fraction. Use #truncateTo: to truncate a number.
Still confused, sorry.
Even with 15 decimal places, a Float-based M set generator is going to
be inaccurate after a few levels of zoom. Fractions don't truncate the
way I need them to.
Lawson
More information about the Squeak-dev
mailing list
|