Hi Lawson,
I'm not a Squeak expert, so I could be wrong, but I think the Squeak float array stores floats as 32 bits. I think so they can be passed to external DLLs or other programs that expect 32 bit floats. If they are 32 bit floats, you can see where your precision is going.
You could try regular arrays or collections, which will use pointers to 64 bit floats. This will take more memory and be a little slower than float arrays but will give you 64 bit precision.
You might also try fractions, which are constructed as two integers. A numerator and a denominator, that can be small or large integers. The precision will be as good as you can get and should be faster than scaled decimal.
Lou
I've been playing around with the Mandelbrot set and I noticed that the implementation using Float looked somewhat vague. After researching a while, it turns out (corrections welcome) that while Float performs 64-bit FP calculations, only a limited number of decimal places are stored for subsequent calculations. Is there a way around this?
I can work up a "fixed" point Complex number simply by writing over the internal ScaledDecimal representation using using z real: (z real) asString , but this is rather a lot of overhead and for precision less than 15ish decimal places, it would be nice to use hardware.
Any relatively easy way to do this (store native precision floating point)?
is there a faster way to truncate a ScaledDecimal than the above?
Lawson
----------------------------------------------------------- Louis LaBrunda Keystone Software Corp. SkypeMe callto://PhotonDemon mailto:Lou@Keystone-Software.com http://www.Keystone-Software.com