On Wed, Mar 10, 2010 at 6:39 AM, Jon Hylands jon@huv.com wrote:
On Fri, 19 Feb 2010 16:25:20 -0500, Jon Hylands jon@huv.com wrote:
I'm getting a weird problem, and its been so many years since I saw this, I'm at a loss as to what to do...
On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and a 32 GB solid state HD. It is running Windows XP Pro.
I'm getting a Low Space Notifier whenever I try and copy text to the clipboard, or when I try and paste. I can do other development things, writing a saving code, and opening browsers and windows with no problems.
Task Manager says Squeak is taking up 37,980 K, and the machine has 1.2 GB of free physical memory.
I'm running the following VM:
Squeak 3.10.6 (release) from Aug 30 2007
And the following image:
Squeak 3.10.2 latest update: #7179
Okay, so this is still bothering me, and I dug a little deeper, and came across something that blew me away...
http://www.huv.com/Clipboard-OutOfMemory.png
Basically, when I do 'copy', the primitive returns a ByteString that is 4 billion bytes long -- 2^32 - 1 to be exact. Clearly that is not really allocated, because even when I have this inspector open, Squeak is still only taking about 37 MB of memory.
However, the method String >> #withSqueakLineEndings is trying to create a new string that is 4 billion bytes long, and that is where I run into the low space warning.
Primitive 141 appears to be causing the issue.
The primitive looks like this in my quite possibly obsolete VM sources:
primitiveClipboardText "When called with a single string argument, post the string to the clipboard. When called with zero arguments, return a string containing the current clipboard contents." | s sz | argumentCount = 1 ifTrue: [s := self stackTop. (self isBytes: s) ifFalse: [^ self primitiveFail]. self successful ifTrue: [sz := self stSizeOf: s. self clipboardWrite: sz From: s + BaseHeaderSize At: 0. self pop: 1]] ifFalse: [sz := self clipboardSize. (self sufficientSpaceToAllocate: sz) ifFalse:[^self primitiveFail]. s := self instantiateClass: (self splObj: ClassString) indexableSize: sz. self clipboardRead: sz Into: s + BaseHeaderSize At: 0. self pop: 1 thenPush: s]
I'm willing to bet that clipboardSize is returning -1.
It probably needs to read something like
primitiveClipboardText "When called with a single string argument, post the string to the clipboard. When called with zero arguments, return a string containing the current clipboard contents." | s sz | argumentCount = 1 ifTrue: [s := self stackTop. (self isBytes: s) ifFalse: [^ self primitiveFail]. self successful ifTrue: [sz := self stSizeOf: s. self clipboardWrite: sz From: s + BaseHeaderSize At: 0. self pop: 1]] ifFalse: [sz := self clipboardSize.
(sz >= 0 and: [self sufficientSpaceToAllocate: sz]) ifFalse:[^self
primitiveFail]. s := self instantiateClass: (self splObj: ClassString) indexableSize: sz. self clipboardRead: sz Into: s + BaseHeaderSize At: 0. self pop: 1 thenPush: s]
I'm not much of a VM person, so hopefully someone who is can look at that primitive and figure out what case is causing it.
One more thing to note - I'm running Windows in a Remote Desktop, from a Linux machine. To clarify, my development machine is a desktop, running Ubuntu linux. The robot is a mini-itx running Windows XP Pro. I use remote desktop from linux to access the Windows GUI on the robot.
The Windows VM's clipboardText function is a tad unsafe:
int clipboardSize(void) { HANDLE h; WCHAR *src; unsigned char *tmp; int i, count, bytesNeeded;
/* Do we have text in the clipboard? */ if(!IsClipboardFormatAvailable(CF_UNICODETEXT)) return 0;
if(!OpenClipboard(stWindow)) return 0;
/* Get it in unicode format. */ h = GetClipboardData(CF_UNICODETEXT); src = GlobalLock(h);
/* How many bytes do we need to store those unicode chars in UTF8 format? */ bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1, NULL, 0, NULL, NULL ); tmp = malloc(bytesNeeded+1);
/* Convert Unicode text to UTF8. */ WideCharToMultiByte(CP_UTF8, 0, src, -1, tmp, bytesNeeded , NULL, NULL);
/* Count CrLfs for which we remove the extra Lf */ count = bytesNeeded; /* ex. terminating zero */ for(i=0; i<count; i++) { if((tmp[i] == 13) && (tmp[i+1] == 10)) bytesNeeded--; } bytesNeeded--; /* discount terminating zero */ free(tmp); /* no longer needed */
GlobalUnlock(h); CloseClipboard();
return bytesNeeded; }
If bytesNeeded is zero then the function will return (int)-1.
From the MS manual page:
Return Value
Returns the number of bytes written to the buffer pointed to by * lpMultiByteStr* if successful. If the function succeeds and *cbMultiByte* is 0, the return value is the required size, in bytes, for the buffer indicated by *lpMultiByteStr*.
The function returns 0 if it does not succeed. To get extended error information, the application can call *GetLastError*, which can return one of the following error codes:
- ERROR_INSUFFICIENT_BUFFER. A supplied buffer size was not large enough, or it was incorrectly set to NULL. - ERROR_INVALID_FLAGS. The values supplied for flags were not valid. - ERROR_INVALID_PARAMETER. Any of the parameter values was invalid. - ERROR_NO_UNICODE_TRANSLATION. Invalid Unicode was found in a string.
So something like
/* How many bytes do we need to store those unicode chars in UTF8 format? */ bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1, NULL, 0, NULL, NULL ); if (bytesNeeded <= 0) return 0;
would be wise.
Later, Jon
The attached is better (remembers to unlock and close clipboard). I don't think I have commit rights for http://squeakvm.org/svn/squeak/trunk. Guess I should go get some :) Who do I talk to?
On Wed, Mar 10, 2010 at 9:25 AM, Eliot Miranda eliot.miranda@gmail.comwrote:
On Wed, Mar 10, 2010 at 6:39 AM, Jon Hylands jon@huv.com wrote:
On Fri, 19 Feb 2010 16:25:20 -0500, Jon Hylands jon@huv.com wrote:
I'm getting a weird problem, and its been so many years since I saw this, I'm at a loss as to what to do...
On my robot Brainbot, I have a 2.83 GHz Core 2 Quad with 2 GB RAM and a 32 GB solid state HD. It is running Windows XP Pro.
I'm getting a Low Space Notifier whenever I try and copy text to the clipboard, or when I try and paste. I can do other development things, writing a saving code, and opening browsers and windows with no problems.
Task Manager says Squeak is taking up 37,980 K, and the machine has 1.2 GB of free physical memory.
I'm running the following VM:
Squeak 3.10.6 (release) from Aug 30 2007
And the following image:
Squeak 3.10.2 latest update: #7179
Okay, so this is still bothering me, and I dug a little deeper, and came across something that blew me away...
http://www.huv.com/Clipboard-OutOfMemory.png
Basically, when I do 'copy', the primitive returns a ByteString that is 4 billion bytes long -- 2^32 - 1 to be exact. Clearly that is not really allocated, because even when I have this inspector open, Squeak is still only taking about 37 MB of memory.
However, the method String >> #withSqueakLineEndings is trying to create a new string that is 4 billion bytes long, and that is where I run into the low space warning.
Primitive 141 appears to be causing the issue.
The primitive looks like this in my quite possibly obsolete VM sources:
primitiveClipboardText "When called with a single string argument, post the string to the clipboard. When called with zero arguments, return a string containing the current clipboard contents." | s sz | argumentCount = 1 ifTrue: [s := self stackTop. (self isBytes: s) ifFalse: [^ self primitiveFail]. self successful ifTrue: [sz := self stSizeOf: s. self clipboardWrite: sz From: s + BaseHeaderSize At: 0. self pop: 1]] ifFalse: [sz := self clipboardSize. (self sufficientSpaceToAllocate: sz) ifFalse:[^self primitiveFail]. s := self instantiateClass: (self splObj: ClassString) indexableSize: sz. self clipboardRead: sz Into: s + BaseHeaderSize At: 0. self pop: 1 thenPush: s]
I'm willing to bet that clipboardSize is returning -1.
It probably needs to read something like
primitiveClipboardText "When called with a single string argument, post the string to the clipboard. When called with zero arguments, return a string containing the current clipboard contents." | s sz | argumentCount = 1 ifTrue: [s := self stackTop. (self isBytes: s) ifFalse: [^ self primitiveFail]. self successful ifTrue: [sz := self stSizeOf: s. self clipboardWrite: sz From: s + BaseHeaderSize At: 0. self pop: 1]] ifFalse: [sz := self clipboardSize.
(sz >= 0 and: [self sufficientSpaceToAllocate: sz]) ifFalse:[^self
primitiveFail]. s := self instantiateClass: (self splObj: ClassString) indexableSize: sz. self clipboardRead: sz Into: s + BaseHeaderSize At: 0. self pop: 1 thenPush: s]
I'm not much of a VM person, so hopefully someone who is can look at that primitive and figure out what case is causing it.
One more thing to note - I'm running Windows in a Remote Desktop, from a Linux machine. To clarify, my development machine is a desktop, running Ubuntu linux. The robot is a mini-itx running Windows XP Pro. I use remote desktop from linux to access the Windows GUI on the robot.
The Windows VM's clipboardText function is a tad unsafe:
int clipboardSize(void) { HANDLE h; WCHAR *src; unsigned char *tmp; int i, count, bytesNeeded;
/* Do we have text in the clipboard? */ if(!IsClipboardFormatAvailable(CF_UNICODETEXT)) return 0;
if(!OpenClipboard(stWindow)) return 0;
/* Get it in unicode format. */ h = GetClipboardData(CF_UNICODETEXT); src = GlobalLock(h);
/* How many bytes do we need to store those unicode chars in UTF8 format? */ bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1, NULL, 0, NULL, NULL ); tmp = malloc(bytesNeeded+1);
/* Convert Unicode text to UTF8. */ WideCharToMultiByte(CP_UTF8, 0, src, -1, tmp, bytesNeeded , NULL, NULL);
/* Count CrLfs for which we remove the extra Lf */ count = bytesNeeded; /* ex. terminating zero */ for(i=0; i<count; i++) { if((tmp[i] == 13) && (tmp[i+1] == 10)) bytesNeeded--; } bytesNeeded--; /* discount terminating zero */ free(tmp); /* no longer needed */
GlobalUnlock(h); CloseClipboard();
return bytesNeeded; }
If bytesNeeded is zero then the function will return (int)-1.
From the MS manual page:
Return Value
Returns the number of bytes written to the buffer pointed to by * lpMultiByteStr* if successful. If the function succeeds and *cbMultiByte* is 0, the return value is the required size, in bytes, for the buffer indicated by *lpMultiByteStr*.
The function returns 0 if it does not succeed. To get extended error information, the application can call *GetLastError*, which can return one of the following error codes:
- ERROR_INSUFFICIENT_BUFFER. A supplied buffer size was not large
enough, or it was incorrectly set to NULL.
- ERROR_INVALID_FLAGS. The values supplied for flags were not valid.
- ERROR_INVALID_PARAMETER. Any of the parameter values was invalid.
- ERROR_NO_UNICODE_TRANSLATION. Invalid Unicode was found in a string.
So something like
/* How many bytes do we need to store those unicode chars in UTF8 format? */ bytesNeeded = WideCharToMultiByte(CP_UTF8, 0, src, -1, NULL, 0, NULL, NULL ); if (bytesNeeded <= 0) return 0;
would be wise.
Later, Jon
On 10.03.2010, at 18:31, Eliot Miranda wrote:
The attached is better (remembers to unlock and close clipboard). I don't think I have commit rights for http://squeakvm.org/svn/squeak/trunk. Guess I should go get some :) Who do I talk to?
Unless I'm mistaken only Ian, John, and Andreas have commit access to the VM trunk.
And it's Ian's private box this is running on.
- Bert -
On Wed, Mar 10, 2010 at 9:38 AM, Bert Freudenberg bert@freudenbergs.dewrote:
On 10.03.2010, at 18:31, Eliot Miranda wrote:
The attached is better (remembers to unlock and close clipboard). I
don't think I have commit rights for http://squeakvm.org/svn/squeak/trunk. Guess I should go get some :) Who do I talk to?
Unless I'm mistaken only Ian, John, and Andreas have commit access to the VM trunk.
And it's Ian's private box this is running on.
Then posting a modified file is fine. Works for me :) Thanks, Bert!
- Bert -
On Wed, Mar 10, 2010 at 10:00:48AM -0800, Eliot Miranda wrote:
On Wed, Mar 10, 2010 at 9:38 AM, Bert Freudenberg bert@freudenbergs.dewrote:
On 10.03.2010, at 18:31, Eliot Miranda wrote:
The attached is better (remembers to unlock and close clipboard). I
don't think I have commit rights for http://squeakvm.org/svn/squeak/trunk. Guess I should go get some :) Who do I talk to?
Unless I'm mistaken only Ian, John, and Andreas have commit access to the VM trunk.
And it's Ian's private box this is running on.
Then posting a modified file is fine. Works for me :) Thanks, Bert!
Actually if you don't mind a few extra steps, it helps a lot if you can open an issue on Mantis under category "VM" and attach it there. Things do have a tendency to get lost if just posted to the list.
Dave
On Wed, Mar 10, 2010 at 10:56 AM, David T. Lewis lewis@mail.msen.comwrote:
On Wed, Mar 10, 2010 at 10:00:48AM -0800, Eliot Miranda wrote:
On Wed, Mar 10, 2010 at 9:38 AM, Bert Freudenberg <bert@freudenbergs.de wrote:
On 10.03.2010, at 18:31, Eliot Miranda wrote:
The attached is better (remembers to unlock and close clipboard). I
don't think I have commit rights for
http://squeakvm.org/svn/squeak/trunk.
Guess I should go get some :) Who do I talk to?
Unless I'm mistaken only Ian, John, and Andreas have commit access to
the
VM trunk.
And it's Ian's private box this is running on.
Then posting a modified file is fine. Works for me :) Thanks, Bert!
Actually if you don't mind a few extra steps, it helps a lot if you can open an issue on Mantis under category "VM" and attach it there. Things do have a tendency to get lost if just posted to the list.
Quite :) Of course.
Dave
On Wed, Mar 10, 2010 at 10:56 AM, David T. Lewis lewis@mail.msen.comwrote:
On Wed, Mar 10, 2010 at 10:00:48AM -0800, Eliot Miranda wrote:
On Wed, Mar 10, 2010 at 9:38 AM, Bert Freudenberg <bert@freudenbergs.de wrote:
On 10.03.2010, at 18:31, Eliot Miranda wrote:
The attached is better (remembers to unlock and close clipboard). I
don't think I have commit rights for
http://squeakvm.org/svn/squeak/trunk.
Guess I should go get some :) Who do I talk to?
Unless I'm mistaken only Ian, John, and Andreas have commit access to
the
VM trunk.
And it's Ian's private box this is running on.
Then posting a modified file is fine. Works for me :) Thanks, Bert!
Actually if you don't mind a few extra steps, it helps a lot if you can open an issue on Mantis under category "VM" and attach it there. Things do have a tendency to get lost if just posted to the list.
Issue 7476. clipboardSize on Win32 can return -1 (4294967295) on failure
Dave
On Wed, Mar 10, 2010 at 11:34:29AM -0800, Eliot Miranda wrote:
On Wed, Mar 10, 2010 at 10:56 AM, David T. Lewis lewis@mail.msen.comwrote:
Actually if you don't mind a few extra steps, it helps a lot if you can open an issue on Mantis under category "VM" and attach it there. Things do have a tendency to get lost if just posted to the list.
Issue 7476. clipboardSize on Win32 can return -1 (4294967295) on failure
Thank you!
vm-dev@lists.squeakfoundation.org