here insert explanation of what you were doing, suspect changes you've made and so forth.
21 February 2005 11:07:59 pm
VM: unix - a SmalltalkImage Image: Squeak3.7 [latest update: #5989]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/rocktiger/squeak/3.7 Trusted Dir /home/rocktiger/squeak/3.7/secure Untrusted Dir /home/rocktiger/squeak/3.7/untrusted
UndefinedObject(Object)>>doesNotUnderstand: #< Receiver: nil Arguments and temporary variables: aMessage: < 64 Receiver's instance variables: nil
Integer class>>byte1:byte2:byte3:byte4: Receiver: Integer Arguments and temporary variables: byte1: nil byte2: nil byte3: nil byte4: nil value: nil Receiver's instance variables: superclass: Number methodDict: a MethodDictionary(#*->a CompiledMethod (474) #+->a CompiledMethod ...etc... format: 2 instanceVariables: nil organization: ('testing' even isInteger isPowerOfTwo) ('arithmetic' * + - / // ...etc... subclasses: #(SmallInteger LargePositiveInteger) name: #Integer classPool: a Dictionary() sharedPools: nil environment: nil category: nil
[] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom') binary. answer := I...]} Arguments and temporary variables: strm: StandardFileStream: '/dev/urandom' answer: nil
BlockContext>>on:do: Receiver: [] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom')...etc... Arguments and temporary variables: exception: FileStreamException handlerAction: [] in UUIDGenerator>>makeUnixSeed {[answer := nil]} handlerActive: true Receiver's instance variables: sender: BlockContext>>on:do: pc: 62 stackp: 0 nargs: 0 startpc: 46 home: UUIDGenerator>>makeUnixSeed
--- The full stack --- UndefinedObject(Object)>>doesNotUnderstand: #< Integer class>>byte1:byte2:byte3:byte4: [] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom') binary. answer := I...]} BlockContext>>on:do: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - UUIDGenerator>>makeUnixSeed UUIDGenerator>>makeSeed UUIDGenerator>>setupRandom UUIDGenerator>>initialize UUIDGenerator class(Behavior)>>new UUIDGenerator class>>generateDefault UUIDGenerator class>>default UUID>>primMakeUUID UUID>>initialize UUID class(Behavior)>>new: UUID class>>fromString: SMSqueakMap>>objectWithId: SMSqueakMap>>packageWithId: SMLoader>>installedReleaseOfMe SMLoader>>installPackageRelease: SMLoader>>installPackageRelease SMLoader>>perform:orSendTo: [] in MenuItemMorph>>invokeWithEvent: {[(selArgCount := selector numArgs) = 0 ifTrue: [target perform: selector] ...]} BlockContext>>ensure: CursorWithMask(Cursor)>>showWhile: MenuItemMorph>>invokeWithEvent: MenuItemMorph>>mouseUp: MenuItemMorph>>handleMouseUp: MouseButtonEvent>>sentTo: MenuItemMorph(Morph)>>handleEvent: MorphicEventDispatcher>>dispatchDefault:with: MorphicEventDispatcher>>dispatchEvent:with: MenuItemMorph(Morph)>>processEvent:using: MorphicEventDispatcher>>dispatchDefault:with: MorphicEventDispatcher>>dispatchEvent:with: MenuMorph(Morph)>>processEvent:using: MenuMorph(Morph)>>processEvent: MenuMorph>>handleFocusEvent: [] in HandMorph>>sendFocusEvent:to:clear: {[ActiveHand := self. ActiveEvent := anEvent. result := focusHolder han...]} [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} BlockContext>>on:do: PasteUpMorph>>becomeActiveDuring: HandMorph>>sendFocusEvent:to:clear: HandMorph>>sendEvent:focus:clear: HandMorph>>sendMouseEvent: HandMorph>>handleEvent: HandMorph>>processEvents [] in WorldState>>doOneCycleNowFor: {[:h | ActiveHand := h. h processEvents. capturingGesture := capturingGest...]} Array(SequenceableCollection)>>do: WorldState>>handsDo: WorldState>>doOneCycleNowFor: WorldState>>doOneCycleFor: PasteUpMorph>>doOneCycle [] in Project class>>spawnNewProcess {[[World doOneCycle. Processor yield. false] whileFalse. nil]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]}
For the record, I've had that happening too intermittently on a debian server. I have never been able to nail it down (or reproduce it consistently), but it looked like some previous UUID generator was locking up /dev/urandom. Simple 'cat'-ing /dev/urandom used to cure the situation, as did an ugly variety of workarounds...
Michal
On Mon, Feb 21, 2005 at 11:32:22PM -1000, hall.cj@verizon.net wrote:
here insert explanation of what you were doing, suspect changes you've made and so forth.
21 February 2005 11:07:59 pm
VM: unix - a SmalltalkImage Image: Squeak3.7 [latest update: #5989]
SecurityManager state: Restricted: false FileAccess: true SocketAccess: true Working Dir /home/rocktiger/squeak/3.7 Trusted Dir /home/rocktiger/squeak/3.7/secure Untrusted Dir /home/rocktiger/squeak/3.7/untrusted
UndefinedObject(Object)>>doesNotUnderstand: #< Receiver: nil Arguments and temporary variables: aMessage: < 64 Receiver's instance variables: nil
Integer class>>byte1:byte2:byte3:byte4: Receiver: Integer Arguments and temporary variables: byte1: nil byte2: nil byte3: nil byte4: nil value: nil Receiver's instance variables: superclass: Number methodDict: a MethodDictionary(#*->a CompiledMethod (474) #+->a CompiledMethod ...etc... format: 2 instanceVariables: nil organization: ('testing' even isInteger isPowerOfTwo) ('arithmetic' * + - / // ...etc... subclasses: #(SmallInteger LargePositiveInteger) name: #Integer classPool: a Dictionary() sharedPools: nil environment: nil category: nil
[] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom') binary. answer := I...]} Arguments and temporary variables: strm: StandardFileStream: '/dev/urandom' answer: nil
BlockContext>>on:do: Receiver: [] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom')...etc... Arguments and temporary variables: exception: FileStreamException handlerAction: [] in UUIDGenerator>>makeUnixSeed {[answer := nil]} handlerActive: true Receiver's instance variables: sender: BlockContext>>on:do: pc: 62 stackp: 0 nargs: 0 startpc: 46 home: UUIDGenerator>>makeUnixSeed
--- The full stack --- UndefinedObject(Object)>>doesNotUnderstand: #< Integer class>>byte1:byte2:byte3:byte4: [] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom') binary. answer := I...]} BlockContext>>on:do:
UUIDGenerator>>makeUnixSeed UUIDGenerator>>makeSeed UUIDGenerator>>setupRandom UUIDGenerator>>initialize UUIDGenerator class(Behavior)>>new UUIDGenerator class>>generateDefault UUIDGenerator class>>default UUID>>primMakeUUID UUID>>initialize UUID class(Behavior)>>new: UUID class>>fromString: SMSqueakMap>>objectWithId: SMSqueakMap>>packageWithId: SMLoader>>installedReleaseOfMe SMLoader>>installPackageRelease: SMLoader>>installPackageRelease SMLoader>>perform:orSendTo: [] in MenuItemMorph>>invokeWithEvent: {[(selArgCount := selector numArgs) = 0 ifTrue: [target perform: selector] ...]} BlockContext>>ensure: CursorWithMask(Cursor)>>showWhile: MenuItemMorph>>invokeWithEvent: MenuItemMorph>>mouseUp: MenuItemMorph>>handleMouseUp: MouseButtonEvent>>sentTo: MenuItemMorph(Morph)>>handleEvent: MorphicEventDispatcher>>dispatchDefault:with: MorphicEventDispatcher>>dispatchEvent:with: MenuItemMorph(Morph)>>processEvent:using: MorphicEventDispatcher>>dispatchDefault:with: MorphicEventDispatcher>>dispatchEvent:with: MenuMorph(Morph)>>processEvent:using: MenuMorph(Morph)>>processEvent: MenuMorph>>handleFocusEvent: [] in HandMorph>>sendFocusEvent:to:clear: {[ActiveHand := self. ActiveEvent := anEvent. result := focusHolder han...]} [] in PasteUpMorph>>becomeActiveDuring: {[aBlock value]} BlockContext>>on:do: PasteUpMorph>>becomeActiveDuring: HandMorph>>sendFocusEvent:to:clear: HandMorph>>sendEvent:focus:clear: HandMorph>>sendMouseEvent: HandMorph>>handleEvent: HandMorph>>processEvents [] in WorldState>>doOneCycleNowFor: {[:h | ActiveHand := h. h processEvents. capturingGesture := capturingGest...]} Array(SequenceableCollection)>>do: WorldState>>handsDo: WorldState>>doOneCycleNowFor: WorldState>>doOneCycleFor: PasteUpMorph>>doOneCycle [] in Project class>>spawnNewProcess {[[World doOneCycle. Processor yield. false] whileFalse. nil]} [] in BlockContext>>newProcess {[self value. Processor terminateActive]}
On Tuesday 22 February 2005 1:10 pm, Michal wrote:
For the record, I've had that happening too intermittently on a debian server. I have never been able to nail it down (or reproduce it consistently), but it looked like some previous UUID generator was locking up /dev/urandom. Simple 'cat'-ing /dev/urandom used to cure the situation, as did an ugly variety of workarounds...
I believe that what was happening may have been that the UTF-8 text converter attached to the stream may have been trying to interpret the bytes.
I just posted a fix for that possibility, but that may not have been the problem, on further consideration.
An aside:
Is it just me, or does anyone else think that the current way we open files that we're going to use as simple streams of bytes is a bit inefficient?
Right now, if someone just says 'readOnlyFileNamed:', this is what happens:
* get the full name from the given name * convert the full name into a local name * compare that with the local name for the sources file * if the names are the same (that is, for any file named 'SqueakV3.sources') then set the converter to a new MacRomanTextConverter, otherwise set it to a new UTF8TextConverter.
But what happens if we just want a stream of uninterpreted bytes? Well, we have change the converter to a Latin1TextConverter, which does nothing to the incoming text, other than slow down the reading. So we go through the construction of a UTF8TextConverter, then throw it away.
Look at the time comparison between two ways of opening a file and reading 1000 uninterpreted binary bytes:
Time millisecondsToRun: [ 1000 timesRepeat: [ | s | s := StandardFileStream readOnlyFileNamed: '/dev/zero'. s binary. s next: 1000. s close ] ] => 119
Time millisecondsToRun: [ 1000 timesRepeat: [ | s | s := FileStream readOnlyFileNamed: '/dev/zero'. s converter: Latin1TextConverter new. s binary. s next: 1000. s close ] ] => 251
40% of the second example's time is spent doing the above comparison of local names with the sources file.
I'm curious as to why this comparison with the local name of the sources file is actually needed; don't we generally know how we're using a file, and so could just set the text converter to the proper one in the case of the sources file?
And it still seems to me that having 'binary' flavors of the file opening routines would help in the cases where we know that we're not dealing with text.
Ned -
For the record, I've had that happening too intermittently on a debian server. I have never been able to nail it down (or reproduce it consistently), but it looked like some previous UUID generator was locking up /dev/urandom. Simple 'cat'-ing /dev/urandom used to cure the situation, as did an ugly variety of workarounds...
I believe that what was happening may have been that the UTF-8 text converter attached to the stream may have been trying to interpret the bytes.
This might well fix the bug that started this thread, but if so, mine was different. The UUID generator bug I am describing has been happening for the last couple of years, well before the UTF stuff. The scenario was always that all 4 bytes read from /dev/urandom appeared as nil, which caused a walkback. (As I said, cat/more/etc. on /dev/urandom unblocked it).
The issue is moot however: since I the server has migrated, I haven't seen it happen (but I haven't had a rush of UUID creation either).
Michal
On Tuesday 22 February 2005 11:59 pm, Michal wrote:
This might well fix the bug that started this thread, but if so, mine was different. The UUID generator bug I am describing has been happening for the last couple of years, well before the UTF stuff. The scenario was always that all 4 bytes read from /dev/urandom appeared as nil, which caused a walkback. (As I said, cat/more/etc. on /dev/urandom unblocked it).
The issue is moot however: since I the server has migrated, I haven't seen it happen (but I haven't had a rush of UUID creation either).
Yes, on thinking about it I realized that the symptom was wrong. Now, we could just test for nils (and keep trying?) but it seems like if we're allowed to open the device we should be allowed to read from it.
Hi!
Eh, I am with Ned on all he says. :) It all seems awkward to say the least.
And also - we chatted (bitmonki, Cees and me and others) on IRC about this and couldn't figure it out (/dev/urandom had the right rights and was readable, yaddayadda - everything was fine on the OS level).
But btw, shouldn't we use #ensure: or something like this:
makeUnixSeed | strm answer | [strm := FileStream readOnlyFileNamed: '/dev/urandom'. [strm binary; converter: Latin1TextConverter new. answer := Integer byte1: strm next byte2: strm next byte3: strm next byte4: strm next] ensure: [strm close]] on: FileStreamException do: [answer := nil]. ^answer
(not compiled or tested)
regards, Göran
Ned,
I believe that what was happening may have been that the UTF-8 text converter attached to the stream may have been trying to interpret the bytes.
Yes, could be.
Is it just me, or does anyone else think that the current way we open files that we're going to use as simple streams of bytes is a bit inefficient?
A bit, yes.
Right now, if someone just says 'readOnlyFileNamed:', this is what happens:
- get the full name from the given name
- convert the full name into a local name
- compare that with the local name for the sources file
- if the names are the same (that is, for any file named 'SqueakV3.sources')
then set the converter to a new MacRomanTextConverter, otherwise set it to a new UTF8TextConverter.
Yes.
But what happens if we just want a stream of uninterpreted bytes? Well, we have change the converter to a Latin1TextConverter, which does nothing to the incoming text, other than slow down the reading. So we go through the construction of a UTF8TextConverter, then throw it away.
Look at the time comparison between two ways of opening a file and reading 1000 uninterpreted binary bytes:
Time millisecondsToRun: [ 1000 timesRepeat: [ | s | s := StandardFileStream readOnlyFileNamed: '/dev/zero'. s binary. s next: 1000. s close ] ] => 119
Time millisecondsToRun: [ 1000 timesRepeat: [ | s | s := FileStream readOnlyFileNamed: '/dev/zero'. s converter: Latin1TextConverter new. s binary. s next: 1000. s close ] ] => 251
40% of the second example's time is spent doing the above comparison of local names with the sources file.
Hmm, not sure if this benchmark really reflect the real world apps.
I'm curious as to why this comparison with the local name of the sources file is actually needed; don't we generally know how we're using a file, and so could just set the text converter to the proper one in the case of the sources file?
My feeling is that once we recteate the .sources file, we can eliminate the check.
And it still seems to me that having 'binary' flavors of the file opening routines would help in the cases where we know that we're not dealing with text.
Yes.
Again, the rule of thumb is to be careful about you are dealing with uninterpreted bytes or text and use #binary and #text properly. I think it is wise to suggest to use #converter: only when it really matters.
-- Yoshiki
On Tuesday 22 February 2005 1:32 am, hall.cj@verizon.net wrote:
[] in UUIDGenerator>>makeUnixSeed {[strm := (FileStream readOnlyFileNamed: '/dev/urandom') binary. answer := I...]} Arguments and temporary variables: strm: StandardFileStream: '/dev/urandom' answer: nil
what do you see when you do
ls -l /dev/urandom
and what user are you running as?
i.e. do you have such a file, and is it readable by the UID running Squeak?
squeak-dev@lists.squeakfoundation.org
-
goran.krampe＠bluefish.se -
hall.cj＠verizon.net -
Michal -
Ned Konz -
Yoshiki Ohshima