When I modify the bzcatAFileToPipe: filename: and return dest...

bzcatAFileToPipe: pathString filename:filestring

        "Pipe bzcat output to some AttachableFileStream...whatever the heck that is...."

        "UnixProcess bzcatAFile"

        | err filename in pipe2 output dest child path  |

        path := pathString.

        filename := filestring.

        in := OSProcess readOnlyFileNamed: path, filename.

        pipe2 := OSPipe nonBlockingPipe.

        output := pipe2 writer.

        dest := pipe2 reader.

  err := self stdErr.

        child := UnixProcess

                forkJob: '/bin/bzcat'

                arguments: nil

                environment: nil

                descriptors: (Array with: in with: output with: err).

        in close.

"        (Delay forSeconds: 1) wait.

        child sigterm."

        ^ dest

I get some data with:

|pipe path filename|

Transcript clear.

path := '/home/wm/' .

filename := 'xmlstart.bz2'.

pipe := UnixProcess bzcatAFileToPipe: path  filename: filename.

pipe inspect.

Transcript show:(pipe upToEnd)

But the amount of data is clearly not all of it as the bzcat itself runs for many minutes.

This is over in a second.

Its as if some "push/pull" processing needs to happen somewhere...and I don't know how to think about it yet.

cordially,

---- On Sun, 16 Jul 2023 12:48:16 -0400 gettimothy <gettimothy@zoho.com> wrote ---

I have used with good success Monty's excellent XML suite.

The use case I used for SAX processing was with a FileStream on an xml file.

result :=

(SAXHandlerSubclass on: xmlStringOrStream)

removeLimits;

documentReadLimit: newReadLimit;

maxEntityReplacementDepth: newMaxEntityDepth;

parseDocument.

What I want to do is replace that 'xmlStringOrStream' with something from the OSProcess suite of tools.

I have been working with  the pipes from the OSProcess using one of the /bin/cat examples in the OSProcess class side.

bzcatAFileToPipe: pathString filename:filestring

        "Pipe bzcat output to some AttachableFileStream...whatever the heck that is...."

        | err filename in pipe2 output dest child path  |

        path := pathString.

        filename := filestring.

        in := OSProcess readOnlyFileNamed: path, filename.

        pipe2 := OSPipe nonBlockingPipe.

        output := pipe2 writer.

        dest := pipe2 reader.

        err := self stdErr.

        child := UnixProcess

                forkJob: '/bin/bzcat'

                arguments: nil

                environment: nil

                descriptors: (Array with: in with: output with: err).

        in close.

"        (Delay forSeconds: 1) wait.

        child sigterm."

        ^ pipe2 

I can successfully stream the output to the terminal by changing the descriptors to:

                descriptors: (Array with: in with: err with: err).

Here is an attempt to just stream the data to a Transcript...

|pipe path filename|

Transcript clear.

path := '/home/wm/' .

filename := 'xmlstart.bz2'.

pipe := UnixProcess bzcatAFileToPipe: path  filename: filename.

[Transcript show:(pipe upToEnd)] forkAt:  Processor userBackgroundPriority named:'Dude'

and nothing happens...

The goal of all this is to be able to replace the FileStream in the below example with "somthing" from the OSProcess.

|ios|

Transcript clear.

ios := (FileStream readOnlyFileNamed:('/home/wm/usr/src/myxmlfile.xml')).

[(DocDemoSaxHandler on: ios) pingevery:100000; debug:true;  optimizeForLargeDocuments;parseDocument] timeProfile .

thx in advance for any pointers.

cordially.