So Metacello is being forced, along with many other packages, to re-implement a compatibility layer because Squeak uses FileDirectory and Pharo uses FileSystem.
Now I completely understand that Pharo's understanding of FileDirectory is about 5 years out of date. By now, countless arguments on pharo-dev have shown that the two packages are roughly isomorphic in functionality and API.
Camillo Bruni's even shown this by implementing a shim exposing a FIleDirectory API on top of FileSystem to ease migration of early Pharo projects to later versions of Pharo.
So I have a pair of questions: * how far have wiresong's FileSystem and Pharo's FileSystem diverged? (And how do we bring the two back together again, if necessary?) * How do we fix the problem?
Pharo has nearly emptied the open source Smalltalk room of oxygen, so there's an argument for saying that FileSystem has won and we should use it. If we did move to FileSystem we _would_ want to keep Cami's shim, because we care about backwards compatibility a whole lot more than Pharo. And of course there's Cuis to consider.
Thoughts?
(Note: please _don't_ talk about the spilt more than absolutely necessary. It's done, it can't be undone, and we should concentrate our energies on moving Squeak forward, and minimising pain for those brave souls who try keep their packages cross platform.)
frank
Quoting Frank Shearar frank.shearar@gmail.com:
So Metacello is being forced, along with many other packages, to re-implement a compatibility layer because Squeak uses FileDirectory and Pharo uses FileSystem.
Now I completely understand that Pharo's understanding of FileDirectory is about 5 years out of date. By now, countless arguments on pharo-dev have shown that the two packages are roughly isomorphic in functionality and API.
...
And of course there's Cuis to consider.
Thoughts? ... frank
Thanks for remembering about Cuis. In Cuis, we haven't payed much attention to FileSystem functionality yet. Most likely our code is rather outdated here. If Squeak decides to switch to FileSystem, I think Cuis can do the same. In any case, we need to review alternatives and pick one for Cuis, but we're not in a hurry.
Cheers, Juan Vuletich
On 24-05-2013, at 1:58 AM, Frank Shearar frank.shearar@gmail.com wrote:
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
- How do we fix the problem?
Without even knowing anything specifically about FileSystem I would claim a very high likelihood that it is better than the old code, because:- Colin wrote at least a good chunk of it it was written recently with experience of just how awful FileDirectory et al have become over years of largely unprincipled hacking
And thus I would strongly suggest it in with a chance of being an improvement. My only caveat would be that I feel reasonably sure it was not written with much of an eye to 'odd' filing systems like RISC OS and thus it may need a bit more work to drag it away from the depressingly mono-cultural ersatz-unix-style that seems to have pervaded the world. I wouldn't mind if were actually a good style...
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Klingon Code Warrior:- 4) "This machine is a piece of GAGH! I need dual ARMv8 processors if I am to do battle with this code!"
tim Rowledge
On 24-05-2013, at 1:58 AM, Frank Shearar frank.shearar@gmail.com wrote:
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
- How do we fix the problem?
Without even knowing anything specifically about FileSystem I would claim
a
very high likelihood that it is better than the old code, because:- Colin
wrote at
least a good chunk of it it was written recently with experience of just
how
awful FileDirectory et al have become over years of largely unprincipled
hacking
And thus I would strongly suggest it in with a chance of being an
improvement.
My only caveat would be that I feel reasonably sure it was not written
with
much of an eye to 'odd' filing systems like RISC OS and thus it may need a
bit
more work to drag it away from the depressingly mono-cultural
ersatz-unix-style
that seems to have pervaded the world. I wouldn't mind if were actually a
good
style...
Also when / if the change is done I recommend that someone that goes through the conversion on production code produce a conversion guide to help people that need to update their systems.
Ron
tim
tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Klingon Code Warrior:- 4) "This machine is a piece of GAGH! I need dual ARMv8
processors if I
am to do battle with this code!"
On Fri, May 24, 2013 at 9:49 AM, tim Rowledge tim@rowledge.org wrote:
ould strongly suggest it in with a chance of being an improvement. My only caveat would be that I feel reasonably sure it was not written with much of an eye to 'odd' filing systems like RISC OS and thus it may need a bit more work to drag it away from the depressingly mono-cultural ersatz-unix-style that seems to have pervaded the world. I wouldn't mind if were actually a good style...
Yeah, Andreas smacked me over forcing unix style on Windows users. Fortunately, he smacked me with code, so there's a bit of agnosticism in there already.
Is it just a matter of generating correct path strings to pass to the primitives, or is there more to it?
Colin
On Sun, May 26, 2013 at 07:14:19PM -0700, Colin Putney wrote:
On Fri, May 24, 2013 at 9:49 AM, tim Rowledge tim@rowledge.org wrote:
ould strongly suggest it in with a chance of being an improvement. My only caveat would be that I feel reasonably sure it was not written with much of an eye to 'odd' filing systems like RISC OS and thus it may need a bit more work to drag it away from the depressingly mono-cultural ersatz-unix-style that seems to have pervaded the world. I wouldn't mind if were actually a good style...
Yeah, Andreas smacked me over forcing unix style on Windows users. Fortunately, he smacked me with code, so there's a bit of agnosticism in there already.
Is it just a matter of generating correct path strings to pass to the primitives, or is there more to it?
There are other things as well. I'm not sure if these apply to FileSystem (sorry I can't look right now) but examples might be:
- Unix views the file system as a tree, regardless of how many file systems and physical devices are involved. Windows models this differently as volumes (C:, D: etc). Other operating systems may have different models.
- The notions of file creation time, last accessed time, last modified time and so forth are done differently (and incompatibly) on different kinds of file system.
- One operating system may simulaneously host multiple types of file system, so some of the differences are associated with the file system and not with the operating system per se.
I'm not sure what might be required to have FileSystem work smoothly with RISC OS as well as Windows/Unix, but I think the effort would be very worthwhile. Windows and Unix file systems are already very similar, so the best way to ensure a good level of abstraction is to make sure that the model also fits file systems such as RISC OS that are not fundamentally unix-based.
So far I have looked at FileSystem only to the extent needed to get OSProcess/CommandShell working on Pharo, so I don't know if the above are really relevant. In any case, I'll be happy to help where I can.
Dave
On Mon, May 27, 2013 at 6:03 AM, David T. Lewis lewis@mail.msen.com wrote:
- Unix views the file system as a tree, regardless of how many file systems
and physical devices are involved. Windows models this differently as volumes (C:, D: etc). Other operating systems may have different models.
The first version of FS that I did tried to model that pretty closely. FS its self supports multiple volumes, and so I created one for each Windows drive, each with it's own CWD, just as Windows shells do. People who run Squeak on Windows (particularly Andreas) didn't like this set up, and so we switched it over to a more Unix-like model where there's a single root, single CWD and special handling for the drive letters in absolute paths. I thought it was cool that we could match the Windows filesystem semantics correctly, but <shrug> I'm not a Windows user, what do I know.
- The notions of file creation time, last accessed time, last modified
time and so forth are done differently (and incompatibly) on different kinds of file system.
The FilePlugin primitives handle the platform differences for us. I haven't run into any issues with this yet.
- One operating system may simulaneously host multiple types of file
system, so some of the differences are associated with the file system and not with the operating system per se.
Filesystem can handle multiple "file systems" at once, so at the image level this isn't an issue (and FS already includes support for file systems other than the one supplied by the operating system—in memory, inside a zip file etc). If the FilePlugin primitives and/or the OS file API don't unify the interface to different filesystems for us, we could use a different set of primitives for each filesystem. I did have an idea for a new set of primitives to go with the new image-level code, but I figured it'd be better to wait and see what issues, if any, surface from real-world usage.
I'm not sure what might be required to have FileSystem work smoothly with RISC OS as well as Windows/Unix, but I think the effort would be very worthwhile. Windows and Unix file systems are already very similar, so the best way to ensure a good level of abstraction is to make sure that the model also fits file systems such as RISC OS that are not fundamentally unix-based.
I think the differences in the path syntax should be easy to deal with, but we'll probably need some way to manipulate the file metadata. (eg. to set the filetype correctly).
So far I have looked at FileSystem only to the extent needed to get
OSProcess/CommandShell working on Pharo, so I don't know if the above are really relevant. In any case, I'll be happy to help where I can.
Cool!
Colin
Hi Frank,
Can you tell me which one has better SUnit coverage? (I'm on a phone.)
If FileSystem doesn't have just some enormous complexity, if the code is at least as good (however we're going to be measuring that) and there are good tests, I can't think of any reason we shouldn't do what the hipsters are doing, except of course the work involved.
<ironic type="hipster">mustache</ironic>
On May 24, 2013, at 1:58 AM, Frank Shearar frank.shearar@gmail.com wrote:
So Metacello is being forced, along with many other packages, to re-implement a compatibility layer because Squeak uses FileDirectory and Pharo uses FileSystem.
Now I completely understand that Pharo's understanding of FileDirectory is about 5 years out of date. By now, countless arguments on pharo-dev have shown that the two packages are roughly isomorphic in functionality and API.
Camillo Bruni's even shown this by implementing a shim exposing a FIleDirectory API on top of FileSystem to ease migration of early Pharo projects to later versions of Pharo.
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
- How do we fix the problem?
Pharo has nearly emptied the open source Smalltalk room of oxygen, so there's an argument for saying that FileSystem has won and we should use it. If we did move to FileSystem we _would_ want to keep Cami's shim, because we care about backwards compatibility a whole lot more than Pharo. And of course there's Cuis to consider.
Thoughts?
(Note: please _don't_ talk about the spilt more than absolutely necessary. It's done, it can't be undone, and we should concentrate our energies on moving Squeak forward, and minimising pain for those brave souls who try keep their packages cross platform.)
frank
Hi Casey,
It looks like FileDirectory has 26% methods covered, which honestly is higher than I'd expected. It looks like FileSystem has 80% code coverage.
"Looks like" because FSFilesystemTest class >> #packageNamesUnderTest returns #('Filesystem') which seems to make TestRunner think there's nothing to cover. Hacking that to return #('FS') and adding an 'FS' package to the image (so that one package "captures" all the FS-* packages) seems to fix the problem.
frank
On 25 May 2013 23:51, Casey Ransberger casey.obrien.r@gmail.com wrote:
Hi Frank,
Can you tell me which one has better SUnit coverage? (I'm on a phone.)
If FileSystem doesn't have just some enormous complexity, if the code is at least as good (however we're going to be measuring that) and there are good tests, I can't think of any reason we shouldn't do what the hipsters are doing, except of course the work involved.
<ironic type="hipster">mustache</ironic>
On May 24, 2013, at 1:58 AM, Frank Shearar frank.shearar@gmail.com wrote:
So Metacello is being forced, along with many other packages, to re-implement a compatibility layer because Squeak uses FileDirectory and Pharo uses FileSystem.
Now I completely understand that Pharo's understanding of FileDirectory is about 5 years out of date. By now, countless arguments on pharo-dev have shown that the two packages are roughly isomorphic in functionality and API.
Camillo Bruni's even shown this by implementing a shim exposing a FIleDirectory API on top of FileSystem to ease migration of early Pharo projects to later versions of Pharo.
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
- How do we fix the problem?
Pharo has nearly emptied the open source Smalltalk room of oxygen, so there's an argument for saying that FileSystem has won and we should use it. If we did move to FileSystem we _would_ want to keep Cami's shim, because we care about backwards compatibility a whole lot more than Pharo. And of course there's Cuis to consider.
Thoughts?
(Note: please _don't_ talk about the spilt more than absolutely necessary. It's done, it can't be undone, and we should concentrate our energies on moving Squeak forward, and minimising pain for those brave souls who try keep their packages cross platform.)
frank
Frank Shearar-3 wrote
FileSystem has won and we should use it
Wow, I'm impressed by the thoughtfulness of the request, as well as the responses - all business :) I've seen similar attitudes on the Pharo side regarding adopting improvements in Squeak. In fact, I've felt the relationship between Squeak and Pharo warming for a while and this confirms that yet again. Thanks to all for being so cool!
- Sean
p.s. I still dream about a day when Squeak and Pharo cooperate more directly, possibly even [gasp] unifying (e.g. Pharo core + Squeak/etoys on top), but I try not to talk about it as not to scare people ;)
----- Cheers, Sean -- View this message in context: http://forum.world.st/FileSystem-tp4689585p4690011.html Sent from the Squeak - Dev mailing list archive at Nabble.com.
On 5/26/13 12:13 PM, "Sean P. DeNigris" sean@clipperadams.com wrote:
p.s. I still dream about a day when Squeak and Pharo cooperate more directly, possibly even [gasp] unifying (e.g. Pharo core + Squeak/etoys on top), but I try not to talk about it as not to scare people ;)
Yes, this could be good. As Pharo 2.0 and 3.0 diverge too much... What about we took Pharo 1.4 Kernel and build on top?
Edgar
On Fri, May 24, 2013 at 1:58 AM, Frank Shearar frank.shearar@gmail.comwrote:
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
I doubt there's much divergence. I synced with the Pharo folks as they were putting it in to their base image, and that was a pretty mature implementation, so I'd be surprised if there was anything difficult to deal with.
- How do we fix the problem?
Well, I'd be happy to see Filesystem in Squeak. :-)
I know Chris is worried about breaking Magma, so we'll need to be careful, but we can do that.
Colin
Well, I'd be happy to see Filesystem in Squeak. :-)
I know Chris is worried about breaking Magma, so we'll need to be careful, but we can do that.
That's appreciated, thanks. You're right, and also about Banyan. And not just breakage but also performance. I'm sure I won't be left behind entirely but I also hope it won't be too painful to switch over.
Banyan makes use of that method in FileDirectory I committed to trunk yesterday (directoryTreeDo:). Important to Banyan would be for FileSystem to be able to do that without creating much (if any) garbage. Is that doable?
On Sun, May 26, 2013 at 7:28 PM, Chris Muller asqueaker@gmail.com wrote:
Banyan makes use of that method in FileDirectory I committed to trunk yesterday (directoryTreeDo:). Important to Banyan would be for FileSystem to be able to do that without creating much (if any) garbage. Is that doable?
Sure. Filesystem does have methods for walking a directory tree, although they're not optimized for memory usage.
Banyan makes use of that method in FileDirectory I committed to trunk yesterday (directoryTreeDo:). Important to Banyan would be for FileSystem to be able to do that without creating much (if any) garbage. Is that doable?
Sure. Filesystem does have methods for walking a directory tree, although they're not optimized for memory usage.
That's what I thought I remember from briefly looking at it before, that the way it did that created lots of garbage. I hope I'm wrong. IMO, FileSystem should be at least equal to FileDirectory across the board, so we leave no reasons at all for someone to want to have FileDirectory in their image (which means they'd have to have both).
On Mon, May 27, 2013 at 12:15 PM, Chris Muller asqueaker@gmail.com wrote:
That's what I thought I remember from briefly looking at it before, that the way it did that created lots of garbage. I hope I'm wrong. IMO, FileSystem should be at least equal to FileDirectory across the board, so we leave no reasons at all for someone to want to have FileDirectory in their image (which means they'd have to have both).
I don't think you can expect a general API that's designed for flexibility and usefulness across a broad range of applications to perform as well as an optimized implementation that you hand-tuned for your specific application. In fact, I'd argue that #directoryTreeDo: shouldn't be part of the trunk, it should be an extension method in Banyan.
Look at it this way: FileDirectory was missing a lot of features that you needed. Instead of using Filesystem, which provides those features, you added them to FileDirectory in a way that's highly specific to the needs of your application. Now you're worried that Filesystem isn't "equal" to FileDirectory, meaning that it's not tuned for your application. But if you put the same effort into optimizing Banyan+Filesystem, you'd be fine.
If you don't want to put in that effort, that's fine too. FileDirectory will be a loadable package, so you can just make Banyan depend on it, and add it to your build script.
Colin
That's what I thought I remember from briefly looking at it before, that the way it did that created lots of garbage. I hope I'm wrong. IMO, FileSystem should be at least equal to FileDirectory across the board, so we leave no reasons at all for someone to want to have FileDirectory in their image (which means they'd have to have both).
I don't think you can expect a general API that's designed for flexibility and usefulness across a broad range of applications to perform as well as an optimized implementation that you hand-tuned for your specific application. In fact, I'd argue that #directoryTreeDo: shouldn't be part of the trunk, it should be an extension method in Banyan.
Colin, may we please not be at odds here? I'm on your side.
Before you said "Sure, FileSystem does that too" but above you're saying this method should not be in trunk? But it should be in trunk as part of FileSystem? I'm confused.
Look at it this way: FileDirectory was missing a lot of features that you needed. Instead of using Filesystem, which provides those features, you added them to FileDirectory in a way that's highly specific to the needs of your application.
"A lot of features?" What are you talking about?
I've presented just one short public method (with one supporting private) from 2007, long before FileSystem, which performs a /very general/ operation, an efficient internal Iterator of a directory tree.
Now you're worried that Filesystem isn't "equal" to FileDirectory, meaning that it's not tuned for your application. But if you put the same effort into optimizing Banyan+Filesystem, you'd be fine. If you don't want to put in that effort, that's fine too. FileDirectory will be a loadable package, so you can just make Banyan depend on it, and add it to your build script.
I don't understand this defensiveness. I'm trying to open up to FileSystem by asking you if this one important function can be made to perform well as in FD.
I figured you'd look at it and, based on your FS knowledge, say "yes, an equivalent method could be spanked out in under 5 minutes" (even if the "efficient" version had to be an extension), so we could be off to the races. Is it not the case? I was hoping to be encouraged by your reply but surprised instead to hear back that I should lower my expectations about performance and appreciate the flexibility..? How can someone be expected to allocate time to convert legacy code if it's known certain functions will end up compromised on performance?
On Mon, May 27, 2013 at 2:45 PM, Chris Muller ma.chris.m@gmail.com wrote:
Colin, may we please not be at odds here? I'm on your side.
I'm sorry, I didn't mean for that post to be so confrontational. I misread your post as shifting the goal-posts so that FileSystem wouldn't be acceptable unless it performs like your tuned-for-Banyan additions to FileDirectory.
There's certainly room to optimize the tree-walking code in Filesystem, so we may be able to meet your needs that way.
On the other hand, there are layers of indirection in Filesystem that aren't present in FileDirectory. Filesystem works with many kinds of directory trees—on disk, in-memory, inside a zip file, inside a git repository etc. It also has whole-tree operations that need to be able to customize the tree walking algorithm. For example, copying a tree needs to visit directories before their contents, while deleting a tree needs to visit directories after their contents. So from the point of view of Filesystem, #directoryTreeDo: is *not* a very general operation, it's quite specific, and tuned for a particular use-case. It may not be possible to optimize Filesystem's tree-walking code to the same level of memory efficiency without sacrificing generality.
But that's not a show-stopper! We could make a method like #directoryTreeDo: just for Banyan, or Banyan could keep on using FileDirectory. It's not like FileDirectory would be removed in 4.5, and even when it finally does get removed, it would still be available as a compatibility package.
Again, my apologies for the over-reaction.
Colin
Could you point me to what's the best way to install latest FS into trunk? Thanks!
On Mon, May 27, 2013 at 7:27 PM, Colin Putney colin@wiresong.com wrote:
On Mon, May 27, 2013 at 2:45 PM, Chris Muller ma.chris.m@gmail.com wrote:
Colin, may we please not be at odds here? I'm on your side.
I'm sorry, I didn't mean for that post to be so confrontational. I misread your post as shifting the goal-posts so that FileSystem wouldn't be acceptable unless it performs like your tuned-for-Banyan additions to FileDirectory.
There's certainly room to optimize the tree-walking code in Filesystem, so we may be able to meet your needs that way.
On the other hand, there are layers of indirection in Filesystem that aren't present in FileDirectory. Filesystem works with many kinds of directory trees—on disk, in-memory, inside a zip file, inside a git repository etc. It also has whole-tree operations that need to be able to customize the tree walking algorithm. For example, copying a tree needs to visit directories before their contents, while deleting a tree needs to visit directories after their contents. So from the point of view of Filesystem, #directoryTreeDo: is *not* a very general operation, it's quite specific, and tuned for a particular use-case. It may not be possible to optimize Filesystem's tree-walking code to the same level of memory efficiency without sacrificing generality.
But that's not a show-stopper! We could make a method like #directoryTreeDo: just for Banyan, or Banyan could keep on using FileDirectory. It's not like FileDirectory would be removed in 4.5, and even when it finally does get removed, it would still be available as a compatibility package.
Again, my apologies for the over-reaction.
Colin
Hi Chris,
Now that I have the CI build working, there's a nice way of telling you: https://github.com/squeak-smalltalk/squeak-ci/blob/master/package-load-scrip...
FileSystem does have a bit of entanglement with Xtreams (mentioned earlier in this thread, IIRC). You can find an Installer for installing Xtreams and FileSystem here though: https://github.com/squeak-smalltalk/squeak-ci/blob/master/package-load-scrip...
frank
On 28 May 2013 02:25, Chris Muller asqueaker@gmail.com wrote:
Could you point me to what's the best way to install latest FS into trunk? Thanks!
On Mon, May 27, 2013 at 7:27 PM, Colin Putney colin@wiresong.com wrote:
On Mon, May 27, 2013 at 2:45 PM, Chris Muller ma.chris.m@gmail.com wrote:
Colin, may we please not be at odds here? I'm on your side.
I'm sorry, I didn't mean for that post to be so confrontational. I misread your post as shifting the goal-posts so that FileSystem wouldn't be acceptable unless it performs like your tuned-for-Banyan additions to FileDirectory.
There's certainly room to optimize the tree-walking code in Filesystem, so we may be able to meet your needs that way.
On the other hand, there are layers of indirection in Filesystem that aren't present in FileDirectory. Filesystem works with many kinds of directory trees—on disk, in-memory, inside a zip file, inside a git repository etc. It also has whole-tree operations that need to be able to customize the tree walking algorithm. For example, copying a tree needs to visit directories before their contents, while deleting a tree needs to visit directories after their contents. So from the point of view of Filesystem, #directoryTreeDo: is *not* a very general operation, it's quite specific, and tuned for a particular use-case. It may not be possible to optimize Filesystem's tree-walking code to the same level of memory efficiency without sacrificing generality.
But that's not a show-stopper! We could make a method like #directoryTreeDo: just for Banyan, or Banyan could keep on using FileDirectory. It's not like FileDirectory would be removed in 4.5, and even when it finally does get removed, it would still be available as a compatibility package.
Again, my apologies for the over-reaction.
Colin
Nice, thank you, Frank
https://github.com/squeak-smalltalk/squeak-ci/blob/master/package-load-scrip...
(Installer wiresong project: 'mc') addPackage: 'FS-Core'; addPackage: 'FS-Disk'; addPackage: 'FS-Memory'; addPackage: 'FS-Zip'; addPackage: 'FS-FileStream'; addPackage: 'FS-Tests-Core'; addPackage: 'FS-Tests-Zip'; addPackage: 'FS-Tests-Disk'; addPackage: 'FS-Tests-FileStream'; install.
WorldState addDeferredUIMessage: [ SmalltalkImage current snapshot: true andQuit: true ].
On 6/18/13, Frank Shearar frank.shearar@gmail.com wrote:
Hi Chris,
Now that I have the CI build working, there's a nice way of telling you: https://github.com/squeak-smalltalk/squeak-ci/blob/master/package-load-scrip...
FileSystem does have a bit of entanglement with Xtreams (mentioned earlier in this thread, IIRC). You can find an Installer for installing Xtreams and FileSystem here though: https://github.com/squeak-smalltalk/squeak-ci/blob/master/package-load-scrip...
frank
On 28 May 2013 02:25, Chris Muller asqueaker@gmail.com wrote:
Could you point me to what's the best way to install latest FS into trunk? Thanks!
On Mon, May 27, 2013 at 7:27 PM, Colin Putney colin@wiresong.com wrote:
On Mon, May 27, 2013 at 2:45 PM, Chris Muller ma.chris.m@gmail.com wrote:
Colin, may we please not be at odds here? I'm on your side.
I'm sorry, I didn't mean for that post to be so confrontational. I misread your post as shifting the goal-posts so that FileSystem wouldn't be acceptable unless it performs like your tuned-for-Banyan additions to FileDirectory.
There's certainly room to optimize the tree-walking code in Filesystem, so we may be able to meet your needs that way.
On the other hand, there are layers of indirection in Filesystem that aren't present in FileDirectory. Filesystem works with many kinds of directory trees—on disk, in-memory, inside a zip file, inside a git repository etc. It also has whole-tree operations that need to be able to customize the tree walking algorithm. For example, copying a tree needs to visit directories before their contents, while deleting a tree needs to visit directories after their contents. So from the point of view of Filesystem, #directoryTreeDo: is *not* a very general operation, it's quite specific, and tuned for a particular use-case. It may not be possible to optimize Filesystem's tree-walking code to the same level of memory efficiency without sacrificing generality.
But that's not a show-stopper! We could make a method like #directoryTreeDo: just for Banyan, or Banyan could keep on using FileDirectory. It's not like FileDirectory would be removed in 4.5, and even when it finally does get removed, it would still be available as a compatibility package.
Again, my apologies for the over-reaction.
Colin
On 27 May 2013 03:04, Colin Putney colin@wiresong.com wrote:
On Fri, May 24, 2013 at 1:58 AM, Frank Shearar frank.shearar@gmail.com wrote:
So I have a pair of questions:
- how far have wiresong's FileSystem and Pharo's FileSystem diverged?
(And how do we bring the two back together again, if necessary?)
I doubt there's much divergence. I synced with the Pharo folks as they were putting it in to their base image, and that was a pretty mature implementation, so I'd be surprised if there was anything difficult to deal with.
- How do we fix the problem?
Well, I'd be happy to see Filesystem in Squeak. :-)
I know Chris is worried about breaking Magma, so we'll need to be careful, but we can do that.
OK, so the first step is to add a CI build for FileSystem. I'm away from my image at the moment, but doesn't Installer have a #wiresong? If you supply me with an Installer script for FileSystem I'll wire up Jenkins.
One thing I've put off doing is getting Magma under CI. It has special ways of setting up tests that don't quite fit our existing CI test infrastructure. But once that's done, Magma would be an excellent guinea pig. (Of course, one could run the Magma test suite manually. I like the CI route because it keeps me honest: you can't fool a CI with accidental local state changes.)
frank
Colin
squeak-dev@lists.squeakfoundation.org
-
Casey Ransberger -
Chris Muller -
Chris Muller -
Colin Putney -
David T. Lewis -
Edgar J. De Cleene -
Frank Shearar -
H. Hirzel -
Juan Vuletich (mail lists) -
Ron Teitelbaum -
Sean P. DeNigris -
tim Rowledge