I have no interest in chopping on the great work being done by you and my other friends in Pharo, but that doesn't mean it is feasible for me to use it in my business. While someone in the Pharo community said FileSystem over FileDirectory is "huge", I see it as an incremental API change, and close to being a matter of preference.
incremental? do you think you can implement a memory-based and/or git-based filesystem or 'remotely connected database-based file system' by just doing incremental changes to FileDirectory? good luck with such 'increments' :)
Yes, of course those capabilities could be added simply with factoring work and little disruption to the API. That's the promise object technology, encapsulation.
Colin made a gorgeous domain model of an abstract FileSystem, no doubt about it. If we switch to it in Squeak I'm sure I will be happy once the conversion is done. I just think its worth asking, though, whether a "core" system should provide that rich a model out-of-the-box or instead just a bland, one-layer-above-the-primitives lightweight model ready to be easily wrapped by the user's _own_ rich model of a FileSystem. Is the core system suitable for tiny embedded programs and will they want a rich model or a basic one? Maybe Spoon will allow us to have our cake and eat it too..?
On 13 December 2012 03:59, Chris Muller asqueaker@gmail.com wrote:
I have no interest in chopping on the great work being done by you and my other friends in Pharo, but that doesn't mean it is feasible for me to use it in my business. While someone in the Pharo community said FileSystem over FileDirectory is "huge", I see it as an incremental API change, and close to being a matter of preference.
incremental? do you think you can implement a memory-based and/or git-based filesystem or 'remotely connected database-based file system' by just doing incremental changes to FileDirectory? good luck with such 'increments' :)
Yes, of course those capabilities could be added simply with factoring work and little disruption to the API. That's the promise object technology, encapsulation.
The main problem with FileDirectory that there's no API. I can hardly call a single class as API... as to me, it is clear that file system domain is a bit more complex and in order to reflect it properly in a language you will need more that a single role (FileDirectory).
So, i really doubt that you end up with 'little disruption to API'. It will be substantial.
Colin made a gorgeous domain model of an abstract FileSystem, no doubt about it. If we switch to it in Squeak I'm sure I will be happy once the conversion is done. I just think its worth asking, though, whether a "core" system should provide that rich a model out-of-the-box or instead just a bland, one-layer-above-the-primitives lightweight model ready to be easily wrapped by the user's _own_ rich model of a FileSystem. Is the core system suitable for tiny embedded programs and will they want a rich model or a basic one? Maybe Spoon will allow us to have our cake and eat it too..?
To me, design comes first. ALWAYS. All of the rest is secondary. FileSystem establishing a clear separation between conceptual model and implementation detail. The value of such improvement is hard to underestimate: users can learn fast & grasp well how to use it (and even understand better how file systems work). Moreover, it makes FileSystem nicely portable among dialects.. So, in future, if you would like to port your application to another dialect, you'll have to port FileSystem backend, instead of walking over your whole application and do patches all over the place.. Because once you have sound API, an effort to port application gravitates down to zero. Also, porting, in broad sense can mean 'porting from version X to version Y' (speaking about 'backwards compatimilimulability).
API establishing a set of rules in a domain & models. The point is that domain does not changes that often, comparing to implementation. That means , if you have a good design, you can be assured that it won't require serious changes and will serve you much much longer without need to change anything in userland code. (Take a Collection protocols for example). Also, once you have such set of rules, it is much easier for people to maintain implementation code, as well as adding extra functionality if needed.
Clearly, VM interfaces (such as primitives) should be not exposed to language 'as is', but always come with API. Because if you don't have rules, then you have very poor maintainability inside of system, as well as forcing users to reinvent their own wheel over and over again.. And next thing, if you would want one day to change primitives or change VM.. you will immediately hit the wall, with bloody 'backwards compatibility' sign on it. Because you will discover, that every single application implements own abstraction layer(s) (or even worse, use low-level functionality directly) on top of what you provided.
So, to me, your statement looks contradictory: from one side, you are seriously concerned about backwards compatibility, but from another one, you trying to ignore/discard the facts that you cannot have good backwards compatibility without having a design and API in a first place.
a little followup..
Please, do not even try to say that FileDirectory and FileSystem is interchangeable :)
Colin laid a foundation in a place where was no foundation at all. Yes, in Pharo we did some changes in order to 'make it work', polishing some rough corners, as well as writing tests and documenting the code.
Without laying foundation, nobody starts to build houses.. (well, some do.. but they do not stay long). We even teach kids about that: http://en.wikipedia.org/wiki/Three_Little_Pigs
On 12-12-2012, at 6:59 PM, Chris Muller asqueaker@gmail.com wrote:
I just think its worth asking, though, whether a "core" system should provide that rich a model out-of-the-box or instead just a bland, one-layer-above-the-primitives lightweight model ready to be easily wrapped by the user's _own_ rich model of a FileSystem.
My experience suggests that the smart approach is to build a core system that has the parts of the rich model that are needed to support the stuff you want in the core and to make sure that the rest can be easily and non-destructively added as part of building up the system. That both allows and discourages the roll-your-own-in-a-hurry; which seems a good compromise between encouraging smart re-use of available code and allowing the truly nuts to invent something even better on the odd occasions they don't utterly crater.
There *should* be a way to provide a compatibility layer of fake-FileDirectory nastiness over the top of a nice FileSystem. It really is about time a good clean design got to have a chance; there's been way too much crapulence in FileDirectory over the decades. I should know - I wrote a fairly large part of it, several times in fact, and always because nobody wanted to make a change to a good system.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Death to all fanatics!
Tim,
I like the points you raise.
On 12/13/12, tim Rowledge tim@rowledge.org wrote:
On 12-12-2012, at 6:59 PM, Chris Muller asqueaker@gmail.com wrote:
I just think its worth asking, though, whether a "core" system should provide that rich a model out-of-the-box or instead just a bland, one-layer-above-the-primitives lightweight model ready to be easily wrapped by the user's _own_ rich model of a FileSystem.
You mean,
a) Colin's FileSystem should be wrapped around FileDirectory? (The is no reference to FileDirectory from any of the new FS packages, This would maintain backward compatibility and at the same time give a more abstract view. )
Or
b) Colin's FileSystem wrapped around a low-level replacement of FileDirectory? plus a compatibility layer of top which gives back FileDirectory
My experience suggests that the smart approach is to build a core system that has the parts of the rich model that are needed to support the stuff you want in the core and to make sure that the rest can be easily and non-destructively added as part of building up the system.
Yes, indeed.
That both allows and discourages the roll-your-own-in-a-hurry; which seems a good compromise between encouraging smart re-use of available code and allowing the truly nuts to invent something even better on the odd occasions they don't utterly crater.
There *should* be a way to provide a compatibility layer of fake-FileDirectory nastiness over the top of a nice FileSystem.
Yes,
Marcus Denker mentions on the Pharo list that this might be an option for Pharo to do
http://forum.world.st/To-try-out-FileSystem-in-2-0-which-download-td4649945.... (full citation see below)
It really is about time a good clean design got to have a chance;
Yes, and the inclusion of Colin's FileSystem into Squeak is planned for 4.5 as I understand the minutes of the board meetings.
there's been way too much crapulence in FileDirectory over the decades.
Yes.
I should know - I wrote a fairly large part of it, several times in fact, and always because nobody wanted to make a change to a good system.
Grate to have you still around here.
In the meantime the attitude has changed and for now (or better early 2013) the addition of FileSystem is planned.
As of now just a SqueakMap Squeak 4.4 entry for the load script of FileSystem might do the job. Could anybody take this on please?
Colin writes that the FileSystem implementation he has been working on for many years (
http://wiresong.ca/filesystem/ ) is pretty close to the Pharo version which is a fork of it.
Colin and the Pharo programmers recently did a sync as Colin writes in http://lists.squeakfoundation.org/pipermail/squeak-dev/2012-November/166519....
A easyt to read tutorial for FileSystem is here http://pharobooks.gforge.inria.fr/PharoByExampleTwo-Eng/latest/FileSystem.pd...
To load FileSystem into Squeak
(Installer wiresong project: 'mc') addPackage: 'FS-Core'; addPackage: 'FS-Disk'; addPackage: 'FS-Memory'; addPackage: 'FS-AnsiStreams'; addPackage: 'FS-Zip'; "addPackage: 'FS-Xtreams';" addPackage: 'FS-FileStream'; addPackage: 'FS-Tests-Core'; addPackage: 'FS-Tests-AnsiStreams'; " addPackage: 'FS-Tests-Xtreams'; " addPackage: 'FS-Tests-Zip'; addPackage: 'FS-Tests-Disk'; addPackage: 'FS-Tests-FileStream'; install.
does the job (script taken from http://lists.squeakfoundation.org/pipermail/squeak-dev/2012-
September/165822.html)
Two packages which depend on the presence of Xtreams are not loaded. As for this thread Xtreams is a different issue.
FileSystem Testrunner results 300 tests, 297 passes, 3 errors.
--Hannes
P.S. Here the citation of Marcus' post
about "libraries as theories"
http://forum.world.st/To-try-out-FileSystem-in-2-0-which-download-td4649945....
(full citation see below)
<citation> Libraries are in the end theories. They model something, e.g. Filesystem access.
The idea here is the following: Libraries are in the end theories. They model something, e.g. Filesystem access. There was an old theory, and it turned out it was not really good. So Colin made a new Theory: FileSystem.
Now there are two philosophies
1) we load the new in addition. It even is compatible to many different versions of the System, and everyone uses it in new code. But the system can not use it, as the developer of course wants to load the latest version of this independently developed library. Some people requested that for FileSystem, and we renamed all classes so that they can continue to load the code themselves.
2) We replace the old. Why?
- to prove the theory. It is amazing, but in all cases, the new needs to be either slightly adapted / improved or bug fixed when you *really* replace the old. It's the ultimate test. - to only have to maintain one code base. - to have a better used and therefore more bug free code base (see e.g. the FilePlugin fixes that had to be done)
and, actually not last but very important: Have examples in the image that show how the library is used. In the end, the system is it's own best documentation. There should only be code in there that shows the beginner how he is supossed to write new code.
</citation>
Minor critique of one statement in this -
On 13-12-2012, at 4:37 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
P.S. Here the citation of Marcus' post [snip]
- to have a better used and therefore more bug free code base (see
e.g. the FilePlugin fixes that had to be done)
I'd suggest a completely new file related plugin - or even split it up into several, possibly? - and leave the old one alone. It could be expensively confusing to have a plugin of the same name intended for a dramatically different system. Far easier to explain to some suffering user that they need to fetch a new version of FileSystemPlugin than FilePlugin but don't forget it has to be from this directory on this site and check this detail….
Whilst the subject is at the front of my mind, let's remember that there is no real need to have identical plugins and low-level image code for all platforms. There is no particular good in having a single class calling a plugin and that plugin having to do large amounts of platform specific mangling in order to conform. Consider the potential for class Blargh having proxies (or bridges, or subclasses, to taste, they can all work) BlarghForWindowsFAT32, BlarghForUnixZFS, BlarghForRISCOSADFS, BlarghForRISCOSNetFS, etc etc. You get to do complicated things in a language that is good for complicated things and pass calls down to simpler, platform/OS/filesystem specific, plugins.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: JSP: Jump on Sexy Programmer
On Thu, Dec 13, 2012 at 6:34 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, and the inclusion of Colin's FileSystem into Squeak is planned for 4.5 as I understand the minutes of the board meetings.
Well, it was proposed, and a few board members were receptive to the idea, but that's not a final decision. Just to be clear, the Board's job is to manage the administrivia of community infrastructure, like servers and mailing lists. Setting the development agenda falls the community at large.
I think the best way to introduce Filesystem in the base image would be to include just the core packages (FS-Core, FS-Disk, FS-Tests-Core and FS-Tests-Disk), and leave the others as optional, loadable additions. That core would be sufficient to cover all the functionality provided by FileDirectory, and parts of the system that deal with files can be gradually migrated over.
I wrote a couple of paragraphs on why it wouldn't make sense to have an implementation FileDirectory as a layer on top of Filesystem, but then I came to the conclusion that it might not be a bad idea after all. The devil is in the details, and we ought to look into the details. Either way, the current implementation of FileDirectory should eventually be available as a loadable package, so that we can still build images with legacy code that depends on it.
Also, I wanted to point out that FS-AnsiStreams is basically obsolete. I wrote it because FileStream and friends couldn't be adapted to work with non-disk filesystems, and I wanted to have a single, uniform interface for reading and writing files, regardless of the type of filesystem that contained them. But then I discovered that Xtreams already does a way better job of that than anything I could have written. There might be some value in FS-AnsiStreams as an ANSI-compliant portability interface, but AFAICT, no dialect supports this part of the ANSI standard correctly anyway, so it wouldn't be much use.
FS-FileStream is more useful, but it's only meant as a stepping stone. It lets existing applications use Filesystem for navigating around the directory hierarchy (which should be a fairly easy port from FileDirectory) while continuing to use FileStream for IO. It provides a FileStream-like interface for IO operations on non-disk files. (Ultimately, a modern application that does a lot of IO should use Xtreams, though. It's a tremendous improvement over FileStream, SocketStream etc.)
So really, the optional bits are FS-Memory, FS-Zip, FS-Git (by Max Leske and Camillo Buni), and FS-Xtreams.
Also, to address Chris' point about tiny images, it would be possible to split FS-Core into two packages, one with just the minimal functionality and then another with the advanced features. I don't think that's a high priority, since we don't have a core image of the size where FS-Core would make a difference. But If/when we do get there, I don't think Filesystem will be an obstacle.
Colin
On 13 December 2012 14:52, Colin Putney colin@wiresong.com wrote:
On Thu, Dec 13, 2012 at 6:34 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
Yes, and the inclusion of Colin's FileSystem into Squeak is planned for 4.5 as I understand the minutes of the board meetings.
Well, it was proposed, and a few board members were receptive to the idea, but that's not a final decision. Just to be clear, the Board's job is to manage the administrivia of community infrastructure, like servers and mailing lists. Setting the development agenda falls the community at large.
I think the best way to introduce Filesystem in the base image would be to include just the core packages (FS-Core, FS-Disk, FS-Tests-Core and FS-Tests-Disk), and leave the others as optional, loadable additions. That core would be sufficient to cover all the functionality provided by FileDirectory, and parts of the system that deal with files can be gradually migrated over.
I wrote a couple of paragraphs on why it wouldn't make sense to have an implementation FileDirectory as a layer on top of Filesystem, but then I came to the conclusion that it might not be a bad idea after all. The devil is in the details, and we ought to look into the details. Either way, the current implementation of FileDirectory should eventually be available as a loadable package, so that we can still build images with legacy code that depends on it.
Also, I wanted to point out that FS-AnsiStreams is basically obsolete. I wrote it because FileStream and friends couldn't be adapted to work with non-disk filesystems, and I wanted to have a single, uniform interface for reading and writing files, regardless of the type of filesystem that contained them. But then I discovered that Xtreams already does a way better job of that than anything I could have written. There might be some value in FS-AnsiStreams as an ANSI-compliant portability interface, but AFAICT, no dialect supports this part of the ANSI standard correctly anyway, so it wouldn't be much use.
And that's pretty much most of the reason for Martin & Mark writing Xtreams - noone implemented streams in quite the same way (and there were things that were just wrong in the spec, but can't recall the details).
frank
Colin Putney-3 wrote
I wrote a couple of paragraphs on why it wouldn't make sense to have an implementation FileDirectory as a layer on top of Filesystem, but then I came to the conclusion that it might not be a bad idea after all. The devil is in the details, and we ought to look into the details. Either way, the current implementation of FileDirectory should eventually be available as a loadable package, so that we can still build images with legacy code that depends on it.
+1. Sounds much better than trying to build a compatible version.
Cheers, - Andreas
-- View this message in context: http://forum.world.st/FileSystem-was-About-backwards-Compatibility-tp4659123... Sent from the Squeak - Dev mailing list archive at Nabble.com.
On 13-12-2012, at 3:34 AM, H. Hirzel hannes.hirzel@gmail.com wrote:
You mean,
a) Colin's FileSystem should be wrapped around FileDirectory? (The is no reference to FileDirectory from any of the new FS packages, This would maintain backward compatibility and at the same time give a more abstract view. )
Or
b) Colin's FileSystem wrapped around a low-level replacement of FileDirectory? plus a compatibility layer of top which gives back FileDirectory
Not quite - a) use FileSystem b) write a new FileDirectory class that provides the same API but redirects everything through FileSystem. Plus, obviously, doing occasional transformations, mangling of filename strings, etc to keep the experience authentic. c) beat up users of FileDirectory to teach them sense.
tim -- tim Rowledge; tim@rowledge.org; http://www.rowledge.org/tim Strange OpCodes: ESR: Emulate Slide Rule
Hi Chris--
I just think its worth asking, though, whether a "core" system should provide that rich a model out-of-the-box or instead just a bland, one-layer-above-the-primitives lightweight model ready to be easily wrapped by the user's _own_ rich model of a FileSystem. Is the core system suitable for tiny embedded programs and will they want a rich model or a basic one? Maybe Spoon will allow us to have our cake and eat it too..?
Yes, they're both straightforward use cases with Naiad modules. And the Flow filesystem support is an option, too.
-C
-- Craig Latta www.netjam.org/resume +31 6 2757 7177 + 1 415 287 3547 (no SMS)
squeak-dev@lists.squeakfoundation.org
-
Andreas.Raab -
Chris Muller -
Colin Putney -
Craig Latta -
Frank Shearar -
H. Hirzel -
Igor Stasenko -
tim Rowledge