Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
On Thu, Feb 14, 2019 at 3:17 PM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Jakob is working on a Tonel port, but I don't know more about it.
Fabio
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
Yes that is true, I have a fork of the Pharo Tonel repository, but I don't work on it often because I do not use it myself.
The current state is that you can have Tonel repositories in Monticello and they handle like FileTree repositories without metadata. Which means pretty awkward, since you cannot access any history from the tools. Moreover there are no method timestamps because Tonel does not store them.
Here it is: https://github.com/j4yk/tonel/tree/squeak I see that I have not adapted the readme file for the proper install procedure yet... sorry.
The plan is to integrate the serializer/deserializer with the Git tooling that you can obtain from the Do menu since Squeak 5.2, which should vastly improve the usability.
Am Do., 14. Feb. 2019, 16:54 hat Fabio Niephaus lists@fniephaus.com geschrieben:
On Thu, Feb 14, 2019 at 3:17 PM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Jakob is working on a Tonel port, but I don't know more about it.
Fabio
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
I think that we should at least support the input/output of tonel packages for inter-operability. Since the git repositories are metadataless, we can't do much more without a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole history to git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then it's currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which at least gives a few sync point in history...).
If we want to perform essential operations like commit/pull/merge or access essential information as history/revisions/diffs then we have to - either reproduce the very hard work that began in Pharo: program a git client inside the image, - or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools: - they scale well even for massive changes (i did check with auto generated Smallapack interface) - they support cherry-picking: you see and control what you commit - they are quasi bugfree - the magma backend provides a few additional features (revisions...) Pharo team has produced huge effort for the github switch, but it does not go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak able to take this major turn in near future.
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing the .last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible, and leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember the discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 févr. 2019 à 15:17, Torsten Bergmann astares@gmx.de a écrit :
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel packages for inter-operability. Since the git repositories are metadataless, we can't do much more without a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole history to git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then it's currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which at least gives a few sync point in history...).
If we want to perform essential operations like commit/pull/merge or access essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does not go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak able to take this major turn in near future.
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing the .last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible, and leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember the discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a ??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3] https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
With these modest requirements, David, please try the port and open issues for anything that bugs you about it.
Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis lewis@mail.msen.com geschrieben:
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel
packages
for inter-operability. Since the git repositories are metadataless, we can't do much more
without
a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole history
to
git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then
it's
currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which at least gives a few sync point in history...).
If we want to perform essential operations like commit/pull/merge or
access
essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto
generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does
not
go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak
able
to take this major turn in near future.
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing the .last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible,
and
leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember
the
discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a
??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format
for
monticello repositories [1].
In Pharo land many projects are already converted to it - as regular
"file
tree" solution faces issues with git handling due to very long file names / deep file hierarchy
(often
leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE
(VisualWorks
Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3]
https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
On Fri, Feb 15, 2019 at 8:58 AM Jakob Reschke forums.jakob@resfarm.de wrote:
With these modest requirements, David, please try the port and open issues for anything that bugs you about it.
Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis lewis@mail.msen.com geschrieben:
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I believe Jakob's Git integration (see below) can also be controlled from within a workspace.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel
packages
for inter-operability. Since the git repositories are metadataless, we can't do much more
without
a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole
history to
git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then
it's
currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which
at
least gives a few sync point in history...).
Agreed. Too bad Tonel has become the default format in Pharo already. If we want to support it, we also have to live with it's flaws...
If we want to perform essential operations like commit/pull/merge or
access
essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto
generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does
not
go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak
able
to take this major turn in near future.
Squeak's MC tools are great and I also thought Squeak can't support Git any time soon. It turns out Jakob has already done most of the heavy lifting as he worked on a Smalltalk implementation of Git. It even comes with a Sourcetree-like UI which supports the most important operations already. In case you haven't seen it, you can install it via "Do" > "Installer installGitInfrastructure" (Squeak-5.2 or later). You can then find it's UI under "Apps" > "Git Browser". Please file bugs at [1]. Unfortunately, the bus factor of this integration is still quite low, but I'd like to see more people working on it.
Fabio
[1] https://github.com/hpi-swa/Squot/issues
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing
the
.last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible,
and
leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember
the
discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a
??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format
for
monticello repositories [1].
In Pharo land many projects are already converted to it - as regular
"file
tree" solution faces issues with git handling due to very long file names / deep file hierarchy
(often
leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE
(VisualWorks
Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3]
https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
Le ven. 15 févr. 2019 à 09:45, Fabio Niephaus lists@fniephaus.com a écrit :
On Fri, Feb 15, 2019 at 8:58 AM Jakob Reschke forums.jakob@resfarm.de wrote:
With these modest requirements, David, please try the port and open issues for anything that bugs you about it.
Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis lewis@mail.msen.com geschrieben:
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I believe Jakob's Git integration (see below) can also be controlled from within a workspace.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel
packages
for inter-operability. Since the git repositories are metadataless, we can't do much more
without
a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole
history to
git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then
it's
currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which
at
least gives a few sync point in history...).
Agreed. Too bad Tonel has become the default format in Pharo already. If we want to support it, we also have to live with it's flaws...
I think that Pharo is on the right path: if you want to be a mainstream language, you cannot always swim against the tide. In this battle, Pharo can't afford waiting for a better solution tomorrow, they prefer a good enough solution today. Perfect MC tools that would not integrate in github development (code review, issue tracker, test bots on feature branches/pull requests, ...) is not interesting them. It's a recipe for being categorized as bizarre and outdated.
It's just that Squeak goals are a bit different. So yes, we have to live with it... Maybe it is time to refresh what these Squeak goals/directions are (based on last year inquiry?)
If we want to perform essential operations like commit/pull/merge or
access
essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a
git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto
generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does
not
go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak
able
to take this major turn in near future.
Squeak's MC tools are great and I also thought Squeak can't support Git any time soon. It turns out Jakob has already done most of the heavy lifting as he worked on a Smalltalk implementation of Git. It even comes with a Sourcetree-like UI which supports the most important operations already. In case you haven't seen it, you can install it via "Do" > "Installer installGitInfrastructure" (Squeak-5.2 or later). You can then find it's UI under "Apps" > "Git Browser". Please file bugs at [1]. Unfortunately, the bus factor of this integration is still quite low, but I'd like to see more people working on it.
Fabio
I'll be very happy to be proven wrong about near future :) I wish I had more time to support Squot... But even with very little time, the least we can do is to try it, report problems, make suggestions. Last time I checked, it was good enough for converting single package history to git, so we can do it. I did not take time to play with git integration though. A bit late for new year resolutions, but I promise I'll try again soon ;)
If we want to have mix development with packages maintained both in a
MC
repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing
the
.last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible,
and
leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use
such
important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember
the
discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a
??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format
for
monticello repositories [1].
In Pharo land many projects are already converted to it - as regular
"file
tree" solution faces issues with git handling due to very long file names / deep file hierarchy
(often
leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE
(VisualWorks
Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3]
https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
Thank you, I will try it.
Sorry - I overlooked your earlier message with the link:
https://github.com/j4yk/tonel/tree/squeak
Thanks,
Dave
On Fri, Feb 15, 2019 at 08:58:16AM +0100, Jakob Reschke wrote:
With these modest requirements, David, please try the port and open issues for anything that bugs you about it.
Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis lewis@mail.msen.com geschrieben:
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel
packages
for inter-operability. Since the git repositories are metadataless, we can't do much more
without
a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole history
to
git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then
it's
currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which at least gives a few sync point in history...).
If we want to perform essential operations like commit/pull/merge or
access
essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto
generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does
not
go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak
able
to take this major turn in near future.
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing the .last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible,
and
leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember
the
discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a
??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format
for
monticello repositories [1].
In Pharo land many projects are already converted to it - as regular
"file
tree" solution faces issues with git handling due to very long file names / deep file hierarchy
(often
leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE
(VisualWorks
Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3]
https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
I am travelling and will not be able to look at this further for a day or two, but just for initial feedback - I loaded the MonticelloTonel packages in my Squeak trunk image, and was able to open NC tools and browse my local cloned repository normally.
This looks really good, thank you :-)
Dave
On Fri, Feb 15, 2019 at 09:02:54AM -0500, David T. Lewis wrote:
Thank you, I will try it.
Sorry - I overlooked your earlier message with the link:
https://github.com/j4yk/tonel/tree/squeakThanks,
Dave
On Fri, Feb 15, 2019 at 08:58:16AM +0100, Jakob Reschke wrote:
With these modest requirements, David, please try the port and open issues for anything that bugs you about it.
Am Fr., 15. Feb. 2019, 02:14 hat David T. Lewis lewis@mail.msen.com geschrieben:
Hi Torsten,
Thank you very much for asking about this, and thanks to Nicolas for the explanations.
As a package maintainer (OSProcess and others), I want to be able to provide continued support for Pharo users, and it would probably be helpful if I could read and write to Tonel as an external format. I am also interested in being able to browse and read the projects (many of them quite excellent) that are being developed in Pharo from my Squeak image.
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Thanks again for asking,
Dave
On Thu, Feb 14, 2019 at 11:15:23PM +0100, Nicolas Cellier wrote:
I think that we should at least support the input/output of tonel
packages
for inter-operability. Since the git repositories are metadataless, we can't do much more
without
a lot of work.
Since tonel has no support for timestamps, it looses authorship (it is replaced by committer-ship). If we don't want to loose authorship, then we need to port whole history
to
git with known initials <-> git(hub) account database. This is currently working for single package with complex graph (to filetree format, but could work for Tonel too). If you want to commit several MC packages in same git repository, then
it's
currently impossible to support complex history. There is generally no information that enables inter-MC-package synchronization (unless some sort of MCConfigurationMap is used, which at least gives a few sync point in history...).
If we want to perform essential operations like commit/pull/merge or
access
essential information as history/revisions/diffs then we have to
- either reproduce the very hard work that began in Pharo: program a git
client inside the image,
- or loose inside image tools.
Squeak MC tools are much more productive than Pharo MC tools:
- they scale well even for massive changes (i did check with auto
generated
Smallapack interface)
- they support cherry-picking: you see and control what you commit
- they are quasi bugfree
- the magma backend provides a few additional features (revisions...)
Pharo team has produced huge effort for the github switch, but it does
not
go without pain. Despite the importance of being visible on github, and profit by state of the art web collaborating tools, i don't see Squeak
able
to take this major turn in near future.
If we want to have mix development with packages maintained both in a MC repository and a git metadataless repository, then we have the problem of finding a common ancestor. I think that this could be doable with some conventions (like storing the .last_known_mc_ancestor in a file) Of course, it's re-introduction of metadata, but the minimal possible,
and
leads to easy to resolve conflicts (that can be automated)...
The last grief I have with Tonel is that it is not syntax agnostic. Since we have a language where we can define compilerClass/parserClass, that's unfortunate (The application i developed in the 90's did use such important feature, it's not just theoretical) It could have been language agnostic with very little effort (remember
the
discussion on Pharo-dev, just using an arbitrary number of [[[ ]]] as method body separators would suffice) For most packages, that will not be a problem though.
Le jeu. 14 f??vr. 2019 ?? 15:17, Torsten Bergmann astares@gmx.de a
??crit :
Hi,
as some of you might already know "Tonel" is a file-per-class format
for
monticello repositories [1].
In Pharo land many projects are already converted to it - as regular
"file
tree" solution faces issues with git handling due to very long file names / deep file hierarchy
(often
leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE
(VisualWorks
Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Thx T.
[1] https://github.com/pharo-vcs/tonel [2] https://github.com/GemTalk/Rowan [3]
https://pharoweekly.wordpress.com/2019/01/20/ann-sett-store-export-to-tonel-...
I can share a bit my experience with tonel and Pharo. I have been using Tonel on Pharo now for DrGeo code. However I don't use git for several reasons, I use Launchpad/Bazzar. So it will be a bit the same situation as Squeak with Tonel content commited to GIT repo.
So all in all from the image I don't have access to versioning, I have to do it externally from the bzr line command or Launchpad web interface[1] and it is pretty nice. So it is not very different when developing with other system.
This is so far an acceptable trade-off, but I am so far only one developer on this repository.
Hilaire
[1] https://bazaar.launchpad.net/~drgeo-developers/drgeo/trunk/files/head:/src/
Le 15/02/2019 à 02:14, David T. Lewis a écrit :
I am a big fan of git, but I do not care about integrating it with the image (Squeak/Pharo) because I prefer using the in-image tools in Squeak, and when I do use git, I prefer to use git tools directly.
I am not well informed about this topic, but overall I would say that being able to read and write Tonel format from Squeak would be very helpful, and I would probably not care very much about version control integration.
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
Hi Eliot, hi all,
(reducing the addressees to squeak-dev again because I suppose the Pharo or VM devs are not that much interested in the Tonel port)
Am So., 17. Feb. 2019 um 21:27 Uhr schrieb Eliot Miranda < eliot.miranda@gmail.com>:
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
Sorry to hear that. I understand your frustration.
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
For anybody who would like to work on this: IIRC the problem is that if you try to load a Tonel project with Metacello, it attempts to load it like a FileTree project, which obviously fails. My as-yet-unverified theory is that there is something in Gofer or about the github:// Monticello repository code that hardcodes the use of FileTree instead of any other reader. If that is correct, one must find this code and extend it to look out for Tonel repositories and use the TonelReader when required. Maybe there are some changes to cherry-pick from Pharo.
Personally, I am very fond of the use of Git instead of Monticello, but not convinced of Tonel so far. Yet we can expect Tonel to be something we will have to live with for some time (just like Monticello's overreliance on unique version names seen in other current threads, though the latter is an old problem and the adoption of Tonel is a new one). And I would not like to see the pool of available libraries for Squeak become even more limited because of the lack of support for a file-out format. I have spent my free time on Squeak only sporadically during the past year. As much as I would have liked to help Eliot further, it didn't appeal to me to spend a free evening or weekend digging through Metacello's code.
Kind regards, Jakob
On Sun, 17 Feb 2019, Jakob Reschke wrote:
Hi Eliot, hi all, (reducing the addressees to squeak-dev again because I suppose the Pharo or VM devs are not that much interested in the Tonel port)
Am So., 17. Feb. 2019 um 21:27 Uhr schrieb Eliot Miranda eliot.miranda@gmail.com: I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance".
Sorry to hear that. I understand your frustration. So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
For anybody who would like to work on this: IIRC the problem is that if you try to load a Tonel project with Metacello, it attempts to load it like a FileTree project, which obviously fails. My as-yet-unverified theory is that there is something in Gofer or about the github:// Monticello repository code that hardcodes the use of FileTree instead of any other reader. If that is correct, one must find this code and extend it to look out for Tonel repositories and use the TonelReader when required. Maybe there are some changes to cherry-pick from Pharo.
Personally, I am very fond of the use of Git instead of Monticello, but not convinced of Tonel so far. Yet we can expect Tonel to be something we will have to live with for some time (just like Monticello's overreliance on unique version names seen in other current threads, though the latter is an old problem and the adoption of Tonel is a new one). And I would not like to see the pool of available libraries for Squeak become even more limited because of the lack of support for a file-out format. I have spent my free time on Squeak only sporadically during the past year. As much as I would have liked to help Eliot further, it didn't appeal to me to spend a free evening or weekend digging through Metacello's code.
Is it a must to use Metacello to load code in Tonel format? What does Metacello add to the mix? Can't we just map each git commit to an mcz?
Levente
Kind regards, Jakob
Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi leves@caesar.elte.hu geschrieben:
Is it a must to use Metacello to load code in Tonel format? What does Metacello add to the mix? Can't we just map each git commit to an mcz?
Levente
Strictly speaking, Metacello is not required. But you would have to resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of Metacello. So I understand that Eliot wants it to work.
There is no mcz to map to. If one were to create a new kind of MCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch multiple packages or none. Each commit describes a particular configuration of package versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser already supports these scenarios for FileTree repositories, albeit without using Monticello for the version control.
On Mon, 18 Feb 2019, Jakob Reschke wrote:
Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi leves@caesar.elte.hu geschrieben:
Is it a must to use Metacello to load code in Tonel format? What does Metacello add to the mix? Can't we just map each git commit to an mcz? LeventeStrictly speaking, Metacello is not required. But you would have to resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of Metacello. So I understand that Eliot wants it to work.
There is no mcz to map to. If one were to create a new kind of MCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch multiple packages or none. Each commit describes a particular configuration of package versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser already supports these scenarios for FileTree repositories, albeit without using Monticello for the version control.
Looking at the repository in question[1], there seem to be multiple packages stored in it[2]: BaselineOfScorch, Scorching, ScorchingDev, ScorchingTests, ScorchingVMTests. My impression is that these could be converted to individual mczs for each commit, which would mean that the same code would be available as MC packages. Then you could load the code with MC/Metacello/whatever from the .mczs.
Am I missing something?
Levente
[1] https://github.com/clementbera/Scorch [2] https://github.com/clementbera/Scorch/tree/master/repository
Yes, it is feasible like that, but not ideal. Not all of these packages are changed with every commit, so you would generate multiple consecutive MCVersions of a package with the unchanged contents in comparison to their immediate ancestor and a log message that does not pertain to the package at all.
Since Squot has already solved this once, the fundamental question is how much effort someone wants to invest to backport features to Monticello and whether the effort is desired and warranted. I've heard multiple voices that do not want to go back to Monticello. And I'm sure there are at least as many on the list that do not want to let it go. ;-) Side note: Squot uses the Monticello model for packages (i. e., MCSnapshot and MCPatch and their contents, and the loader facilities), but it does not use MCVersion or MCRepository.
In any case, you would need something to access the Git history and object database. In Pharo this is the libgit2 binding. In Squot it is FileSystem-Git, which could also be reused to write something for Monticello. But since it is coupled with FileSystem, the next discussion will be whether that should finally be adopted into Trunk, and if so in which form. Again, I've heard multiple proponents to make this move.
After that, the next problem will be that the current MCRepository hierarchy couples file formats and ancestry storage. In the case of Monticello-in-another-VCS, these are orthogonal: you can have FileTree layouts in Git, you can have Tonel layouts in Git, you could have both at the same time, you could have both in a different VCS in the future... So instead of a "FileTree"-Repository and a "Tonel"-Repository, I think there should rather be a "Git"-Repository that picks the correct readers and writers dynamically to process the files and directories in each commit. To make the smell more tangible: I think neither MCFileTreeRepository (or whathever its name actually is) nor TonelRepository should be subclasses of MCFileBasedRepository (they are today), although it sounds crazy.
Am Mo., 18. Feb. 2019 um 11:45 Uhr schrieb Levente Uzonyi < leves@caesar.elte.hu>:
On Mon, 18 Feb 2019, Jakob Reschke wrote:
Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi leves@caesar.elte.hu
geschrieben:
Is it a must to use Metacello to load code in Tonel format? What does Metacello add to the mix? Can't we just map each git commit to an mcz? LeventeStrictly speaking, Metacello is not required. But you would have to
resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of Metacello. So I
understand that Eliot wants it to work.
There is no mcz to map to. If one were to create a new kind of
MCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch multiple packages
or none. Each commit describes a particular configuration of package
versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser already supports
these scenarios for FileTree repositories, albeit without using
Monticello for the version control.
Looking at the repository in question[1], there seem to be multiple packages stored in it[2]: BaselineOfScorch, Scorching, ScorchingDev, ScorchingTests, ScorchingVMTests. My impression is that these could be converted to individual mczs for each commit, which would mean that the same code would be available as MC packages. Then you could load the code with MC/Metacello/whatever from the .mczs.
Am I missing something?
Levente
[1] https://github.com/clementbera/Scorch [2] https://github.com/clementbera/Scorch/tree/master/repository
On Mon, 18 Feb 2019, Jakob Reschke wrote:
Yes, it is feasible like that, but not ideal. Not all of these packages are changed with every commit, so you would generate multiple consecutive MCVersions of a package with the unchanged contents in comparison to their immediate ancestor and a log message that does not pertain to the package at all.
Well, that's just the question of writing the tool. It could easily detect that some commit doesn't modify package X, so there's no need to create a new version of it.
Since Squot has already solved this once, the fundamental question is how much effort someone wants to invest to backport features to Monticello and whether the effort is desired and warranted. I've
That is indeed a question. It seemed to me that even if just temporarily, having .mcz versions would solve Eliot's problem.
heard multiple voices that do not want to go back to Monticello. And I'm sure there are at least as many on the list that do not want to let it go. ;-) Side note: Squot uses the Monticello model for packages (i. e., MCSnapshot and MCPatch and their contents, and the loader facilities), but it does not use MCVersion or MCRepository.
MC is far from perfect. It's a "quick hack". It doesn't scale well. And there were bad design decisions made. But it still works, and many toolchains rely on it, e.g. the Trunk or VMMaker.
In any case, you would need something to access the Git history and object database. In Pharo this is the libgit2 binding. In Squot it is FileSystem-Git, which could also be reused to write something
What I had in my mind was a lot simpler: watch a repository for new commits. When a new commit appears, fetch it, load the tonel code into an image and create the new .mczs. Store the new .mczs in a repository. Repeat.
for Monticello. But since it is coupled with FileSystem, the next discussion will be whether that should finally be adopted into Trunk, and if so in which form. Again, I've heard multiple proponents to make this move.
I'm fine with moving to FileSystem, provided it is ready for production. I see two possible ways to achieve that: 1. create a layer on top of FileSystem which works like FileDirectory, then release it and deprecate it. :) 2. add FileSystem to the Trunk and gradually migrate all uses of FileDirectory to it.
After that, the next problem will be that the current MCRepository hierarchy couples file formats and ancestry storage. In the case of Monticello-in-another-VCS, these are orthogonal: you can have FileTree layouts in Git, you can have Tonel layouts in Git, you could have both at the same time, you could have both in a different VCS in the future... So instead of a "FileTree"-Repository and a "Tonel"-Repository, I think there should rather be a "Git"-Repository that picks the correct readers and writers dynamically to process the files and directories in each commit. To make the smell more tangible: I think neither MCFileTreeRepository (or whathever its name actually is) nor TonelRepository should be subclasses of MCFileBasedRepository (they are today), although it sounds crazy.
I don't think it is a good idea to mix MC and git at all. IMO we only need tools to safely and reliably migrate from one to the other to start a change.
Levente
Am Mo., 18. Feb. 2019 um 11:45 Uhr schrieb Levente Uzonyi leves@caesar.elte.hu: On Mon, 18 Feb 2019, Jakob Reschke wrote:
> Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi <leves@caesar.elte.hu> geschrieben: > > Is it a must to use Metacello to load code in Tonel format? > What does Metacello add to the mix? > Can't we just map each git commit to an mcz? > > Levente > > > Strictly speaking, Metacello is not required. But you would have to resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of Metacello. So I > understand that Eliot wants it to work. > > There is no mcz to map to. If one were to create a new kind of MCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch multiple packages > or none. Each commit describes a particular configuration of package versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser already supports > these scenarios for FileTree repositories, albeit without using Monticello for the version control. Looking at the repository in question[1], there seem to be multiple packages stored in it[2]: BaselineOfScorch, Scorching, ScorchingDev, ScorchingTests, ScorchingVMTests. My impression is that these could be converted to individual mczs for each commit, which would mean that the same code would be available as MC packages. Then you could load the code with MC/Metacello/whatever from the .mczs. Am I missing something? Levente [1] https://github.com/clementbera/Scorch [2] https://github.com/clementbera/Scorch/tree/master/repository > >
In any case, you would need something to access the Git history and object database. In Pharo this is the libgit2 binding. In Squot it is FileSystem-Git, which could also be reused to write something
What I had in my mind was a lot simpler: watch a repository for new commits. When a new commit appears, fetch it, load the tonel code into an image and create the new .mczs. Store the new .mczs in a repository. Repeat.
for Monticello. But since it is coupled with FileSystem, the next discussion will be whether that should finally be adopted into Trunk, and if so in which form. Again, I've heard multiple proponents to make this move.
I'm fine with moving to FileSystem, provided it is ready for production. I see two possible ways to achieve that:
- create a layer on top of FileSystem which works like FileDirectory,
then release it and deprecate it. :)
That's backwards. FileDirectory is the smaller, faster, lower-level, more-utilitarian framework. FileSystem is a "boutique" domain and framework with powerful high-level capabilities, but suffers from a performance-killing design issue. So it'd probably make more sense for FileSystem use FileDirectory as its "primitive", and then we can simply keep FileDirectory around as-is for legacy apps.
- add FileSystem to the Trunk and gradually migrate all uses of
FileDirectory to it.
This would be asking virtually every app to do a similar migration. All because of Squot?
3. Port Squot to use FileDirectory.
4. Don't port anything. Side-by-side existence. People who need Squot simply load Filesystem too.
After that, the next problem will be that the current MCRepository hierarchy couples file formats and ancestry storage. In the case of Monticello-in-another-VCS, these are orthogonal: you can have FileTree layouts in Git, you can have Tonel layouts in Git, you could have both at the same time, you could have both in a different VCS in the future... So instead of a "FileTree"-Repository and a "Tonel"-Repository, I think there should rather be a "Git"-Repository that picks the correct readers and writers dynamically to process the files and directories in each commit. To make the smell more tangible: I think neither MCFileTreeRepository (or whathever its name actually is) nor TonelRepository should be subclasses of MCFileBasedRepository (they are today), although it sounds crazy.
I don't think it is a good idea to mix MC and git at all. IMO we only need tools to safely and reliably migrate from one to the other to start a change.
Levente
Am Mo., 18. Feb. 2019 um 11:45 Uhr schrieb Levente Uzonyi leves@caesar.elte.hu: On Mon, 18 Feb 2019, Jakob Reschke wrote:
> Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi <leves@caesar.elte.hu> geschrieben: > > Is it a must to use Metacello to load code in Tonel format? > What does Metacello add to the mix? > Can't we just map each git commit to an mcz? > > Levente > > > Strictly speaking, Metacello is not required. But you would have to resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of Metacello. So I > understand that Eliot wants it to work. > > There is no mcz to map to. If one were to create a new kind of MCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch multiple packages > or none. Each commit describes a particular configuration of package versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser already supports > these scenarios for FileTree repositories, albeit without using Monticello for the version control. Looking at the repository in question[1], there seem to be multiple packages stored in it[2]: BaselineOfScorch, Scorching, ScorchingDev, ScorchingTests, ScorchingVMTests. My impression is that these could be converted to individual mczs for each commit, which would mean that the same code would be available as MC packages. Then you could load the code with MC/Metacello/whatever from the .mczs. Am I missing something? Levente [1] https://github.com/clementbera/Scorch [2] https://github.com/clementbera/Scorch/tree/master/repository > >
Hi Chris,
On Mon, Feb 18, 2019 at 10:53 AM Chris Muller asqueaker@gmail.com wrote:
In any case, you would need something to access the Git history and
object database. In Pharo this is the libgit2 binding. In Squot it is FileSystem-Git, which could also be reused to write something
What I had in my mind was a lot simpler: watch a repository for new commits. When a new commit appears, fetch it, load the tonel code into an image and create the new .mczs. Store the new .mczs in a repository. Repeat.
for Monticello. But since it is coupled with FileSystem, the next
discussion will be whether that should finally be adopted into Trunk, and if so in which form. Again, I've heard multiple proponents
to make this move.
I'm fine with moving to FileSystem, provided it is ready for production. I see two possible ways to achieve that:
- create a layer on top of FileSystem which works like FileDirectory,
then release it and deprecate it. :)
That's backwards. FileDirectory is the smaller, faster, lower-level, more-utilitarian framework. FileSystem is a "boutique" domain and framework with powerful high-level capabilities, but suffers from a performance-killing design issue. So it'd probably make more sense for FileSystem use FileDirectory as its "primitive", and then we can simply keep FileDirectory around as-is for legacy apps.
I disagree. FIleDirectory is an obsolete mess. FileSystem is a well designed, modern alternative, designed by a Squeaker (Colin). I would much rather see FileSystem as the foundation.
- add FileSystem to the Trunk and gradually migrate all uses of
FileDirectory to it.
This would be asking virtually every app to do a similar migration. All because of Squot?
Port Squot to use FileDirectory.
Don't port anything. Side-by-side existence. People who need
Squot simply load Filesystem too.
After that, the next problem will be that the current MCRepository
hierarchy couples file formats and ancestry storage. In the case of Monticello-in-another-VCS, these are orthogonal: you can have
FileTree layouts in Git, you can have Tonel layouts in Git, you could
have both at the same time, you could have both in a different VCS in the future... So instead of a "FileTree"-Repository and a
"Tonel"-Repository, I think there should rather be a "Git"-Repository
that picks the correct readers and writers dynamically to process the files and directories in each commit. To make the smell more
tangible: I think neither MCFileTreeRepository (or whathever its name
actually is) nor TonelRepository should be subclasses of MCFileBasedRepository (they are today), although it sounds crazy.
I don't think it is a good idea to mix MC and git at all. IMO we only need tools to safely and reliably migrate from one to the other to start
a
change.
Levente
Am Mo., 18. Feb. 2019 um 11:45 Uhr schrieb Levente Uzonyi <
leves@caesar.elte.hu>:
On Mon, 18 Feb 2019, Jakob Reschke wrote: > Am Mo., 18. Feb. 2019, 01:39 hat Levente Uzonyi <leves@caesar.elte.hu> geschrieben:
> > Is it a must to use Metacello to load code in Tonelformat?
> What does Metacello add to the mix? > Can't we just map each git commit to an mcz? > > Levente > > > Strictly speaking, Metacello is not required. But you wouldhave to resolve the package dependencies yourself and gather them from external repositories, which is the main benefit of
Metacello. So I > understand that Eliot wants it to work. > > There is no mcz to map to. If one were to create a new kind ofMCRepository that maps commits to MCVersions, you would have 0..* MCVersions per commit because each commit may touch
multiple packages > or none. Each commit describes a particular configuration ofpackage versions, so one could generate an artificial MCConfiguration for each commit. Note that Squot / the Git Browser
already supports > these scenarios for FileTree repositories, albeit withoutusing Monticello for the version control.
Looking at the repository in question[1], there seem to bemultiple
packages stored in it[2]: BaselineOfScorch, Scorching,ScorchingDev,
ScorchingTests, ScorchingVMTests. My impression is that thesecould be
converted to individual mczs for each commit, which would meanthat the
same code would be available as MC packages. Then you could loadthe code
with MC/Metacello/whatever from the .mczs. Am I missing something? Levente [1] https://github.com/clementbera/Scorch [2] https://github.com/clementbera/Scorch/tree/master/repository > >
Am Mo., 18. Feb. 2019 um 19:53 Uhr schrieb Chris Muller <asqueaker@gmail.com
:
- Port Squot to use FileDirectory.
Part of the fun is that one of its components writes the serialized forms of objects (packages in FileTree format being the almost exclusively used "special" case) to a directory, not even knowing that it is in fact constructing a Git tree in the process. One can try to reimplement that abstraction and call it GitDirectory, or throw over this part of the architecture, of course. Any volunteers? ;-)
On the other hand, this abstraction is one of the parts that will make it quite hard to integrate the Git history in the IDE (that is in the System Browser, for method versions, timestamps, ...) because you would have the cross this barrier to extract any history subsets efficiently. Esteban's mail sounds to me like Iceberg intentionally does not have this abstraction, but I didn't check. One argument in favor of the abstraction is that (in theory) you can swap out Git for something else without touching the file-in and file-out components (in practice, implementing the VCS adapter is the hardest part and even the Git Browser reaches down into the Git layers to accomplish some things, but so it at least deserves its name).
On Mon, 18 Feb 2019 at 05:34, Jakob Reschke forums.jakob@resfarm.de wrote:
Personally, I am very fond of the use of Git instead of Monticello, but not convinced of Tonel so far.
The key things that Tonel addressed for me is: a. Windows FileTree often had problems with "path too long" due to long method names. These occur as semi-random failures when it works for some, and for others located in another path it fails. b. Windows was very slow for many small files
cheers -ben
On 18.02.2019, at 14:48, Ben Coman btc@openInWorld.com wrote:
On Mon, 18 Feb 2019 at 05:34, Jakob Reschke forums.jakob@resfarm.de wrote:
Personally, I am very fond of the use of Git instead of Monticello, but not convinced of Tonel so far.
The key things that Tonel addressed for me is: a. Windows FileTree often had problems with "path too long" due to long method names. These occur as semi-random failures when it works for some, and for others located in another path it fails. b. Windows was very slow for many small files
My fix would be: Don't write to the disk.
Best regards -Tobias
Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <astares@gmx.de mailto:astares@gmx.de> wrote: Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it. So, we drop it… and we add support for the equivalent using the tool we use (iceberg gives you not just the author that modified last the method but all of them, along the history).
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though. There was a time when you, Esteban, were contributing to VMMaker.oscog from Pharo and often there would be many changes where just the timestamp had changed. SO the problem seems to be not with timestamps but with mismanagement of them.
So, we drop it… and we add support for the equivalent using the tool we
use (iceberg gives you not just the author that modified last the method but all of them, along the history).
Who is we? I ask from a different perspective. I ask not too have Pharo use it but to allow Squeak to use it so that I can have timestamps for Squerask and still use Tonel. But still "“Anyway the answer will be no :)".
So, your first statement is false. If things are correct there will *not* be massive generation of conflicts when there is no reason. Second, I am not asking you to change Tonel in Pharo. i am only asking you to allow Tonel to support timestamps in systems that want to use it.
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
Hi Eliot,
Am Mo., 18. Feb. 2019 um 18:34 Uhr schrieb Eliot Miranda < eliot.miranda@gmail.com>:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
When two people's changes to the same method are merged with Git (or on GitHub for that matter) there will be a conflict at the line that contains the timestamp. Even if the method source could be merged conflict-free. Moreover, none of the two original timestamps will be accurate for the merged version of the method, so in fact there *must* be a conflict. This is only alleviated by performing the merge in the IDE where a new timestamp could be generated for the merged method. For otherwise clean merges, this is probably unacceptable for people that just want to press the "merge" button on a pull request.
And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
This is the reason why Pharo cannot just "not use" timestamps and yet support them. Every Pharo user on the team would eliminate the timestamps that the Squeak team members committed. Except if Pharo would allow to carry the existing timestamps around in the code model for Squeak, but it sounds like they are not willing to do that, are they?
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Kind regards, Jakob
PS. Note that I am not on pharo-dev and do not wish to subscribe for just one thread, so I had to omit them from CC.
On Tue, 19 Feb 2019 at 01:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
This is a case of the difference between theory and practice :)
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though.
When I heard of this timestamp conflict problem I began with the same belief to do some digging (and you know how I dig deep into things) with git-only command line experiments and ended up finding its really true. I'll try to remember some examples.
cheers -ben
Still that’s not the point. Not having an agreement is very different that the behaviour you described about me (and that to justify your own actions). You presented my opposition as it would have been “just because” and I presented you my arguments. Even more, I *was* trying to find a solution. I can be right or wrong, but I’m not an idiot trying to be more than I am. Honestly, discussing with you many times feels like it is “my way or the highway”, and it’s hard to follow you :(
Esteban
On 18 Feb 2019, at 18:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano <estebanlm@gmail.com mailto:estebanlm@gmail.com> wrote: Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda <eliot.miranda@gmail.com mailto:eliot.miranda@gmail.com> wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <astares@gmx.de mailto:astares@gmx.de> wrote: Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though. There was a time when you, Esteban, were contributing to VMMaker.oscog from Pharo and often there would be many changes where just the timestamp had changed. SO the problem seems to be not with timestamps but with mismanagement of them.
So, we drop it… and we add support for the equivalent using the tool we use (iceberg gives you not just the author that modified last the method but all of them, along the history).
Who is we? I ask from a different perspective. I ask not too have Pharo use it but to allow Squeak to use it so that I can have timestamps for Squerask and still use Tonel. But still "“Anyway the answer will be no :)".
So, your first statement is false. If things are correct there will *not* be massive generation of conflicts when there is no reason. Second, I am not asking you to change Tonel in Pharo. i am only asking you to allow Tonel to support timestamps in systems that want to use it.
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
Esteban,
On Mon, Feb 18, 2019 at 10:37 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Still that’s not the point. Not having an agreement is very different that the behaviour you described about me (and that to justify your own actions). You presented my opposition as it would have been “just because” and I presented you my arguments. Even more, I *was* trying to find a solution. I can be right or wrong, but I’m not an idiot trying to be more than I am. Honestly, discussing with you many times feels like it is “my way or the highway”, and it’s hard to follow you :(
I am not trying to force my way. I am trying to find a way that works for me. I asked you to support an option in Tonel. I did not ask you8 to change they way Pharo uses Tonel. You can choose to read my request as some kind of threat, or you can choose to read it as a straightforward request. I am not trying to get you to follow me.
I am frustrated at being mistrusted. I have not asked you to follow me. I have tried to work with you. But you don't trust me.
I have done everything I can in the VM (let's ignore source code control and building for now) to support Pharo. I have done *nothing* to interfere with Pharo's use of the VM in the code base itself. have I? Point to actions I have taken, code I have written to try and harm you.
We now have a situation where we support differences in the code base (FilePlugin API, command line argument parsing, version printing) that extend *those I put in at the beginning to support differences* (such as plugins.ext and plugins.int to allow builders to configure the VM as they choose). Where have I tried to block Pharo?
But you left vm-dev. You decided to stop communicating. And now you continue to describe a straight-forward request to allow an optional behavior in Tonel with a flat refusal, plus some BS about me trying to insist that you follow me.
I don't understand Esteban. You and I do not communicate. I feel that you top not want to work with me, that you dislike and distrust me. I cannot work with you. I have tried for years and we are still in difference and disagreement.
Esteban
On 18 Feb 2019, at 18:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though. There was a time when you, Esteban, were contributing to VMMaker.oscog from Pharo and often there would be many changes where just the timestamp had changed. SO the problem seems to be not with timestamps but with mismanagement of them.
So, we drop it… and we add support for the equivalent using the tool we
use (iceberg gives you not just the author that modified last the method but all of them, along the history).
Who is we? I ask from a different perspective. I ask not too have Pharo use it but to allow Squeak to use it so that I can have timestamps for Squerask and still use Tonel. But still "“Anyway the answer will be no :)".
So, your first statement is false. If things are correct there will *not* be massive generation of conflicts when there is no reason. Second, I am not asking you to change Tonel in Pharo. i am only asking you to allow Tonel to support timestamps in systems that want to use it.
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
Fair enough. I should not have step in this discussion.
Esteban
On 18 Feb 2019, at 19:48, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 10:37 AM Esteban Lorenzano <estebanlm@gmail.com mailto:estebanlm@gmail.com> wrote: Still that’s not the point. Not having an agreement is very different that the behaviour you described about me (and that to justify your own actions). You presented my opposition as it would have been “just because” and I presented you my arguments. Even more, I *was* trying to find a solution. I can be right or wrong, but I’m not an idiot trying to be more than I am. Honestly, discussing with you many times feels like it is “my way or the highway”, and it’s hard to follow you :(
I am not trying to force my way. I am trying to find a way that works for me. I asked you to support an option in Tonel. I did not ask you8 to change they way Pharo uses Tonel. You can choose to read my request as some kind of threat, or you can choose to read it as a straightforward request. I am not trying to get you to follow me.
I am frustrated at being mistrusted. I have not asked you to follow me. I have tried to work with you. But you don't trust me.
I have done everything I can in the VM (let's ignore source code control and building for now) to support Pharo. I have done *nothing* to interfere with Pharo's use of the VM in the code base itself. have I? Point to actions I have taken, code I have written to try and harm you.
We now have a situation where we support differences in the code base (FilePlugin API, command line argument parsing, version printing) that extend *those I put in at the beginning to support differences* (such as plugins.ext and plugins.int http://plugins.int/ to allow builders to configure the VM as they choose). Where have I tried to block Pharo?
But you left vm-dev. You decided to stop communicating. And now you continue to describe a straight-forward request to allow an optional behavior in Tonel with a flat refusal, plus some BS about me trying to insist that you follow me.
I don't understand Esteban. You and I do not communicate. I feel that you top not want to work with me, that you dislike and distrust me. I cannot work with you. I have tried for years and we are still in difference and disagreement.
Esteban
On 18 Feb 2019, at 18:34, Eliot Miranda <eliot.miranda@gmail.com mailto:eliot.miranda@gmail.com> wrote:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano <estebanlm@gmail.com mailto:estebanlm@gmail.com> wrote: Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda <eliot.miranda@gmail.com mailto:eliot.miranda@gmail.com> wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann <astares@gmx.de mailto:astares@gmx.de> wrote: Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though. There was a time when you, Esteban, were contributing to VMMaker.oscog from Pharo and often there would be many changes where just the timestamp had changed. SO the problem seems to be not with timestamps but with mismanagement of them.
So, we drop it… and we add support for the equivalent using the tool we use (iceberg gives you not just the author that modified last the method but all of them, along the history).
Who is we? I ask from a different perspective. I ask not too have Pharo use it but to allow Squeak to use it so that I can have timestamps for Squerask and still use Tonel. But still "“Anyway the answer will be no :)".
So, your first statement is false. If things are correct there will *not* be massive generation of conflicts when there is no reason. Second, I am not asking you to change Tonel in Pharo. i am only asking you to allow Tonel to support timestamps in systems that want to use it.
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
On Mon, Feb 18, 2019 at 11:02 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Fair enough. I should not have step in this discussion.
Again, your response is to withdraw and not to engage. I am trying too be honest about my feelings and about the issues. Why don't you try and do the same? Why do you always try and withdraw?
Why do you top post emotionally and run away from a real problem that affects both of us? I do not understand this pattern Esteban. I am not trying to hurt you.
Esteban
On 18 Feb 2019, at 19:48, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 10:37 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Still that’s not the point. Not having an agreement is very different that the behaviour you described about me (and that to justify your own actions). You presented my opposition as it would have been “just because” and I presented you my arguments. Even more, I *was* trying to find a solution. I can be right or wrong, but I’m not an idiot trying to be more than I am. Honestly, discussing with you many times feels like it is “my way or the highway”, and it’s hard to follow you :(
I am not trying to force my way. I am trying to find a way that works for me. I asked you to support an option in Tonel. I did not ask you8 to change they way Pharo uses Tonel. You can choose to read my request as some kind of threat, or you can choose to read it as a straightforward request. I am not trying to get you to follow me.
I am frustrated at being mistrusted. I have not asked you to follow me. I have tried to work with you. But you don't trust me.
I have done everything I can in the VM (let's ignore source code control and building for now) to support Pharo. I have done *nothing* to interfere with Pharo's use of the VM in the code base itself. have I? Point to actions I have taken, code I have written to try and harm you.
We now have a situation where we support differences in the code base (FilePlugin API, command line argument parsing, version printing) that extend *those I put in at the beginning to support differences* (such as plugins.ext and plugins.int to allow builders to configure the VM as they choose). Where have I tried to block Pharo?
But you left vm-dev. You decided to stop communicating. And now you continue to describe a straight-forward request to allow an optional behavior in Tonel with a flat refusal, plus some BS about me trying to insist that you follow me.
I don't understand Esteban. You and I do not communicate. I feel that you top not want to work with me, that you dislike and distrust me. I cannot work with you. I have tried for years and we are still in difference and disagreement.
Esteban
On 18 Feb 2019, at 18:34, Eliot Miranda eliot.miranda@gmail.com wrote:
Esteban,
On Mon, Feb 18, 2019 at 4:06 AM Esteban Lorenzano estebanlm@gmail.com wrote:
Hi,
I’m so not wanting to be part of this discussion, but here I am. Just because it was mentioned my name in a way I consider inaccurate.
On 17 Feb 2019, at 21:27, Eliot Miranda eliot.miranda@gmail.com wrote:
Hi All, Hi Torsten, Hi Jakob,
On Thu, Feb 14, 2019 at 6:17 AM Torsten Bergmann astares@gmx.de wrote:
Hi,
as some of you might already know "Tonel" is a file-per-class format for monticello repositories [1].
In Pharo land many projects are already converted to it - as regular "file tree" solution faces issues with git handling due to very long file names / deep file hierarchy (often leading to trouble on Windows).
Dales's Rowan (a new project/package manager for Smalltalk) supports FileTree and also Tonel repositories [2]. There is also a tool to convert from STORE (VisualWorks Source-Code Versioning System) to Tonel format [3].
So I wonder about adoption of Tonel in Squeak land.
Any status/comments?
Last year I was given contract employment by Tudor Girba's feenk which in part was to involve me porting VMMaker.oscog to Pharo in order to reap the productivity benefits of the Glamorous Toolkit when applied to VMMaker. The major direction was to unify the opensmalltalk-vm repository with a Toned-Format repository holding the Smalltalk code.
This project, though no fault of feenk's did not go well. A big part of it was my own emotional attachment to Squeak, my "muscle memory" with Squeak and my long experience with Smalltalk-80 derived meta programming which I make extensive use of in VMMaker.oscog. But a key issue was that I was not prepared to *move* VMMaker.oscog to Pharo; for me it is key that it can live in both places. In particular I will not risk VMMaker.oscog being stranded on Pharo as it moves away from the "image-based rendering model". This is not because I don't believe in pixel-independent rendering models, but because productivity in VMMaker depends much more than on GToolkit inspection on the key architectural feature of being completely, safely simulateable (not a real English word; but meaning capable of being simulated). As Pharo moves towards external libraries to implement its UI so the degree to which VMMaker is simulated is reduced. Thus crashes in external code cannot be handled with the same power as with errors presented within the Smalltalk environment itself, and this (as harsh experience over thirty five years has taught me) is far more important for overall productivity than having powerful tailorable inspectors.
I therefore wanted a Tonel that could both function for Squeak's Monticello and for Pharo's Iceberg. I had modified the Tonel serialization a little to provide well-formatted support for method timestamps, which are still a key and extremely useful part of Squeak Monticello's "blame" equivalent. Until git author attribution can be imported into Squeak's Monticello (something I don't see happening) I wanted merely to be able to have a variant of Tonel, enabled for example by a class variable, and hence /not/ present in Pharo, that would allow VMMaker.oscog to use Tonel but still be functional inside Squeak. I asked Esteban and was met by a flat no. Not "OK, but this is an option only you will use", but a flat "not under any circumstance”.
This is textually my answer (even with the English errors): … “Anyway the answer will be no :) This was considered, and even the first implementation of tonel had it. Now, this was discarded because it does not works with git (or other file backends). It was not even me who complained first (as I said, I implemented it). It was Dale and other people used to work on git.
Thing is, putting timestamps it will generate conflicts massively when there is no reason to it.
How can this be true? Since a method's time stamp changes only when it is modified it will only produce a change where a method is changed. I see no problem with this.
I do see the possibility of a bug in Pharo when it was using Monticello and timestamps though. There was a time when you, Esteban, were contributing to VMMaker.oscog from Pharo and often there would be many changes where just the timestamp had changed. SO the problem seems to be not with timestamps but with mismanagement of them.
So, we drop it… and we add support for the equivalent using the tool we
use (iceberg gives you not just the author that modified last the method but all of them, along the history).
Who is we? I ask from a different perspective. I ask not too have Pharo use it but to allow Squeak to use it so that I can have timestamps for Squerask and still use Tonel. But still "“Anyway the answer will be no :)".
So, your first statement is false. If things are correct there will *not* be massive generation of conflicts when there is no reason. Second, I am not asking you to change Tonel in Pharo. i am only asking you to allow Tonel to support timestamps in systems that want to use it.
I hope is an understandable reason?”
And then in other mail of same thread I suggested an approach viable for squeak:
"Now, what I’m sure is that if you are trying of using the 100% of Monticello and to use git just as another repository backend, it will not work (we already tried). It will consume far too much effort and you will finish beating the purpose of use git and a file-based-format, and you will be in the worst of two worlds :)
We started like that, and we ended up implementing Iceberg to take benefit of the tools we are using.
In the middle, there was/is gitfiletree. (I think I suggested you to look at it to adapt it for tonel and squeak).
Gitfiletree uses OSProcess to call git as external tool and it makes all the operations needed. “
Now, what I was suggesting was a way to use tonel backed with OSProcess (which works in squeak) instead backing it with libgit2 as us. Since they need the blame support, they could use OSProcess and external git for it.
Of course squeak (and any other dialect) are free to take tonel and adapt it to their needs (in fact, Mariano extended Tonel for VA specific things). And Pharo can read that format but it will ignore all extensions. Now, another problem is to WRITE: Pharo would write those packages in “pharo format” and that information would be lost. And in the case of VMMaker, if the idea is to be able to work on both sides, then Pharo needs to “honour” that format, it cannot just read it and ignore it later. So this creates an incompatibility that needs to be handled.
Anyway, as you can see my answer was not “not under any circumstances”. Was “no, because is not convenient”.
It was still no. And my request was clear. I asked if you would allow Tonel to support timestamps, not that Pharo would use timestamps. This avoids forking Tonel, which I see as essential (it is essential that Tonel *not* fork) for Tonel to be a format that can serve as an interchange between Squeak and Pharo.
Esteban
So here I am joining in this thread reluctantly. I'm extremely frustrated by the state of Tonel in Squeak and between the two dialects. Clément had moved his Scorch optimizer to Tonel/git and I tried to use Metacello and Jakob's Tonel code to at least allow me to load that code and try and push forward on Sista/Scorch. The port *does not work*. Metacello fails to load Scorch. Neither Jakob nor Fabio, after putting in some effort, were able to fix the problem, and it is well beyond me. So I have accepted the fact that I will have to use the Monticello repository for Scorch/Sista.
I had deep concerns that the pursuit of git integration would end up splitting the Pharo and Squeak communities and indeed this is now in progress. I am utterly unmotivated by the lack of cooperation, the sheer arrogance and bullying of those that say "you will move to git/tonel or else", and considering leaving VMMaker altogether. The only things that are keeping me interested are Ron Teitelbaum's Terf and me pursuing a PhD on register allocation in the context of Sista/Scorch with Robert Hirshfeld's group at HPI.
Here's the kind of crap people like Ducasse throw at me:
"Eliot
At the end of the day I will probably ask the two phds that should work on language design to use truffle or pypy because let us face it we cannot work with the Pharo VM. Else we will simply have to fork it (because we do not want to have to jump over cuis, newspeak, squeak code constraints and I do not what) and it will be another drama is in the pico world of the “open” smalltalk VM. "
I am so over this crap. _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
-- _,,,^..^,,,_ best, Eliot
squeak-dev@lists.squeakfoundation.org
-
Ben Coman -
Chris Muller -
David T. Lewis -
Eliot Miranda -
Esteban Lorenzano -
Fabio Niephaus -
Hilaire -
Jakob Reschke -
Levente Uzonyi -
Nicolas Cellier -
Tobias Pape -
Torsten Bergmann