Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is if you compress sources now in the process of finalizing 3.9.
I understand your concern about the changes file going over its limit.
I also understand the great deal of effort you and stef have put it to 3.9 to date. And that getting things done has largely relied on your time and effort. Easy to implement solutions would be appreciated I imagine?
First the task of getting a 3.10 with all changes from 3.9 would start with '3.9a with all changes from 3.0' that Doug Way (bless his heart) produced after I said please three times in one post.
Second condensed sources should come at the beginning of an alpha cycle not at an end. So one alternative is to condense sources with the first version of 3.9 alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from 3.8 (condensed) source. That would not be as nice but it would at least be acceptable.
The full changes versions have never been for users they have been for developers. Their usefulness is very high or I wouldnt be trying so hard to make my point.
A code history project is both far in the future and risky (it might not come off). Do you really want Squeak maintainence to limp along until when and if it comes about?
The easier you make it for me (and others) to find the bugs the quicker, better, and more permanently we can fix them.
You and Stef have been given arbitrary authority to determine how 3.9 gets made. The consequences of how you choose to use that authority and what decisions you make affects the whole community.
I see several of those decisions as increasing the brittleness of squeak and I am voicing my concerns. I ask only that you do your best to address them.
Yours in service, -- Jerome Peace
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On 25.07.2006, at 06:15, Peace Jerome wrote:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is if you compress sources now in the process of finalizing 3.9.
I understand your concern about the changes file going over its limit.
It is not a "concern". The changefile is 25MB in 7048, this is not a "concern", this is a fact. And the other fact is that the maximum size of the changes file is 32MB.
First the task of getting a 3.10 with all changes from 3.9 would start with '3.9a with all changes from 3.0' that Doug Way (bless his heart) produced after I said please three times in one post.
This is technically not possible, as the changes file would go over it's maximum size.
Second condensed sources should come at the beginning of an alpha cycle not at an end. So one alternative is to condense sources with the first version of 3.9 alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from 3.8 (condensed) source. That would not be as nice but it would at least be acceptable.
3.9 is not reproducable from 3.8, so building it on top of some modified 3.8 is impossible, at least if we plan to ship it soon at all.
shiping 3.9 with a .changes file of >25MB is impossible, too.
So what do you suggest?
Marcus
On 25.07.2006, at 08:46, Marcus Denker wrote:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is if you compress sources now in the process of finalizing 3.9.
To make one thing clear: I do understand that code history is *extremely* important. And I do know that we need even more than a simple per-method history can do.
At SCG, we actually do research into analyzing history of code... so be assured that we understand why it's needed, that it is needed, and that without a past there will be no future.
But: Right know the code-history of Squeak is not scalable enough to provide a history for the whole system, and we need to balance between what is possible easily now and what is needed for the future.
Marcus
I would suggest having two versions, the main one having condensed sources. Or how about making 3.9a into 3.10 condense the changes and then start 3.11 right away.
It is very important that we have either a condensed change version, or a condense changes method that works. The limit is already way to close for any real work.
Ron Teitelbaum
From: Marcus Denker Sent: Tuesday, July 25, 2006 2:47 AM
On 25.07.2006, at 06:15, Peace Jerome wrote:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is if you compress sources now in the process of finalizing 3.9.
I understand your concern about the changes file going over its limit.
It is not a "concern". The changefile is 25MB in 7048, this is not a "concern", this is a fact. And the other fact is that the maximum size of the changes file is 32MB.
First the task of getting a 3.10 with all changes from 3.9 would start with '3.9a with all changes from 3.0' that Doug Way (bless his heart) produced after I said please three times in one post.
This is technically not possible, as the changes file would go over it's maximum size.
Second condensed sources should come at the beginning of an alpha cycle not at an end. So one alternative is to condense sources with the first version of 3.9 alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from 3.8 (condensed) source. That would not be as nice but it would at least be acceptable.
3.9 is not reproducable from 3.8, so building it on top of some modified 3.8 is impossible, at least if we plan to ship it soon at all.
shiping 3.9 with a .changes file of >25MB is impossible, too.
So what do you suggest?
Marcus
So what do you suggest?
Marcus
Hi Marcus,
Thanks for your replys.
I had carefully crafted a reply to your first response. Which your most recent reply may have invalidated. Still it sums up my best thinking so far on this and it answers the above question to here it is:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you again for your response. I am feeling reassured by what you say about the importance of code history. And by the fact that YOU are saying it.
The ideal future is far away and uncertain. So I hope you will give some consideration to facilitating what we can produce easily with the tools on hand.
Please give consideration to the formula: -Create a condensed source version for Squeak-6665 (either 3.8 final or 3.9a start). -Provide the load scripts to produce 3.9final with all changes from that source. -And have that as a deliveralble for the squeak maintenence folks.
It will help.
<Current note: So why is it impossible to do this? -- Jer>
Also it would probalbly be good to draw others into the pieces of this project you dont strictly have to do yourself. I suspect there are a good number of squeakers who have enough expericence to condense Squeak-6665. I dont know how many can do what you and stef do with the load scripts. This might be a good excuse to think about how to transfer that knowledge.
I believe what I have suggested is possible. We need to take care of what is needed for right now within the present limitation. The question of what to do about scalability can be postponed til the future.
Yours in service, --Jerome Peace
Marcus Denker denker at iam.unibe.ch Tue Jul 25 08:29:05 UTC 2006 replied:
To make one thing clear: I do understand that code
history
is *extremely* important. And I do know that we need even more than a simple per-method history can do.
At SCG, we actually do research into analyzing
history
of code... so be assured that we understand why it's
needed,
that it is needed, and that without a past there will
be no future.
Marcus
I expect you merely to do your best.
--- Marcus Denker denker@iam.unibe.ch wrote:
On 25.07.2006, at 06:15, Peace Jerome wrote:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is
if
you compress sources now in the process of
finalizing
3.9.
I understand your concern about the changes file
going
over its limit.
It is not a "concern". The changefile is 25MB in 7048, this is not a "concern", this is a fact. And the other fact is that the maximum size of the changes file is 32MB.
First the task of getting a 3.10 with all changes
from
3.9 would start with '3.9a with all changes from
3.0'
that Doug Way (bless his heart) produced after I
said
please three times in one post.
This is technically not possible, as the changes file would go over it's maximum size.
Second condensed sources should come at the
beginning
of an alpha cycle not at an end. So one
alternative
is to condense sources with the first version of
3.9
alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from
3.8
(condensed) source. That would not be as nice but
it
would at least be acceptable.
3.9 is not reproducable from 3.8, so building it on top of some modified 3.8 is impossible, at least if we plan to ship it soon at all.
1) I am talking either 3.8 or 3.9a at 6665 point to present. Is there some practical reason why 6665 would not be the same beast if the changes were compressed. Because in theory it should be the same thing with a different changes and source. Or is it that the MC load scripts can't reproduce the process of change?
2) I want the all changes version for its value in finding the RIGHT fixes to future bug discoveries. This is very valuable. And very important. It is not as urgent as the finalization process. Which means if you can find a way to do it. Or if you can find away to even approximate it. It should be done. It doesn't have to be done immediately. It has to be done eventually. Preferably sooner than later. But even later is valuable.
3) I am speaking up about it now because this is the time to think of the plan to see it happen. Before the bridges are burnt.
shiping 3.9 with a .changes file of >25MB is impossible, too.
Sigh. I'm a programmer, I am used to frustration. That usually happens just before a solution appears. I am a good programmer. I'm not used to defeat. There must be a way to handle both concerns. Ron Teitelbaum has made a suggestion. Does it have any merit?
So what do you suggest?
Marcus
As my first reply ended:
I expect you merely to do your best.
Yours in service, --Jerome Peace
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On 26.07.2006, at 05:41, Peace Jerome wrote:
Please give consideration to the formula: -Create a condensed source version for Squeak-6665 (either 3.8 final or 3.9a start). -Provide the load scripts to produce 3.9final with all changes from that source. -And have that as a deliveralble for the squeak maintenence folks.
How would that be different from what we have now?
Also it would probalbly be good to draw others into the pieces of this project you don’t strictly have to do yourself. I suspect there are a good number of squeakers who have enough expericence to condense Squeak-6665.
Doing a "Smalltalk condenseChanges" is not hard... if it would not be broken in 3.8, that is.
I don’t know how many can do what you and stef do with the load scripts. This might be a good excuse to think about how to transfer that knowledge. I believe what I have suggested is possible.
Possible, but in what timeframe? And who would work on it?
You might have seen that not many people care enough to put real time or money into it... how long do you plan to wait for whom?
We need to take care of what is needed for right now within the present limitation. The question of what to do about scalability can be postponed ‘til the future.
The question if if we want to delay the release of 3.9 forever or not. There are two possible things: -> postpone having 3.9 -> postpone having a full history of 3.9
Which do we do?
Marcus
On Wed, 26 Jul 2006 09:36:19 +0200, Marcus Denker wrote:
The question if if we want to delay the release of 3.9 forever or not. There are two possible things: -> postpone having 3.9 -> postpone having a full history of 3.9
Which do we do?
Easy: the one which affects Squeak as a whole can be postponed. So rush out 3.9, please.
/Klaus
Marcus
Hi Marcus,
Your replys to my last couple of posts have been very illuminating and helpful in clearing up some misunderstandings (mine).
I think I see what can be done at this point.
To restate:
My goal:
1) In order to research fixes to bugs in squeak I would like to have easy access to the version history of changed methods. The images with all changes from 3.0 thru xxxx along with the changes in the current image of 3.9 have been my main tools in doing that research.
2) This is important but not urgent and does not need to delay you in finalizing 3.9
3) If you have complete changes from 6665 (3.8final = 3.9a first ) in the current 3.9a 7048 (or what ever good version beyond ) then with current resources my research would require me to fire up an extra image but would be doable.
I get from what you asked below that 7048 is essentially equivalent to what I was trying to suggest. And that it is not missing any vital historical information.
Is that a correct understanding?
********************** **********************
4) Since it would be REALLY HELPFUL to have access to all version/change infomation in a single search. The problem becomes is there a simple way to do that from where we are now.
**********************
Is 32M a limit for one change file? Can I(or someone with more expericence) modify the version searcher to search more than one changes file.
E.G. "Hi friendly version searcher most of the information you need is right here in the connected changes file but would you kindly also search file AllChangesUpto6665.changes for versions that happened before our time?"
That would be beautiful. Maybe then we could go back to version 1.1 and really see how squeak developed. That would really be educational and helpful.
Thanks again for sticking with me on this. I look forward to your next reply.
Yours in service, -- Jerome Peace
--- Marcus Denker denker@iam.unibe.ch wrote:
On 26.07.2006, at 05:41, Peace Jerome wrote:
Please give consideration to the formula: -Create a condensed source version for Squeak-6665 (either 3.8 final or 3.9a start). -Provide the load scripts to produce 3.9final with
all
changes from that source. -And have that as a deliveralble for the squeak maintenence folks.
How would that be different from what we have now?
Duh! That was the illuminating question.
Cheers --Jer
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
My goal:
- In order to research fixes to bugs in squeak I
would like to have easy access to the version history of changed methods. The images with all changes from 3.0 thru xxxx along with the changes in the current image of 3.9 have been my main tools in doing that research.
- This is important but not urgent and does not need
to delay you in finalizing 3.9
Ok What would be good is to have a DB to which we could push all the changes of squeak since the beginning.
- If you have complete changes from 6665 (3.8final =
3.9a first ) in the current 3.9a 7048 (or what ever good version beyond ) then with current resources my research would require me to fire up an extra image but would be doable.
I get from what you asked below that 7048 is essentially equivalent to what I was trying to suggest.
No because we did not started from a full history image if I remember correctly.
And that it is not missing any vital historical information.
Is that a correct understanding?
- Since it would be REALLY HELPFUL to have access to
all version/change infomation in a single search. The problem becomes is there a simple way to do that from where we are now.
Is 32M a limit for one change file? Can I(or someone with more expericence) modify the version searcher to search more than one changes file.
Sources are hardcoded in Smalltalk look for at: 1 and at: 2 So this should be changed but with care.
E.G. "Hi friendly version searcher most of the information you need is right here in the connected changes file but would you kindly also search file AllChangesUpto6665.changes for versions that happened before our time?"
That would be beautiful. Maybe then we could go back to version 1.1 and really see how squeak developed. That would really be educational and helpful.
yes. The best would be to have an external DB to manage all the versions (in addition to changes).
Thanks again for sticking with me on this. I look forward to your next reply.
Yours in service, -- Jerome Peace
--- Marcus Denker denker@iam.unibe.ch wrote:
On 26.07.2006, at 05:41, Peace Jerome wrote:
Please give consideration to the formula: -Create a condensed source version for Squeak-6665 (either 3.8 final or 3.9a start). -Provide the load scripts to produce 3.9final with
all
changes from that source. -And have that as a deliveralble for the squeak maintenence folks.
How would that be different from what we have now?
Duh! That was the illuminating question.
Cheers --Jer
Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
At 10:37 27.07.2006, Marcus (?) wrote:
My goal:
- In order to research fixes to bugs in squeak I
would like to have easy access to the version history of changed methods. The images with all changes from 3.0 thru xxxx along with the changes in the current image of 3.9 have been my main tools in doing that research.
Our updating scheme in the OOram project may be of interest here. (The code was in VW):
1) Major releases where named by a single letter. A basic release started its life with condensed sources in source file 1. The main change file (source file 2) was empty. 2) Intermediate releases were named with sequence numbers, e.g., G8. The changes from the major release were in a source file number 2. 3) Individual programmers copied the current release with its two read-only source files and added a sources file number 3 to hold his or her private changes. (I found it impractical to add a third source file in Squeak; a great pity)
My current image (I'm still playing with it from time to time. Source no. 1 is 25MB, no. 2 is 7MB, no. 3 (my personal, g09-trygve.st, is 17MB)
Every method source has a header of change notifications, e.g., " 920317 trygve(4.0): Use Cursor execute, not wait. " " 19990125 pst (g09-1): Moved from 'end:' to be shared with subclasses. " " 19990125 pst (g09-1): Superfluous 'end' removed from last page. " A designated "releasebuilder" collected all contributions. Software checked the date sequences for submitted methods. Branches in the modification tree indicated conflicts and could be detected easily. This worked well for us in our closed community of less than a dozen programmers. ctrW created a comment line with date and signature. The cursor left in a position for writing additional comment. (Laziness is the mother of invention)
There has been discussions about adding extra info to the methods. One candidate is the method's life history, automating the discipline required by the OOram scheme. An entry could be added automatically at each compile, giving the date, release name and author name. Compression of the entry list could be automatic at submission time. The programmer could also be invited to add a final comment at this time. Harvesters could esily find conflicts between submissions.
cheers --Trygve
stéphane ducasse ducasse at iam.unibe.ch Thu Jul 27 08:37:36 UTC 2006 wrote:
What would be good is to have a DB to which we could
push all the
changes of squeak since the beginning.
I have problems with this approach as a solution to my current need.
My goal: Is to have AS SOON AS POSSIBLE a way to research fixes to bugs in squeak.
The DB project has too many risks associated with it to be the soluton to that goal. 1) It will take resources. 2) It will take time. 3) There is no guarentee of the quality of the result or that it will even work. 4) You have not stated the scope of the DB project precisely enough to allow further assessment of risk or desirability. 5) The scope even as stated here is too large for one step.
It would be better to do the simplist thing that might possibly work now. Learn from it and reassess the need for an external DB.
My local disk abounds with Squeak .images and .changes.
As a first step, It would seem to me much easier just to teach a version tracker or its equivalent to look for versions in two .changes file. The local one and one of my choosing.
The experience gained from this would inform the larger project you suggest.
An alternate step, 1) If the proposed patch to allow 512MB change files actually works then I would like a 70xx with all changes from 3.0 as a research too for maintainers/developers.
This might be even easier to get done. Especially if a resourse other than you or Marcus could take on the creation of the allChanges build.
I look forward to your response.
Your is service, -- Jerome Peace
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Squeak Maintainence and etc. ; Problem restatement. stéphane ducasse ducasse at iam.unibe.ch Thu Jul 27 08:37:36 UTC 2006 wrote:
My goal:
- In order to research fixes to bugs in squeak I
would like to have easy access to the version
history
of changed methods. The images with all changes
from
3.0 thru xxxx along with the changes in the current image of 3.9 have been my main tools in doing that research.
- This is important but not urgent and does not
need
to delay you in finalizing 3.9
Ok What would be good is to have a DB to which we could
push all the
changes of squeak since the beginning.
- If you have complete changes from 6665 (3.8final
=
3.9a first ) in the current 3.9a 7048 (or what ever good version beyond ) then with current resources
my
research would require me to fire up an extra image but would be doable.
I get from what you asked below that 7048 is essentially equivalent to what I was trying to suggest.
No because we did not start from a full history image
if I remember
correctly.
Uhm. I think weve run into language problems again the suggestion I was refering to was the more recent one:
Please give consideration to the formula: -Create a condensed source version for
Squeak-6665
(either 3.8 final or 3.9a start). -Provide the load scripts to produce 3.9final
with all
changes from that source. -And have that as a deliveralble for the squeak maintenence folks.
not the first wish to have a 7048 with all changes from 3.0. Which is what you would get from loading on top of a full history image.
The important thing is to be able to find (in one image or another ) the versions that lead up to the current one. And to get the most information out of them as to what changed; by whom; and to infer for what purpose.
And that it is not missing any vital historical information.
Is that a correct understanding?
- Since it would be REALLY HELPFUL to have access
to
all version/change infomation in a single search.
The
problem becomes is there a simple way to do that
from
where we are now.
Is 32M a limit for one change file? Can I(or someone with more expericence) modify the version searcher to search more than one changes
file.
Sources are hardcoded in Smalltalk look for at: 1 and at: 2 So this should be changed but with care.
Ok. So I would be possible to experiment,
E.G. "Hi friendly version searcher most of the information you need is right here in the connected changes file but would you kindly also search file AllChangesUpto6665.changes for versions that
happened
before our time?"
That would be beautiful. Maybe then we could go
back
to version 1.1 and really see how squeak developed. That would really be educational and helpful.
yes. The best would be to have an external DB to manage all the versions (in addition to changes).
But, not as a first step. (See my other post on this.)
<snip>
Yours in Service, -- Jerome Peace
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
On Sat, 29 Jul 2006 08:42:09 +0200, Peace Jerome wrote: ...
The important thing is to be able to find (in one image or another ) the versions that lead up to the current one. And to get the most information out of them as to what changed; by whom; and to infer for what purpose.
I doubt that this can be achieved, at least not with the current tools (and .image). Simple counterexample: a method which is no longer active in the image is an orphan in the .sources or the .changes file. Only if the corresponding ChangeRecord instance is still in the image and accessible by a tool, then you'd be able to access it for your investigation.
Example for an orphan in .source : ThreePhaseButtonMorph methodsFor: 'geometry' stamp: 'tk 7/1/97 09:14' #extent. This method is no longer in the image (was probably not distributed with a change set).
Example for an orphan in .changes : ChangeListPlus methodsFor: 'viewing acces' stamp: 'ar 7/15/2005 13:53' #diffedVersionContents. This method has two other change records, one of which should have "prior:" pointing to the orphan.
Attached is a script which does the following (with #7048):
1 - read all chunks from .sources and note the preamble and chunk position 2 - read all chunks from .changes and note the preamble and chunk position
This is done for class definitions (they have no preamble so the chunk position is taken to be := the preamble position), for class comments (have preambles) and for methods (have preambles). Then in a second stage the following is matched against the above:
3 - obtain all class comment ChangeRecords via the existing #scanVersionsOf: routine 4 - obtain all method ChangeRecords via the existing #changeRecordsAt: routine
(both routines are the exclusive methods used by browsers which need version information). This second stage information then overwrites the information obtained during the first stage, by fileIndex and filePosition of their sourcePointer, so that the result tells which items are linked by regular historical "prior:" pointers and which are not.
So when an item from steps 1-2 is not matched during steps 3-4, it must be an orphan.
The script allocates a new global variable SourcesAndChangesPointers by which the results can be accessed (and .image can be saved and analyzed later).
(SourcesAndChangesPointers first) are the results for the .sources file, (SourcesAndChangesPointers second) are the results for the .changes file. Both collections are instances of IdentityDiectionary.
Some datapoints obtained so far:
SourcesAndChangesPointers first size 33979 "# of non-doIt chunks (items) in .sources"
(SourcesAndChangesPointers first associations select: [:assoc | assoc value isInteger and: [assoc key ~= assoc value]]) size 13840 "# of items not covered by ChangeRecord routines"
(SourcesAndChangesPointers first associations select: [:assoc | assoc key == assoc value]) size 1509 "# of class definitions found in .sources"
(SourcesAndChangesPointers first associations reject: [:assoc | assoc value isInteger]) size 18630 "# of items covered by ChangeRecord routines, in .sources"
SourcesAndChangesPointers second size 55200 "# of non-doIt chunks (items) in .changes"
(SourcesAndChangesPointers second associations select: [:assoc | assoc value isInteger and: [assoc key ~= assoc value]]) size 11997 "# of items not covered by ChangeRecord routines"
(SourcesAndChangesPointers second associations select: [:assoc | assoc key == assoc value]) size 170 "# of class definitions found in .changes"
(SourcesAndChangesPointers second associations reject: [:assoc | assoc value isInteger]) size 43033 "# of items covered by ChangeRecord routines, in .changes"
The condition "assoc value isInteger and: [assoc key ~= assoc value]" denotes an item which wasn't assigned a ChangeRecord by some root in the image (root = class comment sourcePointer or method sourcePointer), taking into account all their possible history items, as computed by the respective routine (mentioned above). The condition "assoc key == assoc value" denotes class definitions found during the first stage (they have no source pointer in the current incarnation of the source code subsystem). And the exclusion "assoc value isInteger" denotes items with regular ChangeRecords (possible linked in history with "prior:").
More scripts could be written (but then SourcesAndChangesPointers must become a class), for example for the visualization of orphans and for viewing selections/subsets of the ChangeRecord items by using existing tools, like VersionsBrowser.
Disclaimer: it was carefully tested but the script may contain bugs.
And that it is not missing any vital historical information.
Yes, that'd be great!
/Klaus
Klaus D. Witzel wrote:
On Sat, 29 Jul 2006 08:42:09 +0200, Peace Jerome wrote: ...
The important thing is to be able to find (in one image or another ) the versions that lead up to the current one. And to get the most information out of them as to what changed; by whom; and to infer for what purpose.
I doubt that this can be achieved, at least not with the current tools (and .image).
There may be. Here is a thought that I had for a long time now: Consider that the change history is simply one logical huge file of change records where the .sources and .changes are mere portions of the full "Squeak source file" that logs all the changes that were ever done as the system evolved to your version.
Also consider that if we can describe the name of a *previous* changes (sources) file and if we can describe where in that changes file the previous change record for some code lives, then you could have a situation where -in the condensed changes file of SqueakX.Y.changes- you will find that the "previous version" of some method is to be found in "SqueakX.Y-k.changes". In which case, merely by dropping the appropriate changes file alongside your install you'd have the ability to browse those previous versions - going back all the way to Squeak1.0 if you have all those "chunks of the logical sources of Squeak" (represented by the .changes files).
The (obvious) point being here that if we can keep the information where previous versions ought to be located, it would be trivial to do a condense changes for each new version - if you want history, install the previous versions of Squeak alongside the current one and you got it.
[Disclaimer: The above obviously doesn't deal with packages etc. where due to the current snapshot-nature of Monticello history information is not preserved. In other words: This doesn't work for package history.].
Cheers, - Andreas
On Sat, 29 Jul 2006 18:16:57 +0200, Andreas Raab wrote:
Klaus D. Witzel wrote:
On Sat, 29 Jul 2006 08:42:09 +0200, Peace Jerome wrote: ...
The important thing is to be able to find (in one image or another ) the versions that lead up to the current one. And to get the most information out of them as to what changed; by whom; and to infer for what purpose.
I doubt that this can be achieved, at least not with the current tools (and .image).
There may be. Here is a thought that I had for a long time now: Consider that the change history is simply one logical huge file of change records where the .sources and .changes are mere portions of the full "Squeak source file" that logs all the changes that were ever done as the system evolved to your version.
Yes, like a subset of all change records.
Also consider that if we can describe the name of a *previous* changes (sources) file and if we can describe where in that changes file the previous change record for some code lives, then you could have a situation where -in the condensed changes file of SqueakX.Y.changes- you will find that the "previous version" of some method is to be found in "SqueakX.Y-k.changes".
If the information about "prior:" where accurate, always recorded and reliable: yes, for sure.
In which case, merely by dropping the appropriate changes file alongside your install you'd have the ability to browse those previous versions - going back all the way to Squeak1.0 if you have all those "chunks of the logical sources of Squeak" (represented by the .changes files).
I have converted .sources and .changes to SQLite3 (either using multiple tables or using mutiple db files for different releases, have choice). All that is needed for completing this is the correct "prior:" information (which seems to be rotten, see my previous message on this).
BTW: text queries, i.e. " where sourceCode like '%convert:%from:%to:%using:%' " are easy and fast with SQLite3 :-) This example finds all senders and implementors (and perhaps a bit more, can be further filtered), regardless of whether the method is/was ever in a distributed .image or not.
IMHO the "local" system can have "prior:" as filePosition but when travelling this should be translated to "priorStamp:". Can someone confirm this is sufficient, other opinions and/or experience is appreciated.
The (obvious) point being here that if we can keep the information where previous versions ought to be located, it would be trivial to do a condense changes for each new version - if you want history, install the previous versions of Squeak alongside the current one and you got it.
That'd be easy, I agree.
[Disclaimer: The above obviously doesn't deal with packages etc. where due to the current snapshot-nature of Monticello history information is not preserved. In other words: This doesn't work for package history.].
Similiar to what Trygve described, "external" packages deserve their own fileIndex (or db table or file). That's all about it. If the package maintainer does a change, the (packageFiles at: fileIndex) gets updated. If a package user does a change to a package, her/his own (changeFiles at: fileIndex) gets updated. Can, if someone cares, be replayed on updated packages.
Some support for collaborative work (feedback, unaccepted and accepted changes) would make this an acceptable and usable system, IMO. Very small, very subtle, very easy steps. Not a big bang.
/Klaus
Cheers,
- Andreas
Klaus D. Witzel puso en su mail :
I have converted .sources and .changes to SQLite3 (either using multiple tables or using mutiple db files for different releases, have choice). All that is needed for completing this is the correct "prior:" information (which seems to be rotten, see my previous message on this).
Klaus:
I looking about SQLite3 use from Squeak , but found references to Python, Ruby, etc.
You have any to share or point to? I need a Mac OS X version.
Very thanks.
Edgar
__________________________________________________ Pregunt�. Respond�. Descubr�. Todo lo que quer�as saber, y lo que ni imaginabas, est� en Yahoo! Respuestas (Beta). �Probalo ya! http://www.yahoo.com.ar/respuestas
Edgar,
your Tiger should have SQLite pre-installed, see
- http://www.apple.com/macosx/features/unix/
And on SqMap there is the SQLite3 package (based on the SQLite package from Avi) which was built and tested on Mac OS X Tiger (10.4.2). The SqMap entry has installation/configuration hints for Tiger.
The SQLite3 db software is most current but if needed can be recompiled from source (it is FREE of copyright :)
- http://www.sqlite.org/download.html
Here I use the Win32 pre-compiled on MS$/XP. The commandline utility is great and can open a db in ":memory:" on my 1GB notebook :)
Have fun !
/Klaus
On Sun, 30 Jul 2006 10:53:45 +0200, Lic. Edgar J. De Cleene wrote:
Klaus D. Witzel puso en su mail :
I have converted .sources and .changes to SQLite3 (either using multiple tables or using mutiple db files for different releases, have choice). All that is needed for completing this is the correct "prior:" information (which seems to be rotten, see my previous message on this).
Klaus:
I looking about SQLite3 use from Squeak , but found references to Python, Ruby, etc.
You have any to share or point to? I need a Mac OS X version.
Very thanks.
Edgar
Klaus D. Witzel puso en su mail :
Edgar,
your Tiger should have SQLite pre-installed, see
And on SqMap there is the SQLite3 package (based on the SQLite package from Avi) which was built and tested on Mac OS X Tiger (10.4.2). The SqMap entry has installation/configuration hints for Tiger.
The SQLite3 db software is most current but if needed can be recompiled from source (it is FREE of copyright :)
Here I use the Win32 pre-compiled on MS$/XP. The commandline utility is great and can open a db in ":memory:" on my 1GB notebook :)
Have fun !
Very quick !!
Thanks, very thanks.
Edgar
__________________________________________________ Pregunt�. Respond�. Descubr�. Todo lo que quer�as saber, y lo que ni imaginabas, est� en Yahoo! Respuestas (Beta). �Probalo ya! http://www.yahoo.com.ar/respuestas
On 25 juil. 06, at 06:15, Peace Jerome wrote:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is if you compress sources now in the process of finalizing 3.9.
Why do you say that kind of thing? I really do not get it. This is not true.
I understand your concern about the changes file going over its limit.
I also understand the great deal of effort you and stef have put it to 3.9 to date. And that getting things done has largely relied on your time and effort. Easy to implement solutions would be appreciated I imagine?
First the task of getting a 3.10 with all changes from 3.9 would start with '3.9a with all changes from 3.0' that Doug Way (bless his heart) produced after I said please three times in one post.
But this will not scale!
Second condensed sources should come at the beginning of an alpha cycle not at an end.
So you will not get a true condense version but one with all the changes from that version.
So one alternative is to condense sources with the first version of 3.9 alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from 3.8 (condensed) source. That would not be as nice but it would at least be acceptable.
You and Stef have been given arbitrary authority to determine how 3.9 gets made. The consequences of how you choose to use that authority and what decisions you make affects the whole community.
But are you serious? Do you want us to deliver a system that you CANNOT use? Because you cannot even load any decent package? Is it really what you want?
I see several of those decisions as increasing the brittleness of squeak and I am voicing my concerns. I ask only that you do your best to address them.
Have you checked the number of bug fixes that we integrated/took care? Instead of complaining like that the only way to go is: - build a simple infrastructure so that we can browse any code version - fix the limit of the 32Mb
There are no other choice
Stef
Squeak Maintainence and Condensed Sources stéphane ducasse ducasse at iam.unibe.ch Tue Jul 25 06:54:18 UTC 2006 replied:
On 25 juil. 06, at 06:15, Peace Jerome wrote:
Squeak Maintainence and Condensed Sources
Hi Marcus,
Thank you for your first reply to Condensed Sources vs. Squeak Maintainence
To restate the essesnce of my concern:
I believe a great deal of code will rot and squeak will become a lot more fragile than it already is
if
you compress sources now in the process of
finalizing
3.9.
Why do you say that kind of thing? I really do not get it. This is not true.
It is true that it is my current belief.
In building Squeak you have included several major integrations. -Traits, -UIManager (and tools and plustools systems.) -The Squeakland progress from 3.8 including pieces of stuff from connectors and Kadema -The Smalland stuff from Diego. -My iteration on polygons and curves. -Omnibrowser.
And you have commited to a new way to maintain the source code MC packages.
I cant speak for traits because I have not focused on that aspect of the system.
Omnibrowser from a preliminary look at the code looks like it has been written by people who know good code. And though Ive found some integration bugs they seem to have been easily fixed.
The UIManager was intergrated with several easily findable annoying bugs still present. For a while it broke the sender/implementers buttons in browsers. For these bugs to slip by the maintainers would suggest it was poorly check before being submitted for integration. Those bugs were fixed. There are probably many more subtle bugs attributable to those packages waiting to be found.
The Squeakland stuff is a mismash. The code there has been patched and fixed by people still learning the subtleties of squeak. Bugs have turned up and more are likely to be in that code.
The history was important to me in tracking down why polygons were getting deleted when dropped on the wrong objects. It helped pinpoint what changed and who changed it. It helped me get in touch with the changer. And he used the feedback to improve his code. With out the history will finding the source of the problems be as easy?
The Smalland stuff added a lot to squeak in its usability and visual design. If you look at the implementation you see a lack of consideration for subtleties. In particular code dependencies were increased greatly. That stuff needs to be looked at closely by someone aware of what it means to reduce code surface area. Is that going to be easier to do if the history is lost?
I understand your concern about the changes file
going
over its limit.
I also understand the great deal of effort you and Stef have put it to 3.9 to date. And that getting things done has largely relied on your time and effort. Easy to implement solutions would be appreciated I imagine?
First the task of getting a 3.10 with all changes
from
3.9 would start with '3.9a with all changes from
3.0'
that Doug Way (bless his heart) produced after I
said
please three times in one post.
But this will not scale!
If it doesnt scale then you have to stop at somepoint. The end of a development cycle is the worst place to stop. The beginning of one is better.
Second condensed sources should come at the
beginning
of an alpha cycle not at an end.
So you will not get a true condense version but one
with all the changes
from that version.
Yes. You would have a 3.9 with all changes from Squeak-6665. This gives a way to check on the history of changes in 3.9. Thats what I need. My experience with Doug 6665 with all changes is that it is a BIG help. It meant being able to track down programers intentions and determining where and when things went wrong.
This does not preclude a second condensing of sources at the beginning of 3.10. It solves in a less than ideal but acceptable way the timely need for the historical data.
So one alternative is to condense sources with the first version of
3.9
alpha (or the 3.8 version just before that name change. And produce a 3.9 with all changes from
3.8
(condensed) source. That would not be as nice but
it
would at least be acceptable.
You and Stef have been given arbitrary authority to determine how 3.9 gets made. The consequences of
how
you choose to use that authority and what decisions you make affects the whole community.
But are you serious? Do you want us to deliver a
system that
you CANNOT use?
I think we are having language problems again. How do I make myself clear. What I am asking for is for the sake of maintainence.
3.8 was delivered. Then the version with the full changes was made secondarily. It was made after 3-6 months delay when Doug responded to my second request. I am just asking that my need for this change history be considered. I am hoping it can be honored at an earlier point in time to increase its usefulness.
I expect 3.9 to be delivered. And a 3.9 with full changes to be made available at the same time or shortly there after.
Because you cannot even load any decent package?
Huh? When do you run out of the ability to load decent packages?
Is it really what you want?
If I do not seem to you to be making a reasonable request, then I must be saying something wrong. I appologize for the language barrier. (See my reply to Marcus)
I see several of those decisions as increasing the brittleness of squeak and I am voicing my concerns.
I
ask only that you do your best to address them.
Have you checked the number of bug fixes that we
integrated/took care?
Yes. You and Marcus have been comprehensive in you inclusion of fixes and enhancements into 3dot9. You deserve the credit for the good work and effort.
Instead of complaining like that the only way to go
is: I make my complaints on mantis. This is criticism aimed at appreciating the value of squeak.
- build a simple infrastructure so that we can
browse any code version
- fix the limit of the 32Mb
I do not have the experience to tackle those tasks. Let's look forward to help from the community.
There are no other choice
Not having more choices is a bug ;-) .
Stef
Addendum,
In these posts I have been harsh in my critism of the 3dot9 team. And I have not lightened it by acknowledging the good work you and Marcus have done. You both have labored over a year and a half to deliver a squeak with the seeds of a lot of opportunities for others to build on. Overall its a pretty successful effort. The current 7048 addresses most of the visible problems that arose as you did major integrations. And by doing the work you serve a lot of us who just wanted to use the images as they came out. That was true service. I dont want to loose sight of that in our current discussion.
Thanks for your service.
Yours currently in critism but always in service, -- Jerome Peace.
__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
If it doesn’t scale then you have to stop at somepoint. The end of a development cycle is the worst place to stop. The beginning of one is better.
But we are past the point where we could do it. TO REPEAT: THE CHANGES FILE IS AT 25MB *NOW*. ITS MAXIMUM IS 32MB. THIS IS NOT FIXABLE IN BETA.
Yes. You would have a 3.9 with all changes from Squeak-6665. This gives a way to check on the history of changes in 3.9. That’s what I need. My experience with Doug’ 6665 with all changes is that it is a BIG help. It meant being able to track down programers intentions and determining where and when things went wrong.
But we will reach the limit of 32MB if we build on top of the "all changes from 3.0" version! And "All changes from 3.9" we have.
Squeak3.8a.from3.0.changes 20.7MB Squeak3.8.changes 9MB Squeak3.9b-7048.changes 24.8MB diff to 3.8: 15.8 MB ==================================== Squeak3.8a.from3.0.changes + diff = 20.7 + 15.8 = 36.5
36.6 > 32 ==> We have a problem.
I think we are having language problems again. How do I make myself clear. What I am asking for is for the sake of maintainence.
3.8 was delivered. Then the version with the full changes was made secondarily. It was made after 3-6 months delay when Doug responded to my second request. I am just asking that my need for this change history be considered. I am hoping it can be honored at an earlier point in time to increase its usefulness.
I expect 3.9 to be delivered. And a 3.9 with full changes to be made available at the same time or shortly there after.
But it is technically impossible now, as the changes file is limited to 32MB. so?
Because you cannot even load any decent package?
Huh? When do you run out of the ability to load decent packages?
because the changes file will grow to >32MB and you get an error message.
Marcus
squeak-dev@lists.squeakfoundation.org
-
Andreas Raab -
Klaus D. Witzel -
Lic. Edgar J. De Cleene -
Marcus Denker -
Peace Jerome -
Ron Teitelbaum -
stéphane ducasse -
Trygve Reenskaug