Hi,

Jason Johnson wrote:

On 10/25/07, Peter William Lount <peter@smalltalk.org> wrote:

Hi,

Slicing up the data objects and shipping them in one deep copied parcel
from one Smalltalk image to another isn't a general solution. It may be
a solution for some problems that can be simplified but it will NOT work
for the majority of problems. In fact I'd find a "deep copy and split up
the data objects only solution" quite useless for most of the parallel
problems that I'm working on solving.


Ok, you just made a jump there from "majority of problems" to "most of
the problems that *I'm* working on".  It works just fine for the
problems I'm interested in at the moment, and obviously for most of
the problems businesses, etc. are using Erlang for.

So what are these "majority of problems" you're talking about and who
is solving them now?

You're splitting hairs with my use of English. The vast majority of concurrency problems won't fit well with the process-based model of concurrency (of Erlang and other systems).

Process-based model of concurrency won't solve ALL the problems. Consider that there are an infinite number of concurrency problems. You're telling me that the solution you are proposing solves them all?

Sure if you have a set of problems that you are solving that fit perfectly with the process-based model of concurrency - as used in Erlang - then excellent use that approach to solve your problems. Go for it.

However, please don't force your "process-based ONLY model of concurrency" onto Smalltalk since there are those of us out here with other real world problems that the process-based model of concurrency won't solve easily or at all. This also applies for the deep coping of objects as an only solution - it just works for a tiny subset of problems. Please don't force ill conceived deep copying solutions upon us as the ONLY solution.

One solution won't fit all. That's the point of a programming language - to be general purpose enough to enable programmers or users to solve as wide a range of problems as possible.

It sounds like you need to put your code into a library of objects that are optional rather than forcing them into the virtual machine.

In the general large scale case that I gave as an example (in an earlier
email) the one million data objects could be retrieved or accessed by
any of the 10,000 processes used in the example. While shipping them all
in a deep copied parcel in one message is possible it's not always the
wisest move.


And why would you do that?  No paradigm can remove the responsibility
to think from the programmer.  As I've said I don't know how many
times, the point of this is simply to move this to design where it
belongs instead of implementation where it is now.

Sharing memory is breaking encapsulation.

No, sharing memory does not necessarily break encapsulation. Many things break encapsulation for sure. Blocks for one (see Alan Kay's comments on this).

It's how you look at it. If many processes are accessing an object via the objects methods then encapsulation is not broken since the object is in control of who can see what and when. It doesn't matter how many processes see the object if the object is in control of what is happening to it. That is a current capability of Smalltalk.

A key characteristic of the general problems
is that the data objects can and must be accessible from ANY of the
forked off processes with ANY of them being able to alter the objects at
any point in time with those changes being propagated back to the
central node (assuming there is just one central node) when a commit
occurs and then updating the other forked off processes that have an
interest in seeing updates to objects in mid transaction. Some of these
changes will of course nullify the work of some of these processes
requiring them to abort and possibly start over with the newest changes.
Too many interacting changes between the processes will of course cause
too many aborts and retry (assuming that's the chosen mechanism for
dealing with overlapping changes that are mutually exclusive resulting
in inconsistency to the data objects).


You lost me again.  You seem to come up with a problem *and the way
you want it solved* and then complain that a message passing paradigm
can't solve it.  What does that prove?

I apologize if my writing wasn't clear.

I provide an example and one way (of many ways) to solve it. Yes. That's being responsible from my perspective since I know the a solution that will work I'm not going to hide it as some sort of covert test.

Obviously the message passing paradigm of Smalltalk can solve the problem since that's what I've presented. What's not clear is how the "process-based model of concurrency" - of Erlang - can solve every concurrency problem out there. What's not clear is how Erlang would solve the example I put forward without changing it into a slice and dice of the original one million objects.

The point or "what does it prove" is that no one concurrency solution will solve all the problems. I provided an example to show that and to show the kind of nasty problem that other concurrency problems can solve.

See the "Little Book of Semaphores" that I linked to for many other solutions to concurrency problems.

Solving for 90%
of the cases will thus require much more than what is being proposed by
the simplify concurrency at the loss of capability proponents.


Your 90% is different then mine then.  In fact, I can't think of any
concurrency problems I wish to resolve that this *wont* handle.

As I said in another recent email today your set of problems is of a nature that can be solved that way. That's great. However, please don't force your approach unto us all since it won't work for all of us.

Please put your solution into a library so that those who wish to use it can do so if they so choose. Thanks.

What are needed are solutions that cover various chunks of the solution
space and a way of selecting the correct solution mechanisms either
manually or automatically (preferred if viable). Then the "deep copy and
split up the data objects only solution" may do it's part as a piece in
a wider matrix of solutions. To dispense with the tools we need to solve
the general problems is folly IMHV (In My Humble View).


We already have.

How so?

Or is Squeak/most Smalltalks folly?  There are
certain classes of memory management that don't work well with a GC,
but would with manual memory management.  Is it a mistake to deny it's
use at the language level?

Sure Smalltalk has limits to what it can do. That is very important to recognize it's limits.

However, within the current limits of what Smalltalk can do is working powerfully with concurrency (albeit in a single native thread for most but not all implementations of Smalltalk). To reduce this current capability to solve just a subset of concurrency problems is what I'm calling folly.

If you can provide a proof that your proposed process-based model of concurrency methodology can solve all the concurrency problems that can currently be solved by Smalltalk then you'll have convinced me (and likely others). So far I'm still not even clear what you are proposing.

I work quite hard to implement more power and capability for Smalltalk not take it away. Altering the Smalltalk language - or even one version of it, Squeak - in such a way as to make it less powerful by removing concurrency control capabilities isn't going to fly with me and a large number of users of Squeak.

If you wish to do that then you'll likely need to be very creative and put your concurrency solution into an optional class library somehow. That will fly. Or fork a new version off of Squeak. That would also fly with many or almost all I suspect.

In fact there are many small versions of Smalltalk such as Little Smalltalk or Susie Smalltalk or ... that are quite limited (actually almost all Smalltalks are quite limited when compared with VisualAge with Envy but that's another discussion ;--). They are separate versions of Smalltalk for that reason and it's a good thing too. (I use Susie Smalltalk and Squeak Smalltalk in the Smalltalk.org web site for various jobs - the right tool for the right job).

Some of the innovations that I've encountered or created will enable Smalltalk to be a more powerful language while retaining all aspects of Smalltalk - if they are adopted. (See my other postings in this group and on Smalltalk.org for some of the details of those).

Other innovations that I've encountered or created however, alter the Smalltalk language so much so that it's no longer really Smalltalk - more powerful and highly influenced by Smalltalk but clearly not Smalltalk anymore (at least to me). For this reason and others I'm implementing ZokuScript which is highly influenced by Smalltalk but goes beyond it enough that it isn't Smalltalk anymore. At the same time I'm implementing ZokuTalk which is meant to be a version of Smalltalk. Actually for leverage ZokuTalk will be converted to ZokuScript on the fly and the ZokuScript equivalent will be compiled for execution.

In the meantime I regularly use the following Smalltalks for various projects (paying and otherwise): Squeak, Susie, Dolphin, Visual Age, Visual Works, Coke-Pepsi-idst, and less often others. Naturally as is the way of a mobius system that evolves to it's next level I'm writing ZokuScript/ZokuTalk in Smalltalk. Once the Zoku Execution Engine (not a virtual machine) has reached a stage that it can run by itself development will proceed self hosted by itself.

An excellent book for learning the ins and outs of concurrency control -
and most importantly the common mistakes - is the free PDF book, "The
Little Book of Semaphores",  by Allen B. Downey and his students:
http://www.greenteapress.com/semaphores/downey05semaphores.pdf. Enjoy
the full power of parallel programming.


For me that's just old-think.  Solutions to incredibly complex
problems we never would have had if we simply avoided violating
encapsulation

Well even the Greeks and Babylonians got some things right even if some of their ideas have been refined. As a general principle I concur and support your efforts - as there is some overlap with some of the solutions that I've been working on with Transactions and parallel processing. However, I'm also grounded in the harsh realities of concurrency control in a wide range of contexts - not all problems can be solved by the solution you are presenting. To ignore that is to ignore reality.

If you can however provide a proof I'd have to be adapt since the science aspect of computing implies an evidence based approach.

All the best,

Peter