Discussion:
3.11 and the trunk
(too old to reply)
Andreas Raab
2012-01-28 12:27:26 UTC
Permalink
Folks -

This is going to be lengthy, so I apologize beforehand. But we're in an
unfortunate situation right now, an impasse between the board-approved
3.11 proposal and the new kid in town, the trunk. I'm trying to get out
of this impasse, so here are some thoughts.

First of all, I made a mistake. After my discussions with Matthew (both
on- and off-list) I fully expected that 3.11 was dead when Matthew told
me that he would stop working on it. Keith had been pretty much
invisible in the discussions up to that point so I had (incorrectly)
assumed that he wouldn't have any attachment to the 3.11 label and
consequently I didn't spend any time on thinking about the relationship
between 3.11 and the trunk. Obviously that was wrong. So apologies again
for misinterpreting the situation.

That out of the way, I have to admit that I only realized today just how
different the goals of the 3.11 proposal were from what I have been
reading into them the entire time.

Quoting http://installer.pbworks.com/Squeak311Proposal:
"The technical goal of Squeak 3.11 development is to provide a tool
chain to enable many contributors to add code to the base without
bottlenecks. The validation of this goal being the production of a new
3.11 image built using this tool chain."

What I had missed (literally until today) is that the goal of this
proposal is not to create a new version of "Squeak the software", the
goal of the proposal is to develop a set of tools. The creation of a
3.11 release artifact in this proposal is just a validation step and not
given much further thought (just check how detailed the description of
the tools is and how vague the description of the release artifact itself).

I never got this before. And I'm sure many others didn't. And I'm not
even sure the former board fully realized that they were ordering a
kitchen instead of the entr?e.

But suddenly things make sense that previously didn't. It does explain
for example the contents of http://ftp.squeak.org/3.11 (which had
puzzled me endlessly). Since the goal of the proposal wasn't to create a
3.11 image we find nothing that's labeled 3.11 in the directory. Instead
we find different Squeak versions that the tools were applied to. Makes
sense once you understand that the proposal isn't trying to create a new
release of Squeak but rather a set of tools.

The trunk development on the other hand is all about Squeak the
software, the content. I don't care about tools any more than that I
need them to get the job done, and if I can beat Monticello into
submission and have it give me an incremental update stream with minimal
fuzz I will do that and concentrate on what I consider the important
part: Squeak.

In order to get out of our current impasse, I first and foremost need
people to understand this: The trunk is not about tools or
infrastructure and the 3.11 proposal is not about content.

And perhaps that is also showing the way out. What if we left the tools
work to the tools and the development work to the developers? What I
mean is that we can use the tools that have been developed in the 3.11
process, for example, to do continuous integration from Mantis. All the
bits are there (well, were there, now that Bob got pulled, so perhaps
this won't work) but the tools infrastructure can certainly be used to
take part in the content development which happens in the trunk.

The result of which would be two parts: A loadable set of tools,
developed in the 3.11 process, just as stated in the 3.11 proposal,
applicable to different Squeak versions, just as illustrated by the
content of the 3.11 directory. But in addition to the tools we also get
a rich Squeak 3.11 release artifact the result of work that the tools do
using Mantis, and the work that the developers do using the trunk.

In practice, this would mean we should start the integration process
sooner rather than later, and I'm willing to spend a bit of time in
setting up a server, which, using the tools developed already, find new
stuff on Mantis, load it, test it, provide feedback and recommendations.
Possibly even push it straight into the inbox. The fixes already
screened and marked pending should go into into the trunk ASAP.

Does this make sense? I think it's really important not to miss the
difference in goals between the 3.11 proposal (developing tools for
Squeak) and the trunk (developing Squeak itself). And I think we can do
both at the same time without being in an endless heated conflict.

Comments welcome.

Cheers,
- Andreas
Michael Haupt
2012-01-28 12:27:26 UTC
Permalink
Hi Andreas,

what an interesting turn.

Over the past few weeks, I've been wondering what actually was the
problem, and why (and how) things could possibly have gotten so heated
(the usual OSS community factors aside). My suspicion was that there
must have been some gross misunderstanding, coupled with a lack of
proper communication due to hurt feelings and refusal to respond to
rather personal attacks. I'm sorry to see I seem to have at least
gotten the misunderstanding thing right.

I don't want to start new trouble, but I have an urgent question now.
The decision that has, in part, caused the turmoil was a board
decision, if I recall that correctly. If so, how could the board as a
whole have misinterpreted the 3.11 efforts - was there not enough
discussion?

The past aside, although it is not really up to me to make peace, what
about the news ...
In order to get out of our current impasse, I first and foremost need people
to understand this: The trunk is not about tools or infrastructure and the
3.11 proposal is not about content.
Regarding the trunk, I have had the impression that the entire work
was indeed *just* about software. I may have missed some things, but I
had not quite understood what would eventually constitute release
3.11. That said, I'm a newbie in the development crowd, and this can
well be blamed on my lack of experience.

Having learned about the Installer (and related tools), I was
nevertheless wondering how - in spite of their undoubtable merits -
such tools could possibly have been excluded from usage in ongoing
development. Tools like these are clearly missing.
And perhaps that is also showing the way out. What if we left the tools work
to the tools and the development work to the developers? What I mean is that
we can use the tools that have been developed in the 3.11 process, for
example, to do continuous integration from Mantis. All the bits are there
(well, were there, now that Bob got pulled, so perhaps this won't work) but
the tools infrastructure can certainly be used to take part in the content
development which happens in the trunk.
I, for one, like this idea. The tools are useful for more than just
3.11, right? But still, "getting there" would, I imagine, be easier
*with* them, rather than without.

Is there any chance of getting the entire set back? Or is there too much grudge?

Come on, this is about Squeak, whose progress most of us should be
interested in. ;-)

Best,

Michael
Randal L. Schwartz
2012-01-28 12:27:26 UTC
Permalink
Michael> I don't want to start new trouble, but I have an urgent question now.
Michael> The decision that has, in part, caused the turmoil was a board
Michael> decision, if I recall that correctly. If so, how could the board as a
Michael> whole have misinterpreted the 3.11 efforts - was there not enough
Michael> discussion?

[Donning my board member hat...]

To put it simply, it started with a misunderstanding on my part.

I had seen all of the tools being generated by Keith as "means to an end"... a
new better process for generating 3.11 or 4.0 or whatever else we were aiming
for. What I had missed, despite repeated attempts by Keith to tell me (as I
can now see in retrospect), was that an *actual* new *official* image wasn't
really his personal goal. I think I was confused by it having been called
3.11: I was expecting a 3.11 release "some day".

So, when it came up in the board meeting recently that while we were waiting
for 4.0 to get done (the licensing release), we were losing a way to get
people contributing, and had been for quite some time. 4.0 was getting
delayed week-by-week, and there hadn't been a new "core" release in over a
year. The success of the Pharo fork proved to us that the core of squeak was
stalled. So it was time for a reboot... get people back on.

When Andreas proposed the new trunk using existing tools, it looked like a way
to at least capture the interim desire-to-contribute, realizing that any
patches applied to 3.10.2 would have to be reharvested for 4.1 after 4.0 was
generated. But at least we had something we could bring to squeak-dev.

There was *never* any intent to discount what Keith had accomplished. But at
the end of the day, what mattered was people in squeak-dev being told how to
help, and having a way to help, and that wasn't happening. Part of this was
because the tools weren't ready for prime time yet...Keith told me a few days
ago that he still needs another month or two. Part of this was because of
something I wasn't aware of until recently... Keith hadn't been permitted to
speak on squeak-dev directly because of some work issues.

There's a lot of monday-morning quarterbacking here. We could have spent more
time to understand Keith's tools and goals. We could have made it clear that
work on the trunk *will* need to be reharvested when we get to the 4.0 code
base, either using the same tools again, or using Keith's tools if they're
ready.

But the problem we were solving was the loss of interest in helping the squeak
core move forward. And in that respect, despite some of the negative fallout,
I believe our overall goal has been met, and supported by recent activity.

Going forward, we (as a community) need to figure out if we're generating a
3.11 release from the trunk, or whether this trunk is now just a testbed for
good ideas for the 4.1 release. Part of this depends on the 4.0 release
timing, and that depends on the release team. I'm working with Matthew now to
figure out what resources are needed for that.

I'd also like to get this work back into the hands of the teams. The board
stepped in to solve the problem that didn't appear to be solving itself. But
ultimately, it's the release team(s) who should be driving this.
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Göran Krampe
2012-01-28 12:27:26 UTC
Permalink
Hi!
Post by Andreas Raab
And I think we can do
both at the same time without being in an endless heated conflict.
Comments welcome.
I admit (and I think most of us do) that I haven't fully read all posts
in all this and thus I really don't grok all angles BUT:

- I must say I am surprised to hear that the board, including you
Andreas, have totally "missed" that 3.11 was about tools & process and
not about an artifact. I mean, it must have been said about 137 times!
One can argue about if it was a smart move to designate a release to the
task of finding a process and getting tools in shape - but I find it odd
that this comes as a revelation this late in the game.

- Keith has more or less said that the trunk effort is counter
productive in comparison to his/their 3.11 effort. Personally I might
think it is a gray area, but even so - perhaps once and for all - with a
pedagogic mindset - explaining to us all how this is so and also
describing "trunk" and "bob" in some way that all of us can grok it...
would let us come to some conclusion.


Finally:

At this point I don't really give a rats about the above! :) I just want
us to heal, embrace, drink beer and find a way to move on. This "war"
has hurt us I think, I always brag about Squeak/Smalltalk being a very
friendly place, but it has slipped a bit lately. And thinking that it
might have been my posts setting off the fire doesn't really make it
feel better ;)

BTW, can we please get our current status/process "trunk" or "bob" or
whatever we are using these days written down on squeak.org in an
official manner? Perhaps this is already done.

This also goes for the 3.11 team - does it even exist anymore? Let us
get our act together.

regards, G?ran
Andreas Raab
2012-01-28 12:27:26 UTC
Permalink
Post by Göran Krampe
- I must say I am surprised to hear that the board, including you
Andreas, have totally "missed" that 3.11 was about tools & process and
not about an artifact. I mean, it must have been said about 137 times!
One can argue about if it was a smart move to designate a release to the
task of finding a process and getting tools in shape - but I find it odd
that this comes as a revelation this late in the game.
I can't speak for the board, only for myself. From my perspective this
is an issue of mislabeling. When I see something called "3.11" I expect
this to be about Squeak 3.11, the software, not a set of tools. I'm also
a little at a loss why then, if this is all about tools, so many noises
are made about the 3.11 release artifact. If the entire goal of the
process is to be able to load these tools into arbitrary versions, then
why don't we just load them into the trunk and get over it? There is
clearly still a level of disconnect somewhere but I'm hoping it's not
quite as drastic as the last one.
Post by Göran Krampe
- Keith has more or less said that the trunk effort is counter
productive in comparison to his/their 3.11 effort. Personally I might
think it is a gray area, but even so - perhaps once and for all - with a
pedagogic mindset - explaining to us all how this is so and also
describing "trunk" and "bob" in some way that all of us can grok it...
would let us come to some conclusion.
It's quite simple really. Software development works by developers
working together in a shared repository, right? That's the trunk. The
trunk is nothing but a shared repository that the developers have commit
rights to. In the course of software development, builds happen on a
regular basis. That's Bob. Bob is a tool to assemble various bits of
code and content and compile this into a result. Simple as that.

Mantis role in this process with regards to code is to assemble
contributions from non-core-developers in the project; people who are
not given direct commit rights to the repository. These contributions
need to be reviewed and integrated. That's Installer for you. The result
of the integration process goes back into the repository. There's the
trunk again. All the parts have their place in the process.
Post by Göran Krampe
At this point I don't really give a rats about the above! :) I just want
us to heal, embrace, drink beer and find a way to move on. This "war"
has hurt us I think, I always brag about Squeak/Smalltalk being a very
friendly place, but it has slipped a bit lately. And thinking that it
might have been my posts setting off the fire doesn't really make it
feel better ;)
I'm not unhappy about it. We need to get moving again. Whatever triggers
this process is a good thing. And I never expected the process to be
painfree, but I do think we are making progress if only by illustrating
that other models of contributions can actually be successful. Now we
have to work out how to make this work together to produce an actual
piece of software called Squeak.
Post by Göran Krampe
BTW, can we please get our current status/process "trunk" or "bob" or
whatever we are using these days written down on squeak.org in an
official manner? Perhaps this is already done.
I have nothing on Bob but I think the most relevant links are these:

http://installer.pbworks.com/Squeak311Proposal
http://board.squeak.org/2009/07/02/a-new-community-development-model/
Post by Göran Krampe
This also goes for the 3.11 team - does it even exist anymore?
Let us get our act together.
I leave it to Matthew to respond to this question.

Cheers,
- Andreas
Jecel Assumpcao Jr
2012-01-28 12:27:26 UTC
Permalink
I did not have any problems understanding that Keith had proposed what
Andreas has just described. But when voting for the current process, I
asked if it was going to be compatible with the ongoing 3.11 effort and
was told yes. Even though I now see this was due to some confusion, I
still believe "yes" was actually correct and is what is now being
detailed.

To me, it isn't at all Keith's fault that people didn't understand what
he was trying to do. Nor do I blame anyone who didn't get it before now
(or still doesn't understand). The problem is mostly with the various
communication channels we have. There are several examples I could give
of failing miserably to explain something over email (even with attached
drawings) to really brilliant people while at the same time being able
to explain the exact same thing live to far less brilliant audiences.

Of course, sometimes we think we have understood when we realling
haven't (see people who think they "got" OOP after playing a bit with
C++, for example) and that is when discusions become really complicated.

So I should really say I *think* I have understood Keith's efforts,
except for one detail: in a system that is about automation and
contributions from everyone there seems to be a lot of manually edited
scripts by single authors. The explanation is that I am probably
thinking about things backwards (in terms of which script calls which
script), but that is a another thread.
Post by Andreas Raab
I can't speak for the board, only for myself. From my perspective this
is an issue of mislabeling. When I see something called "3.11" I expect
this to be about Squeak 3.11, the software, not a set of tools. I'm also
a little at a loss why then, if this is all about tools, so many noises
are made about the 3.11 release artifact. If the entire goal of the
process is to be able to load these tools into arbitrary versions, then
why don't we just load them into the trunk and get over it? There is
clearly still a level of disconnect somewhere but I'm hoping it's not
quite as drastic as the last one.
This was caused by a series of steps in which things got renamed. The
whole thing started when the previous board announced "Squeak 3.x is
dead, 4.0 is Spoon" and Keith reacted saying he had already put in a lot
of effort for a process for future 3.x versions and that others felt the
same and would fork if needed. After discussions, the board approved
Keith's project as an official 3.11 with 4.0 now being just a clean
license version of 3.x and Spoon renamed 5.0 (the current board has not
discussed the latter at all).

So Keith was just saying "3.x forever!" (consistent with his proposal)
while the board used the 3.11 name.

Since we are having this discussion on #squeak-dev, it is important for
the community to know that each board member can have their own opinions
but then we reach a common decision. For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever). But if the community likes Monticello,
that will get my vote.

-- Jecel
Colin Putney
2012-01-28 12:27:26 UTC
Permalink
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?

Colin
Jecel Assumpcao Jr
2012-01-28 12:27:27 UTC
Permalink
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
can be found in the Chunky Squeak wiki page:

http://wiki.squeak.org/squeak/584

The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.

-- Jecel
Igor Stasenko
2012-01-28 12:27:27 UTC
Permalink
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods ?are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
Post by Jecel Assumpcao Jr
-- Jecel
--
Best regards,
Igor Stasenko AKA sig.
Eliot Miranda
2012-01-28 12:27:27 UTC
Permalink
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
it is possible to have your cake and eat it too. One can create a
binary format that includes source and includes the meta-source for its
creation. But including a binary representation allows much faster loading,
loading without a
compiler, and source hiding if one choses not to include the source.

There are other advantages, such as not cluttering up the changes file
when one loads a package In the VW parcel system, to which I added
source management, we replaced the SourceFiles with a
SourceFileManager whose job was to manage the sources and changes file
and an arbitrary number of source files for parcels, the binary
format. In
the parcel file the source pointers of compiled methods are the positions of
their source in the parcel source file. When one loads a parcel the
SourceFileManager adds the file to its set of managed files and assigns an
index for the source file. The parcle loader then swizzles all the source
pointers so that they include the source file index along with the position.
So accessing the source for a method loaded form a parcel accesses that
parcel's source file. We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa was
the position in the file.

We didn't create a single file format, having two separate files for binary
and source, which is probably a mistake. A format with a short header,
followed by source, followed by binary, followed by metasource, would be
easier to manage than three separate files.

We didn't include any metasource, but we did include pre-read, load and
unload actions. I did a very bad job on version numbering and prerequisite
selection.

That's not the whole story but enough to start answering your question. If
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.

best
Eliot
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
-- Jecel
--
Best regards,
Igor Stasenko AKA sig.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090819/890645e4/attachment.htm
Igor Stasenko
2012-01-28 12:27:27 UTC
Permalink
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods ?are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
?? ?it is possible to have your cake and eat it too. ?One can create a
binary format that includes source and includes the meta-source for its
creation. ?But including a binary representation allows much faster loading,
loading without a
compiler,?and?source?hiding?if?one?choses?not?to?include?the?source.
There?are?other?advantages,?such?as?not?cluttering?up?the?changes?file?when?one?loads?a?package??In?the?VW?parcel?system,?to?which?I?added?source?management,?we?replaced?the?SourceFiles?with?a?SourceFileManager?whose?job?was?to?manage?the?sources?and?changes?file?and?an?arbitrary?number?of?source?files?for?parcels,?the?binary?format.??In
the parcel file the source pointers of compiled methods are the positions of
their source in the parcel source file. ?When one loads a parcel the
SourceFileManager adds the file to its set of managed files and assigns an
index for the source file. ?The parcle loader then swizzles all the source
pointers so that they include the source file index along with the position.
?So accessing the source for a method loaded form a parcel accesses that
parcel's source file. ?We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa was
the position in the file.
We didn't create a single file format, having two separate files for binary
and source, which is probably a mistake. ?A format with a short header,
followed by source, followed by binary, followed by metasource, would be
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load and
unload actions. ?I did a very bad job on version numbering and prerequisite
selection.
That's not the whole story but enough to start answering your question. ?If
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)

Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.

So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
Eliot Miranda
2012-01-28 12:27:44 UTC
Permalink
Post by Jecel Assumpcao Jr
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would
mention
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
Projects and Etoys here) than the current source based things we
are
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few
drawings)
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or
even
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
it is possible to have your cake and eat it too. One can create a
binary format that includes source and includes the meta-source for its
creation. But including a binary representation allows much faster
loading,
Post by Eliot Miranda
loading without a
compiler, and source hiding if one choses not to include the source.
There are other advantages, such as not cluttering up the changes file when one loads a package In the VW parcel system, to which I added source management, we replaced the SourceFiles with a SourceFileManager whose job was to manage the sources and changes file and an arbitrary number of source files for parcels, the binary format. In
Post by Eliot Miranda
the parcel file the source pointers of compiled methods are the positions
of
Post by Eliot Miranda
their source in the parcel source file. When one loads a parcel the
SourceFileManager adds the file to its set of managed files and assigns
an
Post by Eliot Miranda
index for the source file. The parcle loader then swizzles all the
source
Post by Eliot Miranda
pointers so that they include the source file index along with the
position.
Post by Eliot Miranda
So accessing the source for a method loaded form a parcel accesses that
parcel's source file. We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa
was
Post by Eliot Miranda
the position in the file.
We didn't create a single file format, having two separate files for
binary
Post by Eliot Miranda
and source, which is probably a mistake. A format with a short header,
followed by source, followed by binary, followed by metasource, would be
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load and
unload actions. I did a very bad job on version numbering and
prerequisite
Post by Eliot Miranda
selection.
That's not the whole story but enough to start answering your question.
If
Post by Eliot Miranda
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can
comprehend
Post by Eliot Miranda
the binary package's contents by examining the metasource and can
reproduce
Post by Eliot Miranda
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)
We didn't disallow representation of arbitrary data but we also didn't
support it. The only thing the Parcel system supports (as in the tool set,
rather than what one can extend the framework to do in specific
circumstances) is to represent code, which it does very well.

What are these mistakes? Can you be specific? I think the parcel system
has been a major success. VW is now deployed as a system of components, the
base image and a much larger suite of parcels. Parcels are not tied to a
particular version or implementation and yet are still fast to publish and
load. What's not to like?
Post by Jecel Assumpcao Jr
Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.
So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090824/fd974633/attachment.htm
Igor Stasenko
2012-01-28 12:27:44 UTC
Permalink
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods ?are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
?? ?it is possible to have your cake and eat it too. ?One can create a
binary format that includes source and includes the meta-source for its
creation. ?But including a binary representation allows much faster loading,
loading without a
compiler,?and?source?hiding?if?one?choses?not?to?include?the?source.
There?are?other?advantages,?such?as?not?cluttering?up?the?changes?file?when?one?loads?a?package??In?the?VW?parcel?system,?to?which?I?added?source?management,?we?replaced?the?SourceFiles?with?a?SourceFileManager?whose?job?was?to?manage?the?sources?and?changes?file?and?an?arbitrary?number?of?source?files?for?parcels,?the?binary?format.??In
the parcel file the source pointers of compiled methods are the positions of
their source in the parcel source file. ?When one loads a parcel the
SourceFileManager adds the file to its set of managed files and assigns an
index for the source file. ?The parcle loader then swizzles all the source
pointers so that they include the source file index along with the position.
?So accessing the source for a method loaded form a parcel accesses that
parcel's source file. ?We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa was
the position in the file.
We didn't create a single file format, having two separate files for binary
and source, which is probably a mistake. ?A format with a short header,
followed by source, followed by binary, followed by metasource, would be
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load and
unload actions. ?I did a very bad job on version numbering and prerequisite
selection.
That's not the whole story but enough to start answering your question. ?If
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)
We didn't disallow representation of arbitrary data but we also didn't
support it. ?The only thing the Parcel system supports (as in the tool set,
rather than what one can extend the framework to do in specific
circumstances) is to represent code, which it does very well.
What are these mistakes? ?Can you be specific? ?I think the parcel system
has been a major success. ?VW is now deployed as a system of components, the
base image and a much larger suite of parcels. ?Parcels are not tied to a
particular version or implementation and yet are still fast to publish and
load. ?What's not to like?
I referred mainly to your own statements about mistake(s).
I don't know about parcels so much to tell exactly where is the flaws.
I'm still wondering, how you could unload a parcel if its not longer needed, but
there are still object(s) which used/created by parcel sitting in image.
A basic use case is: developer needs some specific tool (like UI
design tool) when he working
on application. But at the moment when he ships the application, it is
no longer needed.
Post by Eliot Miranda
Post by Igor Stasenko
Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.
So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
Eliot Miranda
2012-01-28 12:27:45 UTC
Permalink
Post by Jecel Assumpcao Jr
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would mention
Projects and Etoys here) than the current source based things we are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but
would
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel
I
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand
crafted
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the
case
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such situation(s).
it is possible to have your cake and eat it too. One can create a
binary format that includes source and includes the meta-source for
its
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
creation. But including a binary representation allows much faster loading,
loading without a
compiler, and source hiding if one choses not to include the source.
There are other advantages, such as not cluttering up the changes file when one loads a package In the VW parcel system, to which I added source management, we replaced the SourceFiles with a SourceFileManager whose job was to manage the sources and changes file and an arbitrary number of source files for parcels, the binary format. In
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
the parcel file the source pointers of compiled methods are the positions of
their source in the parcel source file. When one loads a parcel the
SourceFileManager adds the file to its set of managed files and
assigns
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
an
index for the source file. The parcle loader then swizzles all the source
pointers so that they include the source file index along with the position.
So accessing the source for a method loaded form a parcel accesses
that
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
parcel's source file. We used a floating-point like format for source
pointers, where the exponent was the source file index, and the
mantissa
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
was
the position in the file.
We didn't create a single file format, having two separate files for binary
and source, which is probably a mistake. A format with a short
header,
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
followed by source, followed by binary, followed by metasource, would
be
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load
and
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
unload actions. I did a very bad job on version numbering and prerequisite
selection.
That's not the whole story but enough to start answering your
question.
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
If
there is a well-defined definition of the objects in a package and
that
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
definition is included in the package as metasource, then one can comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)
We didn't disallow representation of arbitrary data but we also didn't
support it. The only thing the Parcel system supports (as in the tool
set,
Post by Eliot Miranda
rather than what one can extend the framework to do in specific
circumstances) is to represent code, which it does very well.
What are these mistakes? Can you be specific? I think the parcel system
has been a major success. VW is now deployed as a system of components,
the
Post by Eliot Miranda
base image and a much larger suite of parcels. Parcels are not tied to a
particular version or implementation and yet are still fast to publish
and
Post by Eliot Miranda
load. What's not to like?
I referred mainly to your own statements about mistake(s).
Ah, ok, Sorry :)
Post by Jecel Assumpcao Jr
I don't know about parcels so much to tell exactly where is the flaws.
I'm still wondering, how you could unload a parcel if its not longer needed, but
there are still object(s) which used/created by parcel sitting in image.
Smalltalk has this problem with or without binary loading; they're called
obsolete classes :) However, the problem of knowing what to remove when the
user says "unload" means that a loaded parcel requires a data structure that
names the classes and methods it loaded. In addition we maintain overrides,
the older versions of methods and class definitions, in a stack, so that
these can be restored when unloading a parcel. I made lots of mistakes here
(not allowing the tools to publish a parcel that has code overridden by
others, not integrating source management and browsing queries with
overridden code, not compressing the changes correctly with overridden code,
etc, etc). Tests would have helped :/

VW did (does?) test for open instances of applications when we unload a
parcel so that if the parcel contains a subclass(s) of ApplicationModel
(VW's top-level GUI app class) all open applications are tested to see if
they contain instances of the class(es) and a warning is issued.

A basic use case is: developer needs some specific tool (like UI
Post by Jecel Assumpcao Jr
design tool) when he working
on application. But at the moment when he ships the application, it is
no longer needed.
Right. I don't know of an automatic solution, but a good convention is to
split all packages into a development and deployment pair where
the deployment half is a prerequisite of the development half. Sticking to
the convention and using good names makes it easier to remember to remove
deevelopment components and to guess which parts of someone else's
components are development only.

I added a bulk instancesOf primitive that answered all instances of an Array
of classes that my colleague Steve Dahl wanted to use in instance migration
on class redefinition. This could be used to look for all instances of the
classes defined by a parcel prior to unload. Do a GC, collect all instaces
of classes defined (rather than redefined) by a parcel and warn if non-empty
(if in a dev image).
Post by Jecel Assumpcao Jr
Post by Eliot Miranda
Obviously one of the side of such problem is uniform object memory,
Post by Igor Stasenko
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.
So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090825/b7e69fb3/attachment.htm
Jecel Assumpcao Jr
2012-01-28 12:27:45 UTC
Permalink
Eliot,

thanks for all your wonderful comments and insights about the Parcel
system in VisualWorks. My experience with it is extremely limited (I
once loaded idass, the chip simulation system, as a parcel into VW 5i
NC) and so I cited V/Win as an existence proof.

You are correct that an object table wouldn't help in general when
comparing two images - I was thinking of the specific case of when one
is known to be directly derived from the other like Squeak 3.8 from 3.7.
This was from the discussion of doing a security audit to allow Squeak
to be included in Debian.
Comparing unordered collections has all the complications you mentioned
and in Smalltalk this is supposed to be solved in #=, which experience
tells us not to trust too much.

For Neo Smalltalk I didn't do pure memory dumps but had a binary format
that was reasonably compressed. And it didn't have small integers but
only variable sized ones and these became SmallIntegers or LargeIntegers
when read in. That made the binary format compatible between the 16 and
36 bit versions of Neo Smalltalk.

One idea for Neo modules that Dan thought a bit excessive was to divide
each into four related modules: the actual objects (I'll call this the
"deployment module" to use your term), the sources (just a bunch of
String objects), the documentation (nicely formated text, with possibly
pictures or even movies) and the tests. In different situations you
might want different subsets of these. For example, while browsing
through SqueakMap you might click on "see more..." and get the full docs
in your machine. Then you might click on an example and create a new
object (in a new module) that would bring in the deployment module to
support it. If you ever try to look at the code for this new object in
the system browser or the debugger then the sources module would get
loaded. There would be links to the tests in the documentation but they
might also get loaded through the SUnit tool. If you close all windows
with the documentation, the doc module will eventually be unloaded.

Of course, I am supposing that objects in a module can point to objects
in a separate module in the above description. And that module
loading/unloading is a kind of crude virtual memory.

-- Jecel
Igor Stasenko
2012-01-28 12:27:46 UTC
Permalink
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would
mention
Projects and Etoys here) than the current source based things we
are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer something
spoonish, where CompiledMethods ?are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had actually
started to explain this a little in the original email, I ended up
deleting it to keep on topic. With a new subject line I don't feel I
have to worry about that. Some details about this (with a few drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or
even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such
situation(s).
?? ?it is possible to have your cake and eat it too. ?One can create a
binary format that includes source and includes the meta-source for its
creation. ?But including a binary representation allows much faster loading,
loading without a
compiler,?and?source?hiding?if?one?choses?not?to?include?the?source.
There?are?other?advantages,?such?as?not?cluttering?up?the?changes?file?when?one?loads?a?package??In?the?VW?parcel?system,?to?which?I?added?source?management,?we?replaced?the?SourceFiles?with?a?SourceFileManager?whose?job?was?to?manage?the?sources?and?changes?file?and?an?arbitrary?number?of?source?files?for?parcels,?the?binary?format.??In
the parcel file the source pointers of compiled methods are the positions of
their source in the parcel source file. ?When one loads a parcel the
SourceFileManager adds the file to its set of managed files and assigns
an
index for the source file. ?The parcle loader then swizzles all the source
pointers so that they include the source file index along with the position.
?So accessing the source for a method loaded form a parcel accesses that
parcel's source file. ?We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa
was
the position in the file.
We didn't create a single file format, having two separate files for binary
and source, which is probably a mistake. ?A format with a short header,
followed by source, followed by binary, followed by metasource, would be
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load and
unload actions. ?I did a very bad job on version numbering and prerequisite
selection.
That's not the whole story but enough to start answering your question.
?If
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this way
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)
We didn't disallow representation of arbitrary data but we also didn't
support it. ?The only thing the Parcel system supports (as in the tool set,
rather than what one can extend the framework to do in specific
circumstances) is to represent code, which it does very well.
What are these mistakes? ?Can you be specific? ?I think the parcel system
has been a major success. ?VW is now deployed as a system of components, the
base image and a much larger suite of parcels. ?Parcels are not tied to a
particular version or implementation and yet are still fast to publish and
load. ?What's not to like?
I referred mainly to your own statements about mistake(s).
Ah, ok, ?Sorry :)
Post by Igor Stasenko
I don't know about parcels so much to tell exactly where is the flaws.
I'm still wondering, how you could unload a parcel if its not longer needed, but
there are still object(s) which used/created by parcel sitting in image.
Smalltalk has this problem with or without binary loading; they're called
obsolete classes :) ?However, the problem of knowing what to remove when the
user says "unload" means that a loaded parcel requires a data structure that
names the classes and methods it loaded. ?In addition we maintain overrides,
the older versions of methods and class definitions, in a stack, so that
these can be restored when unloading a parcel. ?I made lots of mistakes here
(not allowing the tools to publish a parcel that has code overridden by
others, not integrating source management and browsing queries with
overridden code, not compressing the changes correctly with overridden code,
etc, etc). ?Tests would have helped :/
VW did (does?) test for open instances of applications when we unload a
parcel so that if the parcel contains a subclass(s) of ApplicationModel
(VW's top-level GUI app class) all open applications are tested to see if
they contain instances of the class(es) and a warning is issued.
Post by Igor Stasenko
A basic use case is: developer needs some specific tool (like UI
design tool) when he working
on application. But at the moment when he ships the application, it is
no longer needed.
Right. ?I don't know of an automatic solution, but a good convention is to
split all packages into a development and deployment pair where
the?deployment?half is a prerequisite of the?development half. ?Sticking to
the convention and using good names makes it easier to remember to remove
deevelopment components and to guess which parts of someone else's
components are development only.
Yes, and this is what i really missing in smalltalk-80 based
environments: distinction between development
and deployment modes & models.
It would be cool to have some basic things to behave different when in
deployed mode (like preventing access & data overrides).
The main problem in open system (such as smalltalk object memory) is
that when something goes wrong, often you
having two choices: reboot the system or debug and fix the problem in
a living environment.
Often, none of the choices is acceptable, because if we are talking
about end-user application, we don't expect that
user is able to debug & fix the issue. As well as rebooting an image
means loss of data and/or interruption of serving other jobs.

But, if system modelled in modular layers , like kernel -> services ->
interfaces -> working set, then things
would be much easier to handle.
I added a bulk instancesOf primitive that answered all instances of an Array
of classes that my colleague Steve Dahl wanted to use in instance migration
on class redefinition. ?This could be used to look for all instances of the
classes defined by a parcel prior to unload. ?Do a GC, collect all instaces
of classes defined (rather than redefined) by a parcel and warn if non-empty
(if in a dev image).
I think that independent tiny layers (isles/vats) is the future system
organization in smalltalk-like VMs.
First, it gives the strong answer to question, what belongs to what.
There is no possibility to reference a foreign object
other than by far ref. You can count/enumerate them easily, and this
approach also makes possible to run code in vats concurrently.
The problem here is how to handle the shared behavior, like Arrays,
Collections etc in order to avoid duplication. Since in smalltalk
everything is objects, and so methods & classes too, they can belong
only to a single island/vat, and therefore , only owning island can
manipulate with it. This creates a major bottleneck in effective
implementation of concurrently (and independently) running the code.
Trade space for speed? Allow each island to have own Array class with
own implementation?
This question remains open for me.
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
?Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of people.
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains both
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome nowadays.
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.
So, i think , that before entering this domain (allowing binary data),
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could return
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
Eliot Miranda
2012-01-28 12:27:46 UTC
Permalink
Post by Michael Haupt
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Hi Igor,
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
Post by Colin Putney
Post by Jecel Assumpcao Jr
For example, I would far prefer to
see Squeak move to a binary based development model (I would
mention
Projects and Etoys here) than the current source based things
we
are
doing (trunk, bob or whatever).
Forgive me for seizing on a throw-away comment like this, but would
you mind expanding on this a bit? Are you saying you prefer
something
spoonish, where CompiledMethods are passed directly from image to
image? Something else?
Heh, I got asked about this on IRC as well. Though I had
actually
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
started to explain this a little in the original email, I ended
up
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
deleting it to keep on topic. With a new subject line I don't
feel
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
I
have to worry about that. Some details about this (with a few
drawings)
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load
binary
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Jecel Assumpcao Jr
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or
even
some text). This is very simplistic compared to Spoon, and my
proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a
different
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
system. But what you propose to do with pure binary data, a soup
of
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
I don't think that developers of Squeak should be victims of such
situation(s).
it is possible to have your cake and eat it too. One can
create
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
a
binary format that includes source and includes the meta-source for its
creation. But including a binary representation allows much faster
loading,
loading without a
compiler, and source hiding if one choses not to include the source.
There are other advantages, such as not cluttering up the changes file when one loads a package In the VW parcel system, to which I added source management, we replaced the SourceFiles with a SourceFileManager whose job was to manage the sources and changes file and an arbitrary number of source files for parcels, the binary format. In
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
the parcel file the source pointers of compiled methods are the
positions of
their source in the parcel source file. When one loads a parcel
the
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
SourceFileManager adds the file to its set of managed files and assigns
an
index for the source file. The parcle loader then swizzles all the source
pointers so that they include the source file index along with the
position.
So accessing the source for a method loaded form a parcel accesses that
parcel's source file. We used a floating-point like format for source
pointers, where the exponent was the source file index, and the mantissa
was
the position in the file.
We didn't create a single file format, having two separate files
for
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
binary
and source, which is probably a mistake. A format with a short header,
followed by source, followed by binary, followed by metasource,
would
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
be
easier to manage than three separate files.
We didn't include any metasource, but we did include pre-read, load and
unload actions. I did a very bad job on version numbering and prerequisite
selection.
That's not the whole story but enough to start answering your question.
If
there is a well-defined definition of the objects in a package and that
definition is included in the package as metasource, then one can
comprehend
the binary package's contents by examining the metasource and can reproduce
creating the package, provided that the tools are careful to impose
ordering, etc.
best
Eliot
I think you inevitably made wrong decisions, because you went this
way
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
by allowing an
arbitrary binary data , held by package.
In such situations it is much more easier to make a mistakes.
But sure, one who's making no mistakes is one who doing nothing :)
We didn't disallow representation of arbitrary data but we also didn't
support it. The only thing the Parcel system supports (as in the tool set,
rather than what one can extend the framework to do in specific
circumstances) is to represent code, which it does very well.
What are these mistakes? Can you be specific? I think the parcel system
has been a major success. VW is now deployed as a system of
components,
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
the
base image and a much larger suite of parcels. Parcels are not tied
to
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
a
particular version or implementation and yet are still fast to publish and
load. What's not to like?
I referred mainly to your own statements about mistake(s).
Ah, ok, Sorry :)
Post by Igor Stasenko
I don't know about parcels so much to tell exactly where is the flaws.
I'm still wondering, how you could unload a parcel if its not longer needed, but
there are still object(s) which used/created by parcel sitting in image.
Smalltalk has this problem with or without binary loading; they're called
obsolete classes :) However, the problem of knowing what to remove when
the
Post by Eliot Miranda
user says "unload" means that a loaded parcel requires a data structure
that
Post by Eliot Miranda
names the classes and methods it loaded. In addition we maintain
overrides,
Post by Eliot Miranda
the older versions of methods and class definitions, in a stack, so that
these can be restored when unloading a parcel. I made lots of mistakes
here
Post by Eliot Miranda
(not allowing the tools to publish a parcel that has code overridden by
others, not integrating source management and browsing queries with
overridden code, not compressing the changes correctly with overridden
code,
Post by Eliot Miranda
etc, etc). Tests would have helped :/
VW did (does?) test for open instances of applications when we unload a
parcel so that if the parcel contains a subclass(s) of ApplicationModel
(VW's top-level GUI app class) all open applications are tested to see if
they contain instances of the class(es) and a warning is issued.
Post by Igor Stasenko
A basic use case is: developer needs some specific tool (like UI
design tool) when he working
on application. But at the moment when he ships the application, it is
no longer needed.
Right. I don't know of an automatic solution, but a good convention is
to
Post by Eliot Miranda
split all packages into a development and deployment pair where
the deployment half is a prerequisite of the development half. Sticking
to
Post by Eliot Miranda
the convention and using good names makes it easier to remember to remove
deevelopment components and to guess which parts of someone else's
components are development only.
Yes, and this is what i really missing in smalltalk-80 based
environments: distinction between development
and deployment modes & models.
It would be cool to have some basic things to behave different when in
deployed mode (like preventing access & data overrides).
The main problem in open system (such as smalltalk object memory) is
that when something goes wrong, often you
having two choices: reboot the system or debug and fix the problem in
a living environment.
Often, none of the choices is acceptable, because if we are talking
about end-user application, we don't expect that
user is able to debug & fix the issue. As well as rebooting an image
means loss of data and/or interruption of serving other jobs.
Yes, I agree. One of the things the headless support in VW allows which is
quite nice is taking a shapshot which can then be restarted in a headless
mode for debugging. This can easily be mailed or ftp'ed back for analysis.

Not quite the same, but very neat: The other day at Qwaq Craig Latta had a
VM crash while running in a Parallels Linux VM under gdb. He was able to
give me a copy of the VM snapshot at the point where gdb stopped the
process, giving me the opportunity to debug the live app at my leisure. A
cool idea.
Post by Michael Haupt
But, if system modelled in modular layers , like kernel -> services ->
interfaces -> working set, then things
would be much easier to handle.
Yes, yes, yes!! The system should be like an onion where each layer of the
onion is a set of interlocking techtonic plates of modules of functionality.
Post by Michael Haupt
Post by Eliot Miranda
I added a bulk instancesOf primitive that answered all instances of an
Array
Post by Eliot Miranda
of classes that my colleague Steve Dahl wanted to use in instance
migration
Post by Eliot Miranda
on class redefinition. This could be used to look for all instances of
the
Post by Eliot Miranda
classes defined by a parcel prior to unload. Do a GC, collect all
instaces
Post by Eliot Miranda
of classes defined (rather than redefined) by a parcel and warn if
non-empty
Post by Eliot Miranda
(if in a dev image).
I think that independent tiny layers (isles/vats) is the future system
organization in smalltalk-like VMs.
First, it gives the strong answer to question, what belongs to what.
There is no possibility to reference a foreign object
other than by far ref. You can count/enumerate them easily, and this
approach also makes possible to run code in vats concurrently.
The problem here is how to handle the shared behavior, like Arrays,
Collections etc in order to avoid duplication. Since in smalltalk
everything is objects, and so methods & classes too, they can belong
only to a single island/vat, and therefore , only owning island can
manipulate with it. This creates a major bottleneck in effective
implementation of concurrently (and independently) running the code.
Trade space for speed? Allow each island to have own Array class with
own implementation?
This question remains open for me.
Yes, this is a cool radical idea that I haven't got my head around yet. I
need to think about this at length. The obvious approach to the duplication
is copy-on-write where any modifications to the root Array class get
propagated to the copies, assuming there is some hierarchical control
organization. I think this approach is taken in Alex's worlds where
modifications to a parent world are seen my children. But then the merge
problem rears its head when trying to propagate modifications to a child
that has made its own local modifications in the same region.
Post by Michael Haupt
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Obviously one of the side of such problem is uniform object memory,
where each object could
reference any other object and limited only by a imagination of
people.
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
There is no layers or any other means which could establish a certain
barriers (which we calling a modules)
in smalltalk.
It means, that once you integrated the parcel into image, and started
using it, you may have a hard times trying to unload it.
It is possible to develop an image as an artifact, which contains
both
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
binary & sources , but such approach
having a drawbacks, which we, by the way, trying to overcome
nowadays.
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
Practice shows that such approach is credible only
for a small group of individuals, but becomes a bottleneck if you
adopt such scheme for a wider community.
So, i think , that before entering this domain (allowing binary
data),
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
first we should solve more basic problems of smalltalk & its design -
modularity, name spaces, layering & etc etc.. Only the we could
return
Post by Eliot Miranda
Post by Igor Stasenko
Post by Eliot Miranda
Post by Igor Stasenko
to original question and solve it.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
--
Best regards,
Igor Stasenko AKA sig.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090825/ea220f7a/attachment.htm
David Farber
2012-01-28 12:27:46 UTC
Permalink
One of the things the headless support in VW allows which is quite
nice is taking a shapshot which can then be restarted in a headless
mode for debugging.
Eliot - Was this VM or image-side support? Can you describe how it
worked?
Not quite the same, but very neat: The other day at Qwaq Craig
Latta had a VM crash while running in a Parallels Linux VM under
gdb. He was able to give me a copy of the VM snapshot at the point
where gdb stopped the process, giving me the opportunity to debug
the live app at my leisure. A cool idea.
In other words, he was already running the Qwaq VM under gdb, so when
the Qwaq VM crashed (and left him at a gdb prompt), he simply
suspended the Parallels Linux VM and sent you a copy of the suspended
Parallels Linux VM. Is that right?

David
Eliot Miranda
2012-01-28 12:27:47 UTC
Permalink
One of the things the headless support in VW allows which is quite nice is
taking a shapshot which can then be restarted in a headless mode for
debugging.
Eliot - Was this VM or image-side support? Can you describe how it worked?
All image side; no VM support. I can't describe it in detail as I never
worked on it. But the basic idea is to have a switchable headless system in
the image so that in deployment it runs headless and when an error occurs
the error handler temporarily throws the switch, saves a snapshot and resets
the switch. When the snapshot is restarted it will come up headful because
the switch was thrown before the snapshot was taken. In the resuming
snapshot the error handler then does something like an on: .. do: [:ex| self
shapshotHeadful. ex pass] to allow the GUI to display the debugger.

Download VW non-commercial and play with the headless support if you want
details.


Not quite the same, but very neat: The other day at Qwaq Craig Latta had a
VM crash while running in a Parallels Linux VM under gdb. He was able to
give me a copy of the VM snapshot at the point where gdb stopped the
process, giving me the opportunity to debug the live app at my leisure. A
cool idea.
In other words, he was already running the Qwaq VM under gdb, so when the
Qwaq VM crashed (and left him at a gdb prompt), he simply suspended the
Parallels Linux VM and sent you a copy of the suspended Parallels Linux VM.
Is that right?
Right. Very nice, as the patient is on the table.
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090826/e1cc4cce/attachment.htm
David Farber
2012-01-28 12:27:47 UTC
Permalink
One of the things the headless support in VW allows which is quite
nice is taking a shapshot which can then be restarted in a headless
mode for debugging.
Eliot - Was this VM or image-side support? Can you describe how it worked?
All image side; no VM support. I can't describe it in detail as I
never worked on it. But the basic idea is to have a switchable
headless system in the image so that in deployment it runs headless
and when an error occurs the error handler temporarily throws the
switch, saves a snapshot and resets the switch. When the snapshot
is restarted it will come up headful because the switch was thrown
before the snapshot was taken. In the resuming snapshot the error
handler then does something like an on: .. do: [:ex| self
shapshotHeadful. ex pass] to allow the GUI to display the debugger.
I was hoping you were going to say VM side. :) Or, rather, I was
hoping that the context of your original message was initiating a
snapshot from outside a image (i.e. in response to a nightly cron job
to backup the image or a system shutdown event).

But, back to headless/headfull for a moment, doesn't Squeak do this
automatically already? When I run an image in headless mode, all
that happens is that no OS window is opened--but the Squeak UI
objects are all there (and running) in the image. Does VW do
something more, like destroy all the UI objects?
Not quite the same, but very neat: The other day at Qwaq Craig
Latta had a VM crash while running in a Parallels Linux VM under
gdb. He was able to give me a copy of the VM snapshot at the point
where gdb stopped the process, giving me the opportunity to debug
the live app at my leisure. A cool idea.
In other words, he was already running the Qwaq VM under gdb, so
when the Qwaq VM crashed (and left him at a gdb prompt), he simply
suspended the Parallels Linux VM and sent you a copy of the
suspended Parallels Linux VM. Is that right?
Right. Very nice, as the patient is on the table.
Yeah, full OS virtualization has some pretty cool use cases.

David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090826/6e28daa4/attachment.htm
Eliot Miranda
2012-01-28 12:27:47 UTC
Permalink
Post by Eliot Miranda
One of the things the headless support in VW allows which is quite nice
is taking a shapshot which can then be restarted in a headless mode for
debugging.
Eliot - Was this VM or image-side support? Can you describe how it worked?
All image side; no VM support. I can't describe it in detail as I never
worked on it. But the basic idea is to have a switchable headless system in
the image so that in deployment it runs headless and when an error occurs
the error handler temporarily throws the switch, saves a snapshot and resets
the switch. When the snapshot is restarted it will come up headful because
the switch was thrown before the snapshot was taken. In the resuming
snapshot the error handler then does something like an on: .. do: [:ex| self
shapshotHeadful. ex pass] to allow the GUI to display the debugger.
I was hoping you were going to say VM side. :) Or, rather, I was hoping
that the context of your original message was initiating a snapshot from
outside a image (i.e. in response to a nightly cron job to backup the image
or a system shutdown event).
But, back to headless/headfull for a moment, doesn't Squeak do this
automatically already? When I run an image in headless mode, all that
happens is that no OS window is opened--but the Squeak UI objects are all
there (and running) in the image.
Right.
Post by Eliot Miranda
Does VW do something more, like destroy all the UI objects?
No. Its just a switch and routes to either the GUI or e.g. standard i/o.
Post by Eliot Miranda
Not quite the same, but very neat: The other day at Qwaq Craig Latta had
a VM crash while running in a Parallels Linux VM under gdb. He was able to
give me a copy of the VM snapshot at the point where gdb stopped the
process, giving me the opportunity to debug the live app at my leisure. A
cool idea.
In other words, he was already running the Qwaq VM under gdb, so when the
Qwaq VM crashed (and left him at a gdb prompt), he simply suspended the
Parallels Linux VM and sent you a copy of the suspended Parallels Linux VM.
Is that right?
Right. Very nice, as the patient is on the table.
Yeah, full OS virtualization has some pretty cool use cases.
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090826/5a1f3117/attachment.htm
Jecel Assumpcao Jr
2012-01-28 12:27:46 UTC
Permalink
Post by Eliot Miranda
Yes, this is a cool radical idea that I haven't got my head around yet.
?I need to think about this at length. ?The obvious approach to the
duplication is copy-on-write where any modifications to the root
Array class get propagated to the copies, assuming there is some
hierarchical control organization. ?I think this approach is taken in
Alex's worlds where modifications to a parent world are seen my
children. ?But then the merge problem rears its head when trying
to propagate modifications to a child that has made its own local
modifications in the same region.
I have some papers about this (1992 and 1993) but since they are in
Portuguese it makes no sense for me to point them out. The basic idea is
the MESI cache coherence protocol from bus based multiprocessors
(network based multiprocessors normally use directory based schemes
which are closer to what we want but harder to explain so I will start
out with MESI).

Any given cache line, or object in our case, can be either Invalid
(meaning the local node doesn't have a copy), Exclusive (the local node
has a copy and knows that nobody else does), Shared (there is a local
copy and possibly other nodes also have copies) and Modified (the local
copy has been changed and must be saved to main memory).

- You can go from I to either E or S (which one depends on what the
other caches say) by fetching a copy.
- You can go from E to S if you see anybody else fetch a copy.
- You can go from S to E by asking everybody else go from S to I and
inform you they have done so.
- You can go from E to M by writing to your copy.
- You can go from M to E by saving your copy to main memory.
- You can go from E or S to I if you need to reuse the cache line for
other data.

No other transitions are allowed (perhaps this would be far easier to
understand as a drawing?). This scheme doesn't need to merge since there
is at most one changed copy at any given time. This restricts
parallelism compared to multiple worlds, but is compatible with our
current semantics.

Note that David Ungar showed something very similar to this actually
running in Squeak on 56 processors at least year's OOPSLA and the movie
of his demo (thanks, G?ran!) is available online (at
http://siliconsqueak.org among other places).

-- Jecel
Randal L. Schwartz
2012-01-28 12:27:46 UTC
Permalink
Jecel> - You can go from I to either E or S (which one depends on what the
Jecel> other caches say) by fetching a copy.
Jecel> - You can go from E to S if you see anybody else fetch a copy.
Jecel> - You can go from S to E by asking everybody else go from S to I and
Jecel> inform you they have done so.
Jecel> - You can go from E to M by writing to your copy.
Jecel> - You can go from M to E by saving your copy to main memory.
Jecel> - You can go from E or S to I if you need to reuse the cache line for
Jecel> other data.

digraph Jecel {
I -> {E; S} [label = "fetch"];
E -> S [label = "other fetch"];
S -> E [label = "force other S->I"];
E -> M [label = "write"];
M -> E [label = "save"];
{E; S} -> I [label = "reuse"];
}

Name it "jecel.dot", read it into OmniGraffle or any Graphviz Tool.

:)
--
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<***@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.vox.com/ for Smalltalk and Seaside discussion
Jecel Assumpcao Jr
2012-01-28 12:27:27 UTC
Permalink
Post by Igor Stasenko
The central question, which arising immediately is, what is the
credible way(s) to reproduce such artifacts?
When we having a source code, we could (re)compile it on a different
system. But what you propose to do with pure binary data, a soup of
objects, in respect that it is incredibly hard to understand, what
bits you need and what's not, in case if you need to do clean-up ,
refactor, rewrite and simply analyze what is happening.
All true, but that is also the case of the image. Which is why we had
that whole discussion with some Debian guys about Squeak not being free
since you can't generate the current image from text sources like you
can for Self or GNU Smalltalk.

I like to classify computer systems as "blueprint" on one extreme, where
each execution starts wth an empty state and a description of what is to
be done, and as "living" on the other extreme, where once execution is
started it continues forever and accumulates more and more information
not in the original description (you can cheat and interrupt execution
by dumping the full state and later restoring it - like with Squeak
images). Each one has its problems and its advantages. Lots of bugs
which don't matter much in blueprint systems (like memory leaks) are
fatal for living systems. The latter need sophisticated error detection
and correction mechanisms ("lint" for images) which we don't currently
have. On the other hand, developing on living systems is so much more
productive that I am willing to deal with their problems.

Note that PCs were pure blueprint systems when they only had floppy
disks and applications were simply run instead of installed. With
internal fixed disks they have moved more and more towards living
systems (Windows registry and many other things). Many people long so
much for the old days that they cope with the changing behavior of
Windows over time by throwing out their computers every two years and
buying a new one. Certainly such people would not be very happy with the
direction I would like Squeak to go.
Post by Igor Stasenko
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
All of the tools that created the bits in the first place, as well as
the tools to change them are inside the same image as the bits. So I
don't agree with your analogy.

Note that just because you call some bits inside the computer "text"
doesn't mean you can manipulate them without special tools. It is just
that if the list of such tools is large and include things as vi, emacs,
notepad and so on then the tool is invisible to you. But it is there. If
I give you some text I wrote on my ZX81 (odd Sinclair character set) or
on the Burroughs 6700 (EBCDIC) then these tools won't help you very
much.
Post by Igor Stasenko
I don't think that developers of Squeak should be victims of such situation(s).
The original thread was about how I vote for things that will make
Squeakers happy, not for what I would personally like. So nobody will be
a victim because of me. But I see no harm in discussing these things -
most people aren't even aware that we are making a choice.

-- Jecel
K. K. Subramaniam
2012-01-28 12:27:28 UTC
Permalink
Post by Jecel Assumpcao Jr
Post by Igor Stasenko
This is what making a huge difference, for instance, between
applications with open source code and applications shipped in binary
form - you can only report bugs, but can't realy make any suggestions
about what happening.
All of the tools that created the bits in the first place, as well as
the tools to change them are inside the same image as the bits. So I
don't agree with your analogy.
I think a better analogy is the way public key cryptography certificates are
constituted. What matters is not whether a certificate is encoded in ASCII or
binary but whether there is a chain of trust. Should anyone lose their
certificate, it can be reconstituted from its parent cert in the chain. But if
you happen to lose a root cert then no new chains can be reconstituted. You
are stuck with the existing chains originating from this root.

The key tools for a 'binary' encoding are the equality and diff tools. Given
two images A and B, check if they are equivalent. If not, find the difference D
that will reconstitute B from A.

Subbu
Jecel Assumpcao Jr
2012-01-28 12:27:38 UTC
Permalink
K. K. Subramaniam wrote on Thu, 20 Aug 2009 18:19:24 +0530
Post by K. K. Subramaniam
Post by Jecel Assumpcao Jr
All of the tools that created the bits in the first place, as well as
the tools to change them are inside the same image as the bits. So I
don't agree with your analogy.
I think a better analogy is the way public key cryptography certificates are
constituted. What matters is not whether a certificate is encoded in ASCII or
binary but whether there is a chain of trust. Should anyone lose their
certificate, it can be reconstituted from its parent cert in the chain. But if
you happen to lose a root cert then no new chains can be reconstituted. You
are stuck with the existing chains originating from this root.
Hmm... I didn't understand this analogy very well. Don't certificates in
the middle of the chain also involve a pair of public/private keys? If
so, it seems to me that losing the private key in the middle would be as
fatal as losing the root one (though it was affect fewer people).

Was the analogy about how a chain of certificates is like a chain of
images (starting all the way back from Smalltalk-76)?
Post by K. K. Subramaniam
The key tools for a 'binary' encoding are the equality and diff tools. Given
two images A and B, check if they are equivalent. If not, find the difference D
that will reconstitute B from A.
This is what the Debian guy was asking for, but the idea was to convert
images into some kind of XML and then use traditional text diff to deal
with that. I used to have great success with diff but in the past few
years its results have become useless for me (probably some default
settting has been changed and I would have to force it to work the old
way), so I am not sure about the value of this approach. A tool that
actually understood images, as you proposed, might work better. It
wouldn't be too easy to write, however (see other thread about object
identity).

-- Jecel
K. K. Subramaniam
2012-01-28 12:27:39 UTC
Permalink
Post by Jecel Assumpcao Jr
Was the analogy about how a chain of certificates is like a chain of
images (starting all the way back from Smalltalk-76)?
Yes.
Post by Jecel Assumpcao Jr
Post by K. K. Subramaniam
The key tools for a 'binary' encoding are the equality and diff tools.
Given two images A and B, check if they are equivalent. If not, find the
difference D that will reconstitute B from A.
This is what the Debian guy was asking for, but the idea was to convert
images into some kind of XML and then use traditional text diff to deal
with that.
I meant a difference operator, not the diff(1) program. Change set browser is a
good tool but is incomplete. It does not track and log all changes (e.g. class
variables).

Subbu
Jecel Assumpcao Jr
2012-01-28 12:27:44 UTC
Permalink
Post by K. K. Subramaniam
I meant a difference operator, not the diff(1) program. Change set browser is a
good tool but is incomplete. It does not track and log all changes (e.g. class
variables).
I meant the same thing, but mentioned the text diff program as an
example of what some people would like to be able to use. Back when
Smalltalk-80 used an object table it wouldn't have been that hard to
create a difference operator for binary images since objects never
changed their "oop". With direct pointers it is far more complicated to
decide that two objects in separate images are actually the same. The
best strategy is probably to start out with classes and processes and do
a breadth first search.

-- Jecel
Eliot Miranda
2012-01-28 12:27:44 UTC
Permalink
Post by K. K. Subramaniam
Post by K. K. Subramaniam
I meant a difference operator, not the diff(1) program. Change set
browser is a
Post by K. K. Subramaniam
good tool but is incomplete. It does not track and log all changes (e.g.
class
Post by K. K. Subramaniam
variables).
I meant the same thing, but mentioned the text diff program as an
example of what some people would like to be able to use. Back when
Smalltalk-80 used an object table it wouldn't have been that hard to
create a difference operator for binary images since objects never
changed their "oop".
That only works for things that are created in exactly the same order with
no intervening operations. If I were to compile the source for method A
followed by compiling the source for method B I would end up with different
oops for methods A and B than if I were to first compile method B's source
followed by method A's source. Things would also be different if in between
compiling I performed some other arbitrary action that caused allocations
whose results were discarded.

The problem is orthogonal to direct pointers.
Post by K. K. Subramaniam
With direct pointers it is far more complicated to
decide that two objects in separate images are actually the same. The
best strategy is probably to start out with classes and processes and do
a breadth first search.
What is needed is generic structural comparison. One problem in this is
that certain collections are unordered and therefore comparing structure
reachable from unordered collections may involve a combinatoric explosion
(compare all possible pair-wise combinations, succeeding if a match is
found). Another problem is what I'll call incidental concrete difference.
Are these two equivalent or not for the purposes of comparison or not? (1
to: 3) #(1 2 3)? (etc)

The schema for code representation in the system is well-defined and several
ordering operations exist to allow comparison; selector-method-pairs in
method dictionaries can be ordered by lexicographic order of selectors and
sibling subclasses can be ordered by lexicographic order of class names.
Hence structural comparison of Smalltalk code is straight-forward.
Generalising to arbitrary object structures isn't at all straight-forward
unless analogous schema are introduced.

Writing a recursive structural equality tester in Smalltalk is
straight-forward (I have code written for the Newspeak project I could post
if you're interested) but it fails for unordered collections and for
incidental concrete difference.
Post by K. K. Subramaniam
-- Jecel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090824/d91fa78f/attachment.htm
Colin Putney
2012-01-28 12:27:39 UTC
Permalink
Post by Jecel Assumpcao Jr
http://wiki.squeak.org/squeak/584
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or
even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
Interesting.

I note, though, that the wiki page you mention doesn't actually say
much about development. It's mostly concerned with efficient ways of
moving objects between images. Reliably reconstructing part of one
image in another is certainly a crucial part of collaborative
development, but it's not everything.

The other key feature of Monticello is merging. If you and I have the
same chunk in different image, and we make differing but compatible
changes, how can we create a chunk that contains both sets of changes?
I submit that any tool that can do that will have explicit knowledge
of the semantics of objects it's merging, whether Smalltalk code,
Etoys projects or something else.

So the wonderful generality of the Chunky Images idea only gets you so
far, and you still need a tool like Monticello to actually create
collaboratively. In Monticello 2 I've tried to address this idea
explicitly: the core versioning engine is knows nothing about the
semantics of the objects it's versioning, but it does rely on
pluggable domain models that do.

Colin
Ronald Spengler
2012-01-28 12:27:44 UTC
Permalink
We're bumping up against the homoiconicity of the system, aren't we? That
code is really just a kind of data. Has anyone ever done a diff tool for
whole images, not just source methods? It would be fantabulous if I didn't
have to write an installer script for my package, instead having the
necessary objects brought over directly.
Seems like the mother of all problems is: moving things around that way
between images of different formats. Would some future descendant of
SystemTracer perhaps be of use?

- Ron
Post by Jecel Assumpcao Jr
http://wiki.squeak.org/squeak/584
Post by Jecel Assumpcao Jr
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
Interesting.
I note, though, that the wiki page you mention doesn't actually say much
about development. It's mostly concerned with efficient ways of moving
objects between images. Reliably reconstructing part of one image in another
is certainly a crucial part of collaborative development, but it's not
everything.
The other key feature of Monticello is merging. If you and I have the same
chunk in different image, and we make differing but compatible changes, how
can we create a chunk that contains both sets of changes? I submit that any
tool that can do that will have explicit knowledge of the semantics of
objects it's merging, whether Smalltalk code, Etoys projects or something
else.
So the wonderful generality of the Chunky Images idea only gets you so far,
and you still need a tool like Monticello to actually create
the core versioning engine is knows nothing about the semantics of the
objects it's versioning, but it does rely on pluggable domain models that
do.
Colin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090824/27972442/attachment.htm
Jecel Assumpcao Jr
2012-01-28 12:27:44 UTC
Permalink
Post by Ronald Spengler
We're bumping up against the homoiconicity of the system, aren't we?
That code is really just a kind of data. Has anyone ever done a diff tool
for whole images, not just source methods?
Like I said in another reply, given that Squeak objects don't have a
fixed identity (see other thread) it isn't very easy. But I don't think
it is impossible in practice since objects mostly exist in very
stereotyped patterns.
Post by Ronald Spengler
It would be fantabulous if I didn't have to write an installer script for
my package, instead having the necessary objects brought over directly.?
That is what I want. And I partly had it in Smalltalk V/Win (also
released as Smalltalk Express). In that system your image started out as
an essentially empty v.exe file plus a bunch of .dll files with objects
and code. Some of these had to be shipped with the application while
many (with all the development tools) couldn't (due to the license). The
.dll files had lots of stuff you wouldn't need but you had to ship them
even if you only needed a single object. And there were no tools to tell
you that you were using any objects from a .dll so you might ship if
even if you didn't need it. But I don't think all these problems would
be too hard to fix.
Post by Ronald Spengler
Seems like the mother of all problems is: moving things around that way
between images of different formats. Would some future descendant of
SystemTracer perhaps be of use?
If you want to send messages between these different images and you want
to be able to send over some of the arguments (instead of just a far
reference back to the sending image) then this problem has to be solved
anyway. Perhaps we are talking about something more like the Corba
serialization format than the SystemTracer, but it certainly is related
to the latter.

-- Jecel
Eliot Miranda
2012-01-28 12:27:44 UTC
Permalink
Post by Jecel Assumpcao Jr
Post by Ronald Spengler
We're bumping up against the homoiconicity of the system, aren't we?
That code is really just a kind of data. Has anyone ever done a diff tool
for whole images, not just source methods?
Like I said in another reply, given that Squeak objects don't have a
fixed identity (see other thread) it isn't very easy. But I don't think
it is impossible in practice since objects mostly exist in very
stereotyped patterns.
Post by Ronald Spengler
It would be fantabulous if I didn't have to write an installer script for
my package, instead having the necessary objects brought over directly.
That is what I want. And I partly had it in Smalltalk V/Win (also
released as Smalltalk Express). In that system your image started out as
an essentially empty v.exe file plus a bunch of .dll files with objects
and code. Some of these had to be shipped with the application while
many (with all the development tools) couldn't (due to the license). The
.dll files had lots of stuff you wouldn't need but you had to ship them
even if you only needed a single object. And there were no tools to tell
you that you were using any objects from a .dll so you might ship if
even if you didn't need it. But I don't think all these problems would
be too hard to fix.
and our experience at ParcPlace-Digitalk when we compared SLLs (Smalltalk
V's system of object DLLs requiring VM support, quite similar to image
segments) with Parcels (VW's system of a conventional but optimized object
pickling format) was that parcels were as fast, if not faster, and far less
brittle. Making the file format identical to the object format and linking
in objects to the heap instead of parsing them seems like a really cool idea
but it ties the representation far too closely to a particular
implementation of the memory manager and object model. Going with a
"soft"pickling format allows one to concentrate on important issues like
naming assumed structure (what are the prerequisites of a component) and how
to update it in the presence of schema changes, e.g. what if a component
includes an instance of some class which has gained or lost instance
variables in the loading image when compared to the image that published the
component.
Post by Jecel Assumpcao Jr
Post by Ronald Spengler
Seems like the mother of all problems is: moving things around that way
between images of different formats. Would some future descendant of
SystemTracer perhaps be of use?
If you want to send messages between these different images and you want
to be able to send over some of the arguments (instead of just a far
reference back to the sending image) then this problem has to be solved
anyway. Perhaps we are talking about something more like the Corba
serialization format than the SystemTracer, but it certainly is related
to the latter.
Again you should check-out the VW parcel system and the communications
framework OpenTalk. The OpenTalk marshaller is influenced by the parcel
object marshaller but the two have differences.
Post by Jecel Assumpcao Jr
-- Jecel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090824/4031f03f/attachment.htm
Eliot Miranda
2012-01-28 12:27:44 UTC
Permalink
Post by Ronald Spengler
We're bumping up against the homoiconicity of the system, aren't we? That
code is really just a kind of data. Has anyone ever done a diff tool for
whole images, not just source methods? It would be fantabulous if I didn't
have to write an installer script for my package, instead having the
necessary objects brought over directly.
Seems like the mother of all problems is: moving things around that way
between images of different formats. Would some future descendant of
SystemTracer perhaps be of use?
Why is this such a mother of a problem? In VW we used a single parcel
format that could represent compiled code that could be loaded into either a
32-bit or a 64-bit image even when the size of SmallInteger, the number of
tag bits, the existence or not of SmallDouble, the size of identify hashes,
the way class references are encoded in instances, etc all differed between
the 32-bit and 64-bit systems.
Post by Ronald Spengler
- Ron
Post by Jecel Assumpcao Jr
http://wiki.squeak.org/squeak/584
Post by Jecel Assumpcao Jr
The idea is to be more like the Etoys users which can load binary
projects containing not only the code they need but also hand crafted
objects which have no source (like a drawing, some nested Morphs or even
some text). This is very simplistic compared to Spoon, and my proposal
was even more simplistic. In particular, this doesn't handle the case
where any changes to bytecodes or object format are needed.
Interesting.
I note, though, that the wiki page you mention doesn't actually say much
about development. It's mostly concerned with efficient ways of moving
objects between images. Reliably reconstructing part of one image in another
is certainly a crucial part of collaborative development, but it's not
everything.
The other key feature of Monticello is merging. If you and I have the same
chunk in different image, and we make differing but compatible changes, how
can we create a chunk that contains both sets of changes? I submit that any
tool that can do that will have explicit knowledge of the semantics of
objects it's merging, whether Smalltalk code, Etoys projects or something
else.
So the wonderful generality of the Chunky Images idea only gets you so
far, and you still need a tool like Monticello to actually create
the core versioning engine is knows nothing about the semantics of the
objects it's versioning, but it does rely on pluggable domain models that
do.
Colin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090824/3bd5165a/attachment.htm
Jecel Assumpcao Jr
2012-01-28 12:27:44 UTC
Permalink
Post by Colin Putney
I note, though, that the wiki page you mention doesn't actually say
much about development.
That is left up to other tools in this proposal.
Post by Colin Putney
It's mostly concerned with efficient ways of
moving objects between images.
It can't even do that either - an object can only be reloaded into the
exact same image from which it was extracted (unlike ImageSegments). The
idea is that by letting several "images" live side by side in memory and
disk without taking up too much space then you won't mind dedicating an
"image" for each Squeak application you use.
Post by Colin Putney
Reliably reconstructing part of one
image in another is certainly a crucial part of collaborative
development, but it's not everything.
You are right, but as I mentioned above my proposal doesn't even try to
do that much.
Post by Colin Putney
The other key feature of Monticello is merging. If you and I have the
same chunk in different image, and we make differing but compatible
changes, how can we create a chunk that contains both sets of changes?
You would have to use Monticello or something similar (in which case you
would be limited to source code). For "merging" generic objects my idea
was to use Croquet, but then you would be limited to the equivalent of
instant messaging rather than email.
Post by Colin Putney
I submit that any tool that can do that will have explicit knowledge
of the semantics of objects it's merging, whether Smalltalk code,
Etoys projects or something else.
Yes. And I have given up on automatic conflict resolution after working
many years on the problem (for the more general Neo Smalltalk modules,
since for Chunky Squeak you can't even get conflicts in the first place
as it is so limited).
Post by Colin Putney
So the wonderful generality of the Chunky Images idea only gets you so
far, and you still need a tool like Monticello to actually create
collaboratively. In Monticello 2 I've tried to address this idea
explicitly: the core versioning engine is knows nothing about the
semantics of the objects it's versioning, but it does rely on
pluggable domain models that do.
I should have mentioned http://wiki.squeak.org/squeak/5637 (Neo
Smalltalk "groups") as well since Chunky Squeak is just a very stripped
down version of it. In that system you do have merging in something that
is similar to "commit" in transactional systems.

-- Jecel
Göran Krampe
2012-01-28 12:27:27 UTC
Permalink
Hi!
Post by Andreas Raab
Post by Göran Krampe
- Keith has more or less said that the trunk effort is counter
productive in comparison to his/their 3.11 effort. Personally I might
think it is a gray area, but even so - perhaps once and for all - with
a pedagogic mindset - explaining to us all how this is so and also
describing "trunk" and "bob" in some way that all of us can grok it...
would let us come to some conclusion.
It's quite simple really. Software development works by developers
working together in a shared repository, right? That's the trunk. The
trunk is nothing but a shared repository that the developers have commit
rights to. In the course of software development, builds happen on a
regular basis. That's Bob. Bob is a tool to assemble various bits of
code and content and compile this into a result. Simple as that.
Mantis role in this process with regards to code is to assemble
contributions from non-core-developers in the project; people who are
not given direct commit rights to the repository. These contributions
need to be reviewed and integrated. That's Installer for you. The result
of the integration process goes back into the repository. There's the
trunk again. All the parts have their place in the process.
I really, really would like to hear Keith comment on the above because
if it is indeed like you describe it- then I am at a total loss over the
whole hoopla. I suspect it is not as simple as you describe above.
Post by Andreas Raab
Post by Göran Krampe
At this point I don't really give a rats about the above! :) I just
want us to heal, embrace, drink beer and find a way to move on. This
"war" has hurt us I think, I always brag about Squeak/Smalltalk being
a very friendly place, but it has slipped a bit lately. And thinking
that it might have been my posts setting off the fire doesn't really
make it feel better ;)
I'm not unhappy about it. We need to get moving again. Whatever triggers
this process is a good thing. And I never expected the process to be
painfree, but I do think we are making progress if only by illustrating
that other models of contributions can actually be successful. Now we
have to work out how to make this work together to produce an actual
piece of software called Squeak.
True.
Post by Andreas Raab
Post by Göran Krampe
BTW, can we please get our current status/process "trunk" or "bob" or
whatever we are using these days written down on squeak.org in an
official manner? Perhaps this is already done.
http://installer.pbworks.com/Squeak311Proposal
http://board.squeak.org/2009/07/02/a-new-community-development-model/
Right, and neither is really on www.squeak.org. I know Keith couldn't
understand why his 3.11 stuff needed to be on www.squeak.org - but it
really makes it much more official and easy for anyone to find.
Post by Andreas Raab
Post by Göran Krampe
This also goes for the 3.11 team - does it even exist anymore? Let us
get our act together.
I leave it to Matthew to respond to this question.
Right.

regards, G?ran

PS. ...now will focus on DS hacking instead...
Ian Trudel
2012-01-28 12:27:27 UTC
Permalink
Post by Göran Krampe
Post by Andreas Raab
http://installer.pbworks.com/Squeak311Proposal
http://board.squeak.org/2009/07/02/a-new-community-development-model/
Right, and neither is really on www.squeak.org. I know Keith couldn't
understand why his 3.11 stuff needed to be on www.squeak.org - but it really
makes it much more official and easy for anyone to find.
I agree with you, G?ran. Moreover, it should no longer be a "proposal"
if it takes any official position in the community. The proposal
should be revamped, adapted to the new direction taken, and clarified
before making it to the official web page.

Ian.
--
http://mecenia.blogspot.com/
Andreas Raab
2012-01-28 12:27:27 UTC
Permalink
Post by Göran Krampe
Post by Andreas Raab
Post by Göran Krampe
- Keith has more or less said that the trunk effort is counter
productive in comparison to his/their 3.11 effort. Personally I might
think it is a gray area, but even so - perhaps once and for all -
with a pedagogic mindset - explaining to us all how this is so and
also describing "trunk" and "bob" in some way that all of us can grok
it... would let us come to some conclusion.
It's quite simple really. Software development works by developers
working together in a shared repository, right? That's the trunk. The
trunk is nothing but a shared repository that the developers have
commit rights to. In the course of software development, builds happen
on a regular basis. That's Bob. Bob is a tool to assemble various bits
of code and content and compile this into a result. Simple as that.
Mantis role in this process with regards to code is to assemble
contributions from non-core-developers in the project; people who are
not given direct commit rights to the repository. These contributions
need to be reviewed and integrated. That's Installer for you. The
result of the integration process goes back into the repository.
There's the trunk again. All the parts have their place in the process.
I really, really would like to hear Keith comment on the above because
if it is indeed like you describe it- then I am at a total loss over the
whole hoopla. I suspect it is not as simple as you describe above.
From my perspective it is. I didn't make this up. Replace Bob with ant
and I've described the product development process we use at Qwaq.
Literally.

Cheers,
- Andreas
Ian Trudel
2012-01-28 12:27:26 UTC
Permalink
Hello Andreas,

I have always thought from the beginning that both could be integrated
with each other, the process and the trunk. They are not mutually
exclusive. There are however few problems:

* The 3.11 proposal is unclear if it takes people weeks or months
to understand important parts of it.
* Keith made it feel as such the 3.11 proposal and the trunk could
never live along side.
* Bob is closed source and unlikely to be otherwise.

It's certainly possible to create an infrastructure based on ideas
developed in 3.11 proposal and using existing tools applied on current
trunk and/or other means. This could further be supported by a small
unpretentious Release Builder as a replacement to Bob, something that
would not have to harvest into Mantis and alike (considering the
amount of work put into Bob, it should probably focus on simplicity).

Regards,
Ian.
--
http://mecenia.blogspot.com/
Igor Stasenko
2012-01-28 12:27:26 UTC
Permalink
It is clear that for different people, who get used to one or another tool,
sometimes, it is very hard to convince them to change their habits and
stop using old tool and start using a new one
immediately.
This is lenghtly process which could take years , and we should be patient.

I am understanding the 3.11 effort first, as a set of tools which has
to be written in order to simplify/automate
an image building process and also enable a more open, friendly
contribution process with less bottlenecks, with easily crossing fork
'barriers'.
So, i had never underestimated the value of such tools. But as i said,
it could take years before people take a full grasp of the idea. So,
patience, patience and patience.

Both 'trunk' and '3.11' depicting some kind of process.
And as Andreas stated, they are perpendicular in respect that they
having different targets/goals.
It may be hard to see, because both are about software development..
And i must say that 3.11 cares more about integration of already
existing artifact(s), not really matter what they are, while trunk
process is cares more about continuous development of a single
artifact (a Squeak release).

I am proposed already, to look for a ways, how Sake could help us in
developing a squeak release. I thought that it will be a first, little
baby-steps of all devs who involved currently in trunk to get
accustomed with new tool(s) , provided by Keith for us.
At first, i propose to write down a task for building/updating the
image from trunk , as an alternative to MCMUpdater.
I am really interesting in what places we could easily replace one
tool by another, but it is not really necessary to replace it, there
is _absolutely_ could be a cases where we could use both of them. Sake
is 'Make', which could use __any existing tool__ in order to build a
final artifact.
By writting down a set of automated, test-driven and repeatable tasks,
we will entering another dimension - where we could have a fully
automated build farm which doing nightly boring job for us. Hence the
Bob.

I seen a very good progress last couple of months around Squeak, since
we established a trunk process. And i am sure, more fun is waiting for
us over a next months, when we start using a new tools, such as
Bob/Sake, DeltaStreams and MC 1.6 / MC 2.0.
--
Best regards,
Igor Stasenko AKA sig.
Steve McCusker
2012-01-28 12:27:26 UTC
Permalink
I've been watching this list for a while, and playing with Squeak & Seaside.
I have been a software developer for 40 odd years and have used various
Smalltalks during that time as well lots of other tools and Smalltalk is my
favourite development language where constraints (money, employer's whims
etc) permit.

I was about ready to forget Squeak because of the sudden abandonment o f
Matthew & Keith's efforts to create a sensible set of tools to take Squeak
development onwards.

So I was really pleased to see Andreas' email.

Sad it took so long him and the board to realise what Keith was trying to
do, and yes he is a bit prickly sometimes, but I think a convincing apology
from Andreas and the rest of the board might help heal wounds and get a
better Squeak development going.

Cheers from the far side of the world.

Steve McCusker
Hobart
Tasmania
Simon Michael
2012-01-28 12:27:26 UTC
Permalink
This seems to clarify quite a bit, and the proposal seems a good and natural next step. Also I think the board has been
doing an excellent job, both in moving things forward and in their handling of serious flamage from Keith.
Eliot Miranda
2012-01-28 12:27:26 UTC
Permalink
Post by Simon Michael
This seems to clarify quite a bit, and the proposal seems a good and
natural next step. Also I think the board has been doing an excellent job,
both in moving things forward and in their handling of serious flamage from
Keith.
Can we please refrain from inflaming the situation further. If one reads
Keith's postings in their entirety they seem extremely patient and
discursive. Accusing Keith (unjustly IMO) of flamage isn't going to help us
resolve these issues and get on with things.

best
Eliot
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20090819/7906be99/attachment.htm
Simon Michael
2012-01-28 12:27:26 UTC
Permalink
I have been reading and have a different impression, and I wanted that on record. Perhaps I should have stuck to
expressing support. I apologise to all for adding heat.
Continue reading on narkive:
Loading...