Discussion:
The Trunk: Kernel-mt.1009.mcz
(too old to reply)
c***@source.squeak.org
0000-04-08 16:13:59 UTC
Permalink
Marcel Taeumel uploaded a new version of Kernel to project The Trunk:
http://source.squeak.org/trunk/Kernel-mt.1009.mcz

==================== Summary ====================

Name: Kernel-mt.1009
Author: mt
Time: 2 April 2016, 12:23:20.21041 pm
UUID: bd1b45d5-cb4d-41b0-890b-c8f70de70b1c
Ancestors: Kernel-ul.1008

Recent CogVMs do not yield preempted processes. Fixes the behavior where "[ [] repeat] fork" could not be interrupted anymore via CMD+. and hence locked the image.

=============== Diff against Kernel-ul.1008 ===============

Item was changed:
----- Method: ProcessorScheduler>>preemptedProcess (in category 'accessing') -----
preemptedProcess
"Return the process that the currently active process just preempted."
+
+ self activeProcess priority to: 1 by: -1 do: [:priority |
+ (quiescentProcessLists at: priority) ifNotEmpty: [:list |
+ ^ Smalltalk processPreemptionYields
+ ifTrue: [list last]
+ ifFalse: [list first]]].
+ ^ nil
+
- | list |
- activeProcess priority to: 1 by: -1 do:[:priority|
- list := quiescentProcessLists at: priority.
- list isEmpty ifFalse:[^list last].
- ].
- ^nil
-
"Processor preemptedProcess"!
Eliot Miranda
2016-04-02 16:16:05 UTC
Permalink
Hi Marcel,
Post by c***@source.squeak.org
http://source.squeak.org/trunk/Kernel-mt.1009.mcz
==================== Summary ====================
Name: Kernel-mt.1009
Author: mt
Time: 2 April 2016, 12:23:20.21041 pm
UUID: bd1b45d5-cb4d-41b0-890b-c8f70de70b1c
Ancestors: Kernel-ul.1008
Recent CogVMs do not yield preempted processes. Fixes the behavior where "[ [] repeat] fork" could not be interrupted anymore via CMD+. and hence locked the image.
Just to be clear, this is not a VM change. The VM supports either yielding on preemption (the original behaviour) or preemption not yielding (the new, IMO correct, behaviour), based on a flag stored in the image header. So your comment should have read

Squeak 5, as of Kernel-eem.??? does not yield preempted processes. Fixes the behavior where "[ [] repeat] fork" could not be interrupted anymore via CMD+. and hence locked the image.

I'll figure out which version when "away from my phone" ;-)
Post by c***@source.squeak.org
=============== Diff against Kernel-ul.1008 ===============
----- Method: ProcessorScheduler>>preemptedProcess (in category 'accessing') -----
preemptedProcess
"Return the process that the currently active process just preempted."
+
+ self activeProcess priority to: 1 by: -1 do: [:priority |
+ (quiescentProcessLists at: priority) ifNotEmpty: [:list |
+ ^ Smalltalk processPreemptionYields
+ ifTrue: [list last]
+ ifFalse: [list first]]].
+ ^ nil
+
- | list |
- activeProcess priority to: 1 by: -1 do:[:priority|
- list := quiescentProcessLists at: priority.
- list isEmpty ifFalse:[^list last].
- ].
- ^nil
-
"Processor preemptedProcess"!
marcel.taeumel
2016-04-02 16:15:24 UTC
Permalink
Hi Eliot,

thanks for clarification. Unfortunately, I cannot update the commit message
-- can I?

Best,
Marcel



--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4887941.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
Eliot Miranda
2016-04-02 16:54:52 UTC
Permalink
Hi Marcel,
Post by marcel.taeumel
Hi Eliot,
thanks for clarification. Unfortunately, I cannot update the commit message
-- can I?
Not easily. But it's not important. I only wanted to point out that it's not a VM change, but a system setting/mode change.
Post by marcel.taeumel
Best,
Marcel
_,,,^..^,,,_ (phone)
Post by marcel.taeumel
--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4887941.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
Bert Freudenberg
2016-04-03 13:49:50 UTC
Permalink
Post by Eliot Miranda
Hi Marcel,
[...]
I only wanted to point out that it's not a VM change, but a system setting/mode change.
Can it be queried and changed at runtime?

- Bert -
Bert Freudenberg
2016-04-03 13:52:26 UTC
Permalink
Post by Bert Freudenberg
Post by Eliot Miranda
Hi Marcel,
[...]
I only wanted to point out that it's not a VM change, but a system setting/mode change.
Can it be queried and changed at runtime?
Ah, it can. Just found processPreemptionYields.

- Bert -
Eliot Miranda
2016-04-03 23:47:52 UTC
Permalink
Hi Bert,
Post by Bert Freudenberg
Post by Eliot Miranda
Hi Marcel,
[...]
I only wanted to point out that it's not a VM change, but a system setting/mode change.
Can it be queried and changed at runtime?
Yes. There are even a setter and a getter in trunk. See SmalltalkImage>>#processPreemptionYields[:].
Post by Bert Freudenberg
- Bert -
marcel.taeumel
2016-04-04 06:33:57 UTC
Permalink
We might expose this as a preference?

Best,
Marcel



--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888130.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
Eliot Miranda
2016-04-04 23:15:17 UTC
Permalink
Hi Marcel,
Post by marcel.taeumel
We might expose this as a preference?
I think it has too deep and pervasive effects to be a preference. Basically we're still fixing the system after making this change (but the change is important for concurrency). Making it a preference would cause a lot of uncertainty in debugging. Please let's keep it the way it is.
Post by marcel.taeumel
Best,
Marcel
--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888130.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
marcel.taeumel
2016-04-05 16:27:28 UTC
Permalink
Hi Eliot,

with "Smalltalk processPreemptionYields == false", is there a Smalltalk way
to implement ProcessorScheduler >> #yield?

At the moment, it looks like this:

| semaphore |
<primitive: 167>
semaphore := Semaphore new.
[semaphore signal] fork.
semaphore wait.

And another question: How to "yield" a process (i.e. put back in line) that
is not currently running but runnable?

Best,
Marcel



--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888435.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
Levente Uzonyi
2016-04-05 17:44:07 UTC
Permalink
The line

[] forkAt: Processor activePriority + 1

should interrupt the current process without any Semaphores involved.
Except for the case when the process has the highest possible priority,
but yielding from that process, assuming there's only one, wouldn't make
any sense anyway.

Levente
Post by marcel.taeumel
Hi Eliot,
with "Smalltalk processPreemptionYields == false", is there a Smalltalk way
to implement ProcessorScheduler >> #yield?
| semaphore |
<primitive: 167>
semaphore := Semaphore new.
[semaphore signal] fork.
semaphore wait.
And another question: How to "yield" a process (i.e. put back in line) that
is not currently running but runnable?
Best,
Marcel
--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888435.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
Eliot Miranda
2016-04-05 18:17:04 UTC
Permalink
Post by Levente Uzonyi
The line
[] forkAt: Processor activePriority + 1
should interrupt the current process without any Semaphores involved.
Yes, but if Smalltalk processPremeptionYields is false, this will not cause
the current process to yield (which is to be desired).
Post by Levente Uzonyi
Except for the case when the process has the highest possible priority,
but yielding from that process, assuming there's only one, wouldn't make
any sense anyway.
Levente
Thank you, Levente, that has shown me to write a much simpler example.

The following doit should answer false:

| yielded |
yielded := false.
[yielded := true] fork.
yielded

But this should answer true:

| yielded |
yielded := false.
[yielded := true] fork.
Processor yield.
yielded

Now try the following with processPreemptionYields either true or false:

| yielded |
yielded := false.
[yielded := true] fork.
[] forkAt: Processor activePriority + 1.
yielded

This will answer what Smalltalk processPreemptionYields adds, i.e.

| yielded |
yielded := false.
[yielded := true] fork.
[] forkAt: Processor activePriority + 1.
self assert: yielded = Smalltalk processPreemptionYields



Extra examples attached...
Post by Levente Uzonyi
Hi Eliot,
Post by marcel.taeumel
with "Smalltalk processPreemptionYields == false", is there a Smalltalk way
to implement ProcessorScheduler >> #yield?
| semaphore |
<primitive: 167>
semaphore := Semaphore new.
[semaphore signal] fork.
semaphore wait.
And another question: How to "yield" a process (i.e. put back in line) that
is not currently running but runnable?
Best,
Marcel
--
http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888435.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
HTH
_,,,^..^,,,_
best, Eliot
marcel.taeumel
2016-04-06 06:58:30 UTC
Permalink
Thank you, Eliot. I was mistaken that "[ ] fork" (resp. Process >> #resume)
adds it to the front, when it actually adds it to the back of the queue.

Best,
Marcel



--
View this message in context: http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888531.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.

Eliot Miranda
2016-04-05 18:06:21 UTC
Permalink
Hi Marcel,
Post by marcel.taeumel
Hi Eliot,
with "Smalltalk processPreemptionYields == false", is there a Smalltalk way
to implement ProcessorScheduler >> #yield?
| semaphore |
<primitive: 167>
semaphore := Semaphore new.
[semaphore signal] fork.
semaphore wait.
So that works, right? Remember what the setting does:

If Smalltalk processPreemptionYields == false then when preempted by a
higher-priority process, the current process stays at the head of its run
queue. But when a process waits on a semaphore it is removed from its run
queue. When a process resumes it always gets added to the back of its run
queue. So in the above nothing changes. Either the primitive puts the
calling process to the back of its run queue, or (if the primitive is not
implemented):

1. the fork creates a new process, adding it to the end of the active
process's run queue
2. the wait in "semaphore wait" removes the active process from its run
queue and adds it to the semaphore, so the active process is now not
runnable, waiting on the semaphore, which
3. allows the next process in the run queue (if any) to run, and eventually
4. allows the newly forked process to run, and
5. the signal in "semaphore signal" removes the process from the semaphore
and adds it to the back of the run queue, so
6. all processes at the same priority level as the process that called
yield have run

Note that all the primitive does is to circumvent having to create a
semaphore and create and schedule a process, and do a signal and a wait to
move a process to the back of its run queue. This is worth while because
most of the time a process's run queue is empty, it being the only runnable
process at that priority.

Test it:
| yielded |
yielded := false.
[yielded := true] fork.
yielded


| yielded |
yielded := false.
[yielded := true] fork.
Processor yield.
yielded


OK, so there's clearly a lot of confusion about process preemption
yielding. I hope that the following two examples show what's different
between the two regimes. In the first example we merely create two
processes at a lower priority than the active process, at a priority where
there are no other processes:

| run priority process1 process2 |
run := true.
"*find an empty priority level at a priority lower than the active process*"
priority := Processor activePriority - 1.
[(Processor waitingProcessesAt: priority) isEmpty] whileFalse:
[priority := priority - 1].

"*Create two processes at that priority*"
process1 := [[run] whileTrue] forkAt: priority.
process2 := [[run] whileTrue] forkAt: priority.

"*Check that their order in the list is the same as the order in which they
were created*"
self assert: (Processor waitingProcessesAt: priority) first == process1.
self assert: (Processor waitingProcessesAt: priority) last == process2.

"*set the flag to zero and block on a delay, allowing the processes to run
to termination*"
run := false.
(Delay forMilliseconds: 50) wait.

"*Check that they have indeed terminated*"
self assert: (Processor waitingProcessesAt: priority) isEmpty


So far so good. Now let's preempt process1 while it is running, by waiting
on a delay without setting run to false:

| run priority process1 process2 |
run := true.
"*find an empty priority level at a priority lower than the active process*"
priority := Processor activePriority - 1.
[(Processor waitingProcessesAt: priority) isEmpty] whileFalse:
[priority := priority - 1].

"*Create two processes at that priority*"
process1 := [[run] whileTrue] forkAt: priority.
process2 := [[run] whileTrue] forkAt: priority.

"*Check that their order in the list is the same as the order in which they
were created*"
self assert: (Processor waitingProcessesAt: priority) first == process1.
self assert: (Processor waitingProcessesAt: priority) last == process2.

"*Now block on a delay, allowing the first one to run, spinning in its
loop.*
* When the delay ends the current process will preempt process1, because *
*process1** is at a lower priority.*"
(Delay forMilliseconds: 50) wait.

Smalltalk processPreemptionYields
ifTrue: "*If process preemption yields, process1 will get sent to the back
of the run queue (which is wrong; no process explicitly yielded)*"
[self assert: (Processor waitingProcessesAt: priority) first == process2.
self assert: (Processor waitingProcessesAt: priority) last == process1]
ifFalse: "*If process preemption doesn't yield, the processes retain their
order (yay! we indeed have cooperative scheduling within a priority).*"
[self assert: (Processor waitingProcessesAt: priority) first == process1.
self assert: (Processor waitingProcessesAt: priority) last == process2].

"*set the flag to zero and block on a delay, allowing the processes to run
to termination*"
run := false.
(Delay forMilliseconds: 50) wait.
"*Check that they have indeed terminated*"
self assert: (Processor waitingProcessesAt: priority) isEmpty

Run the above after trying both "Smalltalk processPreemptionYields: false"
and "Smalltalk processPreemptionYields: true"

Does the above clarify things? Nothing changes within priorities. What
the setting controls is what happens when a process is preempted by a
higher-priority process. The old code did an implicit yield of the
preempted process, destroying the guarantee of cooperative scheduling
within a priority.


I've attached the two examples above in a workspace text (in which
"Processor waitingProcessesAt: priority" is cached in a variable).


And another question: How to "yield" a process (i.e. put back in line) that
Post by marcel.taeumel
is not currently running but runnable?
From a higher priority process one can directly manipulate the lists of
lower priority processes. So, e.g. if in the above we've created process1
and process2 and they're on the list, we can do things like

(Processor waitingProcessesAt: priority) addLast: (Processor
waitingProcessesAt: priority) removeFirst

Again, this is easier if Smalltalk processPreemptionYields is false. You
know that the only thing that can change the order of processes in a list
is either cooperative scheduling within those processes or explicit
manipulation of the list by higher priority processes.


Do you have a good model of the runnable processes? The active process is
always not on its list because it is running. There may be runnable, but
not running, processes at its priority which will be on the list at the
active process's priority. All other runnable processes are on lists at
lower priorities. Each priority has one list. Resuming a process (either
by resume or by aSemaphore signal) adds a previously not runnable process
to the end of the list at its priority (unless it is the highest-priority
runnable process, in which case it will become the active process).
Suspending a process, or causing it to wait on a semaphore, removes the
process from its list (unless it is the active process, in which case it
does not need to be removed).

This is interesting and important. Having implemented the Blue Book VM and
worked on the innards of the scheduler for more years than I'd care to
remember I have a very clear picture of the scheduler; it's simple, and
well designed. But Marcel, you're not sure of a number of things, which
makes me think you don't have that model in your head, and given that
you're an extremely strong Smalltalker, that implies that many other
Smalltalk programmers probably don't have that model in their heads
either. What can we do to explain the scheduler more clearly to Smalltalk
programmers as opposed to VM implementors?


Best,
Post by marcel.taeumel
Marcel
--
http://forum.world.st/The-Trunk-Kernel-mt-1009-mcz-tp4887889p4888435.html
Sent from the Squeak - Dev mailing list archive at Nabble.com.
_,,,^..^,,,_
best, Eliot
Chris Muller
2016-04-02 18:58:16 UTC
Permalink
Post by marcel.taeumel
thanks for clarification. Unfortunately, I cannot update the commit message
-- can I?
Yes, you can load the prior version, file-in your change, re-save it
as +1 higher with the correct comment.
Loading...