Hi Jaromir,
very weird ... I was able to reproduce the sporadic stack dump with your second example, and in one situation I also got another error (I forgot to take a screenshot but it said something like errorSubscriptBounds on a BlockCannotReturn instance).
I think this is either a bug in the VM or the TraceDebugger is abusing a set of primitives (see the comment in Sandbox2>>#context:doPrimitiveNew:receiver:args: for a similar situation). It definitely will be hard to debug. How does your crash dump look like (if the VM has created one)? (Note that the latest stack always is appended to that file.) If you could collect a few SqueakDebug.logs and crash dumps, maybe that would help. I will try the same.
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
By the way, I think we're at 22933. Note that the files on files.squeak.org are only updated once a day (provided that nothing goes wrong).
Best, Christoph
--- Sent from Squeak Inbox Talk
On 2024-01-03T23:19:53+00:00, mail@jaromir.net wrote:
Hi Christoph,
On 02-Jan-24 9:28:51 PM, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger?
Sorry, that must have been an image before the latest update. In a fresh, latest image it works as you describe.
However - and forgive me if this is some kind of problem at my end too - I'm observing this:
I downloaded a fresh image 22929, installed the Trace debugger, turned off the tree view and tried to run this example:
[[self halt. ^ 1] on: BlockCannotReturn do: [:ex | ex resume] ] fork
The exact steps are:
- do it in workspace
- select Doit context (second from top)
- switch to Trace debugger
- click step Over
and this just happened:
The weird thing is next time I open the same image again, nothing bad can happen many times but all of a sudden it happens again.
A few times I also saw this weird error pop up:
In this case I used another example: [[true ifTrue: [self halt. ^ 1]] on: BlockCannotReturn do: [:ex | ex resume] ] fork
What could cause such intermittent failures? I tried downloading a new image again but the same happens.
I hope I'm not wasting your time with something happening just at my end. The only irregular thing I noticed is it's difficult to install or update the baseline the same github cache issue I had when I installed Squot. The installation fails a few times before it goes through successfully.
Thanks! best, Jaromir
However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcesssuspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
The TraceDebugger now honors the suspended/terminated state of the debugged process so you cannot step beyond Processor activeProcesssuspend any longer, like in a regular debugger.
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2024-01-02T19:53:31+01:00, christoph.thiede(a)student.hpi.uni-potsdam.de wrote:
Hi Jaromir,
thanks again for your messages, please keep them coming! And
squeak-dev is totally the right place for them IMO. :-)
Regarding the openPath bug: Thanks for the pointer, I have uploaded
Collections-ct.1061 to the inbox which would fix it. However, it is a bit surprising that PluggableTreeItemNode>>#asString (and also Context>>#printString by the way) answers texts not strings, so maybe this needs further discussion.
Or when you open an inspector on a context the context's state gets
frozen in time and won't change when you proceed debugging - another groundbreaking change!
Yes, these "snapshot inspectors" (or also "snapshot explorers") have
their pros and cons. As a con, I often found it inconvenient that I cannot watch the changing state of certain objects in extra windows as I am stepping through a TraceDebugger. But always updating these inspectors depending on the current time of the TraceDebugger might be confusing as well because there is no clear visual connection ... It's an unsolved UX problem for me. But I'm glad they work well for you. :-) If you have any better ideas, let me know!
Regarding your questions about the behavior of code when being run in
the TraceDebugger:
If I turn off the preference "Show call tree in TraceDebugger" am I
right to expect the Trace debugger behavior would be equivalent to the traditional one?
Yes and no. :-) First, the representation of the traced program in
the TraceDebugger (stack vs tree) does not influence the execution semantics of the program. It's just that in rare situations with irregular context switches, the tree model is currently unable to locate certain contexts at certain points in time. That's why these contexts are skipped as you step through a program in the TraceDebugger with the context tree activated.
Second, code that is simulated inside the TraceDebugger is (or
should) behave exactly as the same code being run in a normal simulator (like when you step through an expression or use Context class>>#runSimulated:). There are however two exceptions to this invariant:
(1) Bugs in the simulation engine: We (that's an including we!) have
been working on fixing these bugs so that all code can behave exactly then same when being simulated. Still, there are some open known (and likely further unknown) issues (e.g., you cannot simulate a simulator which is executing a failed primitive: Context runSimulated: [Context runSimulated: [#() tryPrimitive: 60 withArgs: #(0)]]), so this delightful quest is still going on. :-)
(2) Context primitives 195-197 (#findNextUnwindContextUpTo:,
#terminateTo:, #findNextHandlerContextStarting) always fail when the context is executed in SimulationStudio (which also includes the TraceDebugger): This is due to the nature of SimulationStudio, which subclasses from Context (see SimulationContext) to make parts of the simulated code execution customizable. The VM, however, is not prepared to the existence of such subclass objects of Context and will always fail when these primitives are invoked on an object that is not exactly of the class Context, so the methods execute their fallback code instead. So this is a visible difference in the execution semantics between normal VM and SimulationStudio/TraceDebugger.
However, now you might say: This makes sense when I evaluate
Simulator debug: [thisContext findNextHandlerContextStarting] because when I inspect thisContext in that debugger, it shows a subclass of Context; but when I do [thisContext findNextHandlerContextStarting] debugTrace, thisContext actually is an instance of Context itself, so how can the VM detect this? And you would be right, because when you *inspect* a context in the TraceDebugger, it is a Context instance indeed, but not when you actually *execute* it in the TraceDebugger, as you can see when you evaluate [thisContext class] debugTrace ... The explanation for that lies in TDBTrace>>#enableSimulatorDuring:, but to cut it short, we convert all (non-dead) Context instances from the TraceDebugger's tree to a subclass of SimulationContext temporarily during each step to achieve two things: First, to not confuse observant users like you with the existence of these subclasses (well, maybe that did not work too well), and second, to make it possible to resume from a trace debugger at any point, which will execute the process in the regular VM; and as noted before, the VM can only handle Context instances, so it would fail when scheduling the process otherwise (you can actually observe that when trying to proceed from Simulator debug: [thisContext findNextHandlerContextStarting]). (Fun fact: Not all VMs handle Context subinstances that carefully: SqueakJS will seriously mix up the context/object layout, while TruffleSqueak will terminate as soon as you instantiate (!) any subinstance of Context, so I'm gladful that the OpenSmalltalk VM is as tolerant as it is.)
I hope this was a bit interesting to you!
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as if the computation just ran until the end.
Hm, I cannot reproduce this. If I step through ^2 and then step
through again, I land in Context>>terminateTo:. Are you using the latest version of trunk and TraceDebugger? However, you currently end up in #cannotReturn: when stepping beyond the Processor activeProcess suspend in the bottom context of a process using the TraceDebugger. This is because other than the normal debugger, the TraceDebugger does not yet honor the suspended/terminated state of the interrupted process. Maybe it should ...
Thanks for your thoughts and I'm always happy about more! :-)
Best, Christoph
Sent from Squeak Inbox Talk
On 2024-01-01T19:25:04+00:00, mail(a)jaromir.net wrote:
Hi Christoph,
sorry, a follow-up question :)
If I turn off the preference "Show call tree in TraceDebugger" am I right to expect the Trace debugger behavior would be equivalent to
the
traditional one?
In other words: if I run the traditional debugger and the Trace one
side
by side, should they display analogous steps?
In the other message I wrote about a different way to simulate
primitive
calls.
However, I've noticed other irregularities so that's why I started wondering maybe my assumption was wrong and the Trace debugger is designed to present the simulation differently. Please advise.
Example: do step through to the [^2] block and then step through
again
[^2] ensure: []
Traditionally, you end up in the unwind block. In the new Trace debugger you end up with the #cannotReturn:
context as
if the computation just ran until the end.
Is this expected? (My guess is it isn't but can't figure out why)
Thanks again, Jaromir
On 01-Jan-24 3:31:29 PM, "Jaromir Matas" <mail(a)jaromir.net>
wrote:
Hi Christoph,
Is it ok that I ask questions about the new debugger? What would
be the
best format for such a "Q&A" - here or perhaps within a topic on squeak-smalltalk/squeak-object-memory? I don't expect a flood of questions but to get a bit familiar with your debugger it would
help
tremendously to be able to ask right away instead of trudging
through
the code/help :) The code usually helps to understand **how**
things
work, the mechanics, but rarely **why**, the intentions.
you can also turn off the preference "Show call tree in
TraceDebugger"
Thanks, that helps to familiarize myself with the new
functionalities
"step-by-step", and not be overwhelmed by all the might of the
call
tree :) Being able to go back is already a hell of an improvement!
Or
when you open an inspector on a context the context's state gets
frozen
in time and won't change when you proceed debugging - another groundbreaking change!
Question: In the traditional debugger, when you step into a primitive, the primitive gets executed and the simulation moves over the
primitive
call. The Trace debugger, however, starts executing the fallback
code
of the primitive call - why is that?
Screenshot after step into #terminateTo:
Thanks again, Jaromir
On 31-Dec-23 2:16:32 AM,
christoph.thiede(a)student.hpi.uni-potsdam.de
wrote:
Hi Jaromir,
thanks a lot for trying it out!! Your feedback means a lot to me.
:-)
Indeed you hit a pretty unfortunate example. The TraceDebugger is
not
ready for all your clever investigations regarding non-local
returns
and unwinding. ;-) In fact, your example reveals another
limitation
that I forgot to mention in the announcement, which regards
programs
with irregular context switches - e.g., generators/coroutines,
but
also non-local returns through unwind contexts. This is because
the
TraceDebugger stores and displays all method invocations in a
tree,
but in the case of manual context switches, there is no single
global
tree - its structure would change over the execution time, and
when
selecting a method invocation, it is not even clear to what
parent
(sender) it would belong, as there might be multiple. The current solution is to display the tree from the perspective of the stack
at
the viewed point in time (see also the '@ <timeIndex>' in the
window
title), so it looks corrupted while stepping through Context>>#terminateTo: as the stack is being manipulated. (You
would
notice the same in a normal debugger if you turned off the
optional
primitive 196 in this method - for SimulationContexts this method always uses the fallback code.)
Nevertheless, I have pushed some changes that should allow you to
step
out of #terminateTo: again. (You can update the TraceDebugger
from the
window menu icon at the right top, like all of my tools.) At some point there will no method be displayed, but you can just step
further
and eventually return back to the starting point. :-) If you want
to,
you can also turn off the preference "Show call tree in
TraceDebugger"
to make the TraceDebugger look more like a normal debugger, which
also
solves the context switches issue. But in general - unless you
are
debugging unwinding stuff - I would not recommend that as it
removes
one important strength of the TraceDebugger. :-)
But again, this is really not a prime example for the
TraceDebugger.
Better use it to explore how the simulator works. :-) For
example, you
could do the following:
[ContextTest debug: #testBlockCannotReturn] debugTrace.
And in that trace debugger, you could select the start method,
press
Cmd + f(ind), and type "return:from:" to investigate the behavior
of
your solution there again, etc.
Thanks for your comments! This was a good chance for me to sort
some
things out! :-)
Best, Christoph
Sent from Squeak Inbox Talk https://github.com/hpi-swa-lab/squeak-inbox-talk
On 2023-12-30T19:04:07+00:00, mail(a)jaromir.net wrote:
> Hi Christoph, > > This indeed sounds like a GREAT idea! I look forward to seeing
your
use > cases to build the right intuition. > > In the meantime I've tried to debug/trace this example I've
been
working > with lately: > > [^2] ensure: [] > > If I start the debugger, hit `trace it` and then `step over`,
it
stops > at Context>>terminate and the view gets corrupted (the initial
part
of > the trace is hidden and can't be made visible unless clicking
on
some of > the pink lines - but not every line does it...) > > > > > If I then continue stepping over it ends up with some kind of
error:
> > > Maybe this is just an unfortunate example... Or maybe I'm just
doing
> something wrong... > > At any rate - THANKS for your effort!! > > > On 30-Dec-23 4:37:28 PM, christoph.thiede(a)student.hpi.uni-potsdam.de > wrote: > > >Thanks for the reply, Dave! I will try to post one or two
concrete
use > >cases about the TraceDebugger in the next couple of days, so
stay
> >tuned. :-) > > > >Best, > >Christoph > > > >--- > >Sent from Squeak Inbox Talk > >https://github.com/hpi-swa-lab/squeak-inbox-talk > > > >On 2023-12-29T11:01:10-06:00, lewis(a)mail.msen.com wrote: > > > > > This sounds like really interesting work! I love the idea
of
being > > > able to interactively go back in "oops, I‘ve stepped too
far,
let‘s > > > start all over again" situations. It will probably take
some
time for > > > me and others to wrap our heads around the things you have
done,
so > > > don't be surprised if you get a delayed response to this announcement > > > :-) > > > > > > Congratulations! > > > Dave > > > > > > > > > On Fri, Dec 29 2023 at 01:42:16 AM +0100, > > > christoph.thiede(a)student.hpi.uni-potsdam.de wrote: > > > > Hi all! > > > > > > > > I‘m very excited to announce a project today that we
have been
> > > > working on over the past two years: The *TraceDebugger*
[1] is
a > >new > > > > back-in-time/time-travel/omniscient debugging tool for
Squeak
that > > > > allows you to record past method activations and states
during
> > > > execution and explore them later. > > > > > > > > https://github.com/hpi-swa-lab/squeak-tracedebugger > > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > **What can it do? (Features)** > > > > > > > > - *Record all method activations and historic states:* Normally > >step > > > > through a program in the debugger while automatically recording its > > > > execution. > > > > - *Replay execution of a traced program:* Navigate
through all
> > > > method invocations using the /*context tree*/ or the
/*Step
> >Back/Step > > > > Forward*/ buttons (to avoid these "oops, I‘ve stepped
too far,
> > > > let‘s start all over again" situations). > > > > - *Interact with historic states:* Inspect/explore
snapshots
of > > > > objects or send them any message. > > > > - *State-centric debugging using the ***/History Explorer*/**:* > > > > Gather, explore, and visualize all changes to an object/expression > > > > over the recorded time ("When did this > >variable/collection/screenshot > > > > change?"). > > > > - *Additional navigation tools* for searching and
filtering
the > > > > context tree. > > > > - *Focus on interactivity:* No hours of recording, no
GBs of
mem > > > > consumption - at least for common small to medium
programs.
> > > > - *UI resembles the classic Smalltalk debugger:* You'll
find
your > > > > familiar stepping buttons, code browsing tools,
inspectors,
and > > > > shortcuts - plus more. > > > > > > > > The TraceDebugger is a general-purpose tool and not tied
to
> > > > particular domains. In the past months, we have
successfully
used > >it > > > > to understand several bugs and interaction patterns in
the
Trunk > > > > (Morphic layout/rendering, compiler/decompiler, code simulation, > > > > …). The tool is also self-supporting, so you can debug a > > > > TraceDebugger from another TraceDebugger. :-) > > > > > > > > **What can‘t it do (yet)? (Limitations and future
work)**
> > > > > > > > - *High performance:* While (sufficiently) fast enough
for
most > > > > small to medium workloads, tracing very compute- or mem-intensive > > > > operations may require more time (ex.:
compiler/decompiler
> > > > invocation: <1s, HTTPS request: <10s, tool building:
<5m,
complex > > > > rendering: minutes up to hours). > > > > - *Not a dataflow analyzer:* The TraceDebugger does not
track
> > > > dataflow events (e.g., argument passing) but only state changes. > > > > - *No tracing of external states/events* for
FFI/OSProcess or
> >custom > > > > VM modules. > > > > - *No support for advanced language concepts* such as
identity
> > > > forwarding/write barriers. > > > > > > > > **How does it work? (Implementation)** > > > > > > > > In one sentence: To record message sends and side
effects, we
> > > > decorate the execution of certain bytecodes with tracing extensions > > > > by modifying the code simulation using SimulationStudio
[2].
> > > > > > > > In one paragraph: The program is executed in a
specialized
code > > > > simulator that overrides instructions for sending
messages
(e.g., > > > > send, superSend) and for performing side-effects (e.g., > >popIntoRcvr, > > > > primitiveAtPut, push). All message sends are recorded in
a
tree and > > > > all changed object slots are stored in a sparse
time-dependent
> >memory > > > > structure before they are overwritten. For
time-traveling, the
tree > > > > is traversed using a cursor. For accessing historic
objects, a
> >proxy > > > > evaluates all messages sent to an object in another specialized > > > > simulator (retracing simulator) that emulates historic
states
for > >the > > > > requested point in time by forwarding read primitives
(e.g.,
> > > > pushRcvr, primitiveAt) to the recorded memory. For
gathering
state > > > > changes in the History Explorer efficiently, the query
is
evaluated > > > > in a range retracing simulator with vectorization and
fork
> >semantics. > > > > > > > > In academic terms: We have published two papers about
the
> > > > TraceDebugger that provide further details about its implementation > > > > and its applications for program exploration,
"Object-Centric
> > > > Time-Travel Debugging: Exploring Traces of Objects" [3]
and
> > > > "Time-Awareness in Object Exploration Tools: Toward In
Situ
> > > > Omniscient Debugging" [4]. > > > > > > > > In Smalltalk: Just check out the code base and explore
it by
> > > > yourself! The class comments in TraceDebugger > >code://TraceDebugger > > > > and TDBCursor code://TDBCursor should provide good
starting
> >points. > > > > > > > > **How can I use it?** > > > > > > > > Please try it out and report feedback! The TraceDebugger supports > > > > the latest Squeak Trunk and Squeak 6.0. You can either download a > > > > prepared all-in-one bundle on GitHub: > > > > > > > >
https://github.com/hpi-swa-lab/squeak-tracedebugger/releases
> > > > > > > > Or you can install it into your own image using
Metacello:
> > > > > > > > Metacellonew > > > > baseline:'TraceDebugger'; > > > > repository:'github://hpi-swa-lab/squeak-tracedebugger'; > > > > /"repository: > > > >
'github://hpi-swa-lab/squeak-tracedebugger:squeak60';"//"for
Squeak > > > > 6.0"/ > > > > get; > > > > load. > > > > > > > > To get started, just open a normal debugger (e.g., by selecting an > > > > expression and pressing Cmd+Shift+D to debug it) and
then
press the > > > > "Trace It" button on the right. There‘s also some pretty detailed > > > > documentation in the Help Browser <code:// TraceDebugger showHelp> > > > > that covers everything you should know. > > > > > > > > My goal is to improve convenience and provide a useful
tool
for the > > > > community, so I‘m very excited to hear your impressions, ideas, and > > > > thoughts. Here, on GitHub, or in a private message.
Let‘s have
a > > > > great discussion! :-) > > > > > > > > Best, > > > > > > > > Christoph (and Marcel) > > > > > > > > PS: Props to Eliot who brought up the original idea of "subclassing > > > > from Context" for other reasons four years ago. [5] > > > > > > > > [1]
https://github.com/hpi-swa-lab/squeak-tracedebugger
> > > > [2] https://github.com/LinqLover/SimulationStudio > > > > [3] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Object-Centric Time-Travel Debugging: Exploring Traces
of
Objects. > > > > https://doi.org/10.1145/3594671.3594678 In /Companion Proceedings > > > > of the 7th International Conference on the Art, Science,
and
> > > > Engineering of Programming/ (/<Programming>'23
Companion/),
March > > > > 13–17, 2023, Tokyo, Japan. ACM, New York, NY, USA, 7
pages.
DOI: > > > > 10.1145/3594671.3594678 https://doi.org/10.1145/3594671.3594678. > > > > PDF:
https://dl.acm.org/doi/pdf/10.1145/3594671.3594678
> > > > [4] Christoph Thiede, Marcel Taeumel, and Robert
Hirschfeld.
> > > > Time-Awareness in Object Exploration Tools: Toward In
Situ
> >Omniscient > > > > Debugging.
https://dl.acm.org/doi/10.1145/3622758.3622892 In
> > > > /Proceedings of the 2023 ACM SIGPLAN International
Symposium
on New > > > > Ideas, New Paradigms, and Reflections on Programming and Software/ > > > > (/Onward! '23/), October 25–27, 2023, Cascais, Portugal.
ACM,
New > > > > York, NY, USA, 14 pages. DOI: 10.1145/3622758.3622892 > > > > https://doi.org/10.1145/3622758.3622892. PDF: > > > > https://dl.acm.org/doi/pdf/10.1145/3622758.3622892 > > > > [5] > > > > >
http://lists.squeakfoundation.org/pipermail/squeak-dev/2019-October/204803.html
> > > > > > > > --- > > > > /Sent from//Squeak Inbox Talk > > > > https://github.com/hpi-swa-lab/squeak-inbox-talk/