I've been saying for a while[1][2][3] that adding extra abstractions makes
things harder for newcomers, and adding new notations doubly so. And then
I notice this DSL in my own backyard. Makes me feel like a hypocrite.
[1] https://news.ycombinator.com/item?id=13565743#13570092
[2] https://lobste.rs/s/to8wpr/configuration_files_are_canary_warning
[3] https://lobste.rs/s/mdmcdi/little_languages_by_jon_bentley_1986#c_3miuf2
The implementation of the DSL was also highly hacky:
a) It was happening in the tangle/ tool, but was utterly unrelated to tangling
layers.
b) There were several persnickety constraints on the different kinds of
lines and the specific order they were expected in. I kept finding bugs
where the translator would silently do the wrong thing. Or the error messages
sucked, and readers may be stuck looking at the generated code to figure
out what happened. Fixing error messages would require a lot more code,
which is one of my arguments against DSLs in the first place: they may
be easy to implement, but they're hard to design to go with the grain of
the underlying platform. They require lots of iteration. Is that effort
worth prioritizing in this project?
On the other hand, the DSL did make at least some readers' life easier,
the ones who weren't immediately put off by having to learn a strange syntax.
There were fewer quotes to parse, fewer backslash escapes.
Anyway, since there are also people who dislike having to put up with strange
syntaxes, we'll call that consideration a wash and tear this DSL out.
---
This commit was sheer drudgery. Hopefully it won't need to be redone with
a new DSL because I grow sick of backslashes.
I've extracted it into a separate binary, independent of my Mu prototype.
I also cleaned up my tracing layer to be a little nicer. Major improvements:
- Realized that incremental tracing really ought to be the default.
And to minimize printing traces to screen.
- Finally figured out how to combine layers and call stack frames in a
single dimension of depth. The answer: optimize for the experience of
`browse_trace`. Instructions occupy a range of depths based on their call
stack frame, and minor details of an instruction lie one level deeper
in each case.
Other than that, I spent some time adjusting levels everywhere to make
`browse_trace` useful.
So far we only checked if a single recipe used a variable with multiple
types in any single space. Now we also ensure that the types deduced for
a variable in a space are identical across recipes.
The drawback of this is that we forget to initialize old_name when we
create instructions out of whole cloth in a few places. But this problem
already existed..
Be more disciplined about tagging 2 different concepts in the codebase:
a) Use the phrase "later layers" to highlight places where a layer
doesn't have the simplest possible self-contained implementation.
b) Use the word "hook" to point out functions that exist purely to
provide waypoints for extension by future layers.
Since both these only make sense in the pre-tangled representation of
the codebase, using '//:' and '#:' comments to get them stripped out of
tangled output.
(Though '#:' comments still make it to tangled output at the moment.
Let's see if we use it enough to be worth supporting. Scenarios are
pretty unreadable in tangled output anyway.)
Stop requiring jump instructions to explicitly provide a ':label' type
for jump targets.
This has been a source of repeated confusion for my students:
a) They'd add the ':label' to the label definition rather than the
jump target (label use)
b) They'd spend time thinking about whether the initial '+' prefix was
part of the label name.
In the process I cleaned up a couple of things:
- the space of names is more cleanly partitioned into labels and
non-labels (clarifying that '_' and '-' are non-label prefixes)
- you can't use label names as regular variables anymore
- you can infer the type of a label just from its name
More consistent definitions for jump targets and waypoints.
1. A label is a word starting with something other than a letter or
digit or '$'.
2. A waypoint is a label that starts with '<' and ends with '>'. It has
no restrictions. A recipe can define any number of waypoints, and
recipes can have duplicate waypoints.
3. The special labels '{' and '}' can also be duplicated any number of
times in a recipe. The only constraint on them is that they have to
balance in any recipe. Every '{' must be followed by a matching '}'.
4. All other labels are 'jump targets'. You can't have duplicate jump
targets in a recipe; that would make jumps ambiguous.
Always check if next_word() returned an empty string (if it hit eof).
Thanks Rebecca Allard for running into a crash when a .mu file ends with
'{' (without a following newline).
Open question: how to express the constraint that next_word() should
always check if its result is empty? Can *any* type system do that?!
Even the usual constraint that we must use a result isn't iron-clad: you
could save the result in a variable but then ignore it. Unless you go to
Go's extraordinary lengths of considering any dead code an error.
Undo 3272. The trouble with creating a new section for constants is that
there's no good place to order it since constants can be initialized
using globals as well as vice versa. And I don't want to add constraints
disallowing either side.
Instead, a new plan: always declare constants in the Globals section
using 'extern const' rather than just 'const', since otherwise constants
implicitly have internal linkage (http://stackoverflow.com/questions/14894698/why-does-extern-const-int-n-not-work-as-expected)
Move global constants into their own section since we seem to be having
trouble linking in 'extern const' variables when manually cleaving mu.cc
into separate compilation units.
Prefer preincrement operators wherever possible. Old versions of
compilers used to be better at optimizing them. Even if we don't care
about performance it's useful to make unary operators look like unary
operators wherever possible, and to distinguish the 'statement form'
which doesn't care about the value of the expression from the
postincrement which usually increments as a side-effect in some larger
computation (and so is worth avoiding except for some common idioms, or
perhaps even there).
This is part of efforts to allow students to transition gradually from
the sandbox to running programs directly on the commandline, writing
real scenarios, etc. Running on the commandline requires 'main', but
overriding 'main' would mess up edit/ which is itself a Mu program.
Always show instruction before any transforms in error messages.
This is likely going to make some errors unclear because they *need* to
show the original instruction. But if we don't have tests for those
situations did they ever really work?
I'd been toying with this idea for some time now given how large the
repo had been growing. The final straw was noticing that people cloning
the repo were having to wait *5 minutes*! That's not good, particularly
for a project with 'tiny' in its description. After purging .traces/
clone time drops to 7 seconds in my tests.
Major issue: some commits refer to .traces/ but don't really change
anything there. That could get confusing :/
Minor issues:
a) I've linked inside commits on GitHub like a half-dozen times online
or over email. Those links are now liable to eventually break. (I seem
to recall GitHub keeps them around as long as they get used at least
once every 60 days, or something like that.)
b) Numbering of commits is messed up because some commits only had
changes to the .traces/ sub-directory.
Standardize quotes around reagents in error messages.
I'm still sure there's issues. For example, the messages when
type-checking 'copy'. I'm not putting quotes around them because in
layer 60 I end up creating dilated reagents, and then it's a bit much to
have quotes and (two kinds of) brackets. But I'm sure I'm doing that
somewhere..
As outlined at the end of 2797. This worked out surprisingly well. Now
the snapshotting code touches fewer layers, and it's much better
behaved, with less need for special-case logic, particularly inside
run_interactive(). 30% slower, but should hopefully not cause any more
bugs.
I'm dropping all mention of 'recipe' terminology from the Readme. That
way I hope to avoid further bike-shedding discussions while I very
slowly decide on the right terminology with my students.
I could be smarter in my error messages and use 'recipe' when code uses
it and 'function' otherwise. But what about other words like ingredient?
It would all add complexity that I'm not yet sure is worthwhile. But I
do want separate experiences for veteran programmers reading about Mu on
github and for people learning programming using Mu.
All my attempts at staging this change failed with this humongous commit
that took all day and involved debugging three monstrous bugs. Two of
the bugs had to do with forgetting to check the type name in the
implementation of shape-shifting recipes. Bug #2 in particular would
cause core tests in layer 59 to fail -- only when I loaded up edit/! It
got me to just hack directly on mu.cc until I figured out the cause
(snapshot saved in mu.cc.modified). The problem turned out to be that I
accidentally saved a type ingredient in the Type table during
specialization. Now I know that that can be very bad.
I've checked the traces for any stray type numbers (rather than names).
I also found what might be a bug from last November (labeled TODO), but
we'll verify after this commit.
Stack of plans for cleaning up replace_type_ingredients() and a couple
of other things, from main problem to subproblems:
include type names in the type_tree rather than in the separate properties vector
make type_tree and string_tree real cons cells, with separate leaf nodes
redo the vocabulary for dumping various objects:
do we really need to_string and debug_string?
can we have a version with *all* information?
can we have to_string not call debug_string?
This commit nibbles at the edges of the final task, switching from
member method syntax to global function like almost everything else. I'm
mostly using methods just for STL in this project.
When we stash a value, mu does several levels of work for us:
a) First it inserts instructions above the stash to convert the value to
text using to-text-line.
b) to-text-line calls to-text. Both are shape-shifting, so multiple
levels of specialization happen.
To give a good error message, we track the 'stack' of current
specializations at the time of the error, and also check if the
offending instruction at the top-most level looks like it was inserted
while rewriting stash instructions.
Manual example (since booleans can't be stashed at the moment):
x:boolean <- copy 1/true
stash x
- notes
bug in edit/ triggers in immutable but not master branch
bug triggered by changes to layer 059: we're finding an unspecialized call to 'length' in 'append_6'
hard to debug because trace isn't complete
just bring out the big hammer: use a new log file
length_2 from recipes.mu is not being deleted (bug #1)
so reload doesn't switch length to length_2 when variant_already_exists (bug #2)
so we end up saving in Recipe for a primitive ordinal
so no valid specialization is found for 'length' (bug #3)
why doesn't it trigger in a non-interactive scenario?
argh, wasn't checking for an empty line at end. ok, confidence restored.
We don't actually need skip_whitespace_AND_comments_BUT_NOT_newline
anywhere except next_word(). Perhaps what I should really do is split
the definition of next_word() into two variants..
When skipping past some text (usually whitespace, but also commas and
comments) I need to always be aware of whether it's ok to switch to the
next line or not.
Another gotcha uncovered in the process of sorting out the previous
commit: I keep using eof() but forgetting that there are two other
states an istream can get into. Just never use eof().
Kartik Agaram