We now have a notion of libraries that we load after app code, to
prevent them from getting overridden.
Should I just load all libraries after the app? There might be value in
allowing apps to override library functions. Disallowing that too much
may be going against Lua's dynamic nature.
When should code go in the template used by new apps vs the .lua files
distributed with Teliva?
- from a privilege perspective there's no difference
- from a compatibility perspective stuff in .tlv will not get upgraded
with Teliva.
- for me the maintainer, functions in .lua files are easier to upgrade
in a single place.
- for the reader of an app, functions in .lua files will not show up to
be edited. They can still be overloaded, but the current version isn't
as discoverable. Putting something in the app is a slight nudge to
readers that they're encouraged to mess with it.
- Stuff in .lua files can use local functions and so have more internal
complexity. Apps can also hide details within functions, but that'll
make them more likely to run into limitations with Teliva's editing
environment.
I'm not yet sure how to reason about the second point in practice.
- Stuff in .tlv files I don't have to worry about compatibility
guarantees for.
- Stuff in .lua files I _do_ have to worry about compatibility
guarantees for.
Perhaps this means I'm doing things exactly wrong in this commit?
Functions like map/reduce/filter/append seem more timeless, whereas I'm
still just feeling my way around with start_reading and start_writing.
We'll see. For now I'm ruled by the fourth point. Messing with tasks and
the scheduler is much more advanced than anything else in template.tlv;
it seems to make sense to add some friction to modifying them.
Bottomline: Complex sub-systems go in .lua files. Simple, self-contained
definitions go into apps. Both are probably equally burdensome now from
a compatibility perspective.
NetBSD still uses curses by default. One _could_ install ncurses, but I
don't have access to a NetBSD box with permissions to install ncurses,
so I'm experimenting to see how far we can get with just curses. So far
most of the apps seem to work, with the exception of one bug that I'll
commit next.
I should have documented that I'd never actually seen that code path
trigger before. Here's a minimal test that did it just now:
function test_foo()
return a+1
end
E2: [string "test_foo"]:2: attempt to perform arithmetic on global 'a' (a nil value)
A simple missing variable doesn't do it since it just evaluates to nil.
Without this commit, the above test was silently continuing to the main
app after failing tests.
In each session, Teliva has to bootstrap a trusted channel with the
computer owner while running arbitrarily untrusted code. So let's get
really, really precise about what the trusted channel consists of:
- the bottom-most row of screen containing the menu
- the keystrokes the owner types in
- ncurses COLOR_PAIR slots 254 (menu) and 255 (error)
One reason the menu colors are important: we don't want people to get
used to apps that hide the menu colors by setting default
foreground/background to invisible and then drawing their own menu one
row up.
The error COLOR_PAIR I don't see any reason to carve out right now, but
it seems like a good idea for Teliva the framework to not get into the
habit of apps doing some things for it.
I'm not sure how realistic all this is (I feel quite ill-equipped to
think about security), but it seems worthwhile to err on the side of
paranoia. Teliva will be paranoid so people don't have to be.
Examples:
- you try to write file but disk is full
- you have two Teliva files being edited at the same time
Both are situations where it's impossible to avoid some data loss.
However, we should now at least have some valid state of the .tlv file
saved to disk where we'd previously end up with a zero-size file or
garbage.
Teliva emits timestamps in multi-line format end in a newline. As a
result, notes get rendered on the next line and are then immediately
overwritten by the contents of the definition.
This bug was masked by my hacky 'original' timestamps which don't use
multi-line format.
This reverts commit 7c1b9d0b91.
The 'big hammer' isn't good enough. The recent changes view seems to
need state on the stack across invocations of the editor.
This is a complete mess. I want to abstract reading multiline strings
behind a function, but the lookahead requirements for that are quite
stringent. What's a reasonable abstraction here?
I really wanted to avoid getting into defining or parsing new file
formats. However, using the entire power of Lua is not ideal, as
described earlier in Konrad Hinsen's bug. In addition to everything
else, it's a vector for arbitrary code execution when someone loads an
untrusted image.
I could use JSON, but it requires ugly string escaping. Seems cleaner to
just use YAML. But YAML is complex and needs its own dependencies. If
I'm going to do my own, might as well make the multi-line string format
really clear.
I can't yet write the new format.
Steps to reproduce:
* Run teliva with some app.
* Press ctrl-e to edit the app.
* Select some function.
* Press ctrl-g and type in some Lua keyword like 'function' or 'while'
(Since the first word in a function is often 'function', it becomes
the default if you press ctrl-g immediately after entering the editor
for a function.)
* Type nothing. Run the app.
Desired behavior: app continues to run. The definition for the keyword
is silently ignored (in future we may want to provide an error message)
Behavior before this commit: app silently exited with non-zero status,
and refused to restart thereafter until the .tlv file was manually
edited to delete the definition for the Lua keyword.
Behavior after this commit: app throws an error message like these:
* For `function`:
```
src/teliva: x.tlv:99: '(' expected near '='
sorry, you'll need to edit the image directly. press any key to exit.
```
* For `while`:
```
src/teliva: x.tlv:99: unexpected symbol near 'while'
sorry, you'll need to edit the image directly. press any key to exit.
```
You still need to edit the .tlv file manually, but the steps for
recovery are a bit more discoverable.
To fix this properly I also need to fix a looming security hole I've
been thinking about for some time. The long-term goal of Teliva is to
put the human running apps in control of what they do, by sandboxing
accesses to the file system, network and so on. However, even after we
build gates on all of Lua's standard libraries, we're still parsing .tlv
files as Lua, with all of its power available.
Solution: load .tlv files as some sort of JSON-like subset of Lua. Maybe
I should just use JSON, and rely on code that's already in Teliva, even
if I'm introducing a new notation in the process.
Kartik K. Agaram