There's a separate open question here of where Teliva should store files
like teliva_editor_state and teliva_editor_buffer. One school of thought
is that apps should never be dropping crud into people's directories. On
the other hand, I'm kinda encouraging people so far to just run apps
from Teliva's directory. Perhaps that makes it ok?
We're now back to the problem of how to transparently allow Teliva to
create temporary filenames without every app having to explicitly allow
them.
I think I may need to define start_writing in C, so that it can use a
non-sandboxed version of io.open.
The trouble with os.tmpname() is that it always creates in /tmp.
If /tmp is in a different volume from our real filename, os.rename()
will fail.
This new primitive doesn't support primitive paths yet.
I'm also again nervous about the security implications of my whole
approach. What if we create an inner function called start_writing?
Would we be able to do anything inside it? I'm starting to suspect this
whole approach of going by caller name is broken. An app could also
create inner functions called 'main'..
We now have a notion of libraries that we load after app code, to
prevent them from getting overridden.
Should I just load all libraries after the app? There might be value in
allowing apps to override library functions. Disallowing that too much
may be going against Lua's dynamic nature.
We aren't actually secure as the previous commit said. The hole here is
that you can't override start_writing by typing in 'start_writing' into
the big picture. However you _can_ override it by typing in _anything
else_.
The interface for apps looks much nicer now, see 'main' in zet.tlv.
However there are some open issues:
- It can still be confusing to the computer owner that an app tries to
write to some temporary file that isn't mentioned anywhere.
- File renames can fail if /tmp is on a different volume.
- What happens if an app overrides start_writing()? The computer owner
may think they've audited the caller of start_writing and give it
blanket file permissions. Teliva tunnels through start_writing when
computing the caller. If the app can control what start_writing does,
the app could be performing arbitrary malicious file operations.
Right now things actually seem perfectly secure. Overriding
start_writing has no effect. Our approach for loading .tlv files (in
reverse chronological order, preventing older versions from overriding
newer ones) has the accidentally _great_ property that Teliva apps can
never override system definitions.
So we have a new reason to put standard libraries in a .lua file: if
we need to prevent apps from overriding it.
This feels like something that needs an automated test, both to make
sure I'm running the right experiment and to ensure I don't
accidentally cause a regression in the future. I can totally imagine a
future rewrite that tried a different approach than
reverse-chronological.
Perhaps this is a bad idea. It feels arbitrary, what methods Lua happens
to include in string and table objects without having to go through the
respective modules.
When should code go in the template used by new apps vs the .lua files
distributed with Teliva?
- from a privilege perspective there's no difference
- from a compatibility perspective stuff in .tlv will not get upgraded
with Teliva.
- for me the maintainer, functions in .lua files are easier to upgrade
in a single place.
- for the reader of an app, functions in .lua files will not show up to
be edited. They can still be overloaded, but the current version isn't
as discoverable. Putting something in the app is a slight nudge to
readers that they're encouraged to mess with it.
- Stuff in .lua files can use local functions and so have more internal
complexity. Apps can also hide details within functions, but that'll
make them more likely to run into limitations with Teliva's editing
environment.
I'm not yet sure how to reason about the second point in practice.
- Stuff in .tlv files I don't have to worry about compatibility
guarantees for.
- Stuff in .lua files I _do_ have to worry about compatibility
guarantees for.
Perhaps this means I'm doing things exactly wrong in this commit?
Functions like map/reduce/filter/append seem more timeless, whereas I'm
still just feeling my way around with start_reading and start_writing.
We'll see. For now I'm ruled by the fourth point. Messing with tasks and
the scheduler is much more advanced than anything else in template.tlv;
it seems to make sense to add some friction to modifying them.
Bottomline: Complex sub-systems go in .lua files. Simple, self-contained
definitions go into apps. Both are probably equally burdensome now from
a compatibility perspective.
I'm starting to get quite indisciplined about where I introduce global
bindings. Seems reasonable since any modules in Teliva will come from
within the framework.
They may take more or less from it (sieve.tlv in particular takes
nothing since call depth doesn't help at all there), but what they take
is in the right order so that you can compare across apps.
Teliva isn't yet smart enough to know the caller of an indirect function
where the function being called goes through a local variable.
I'd expected fixing this to be a long death march. However, there's a
shockingly easy fix: just make every indirect call go through an
additional direct function call.
My policy for zet.tlv was that function 'main' could open any file. This
stopped working since I introduced spawn_main. But with this commit it's
working again.
I can also drop all my special-casing of 'main' since it's now a regular
Lua call.
We still can't rely on the caller of an indirect call. That affects
start_reading and start_writing, which really need to be part of the
framework.
File operations now always return a channel (or nil on error or
permission denied).
When start_reading() from a filename, you can repeatedly :recv() from
the channel it returns.
When :recv() returns nil, you're at the end of the file. Stop.
When you start_writing() to a filename, you can repeatedly :send() to
the channel it returns.
When you're done writing, :close() the channel. Writes to the file won't
be externally visible until you do.
To make this work I'm now always starting up the scheduler, so I need to
fix sieve.tlv.
Transparently running the scheduler is an abstraction, and whenever I
create an abstraction I always worry about how it might fail. There's
a hopefully-clear error when you read past end of a file.
If we press a key the computation now restarts instantly.
There's no fiction of multi-threading in Teliva. If the application
doesn't work right, it beach-balls. If it doesn't beach-ball under
normal circumstances you're more certain it'll never beach-ball. It's
more work up-front, but there's less variability in outcomes.
Until now you had to press ctrl-x twice in rapid succession to exit if
an app turned on non-blocking keyboard with nodelay(true). This became
particularly noticeable after the previous change to anagrams.tlv, which
could no longer exit.
Tasteful apps should only perform side-effects through 'window'
arguments rather than the 'curses' module directly. It's ok however to
read constants like curses.A_NORMAL or curses.stdscr().
There are some limitations, unfortunately. Ncurses wasn't designed with
testability in mind. For example, there's no way to curs_set or
assume_default_colors without the 'curses' module. Oh well.
They make it seem like you can use them to create simple REPL apps, but
you can't, because standard Teliva shortcuts won't work.
I _could_ make them work by emulating them using getch(), but that feels
like an unnecessary abstraction for now.
Kartik K. Agaram