There's a separate open question here of where Teliva should store files
like teliva_editor_state and teliva_editor_buffer. One school of thought
is that apps should never be dropping crud into people's directories. On
the other hand, I'm kinda encouraging people so far to just run apps
from Teliva's directory. Perhaps that makes it ok?
We're now back to the problem of how to transparently allow Teliva to
create temporary filenames without every app having to explicitly allow
them.
I think I may need to define start_writing in C, so that it can use a
non-sandboxed version of io.open.
The trouble with os.tmpname() is that it always creates in /tmp.
If /tmp is in a different volume from our real filename, os.rename()
will fail.
This new primitive doesn't support primitive paths yet.
I'm also again nervous about the security implications of my whole
approach. What if we create an inner function called start_writing?
Would we be able to do anything inside it? I'm starting to suspect this
whole approach of going by caller name is broken. An app could also
create inner functions called 'main'..
We now have a notion of libraries that we load after app code, to
prevent them from getting overridden.
Should I just load all libraries after the app? There might be value in
allowing apps to override library functions. Disallowing that too much
may be going against Lua's dynamic nature.
The interface for apps looks much nicer now, see 'main' in zet.tlv.
However there are some open issues:
- It can still be confusing to the computer owner that an app tries to
write to some temporary file that isn't mentioned anywhere.
- File renames can fail if /tmp is on a different volume.
- What happens if an app overrides start_writing()? The computer owner
may think they've audited the caller of start_writing and give it
blanket file permissions. Teliva tunnels through start_writing when
computing the caller. If the app can control what start_writing does,
the app could be performing arbitrary malicious file operations.
Right now things actually seem perfectly secure. Overriding
start_writing has no effect. Our approach for loading .tlv files (in
reverse chronological order, preventing older versions from overriding
newer ones) has the accidentally _great_ property that Teliva apps can
never override system definitions.
So we have a new reason to put standard libraries in a .lua file: if
we need to prevent apps from overriding it.
This feels like something that needs an automated test, both to make
sure I'm running the right experiment and to ensure I don't
accidentally cause a regression in the future. I can totally imagine a
future rewrite that tried a different approach than
reverse-chronological.
Perhaps this is a bad idea. It feels arbitrary, what methods Lua happens
to include in string and table objects without having to go through the
respective modules.
When should code go in the template used by new apps vs the .lua files
distributed with Teliva?
- from a privilege perspective there's no difference
- from a compatibility perspective stuff in .tlv will not get upgraded
with Teliva.
- for me the maintainer, functions in .lua files are easier to upgrade
in a single place.
- for the reader of an app, functions in .lua files will not show up to
be edited. They can still be overloaded, but the current version isn't
as discoverable. Putting something in the app is a slight nudge to
readers that they're encouraged to mess with it.
- Stuff in .lua files can use local functions and so have more internal
complexity. Apps can also hide details within functions, but that'll
make them more likely to run into limitations with Teliva's editing
environment.
I'm not yet sure how to reason about the second point in practice.
- Stuff in .tlv files I don't have to worry about compatibility
guarantees for.
- Stuff in .lua files I _do_ have to worry about compatibility
guarantees for.
Perhaps this means I'm doing things exactly wrong in this commit?
Functions like map/reduce/filter/append seem more timeless, whereas I'm
still just feeling my way around with start_reading and start_writing.
We'll see. For now I'm ruled by the fourth point. Messing with tasks and
the scheduler is much more advanced than anything else in template.tlv;
it seems to make sense to add some friction to modifying them.
Bottomline: Complex sub-systems go in .lua files. Simple, self-contained
definitions go into apps. Both are probably equally burdensome now from
a compatibility perspective.
I'm starting to get quite indisciplined about where I introduce global
bindings. Seems reasonable since any modules in Teliva will come from
within the framework.
Teliva isn't yet smart enough to know the caller of an indirect function
where the function being called goes through a local variable.
I'd expected fixing this to be a long death march. However, there's a
shockingly easy fix: just make every indirect call go through an
additional direct function call.
My policy for zet.tlv was that function 'main' could open any file. This
stopped working since I introduced spawn_main. But with this commit it's
working again.
I can also drop all my special-casing of 'main' since it's now a regular
Lua call.
We still can't rely on the caller of an indirect call. That affects
start_reading and start_writing, which really need to be part of the
framework.
File operations now always return a channel (or nil on error or
permission denied).
When start_reading() from a filename, you can repeatedly :recv() from
the channel it returns.
When :recv() returns nil, you're at the end of the file. Stop.
When you start_writing() to a filename, you can repeatedly :send() to
the channel it returns.
When you're done writing, :close() the channel. Writes to the file won't
be externally visible until you do.
To make this work I'm now always starting up the scheduler, so I need to
fix sieve.tlv.
Transparently running the scheduler is an abstraction, and whenever I
create an abstraction I always worry about how it might fail. There's
a hopefully-clear error when you read past end of a file.
Until now you had to press ctrl-x twice in rapid succession to exit if
an app turned on non-blocking keyboard with nodelay(true). This became
particularly noticeable after the previous change to anagrams.tlv, which
could no longer exit.
They make it seem like you can use them to create simple REPL apps, but
you can't, because standard Teliva shortcuts won't work.
I _could_ make them work by emulating them using getch(), but that feels
like an unnecessary abstraction for now.
I introduced this ability when I packaged up the lfs directory, but it
can enable apps to circumvent sandboxing rules in some situations. If
you can socially engineer someone to allow reading a file called
'passwd' in the current directory, you can now change directory to /etc
and read something sensitive.
Protecting against stuff like this gets subtle. It's easy for people to
create policies that aren't robust to changing directories. Requiring
absolute paths is also pretty unfriendly. So the whole notion of current
directory is perhaps implicit state that is confusing to manage. Fix it
in the context of a single session.
When I started logging getch() events (which are just to help the reader
orient on the log), this suddenly became more urgent.
Now the log is larger, and it's also a circular buffer that rolls back
to the start when it fills up.
The next failure mode will be if we see the buffer filled up with just
getch() calls, reducing visibility over real file and network
operations. In which case we'll need to start coalescing getch() events.
Kartik K. Agaram