spell-cards.love/Manual_tests.md

I care a lot about being able to automatically check _any_ property about my
program before it ever runs. However, some things don't have tests yet, either
because I don't know how to test them or because I've been lazy. I'll at least
record those here.

Initializing settings:
  - from previous session
  - from defaults
    - `default_map` absent/present

  - run with an untested version of LÖVE. Error message pops up and waits for a key. The app attempts to continue, and doesn't receive the key.
  - run with a LÖVE v12 release candidate. No errors; it is a supported version. All tests pass.
  - create a couple of spuriously failing tests. Run with an untested version of LÖVE. Error message includes message about untested version.

Code loading:
* run love with directory; text editor runs
* run love with zip file; text editor runs

* How the screen looks. Our tests use a level of indirection to check text and
  graphics printed to screen, but not the precise pixels they translate to.
    - where exactly the cursor is drawn to highlight a given character
    - analogously, how a shape precisely looks as you draw it

### Protocol with driver; error-handling

* clone this repo to a new client app, clear its save dir[1], run it, run the
  driver, add a definition:
    ```
    on.draw = function()
    end
    ```
  Hit F4. No error.
  Quit driver, quit client app. Restart client app. No error.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, add a definition that draws to screen:
    ```
    on.draw = function()
      love.graphics.print('hello!', 50,50)
    end
    ```
  Hit F4. Client app shows 'hello!'
  Quit driver, quit client app. Restart client app. No error. Client app shows 'hello!'

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, add a definition containing invalid Lua:
    ```
    on.draw = function(
    ```
  Hit F4. Driver shows an error under the definition.
  Fix the definition:
    ```
    on.draw = function()
    end
    ```
  Hit F4. The error disappears.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, add a definition containing invalid Lua:
    ```
    on.draw = function(
    ```
  Hit F4. Driver shows an error under the definition as before.
  Quit the client app.
  Restart the client app. It loads up without error.
  Switch back to the driver. Fix the definition.
    ```
    on.draw = function()
    end
    ```
  Hit F4. The error disappears.

Driver can connect to app on errors in on.initialize (and `on.load_settings`).
* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define `on.initialize` with a run-time error:
    ```
    on.initialize = function()
        foo = bar+1
    end
    ```
  Hit F4.
  Quit the client app and restart. App shows an error.
  Edit `on.initialize` in the driver and remove the error:
    ```
    on.initialize = function()
    end
    ```
  Hit F4. The error disappears from the app and driver.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define `on.initialize` with a run-time error:
    ```
    on.initialize = function()
        foo = bar+1
    end
    ```
  Hit F4.
  Quit the client app and restart. App shows an error.
  Hit F4 again in the driver (without fixing the error).
  The client app continues to show the error.

Driver can connect to app on errors in `on.quit` (and `on.save_settings`).
* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define `on.quit` with a run-time error:
    ```
    on.quit = function()
        foo = bar+1
    end
    ```
  Hit F4.
  Try to quit the client app. It shows an error and refuses to quit.
  Edit `on.quit` in the driver and remove the error:
    ```
    on.quit = function()
    end
    ```
  Hit F4. The error disappears from the app and driver.
  Try to quit the client app. Now the quit succeeds.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define `on.quit` with a run-time error:
    ```
    on.quit = function()
        foo = bar+1
    end
    ```
  Hit F4.
  Try to quit the client app. It shows an error and refuses to quit.
  Hit F4 again in the driver (without fixing the error).
  Try to quit the client app again. It continues to show the error.

Driver can connect to app that contains test failures on startup.
* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define a new test with an invalid assertion:
    ```
    test_foo = function()
        check(nil, 'foo')
    end
    ```
  Hit F4. The test fails.
  Quit the client app and restart. App shows an error.
  Edit `test_foo` in the driver and remove the error:
    ```
    test_foo = function()
    end
    ```
  Hit F4. The error disappears from the app and driver.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, define a new test with an invalid assertion:
    ```
    test_foo = function()
        check(nil, 'foo')
    end
    ```
  Hit F4.
  Quit the client app and restart. App shows an error.
  Hit F4 again in the driver (without fixing the error).
  The client app continues to show the error.

* clone this repo to a new client app, clear its save dir, run it, run the
  driver, add a definition containing invalid Lua:
    ```
    on.draw = function(
    ```
  Hit F4. Driver shows an error under the definition as before.
  Quit the client app.
  Switch back to the driver. Hit F4. The driver hangs and needs to be
  force-quit. [This is not ideal, but how things are today.]

[1] We never clear the app from the driver's config. driver.love needs to be
robust to apps changing out from under it.

### Other compromises

Lua is dynamically typed. Tests can't patch over lack of type-checking.

* All strings are UTF-8. Bytes within them are not characters. I try to label
  byte offsets with the suffix `_offset`, and character positions as `_pos`.
  For example, `string.sub` should never use a `_pos`, only an `_offset`.

* Some ADT/interface support would be helpful in keeping per-line state in
  sync. Any change to line data should clear the derived line property
  `screen_line_starting_pos`.

* Some inputs get processed in love.textinput and some in love.keypressed.
  Several bugs have arisen due to destructive interference between the two for
  some key chord. I wish I could guarantee that the two sets are disjoint. But
  perhaps I'm not thinking about this right.

* Like any high-level language, it's easy to accidentally alias two non-scalar
  variables. I wish there was a way to require copy when assigning.

* I wish I could require pixel coordinates to be integers. The editor
  defensively converts input margins to integers.

* My test harness automatically runs `test_*` methods -- but only at the
  top-level. I wish there was a way to raise warnings if someone defines such
  a function inside a dict somewhere.