driver.love/Manual_tests.md

I care a lot about being able to automatically check any property about my program before it ever runs. However, some things don't have tests yet, either because I don't know how to test them or because I've been lazy. I'll at least record those here.

Initializing settings:

• from previous session

• from defaults

  • default_map absent/present
  • when connecting to a new app

• run with an untested version of LÖVE. Error message pops up and waits for a key. The app attempts to continue, and doesn't receive the key.

• run with a LÖVE v12 release candidate. No errors; it is a supported version. All tests pass.

• create a couple of spuriously failing tests. Run with an untested version of LÖVE. Error message includes message about untested version.

Code loading:

• run love with directory; text editor runs

• run love with zip file; text editor runs

• How the screen looks. Our tests use a level of indirection to check text and graphics printed to screen, but not the precise pixels they translate to.

  • where exactly the cursor is drawn to highlight a given character
  • analogously, how a shape precisely looks as you draw it

Panning:

• Zoom = 1 short node entirely within viewer, cursor in middle of screen ✓ arrow keys ✓ page-up positions cursor on top line ✓ page-down positions cursor on bottom line ✓ mouse panning tall node extending below viewport ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated down arrow ✓ repeated pagedown ✓ pageup ✓ mouse panning tall node extending above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated up arrow ✓ pagedown ✓ repeated pageup ✓ mouse panning tall node extending both below and above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated pagedown ✓ repeated pageup ✓ mouse panning

• Zoom > 1 (e.g. ctrl+0 ctrl+= ctrl+=) short node entirely within viewer, cursor in middle of screen ✓ arrow keys ✓ page-up positions cursor on top line ✓ page-down positions cursor on bottom line ✓ mouse panning tall node extending below viewport ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated down arrow ✓ repeated pagedown ✓ pageup ✓ mouse panning tall node extending above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated up arrow ✓ pagedown ✓ repeated pageup ✓ mouse panning tall node extending both below and above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated pagedown ✓ repeated pageup ✓ mouse panning

• Zoom < 1 (e.g. ctrl+0 ctrl+- ctrl+-) short node entirely within viewer, cursor in middle of screen ✓ arrow keys ✓ page-up positions cursor on top line ✓ page-down positions cursor on bottom line ✓ mouse panning tall node extending below viewport ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated down arrow ✓ repeated pagedown ✓ repeated pageup ✓ mouse panning tall node extending above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated up arrow ✓ repeated pagedown ✓ repeated pageup ✓ mouse panning tall node extending both below and above viewport ✓ repeated up arrow near top pans up ✓ repeated down arrow ✓ repeated down arrow near bottom pans down ✓ repeated up arrow ✓ repeated pagedown ✓ repeated pageup ✓ mouse panning

Protocol with driver; error-handling

• clone this repo to a new client app, clear its save dir[1], run it, run the driver, add a definition:

  on.draw = function()
  end
  

  Hit F4. No error. Quit driver, quit client app. Restart client app. No error.

[1] We never clear the app from the driver's config. driver.love needs to be robust to apps changing out from under it.

• clone this repo to a new client app, clear its save dir, run it, run the driver, add a definition that draws to screen:

  on.draw = function()
    love.graphics.print('hello!', 50,50)
  end
  

  Hit F4. Client app shows 'hello!' Quit driver, quit client app. Restart client app. No error. Client app shows 'hello!'

• clone this repo to a new client app, clear its save dir, run it, run the driver, add a definition containing invalid Lua:

  on.draw = function(
  

  Hit F4. Driver shows an error under the definition. Fix the definition:

  on.draw = function()
  end
  

  Hit F4. The error disappears.

• clone this repo to a new client app, clear its save dir, run it, run the driver, add a definition containing invalid Lua:

  on.draw = function(
  

  Hit F4. Driver shows an error under the definition as before. Quit the client app. Restart the client app. It loads up without error. Switch back to the driver. Fix the definition.

  on.draw = function()
  end
  

  Hit F4. The error disappears.

Driver can connect to app on errors in on.initialize (and on.load_settings).

• clone this repo to a new client app, clear its save dir, run it, run the driver, define on.initialize with a run-time error:

  on.initialize = function()
      foo = bar+1
  end
  

  Hit F4. Quit the client app and restart. App shows an error. Edit on.initialize in the driver and remove the error:

  on.initialize = function()
  end
  

  Hit F4. The error disappears from the app and driver.

• clone this repo to a new client app, clear its save dir, run it, run the driver, define on.initialize with a run-time error:

  on.initialize = function()
      foo = bar+1
  end
  

  Hit F4. Quit the client app and restart. App shows an error. Hit F4 again in the driver (without fixing the error). The client app continues to show the error.

Driver can connect to app on errors in on.quit (and on.save_settings).

• clone this repo to a new client app, clear its save dir, run it, run the driver, define on.quit with a run-time error:

  on.quit = function()
      foo = bar+1
  end
  

  Hit F4. Try to quit the client app. It shows an error and refuses to quit. Edit on.quit in the driver and remove the error:

  on.quit = function()
  end
  

  Hit F4. The error disappears from the app and driver. Try to quit the client app. Now the quit succeeds.

• clone this repo to a new client app, clear its save dir, run it, run the driver, define on.quit with a run-time error:

  on.quit = function()
      foo = bar+1
  end
  

  Hit F4. Try to quit the client app. It shows an error and refuses to quit. Hit F4 again in the driver (without fixing the error). Try to quit the client app again. It continues to show the error.

Driver can connect to app that contains test failures on startup.

• clone this repo to a new client app, clear its save dir, run it, run the driver, define a new test with an invalid assertion:

  test_foo = function()
      check(nil, 'foo')
  end
  

  Hit F4. The test fails. Quit the client app and restart. App shows an error. Edit test_foo in the driver and remove the error:

  test_foo = function()
  end
  

  Hit F4. The error disappears from the app and driver.

• clone this repo to a new client app, clear its save dir, run it, run the driver, define a new test with an invalid assertion:

  test_foo = function()
      check(nil, 'foo')
  end
  

  Hit F4. Quit the client app and restart. App shows an error. Hit F4 again in the driver (without fixing the error). The client app continues to show the error.

• clone this repo to a new client app, clear its save dir, run it, run the driver, add a definition containing invalid Lua:

  on.draw = function(
  

  Hit F4. Driver shows an error under the definition as before. Quit the client app. Switch back to the driver. Hit F4. The driver hangs and needs to be force-quit. [This is not ideal, but how things are today.]

• press C-n, add an empty definition:

  foo = function()
  end
  

  Press C-d, select foo to delete its definition. Driver doesn't hang, no errors in logs.

• press C-d, select on to try to delete its definition. Driver shows an error that it's not allowed.

Other compromises

Lua is dynamically typed. Tests can't patch over lack of type-checking.

• All strings are UTF-8. Bytes within them are not characters. I try to label byte offsets with the suffix _offset, and character positions as _pos. For example, string.sub should never use a _pos, only an _offset.

• Some ADT/interface support would be helpful in keeping per-line state in sync. Any change to line data should clear the derived line property screen_line_starting_pos.

• Some inputs get processed in love.textinput and some in love.keypressed. Several bugs have arisen due to destructive interference between the two for some key chord. I wish I could guarantee that the two sets are disjoint. But perhaps I'm not thinking about this right.

• Like any high-level language, it's easy to accidentally alias two non-scalar variables. I wish there was a way to require copy when assigning.

• I wish I could require pixel coordinates to be integers. The editor defensively converts input margins to integers.

• My test harness automatically runs test_* methods -- but only at the top-level. I wish there was a way to raise warnings if someone defines such a function inside a dict somewhere.