text.love/Manual_tests.md

62 lines
4.2 KiB
Markdown
Raw Permalink Normal View History

I care a lot about being able to automatically check _any_ property about my
2022-06-15 05:47:49 +00:00
program before it ever runs. However, some things don't have tests yet, either
because I don't know how to test them or because I've been lazy. I'll at least
record those here.
Initializing settings:
- delete app settings, start with a filename; window opens running the text editor with cursor at top of file
- run with absolute file path; quit; restart; window opens running the text editor in same position+dimensions
- run with relative file path; quit; switch to new directory; restart without a filename; window opens running the text editor in same file in same position+dimensions
- run with a filename on commandline, scroll around, quit; restart without a filename; window opens running the text editor in same position+dimensions
- run with a filename on commandline, scroll around, quit; restart with same filename; window opens running the text editor in same position+dimensions
- run with a filename on commandline, scroll around, quit; restart with new filename; window opens new filename with cursor up top
- run editor, scroll around, move cursor to end of some line, quit; restart with new filename; window opens running the text editor in same position+dimensions
2023-03-29 05:00:43 +00:00
- quit while running the text editor, restart; window opens running the text editor in same position+dimensions
2023-03-31 06:12:01 +00:00
- quit while editing source (color; no selection), restart; window opens editing source in same position+dimensions
2023-03-29 05:00:43 +00:00
- start out running the text editor, move window, press ctrl+e twice; window is running text editor in same position+dimensions
- start out editing source, move window, press ctrl+e twice; window is editing source in same position+dimensions
- no log file; switching to source works
- run with an untested version of LÖVE. Error message pops up and waits for a key. The app attempts to continue, and doesn't receive the key.
_yet another_ bugfix to the version check X-( When I stopped running the version check before the tests I also stopped initializing Version, which can be used in tests to watch out for font changes across versions. As a result I started seeing a test failure with LÖVE v12. It looks like all manual tests pass now. And we're also printing the warning about version checks before running tests, which can come in handy if a new version ever causes test failures. The only thing that makes me unhappy is the fact that we're calling the version check twice. And oh, the fact that this part around initialization and version management is clearly still immature. I'll capture some desires and fragmentary thought processes around them: * If there's an error, go to the source editor. * But oh, don't go to source editor on some unactionable errors, so we include a new `Current_app` mode for them: * Unsupported version requires an expert. Just muddle through if you can or give a warning someone can send me. * A failing test might be spurious depending on the platform and font rendering scheme. So again just provide a warning someone can send me. [Source editor can be confusing for errors. Also an editor! But not showing the file you asked for!] * But our framework clears the warning after running tests: * If someone is deep in developing a new feature and quits -> restore back in the source editor. [Perhaps `Current_app` is the wrong place for this third hacky mode, since we actually want to continue running. Perhaps it's orthogonal to `Current_app`.] [Ideally I wouldn't run the tests after the version check. I'd pause, wait for a key and then resume tests? "Muddle through" is a pain to orchestrate.] * We store `Current_app` in settings. But we don't really intend to persist a `Current_app` of 'error'. Only the main app or 'source' editor. [Another vote against storing 'error' in `Current_app`.] * So we need to rerun the version check after running tests to actually show the warning. [Perhaps I need to separate out the side-effect of setting `Version` from the side-effect of changing `Current_app`. But that's not right either, because I do still want to raise an error message if the version check fails before running tests. Which brings us back to wanting to run the tests after raising the version check..] One good thing: none of the bugs so far have been about silently ignoring test failures. I thought that might be the case for a bit, which was unnerving. I grew similar muddiness in Mu's bootstrap system over time, with several surrounding modes around the core program that interacted poorly or at least unsatisfyingly with each other. On one level it just feels like this outer layer reflects muddy constraints in the real world. But perhaps there's some skill I still need to learn here.. Why am I even displaying this error if we're going to try to muddle through anyway? In (vain) hopes that someone will send me that information. It's not terribly actionable even to me. But it's really intended for when making changes. If a test fails then, you want to know. The code would be cleaner if I just threw an unrecoverable error from the version check. Historically, the way I arrived at this solution was: * I used the default love.errorhandler for a while * I added xpcall and error recovery, but now I have situations where I would rather fall back on love.errorhandler. How to tell xpcall that? But no, this whole line of thought is wrong. LÖVE has a precedent for trying to muddle through on an unexpected version. And spurious test failures don't merit a hard crash. There's some irreducible requirement here. No point making the code simplistic when the world is complex. Perhaps I should stop caching Version and just recompute it each time. It's only used once so far, hardly seems worth the global. We have two bits of irreducible complexity here: * If tests fail it might be a real failure, or it might not. * Even if it's an unexpected version, everything might be fine. And the major remaining problem happens at the intersection of these two bits. What if we get an unexpected version with some difference that causes tests to fail? But this is a hypothetical and not worth thinking about since I'll update the app fairly quickly in response to new versions.
2023-12-07 01:34:04 +00:00
- run with a LÖVE v12 release candidate. No errors; it is a supported version. All tests pass.
- create a couple of spuriously failing tests. Run with an untested version of LÖVE. Error message includes message about untested version.
2023-12-03 20:11:17 +00:00
Code loading:
* run love with directory; text editor runs
* run love with zip file; text editor runs
2022-07-20 15:53:31 +00:00
* How the screen looks. Our tests use a level of indirection to check text and
graphics printed to screen, but not the precise pixels they translate to.
- where exactly the cursor is drawn to highlight a given character
- analogously, how a shape precisely looks as you draw it
* start out running the text editor, press ctrl+e to edit source, make a change to the source, press ctrl+e twice to return to the source editor; the change should be preserved.
* run with an untested version of LÖVE. Error message pops up. Press a key. Text editor comes up, and doesn't receive the key. Press ctrl+e. Error pops up. Press a key. Source editor opens up. Press ctrl+e. Error pops up. Press a key. Text editor returns.
2022-07-20 15:53:31 +00:00
### Other compromises
Lua is dynamically typed. Tests can't patch over lack of type-checking.
* All strings are UTF-8. Bytes within them are not characters. I try to label
2022-06-20 15:24:56 +00:00
byte offsets with the suffix `_offset`, and character positions as `_pos`.
For example, `string.sub` should never use a `_pos`, only an `_offset`.
* Some ADT/interface support would be helpful in keeping per-line state in
2023-10-28 06:57:18 +00:00
sync. Any change to line data should clear the derived line property
`screen_line_starting_pos`.
* Some inputs get processed in love.textinput and some in love.keypressed.
Several bugs have arisen due to destructive interference between the two for
some key chord. I wish I could guarantee that the two sets are disjoint. But
perhaps I'm not thinking about this right.
* Like any high-level language, it's easy to accidentally alias two non-scalar
variables. I wish there was a way to require copy when assigning.
2023-03-29 05:00:43 +00:00
* I wish I could require pixel coordinates to be integers. The editor
defensively converts input margins to integers.
* My test harness automatically runs `test_*` methods -- but only at the
top-level. I wish there was a way to raise warnings if someone defines such
a function inside a dict somewhere.