1. implement point restore for =chronometrist=/=chronometrist-report=/=chronometrist-statistics= in a kill-buffer-hook, so it works whenever the buffer is killed, not just when we call the command.
1. [ ] Hook enhancement - can we supply the whole plist (including tags and key-values) to the task-start hooks, so users can have even smarter hook functions?
2. [X] Revisit 'no reason' commands - maybe we should ask for tags and key-values with the regular commands, and skip them with the 'no reason' variants?
5. [ ] when reading values, add quit keybinding (consistent with output of ~chronometrist-kv-completion-quit-key~) by passing MAP to ~read-from-minibuffer~
6. [X] bug - missing values in history
7. [X] enh - remove key-values from suggestions which have already been added
8. [ ] enh - create custom variable to auto-insert key-values used in previous task of same :name in the key-value buffer.
9. [X] bug - incorrect indenting in -kv-buffer
10. [X] bug - I think -append-to-last-expr is eating key-values
* some way to require the user to enter a non-empty value (e.g. like the 'require-match' argument to completing-read, except read-from-minibuffer doesn't have that...)
2. refactor repetitive calls to (format "%04d-%02d-%02d" (elt seq a) (elt seq b) (elt seq c))
3. See if it is possible to store buttons in a variable, so *-print-non-tabular functions can be made shorter and less imperative. (see ~make-text-button~)
4. Merge all event-querying functions so that they always operate on an entire hash table (so no 'day' variants),
5. [ ] Use ~substitute-command-keys~ instead of ~chronometrist-format-keybinds~
6. [ ] recreate -events-clean, remove splitting code from -events-populate
- I don't know of a way to know the project being clocked into using timeclock hooks.
- With v0.2.0 Chronometrist also has a before-project-stop-functions, which runs before the project is stopped, and can control whether the project actually is stopped.
** DONE Deferred (tag/key/value) history generation
Defer (tag/key/value) history generation from file-change-time to prompt-time, and make it per-task instead of all tasks at once
+ The biggest resource hog is splitting of midnight-spanning intervals, however.
+ Reduce memory use by allowing user to restrict number of s-expressions read.
+ Per-task history generation will create problems - e.g. values for a given key for one task won't be suggested for values for the same key in another 🤦
* Tags and keys are already task-sensitive; just don't make values task-sensitive.
** DONE Hash file contents to optimize for common changes
Compare partial hashes of file to know what has changed - only update memory when necessary.
** In-memory cache
Don't store entire file into memory; instead, split midnight-spanning intervals just for the requested data.
+ Will increase load time for each forward/backward command in =chronometrist-report= and =chronometrist-statistics=
+ Will reduce memory used by =chronometrist-events=.
* Further reductions can take place, if we automatically discard cache entries past a certain limit. (perhaps excluding data for the current day, week, or month)
** Split and save midnight-spanning intervals to disk
It will remove the need for an in-memory version of data with split midnight-spanning intervals.
+ Least memory use?
+ Might make the file harder for a user to edit.
** Save timestamps as UNIX epoch time
+ Will (probably) greatly speed up time parsing and interval splitting.
+ Will greatly impede human editing of the file, too. 🤔
* An editing UI could help - pretty sure every timestamp edit I've ever made has been for the last interval, or at most an interval in today's data.
- The editing UI could have commands for next/previous interval; one could also have a command which, in the file, opens the plist at point for editing.
** Use an SQL database instead of a text file
That is, assuming SQL can
1. find the difference between ISO-8601 timestamps
2. compare ISO-8601 timestamps, and
3. do 1 and 2 faster than Elisp.
** Change data structure
Instead of storing each plist as-is, split each into two, one with the =:start= and one with the =:end=. Now we have the elegance of the one-plist-is-a-complete-interval schema in the file, and the ease and speed of detection of midnight spanning intervals in memory.
1. Easy to read data for a single date - Chronometrist's most common operation.
2. No more expensive operations on each startup (date checking, splitting of midnight spanning intervals).
3. May be easier to check for midnight-spanning intervals - plists are already grouped by date, just check the first and the last plist per date. Which means, if a user neglects to split a midnight-spanning interval, we can do it for them.
* Make this checking - and also checking for ordering of keys and plists - a command which the user can run at will, rather than something being run automatically.
4. May reduce memory and disk space used - hash table keys are already dates, so =:start= and =:stop= can just be ISO times rather than date-times.
Cons
1. Opting to remove dates from intervals might make it difficult for users to recover from erroneous edits.
1. Users will have to be disciplined while editing, and ensure that events are split on day boundaries. If not, we will have to check each time the file changes, defeating the entire optimization.
* Not quite - it's less work to test that, when the intervals are grouped by date.
- This may also be possible with [[*Store plists grouped by date][plists grouped by date]], if we serialize the hash table as plists rather than - as we do now - directly adding individual plists to the file.
Cons
1. Read syntax not portable to other dialects or languages, e.g. if one wants to make an Android frontend in LambdaNative (Gambit Scheme)
* A language with reader macros, such as Common Lisp, might work.
* Or, use [[*Store plists grouped by date][plists grouped by date]].
2. Slightly noisier syntax than plain plists.
3. Users cannot create comments within the hash table expression.
* Couldn't they use plist keys for comments instead? 🤔
4. May increase memory consumption - the entire hash table will be loaded into memory.
+ Fill =chronometrist-events= only as much as the buffer needing split intervals requires. e.g. for =chronometrist=, just a day; for =chronometrist-report=, a week; etc.
+ Anything requiring split intervals will first look in =chronometrist-events=, and if not found, will read from the file and update =chronometrist-events=.
5. What if commands both write to the file /and/ add to the hash table, so we don't have to re-read the file and re-populate the table for commands? The expensive reading+parsing could be avoided for commands, and only take place for the user changing the file.
* Rather than storing and hashing the full length, we could do it until (before) the last s-expression (or last N s-expressions?). That way, we know if the last expression (or last N expressions) have changed.
* Or even the first expression of the current date. That way, we just re-read the intervals for today. Because chronometrist-events uses dates as keys, it's easy to work on the basis of dates.
6. [ ] Don't generate tag/keyword/value history from the entire log, just from the last N days (where N is user-customizable).
7. [ ] Just why are we reading the whole file? ~chronometrist~ should not read more than a day; ~chronometrist-report~ should not read more than a week at a time, and so on. Make a branch which works on this logic, see if it is faster.
* While also handling alist members which are proper lists
3. [X] Add variable (to chronometrist-sexp.el) to set pretty-printing function. Default to ppp.el if found, fallback to internal Emacs pretty printer, and let users set their own pretty printing function.
1. [ ] With tags and key-value query functions in before-out-functions, clock in Task A -> clock in Task B -> prompted for tags and key values for Task A, add some -> they get added to Task B 😱
2. [ ] I clocked into a task -
#+BEGIN_SRC
(:name "Arrangement/new edition"
:tags (new edition)
:start "2020-08-17T00:33:24+0530")
#+END_SRC
I added some key values to it. What it should have looked like -
#+BEGIN_SRC
(:name "Arrangement/new edition"
:tags (new edition)
:composer "Schubert, Franz"
:song "Die schöne Müllerin"
:start "2020-08-17T00:33:24+0530"
:stop "2020-08-17T01:22:40+0530")
#+END_SRC
What it actually looked like -
#+BEGIN_SRC
(:name "Arrangement/new edition"
:tags (new edition)
:composer "Schubert, Franz"
:song "Die schöne Müllerin"
:start "2020-08-17T00:33:24+0530"
...)
#+END_SRC
And of course, that results in an error trying to process it.
1. [ ] Add =:stop= time when we call =chronometrist-kv-accept=, not when we quit the key-value prompt with a blank input.
* It might be nice to be able to quit =chronometrist-kv-add= with C-g instead, actually.
+ =C-g= stops execution of =chronometrist-run-functions-and-clock-in=/=chronometrist-run-functions-and-clock-out=, so they can't reach the calls for =chronometrist-in=/=chronometrist-out=.
We can make the clock-in/out happen in an =unwind-protect=, but that means clock-in/out /always/ takes place, e.g. even when a function asks if you'd really like to clock out (like the Magit commit prompt example does), and you respond with "no".
- What if we call =chronometrist-before-out-functions= with =run-hook-with-args= like all other hooks, so it runs all functions unconditionally and any function wishing to abort clocking in/out can use catch/throw?
=chronometrist-kv-add= could quit nonlocally when the user enters a blank input (or hits C-g? Maybe by using =unwind-protect=?), cancelling the clock in/out, and thereby letting =chronometrist-kv-accept= resume clock in/out. (It can determine whether to clock in or out using =chronometrist-current-task=)
2. [ ] Implement undo/redo by running undo-tree commands on chronometrist.sexp
* [ ] Possibly show what changes would be made, and prompt the user to confirm it.
* How will this work with the SQLite backend? Rollbacks?
* It might be easier to just have a 'remove last interval' (the operation I use undo for most often), so we don't reimplement an undo for SQLite.
3. [X] Enhanced tag/key-value prompt - before asking for tags/key-values, if the last occurence of task had tags/key-values, ask if they should be reused. y - yes, n - no (continue to usual prompts).
* "Explain" command - show intervals for task today
* "Day summary" - for users who use the "reason" feature to note the specifics of their actual work. Combine the reasons together to create a descriptive overview of the work done in the day.
* [ ] Add support for using key-values to point to a commit (commit hash + repo path?)
+ Need some way to extend the key-value prompt, so we can provide completion for commit hashes + commit messages...
* [ ] Add command to open the commit associated with the interval in Magit
* [ ] Make a user-customizable alist of project names and repo locations (local or remote), so shorter project names can be used instead of repo locations, saving space and reducing duplication.
8. [ ] key-values - create transformer for key-values, to be run before they are added to the file. This will allow users to do cool things like sorting the key-values.
9. [ ] *Convert* current interval - change the =:name= of the currently clocked-in interval. Tags and key-values may be re-queried. Clock-in hook functions will be run again with the new task as the argument.
10. [ ] *Rename* a project (updating all records)
11. [ ] *Delete* a project (erasing all records)
12. [ ] *Hide* a project (don't show it in any Chronometrist-* buffer, effectively deleting it non-destructively)
13. [ ] *Reset* current interval - update the =:start= time to the current time.
14. [X] Alternative query function for tags and key-values - a single query. Either with tags and key-values as a single plist, or something like the multi-field query-replace prompt.
15. [ ] Customizable field widths
16. [ ] Ask existing users if they'd like to have a prop line added to =chronometrist-file=
1. [ ] Remove duplication between =chronometrist-toggle-task= and =chronometrist-toggle-task-button=
2. [ ] Make functions more test-friendly. Quite a few can get away with returning values instead of writing to a file - this will make it easier to test them. Other functions can handle the file operations for them.
* Could make two "transformer lists" - =chronometrist-list-format-transformers= and =chronometrist-entry-transformers=.
The former would be called before =tabulated-list-format= is set. The latter would be called by =chronometrist-entries=, with each individual entry as an argument.
=chronometrist-goal= will simply added a function to each of those.
* Actually, are transformers really necessary? It could be done with a function inserted by =chronometrist-goal= into =chronometrist-mode-hook=. The function itself would become a little more complex, but it would remove the two transformer lists + the =call-transformers= dependency from the code. =chronometrist-mode-hook= is required either way, to set up =chronometrist-goal=.
+ Turns out, they are. We set =tabulated-list-entries= to a function. To modify the value, we must hook into that function in some way. =tabulated-list-format= could be modified in a regular hook, but it feels more consistent to make it a transformer too 🤔
* [X] BUG - if something was removed from the last expression (thereby decreasing the length of the file), =chronometrist-file-change-type= returns =t= instead of =:last=
* [X] BUG - args out of range error when last plist is removed
3. [ ] Don't suggest nil when asking for first project on first run
4. [ ] When starting a project with time of "-" (i.e. not worked on today until now), immediately set time to 0 instead of waiting for the first timer refresh
5. [ ] Mouse commands should work only on buttons.
6. [X] Button actions should accept prefix arguments and behave exactly like their keyboard counterparts.
7. [ ] mouse-3 should clock-out without asking for reason.
8. [ ] Some way to ask for the reason just before starting a project. Even when clocking out, the reason is asked /before/ clocking out, which adds time to the project.
9. [ ] Allow calling chronometrist-in/out from anywhere-within-Emacs (a la timeclock) as well as from the chronometrist buffer.
10. [ ] =chronometrist-timer= - if =chronometrist-file= is being edited (buffer exists and modified), don't refresh - this will (hopefully) prevent Emacs from going crazy with errors in trying to parse malformed data.
1. [ ] Interact with Chronometrist from phone, tablet, or smart watches. (Help needed, I'm a total strange to mobile development and don't own any wearables.)
1.+=:tag-prompt=, =:key-prompt= - values can be nil, t (the default), or a function. If nil, don't ask for tags/keys for this task. If t, ask for tags/keys for this task using =chronometrist-tags-add=/=chronometrist-key-add=. If it's a function, use that as the prompt.+
* Tags and key-values are optional extensions; we don't want Chronometrist to know about them.
+ Well, even with this style of configuration, Chronometrist doesn't necessarily have to...it could use the fields it knows about, ignoring the rest; the extensions could check for the fields they know about.
* Instead of setting the prompt function, set hooks (=chronometrist-before-in-functions=/=chronometrist-after-in-functions=/=chronometrist-before-out-functions=/=chronometrist-after-out-functions=) per-task. This is preferable, because if you define a custom prompt function, you probably also want to remove certain functions coming earlier in the hook, such as =chronometrist-skip-query-prompt=, for that task.
2.=:hide= - values can be nil (the default) or t - if t, hide this task from being displayed in =chronometrist=/=chronometrist-report=/=chronometrist-statistics= buffers. (effectively a non-destructive deletion of all intervals of the task)
1. Adding tasks without clocking into them (the list is stored in a separate file)
2. Not asking for tags and/or key-values for a particular task, or having a special behaviour for a task. (e.g. some tasks I use follow certain patterns, which I'd like to automate away)
2. [ ] Do basic checks on values of all customizable variables when they are changed by the user, and provide meaningful errors if they can't be used by the program.
3. [ ] Task-sensitive value suggestions - if you use the key =:key= for two different tasks, and don't want the values for =:key= in one task being suggested for =:key= in another...
* ...or a list of keys to exclude from task-sensitivity?
+ So =chronometrist-value-history= will have =("task" . "key")= as hash key and =("value" ...)= as hash value. Keys which are present in the 'blacklist' are stored the same way as now - ="key"= as hash key, =("value" ...)= as hash value.
* Can we figure it out automatically, without requiring configuration? 🤔
* Maybe suggest values for the current task first, and only after that for other tasks? Solves the problem of 'mixed up' value histories, removes the need for a switch to turn it off/have the user configure a blacklist of keys...
* Lack of task-sensitive value suggestions (#3) is an inconsistency, because tags and keys are already task-sensitive. From that perspective, tag-sensitive key and value suggestions are a whole new can of worms.
2. narrow to specific task - average time spent in $TIMEPERIOD, average days worked on in $TIMEPERIOD, current/longest/last streak, % of $TIMEPERIOD, % of active (tracked) time in $TIMEPERIOD, ...
3. general - most productive $TIMEPERIOD, GitHub-style work heatmap calendar, ...
4. press 1 for weekly stats, 2 for monthly, 3 for yearly
Some way to use markup (Markdown, Org, etc) for certain plist values.
** Implementation ideas
1. A list of keys whose values are to be edited in a user-specified major mode.
* Multiple windows - instead of a single key-value buffer, we'll have multiple buffers in multiple visible windows, somewhat like =ediff=. The =accept= command will use the data from all involved buffers.
+ The buffer and window will be created when a keyword associated with that mode is selected at the prompt.
* Alternatively, the whole plist goes into a single buffer of the markup's major mode - the markup bits as markup, the rest of the plist in a code block 🤷
* poly-mode to mix different modes
* see [[info:elisp#Swapping Text][swapping text]]
2. "Input frontends" - a way to represent s-expressions as Markdown, Org, etc, so the entire plist can be edited in that mode. As a side-effect, this will permit use of Markdown, Org, etc in keyword-values - e.g. to use markup in comments or notes.
3. A binding in the key-value buffer, which will insert the string at point in a buffer of a certain mode.
[2021-06-02 13:49:41] rnkn: contrapunctus: have a menu item that opens the org file \\
[2021-06-02 13:50:49] contrapunctus: rnkn: hm, okay, I'll look into that...any others, for those who always have menus disabled? \\
[2021-06-02 13:51:43] rgr: why are you shipping as an org and not an info? Not that I have any skin in the game. But just link it in your mode/function docstring maybe. \\
[2021-06-02 13:52:15] rnkn: not the docstring \\
[2021-06-02 13:52:24] rnkn: use the :link keyword for the defgroup \\
[2021-06-02 13:53:35] contrapunctus: rgr: Org [...] can do a ton more stuff than [the] info [viewer]. To the point that even HTML export (even with infojs) would be a disgrace to the file ;) \\
[2021-06-02 13:53:37] rnkn: although you will probably need a function link instead to find the org file \\
[2021-06-02 13:54:30] contrapunctus: Although I guess the manual.org does not really need those fancy features...info export could work for it. \\
+ =EXTENSION= custom group and =EXTENSION-cell-function= custom variable (with your provided function as the default value)
+ the appropriate =EXTENSION-row-transformer= and =EXTENSION-schema-transformer= functions; the former calls =EXTENSION-cell-function= and inserts the string into the specified position in each row
+ =EXTENSION-setup= and =EXTENSION-teardown= functions, which
- add/remove the generated =EXTENSION-row-transformer= and =EXTENSION-schema-transformer= functions to =chronometrist-row-transformers= and =chronometrist-schema-transformers=, and
- run the =:setup= and =:teardown= forms if supplied.
A macro to create Chronometrist frontends (based on =tabulated-list-mode=). If implemented, would shorten code for all four existing and two planned frontends.
Programmer specifies -
1. frontend command name (a symbol)
2. name (string, passed to =define-derived-mode=)
3. rows function
4. schema value
5. name of buffer created by command
Macro creates -
1.=frontend= interactive command, which behaves like a toggle
Currently we have at least three ways of displaying durations - ="HH:MM:SS"= , ="XhYm"= , and =X hour(s), Y minutes(s)"= . Make a single function similar to =format-time-string=, but for durations. =ts-human-format-duration= from =ts.el= is not nearly as flexible as I'd like. When completed, we can have a single custom variable accepting a format string, which can be used to customize display of durations for the entire application at once.
+ ="%Y"= , ="%O"= , ="%W"= , ="%D"= , ="%H"= , ="%M"= , ="%S"= - same as above, but with string units, e.g. ="hour(s)"= , ="minute(s)"= , and ="second(s)"=
- separator can be specified like this - ="%<SEP><CODE>"= , e.g. ="%-T"=; only entered if there is a value present to the left; if unspecified, it is a space
#f(compiled-function (&optional remember-pos update) "Populate the current Tabulated List mode buffer.\nThis sorts the `tabulated-list-entries' list if sorting is\nspecified by `tabulated-list-sort-key'. It then erases the\nbuffer and inserts the entries with `tabulated-list-printer'.\n\nOptional argument REMEMBER-POS, if non-nil, means to move point\nto the entry with the same ID element as the current line and\nrecenter window line accordingly.\n\nNon-nil UPDATE argument means to use an alternative printing\nmethod which is faster if most entries haven't changed since the\nlast print. The only difference in outcome is that tags will not\nbe removed from entries that haven't changed (see\n`tabulated-list-put-tag'). Don't use this immediately after\nchanging `tabulated-list-sort-key'." #<bytecode 0x1fdbee9d8f65>)(t nil)
ad-Advice-tabulated-list-print(#f(compiled-function (&optional remember-pos update) "Populate the current Tabulated List mode buffer.\nThis sorts the `tabulated-list-entries' list if sorting is\nspecified by `tabulated-list-sort-key'. It then erases the\nbuffer and inserts the entries with `tabulated-list-printer'.\n\nOptional argument REMEMBER-POS, if non-nil, means to move point\nto the entry with the same ID element as the current line and\nrecenter window line accordingly.\n\nNon-nil UPDATE argument means to use an alternative printing\nmethod which is faster if most entries haven't changed since the\nlast print. The only difference in outcome is that tags will not\nbe removed from entries that haven't changed (see\n`tabulated-list-put-tag'). Don't use this immediately after\nchanging `tabulated-list-sort-key'." #<bytecode 0x1fdbee9d8f65>) t nil)
apply(ad-Advice-tabulated-list-print #f(compiled-function (&optional remember-pos update) "Populate the current Tabulated List mode buffer.\nThis sorts the `tabulated-list-entries' list if sorting is\nspecified by `tabulated-list-sort-key'. It then erases the\nbuffer and inserts the entries with `tabulated-list-printer'.\n\nOptional argument REMEMBER-POS, if non-nil, means to move point\nto the entry with the same ID element as the current line and\nrecenter window line accordingly.\n\nNon-nil UPDATE argument means to use an alternative printing\nmethod which is faster if most entries haven't changed since the\nlast print. The only difference in outcome is that tags will not\nbe removed from entries that haven't changed (see\n`tabulated-list-put-tag'). Don't use this immediately after\nchanging `tabulated-list-sort-key'." #<bytecode 0x1fdbee9d8f65>) (t nil))
2. [ ] bind context menu to right click outside of buttons
3. [ ] bind toggle task with/without hooks to left/right-click on buttons
1. [ ] keybindings in tooltips
4.+print keybindings in columns (following the table width)+
* Perhaps I needn't worry too much. =menu-bar-mode= is enabled by default, and it makes #1 and #2 easy. I think a significant amount of the userbase disables =menu-bar-mode=, but they also have things like =counsel-M-x=, =describe-=.
1. The result data may just be plists, which could be displayed/edited directly from the search results ([[info:elisp#Invisible Text][invisible text]] or [[info:elisp#Selective Display][selective display]] + [[info:elisp#Narrowing][narrowing]]?)
2. The result data may be something which corresponds to the input data, in which case we could jump to the corresponding plist.
3. The result data may be impossible to trace back to the input data (e.g. a sum of intervals from many plists), in which case we cannot provide direct editing.
* Incorrect time displayed when midnight crossed with a task active :bug:
Midnight-spanning intervals are split in the hash table when the file is read, but not when a task is started and not stopped before midnight. Run a function to check at midnight?
13. [X] interactive backend-switching command with completion; reset backend state after switching to account for changes made to the file in the meantime
14. [ ] (maybe) handle existing (in-file) split tasks. Currently, if an interval is split, commands such as =chronometrist-key-values-unified-prompt= operate on the second split interval, but not the first. Instead, plist-group methods for =chronometrist-replace-last= and =chronometrist-remove-last= could check if the last interval is split, and operate on both of them. (=chronometrist-insert= already splits plists when inserting.)
* Don't modify =chronometrist-latest-record=; code using it generally only wants the latter half of a split plist.