Combine tabular views into more general graph view

This commit is contained in:
contrapunctus 2022-04-10 18:54:26 +05:30
parent 1a7ebecb2e
commit 80f08aabdf
1 changed files with 64 additions and 55 deletions

119
TODO.org
View File

@ -8,10 +8,12 @@
* v0.3.0 [100%]
1. [X] +Release or+ bundle plist-pp.el
2. [X] -report - error opening when clocked in
* MELPA, hosting [0%]
1. [ ] merge =chronometrist-goal= into this repository
2. [X] move project to codeberg.org or tildegit.org
3. [X] ...and then, update the MELPA recipes.
* Chronometrist
1. implement point restore for =chronometrist=/=chronometrist-report=/=chronometrist-statistics= in a kill-buffer-hook, so it works whenever the buffer is killed, not just when we call the command.
@ -34,11 +36,13 @@
5. Delete =:stop= time and save
6. Clock out again
7. Accept the already-inserted key-value. Duplicate!
** Features [50%]
1. [ ] Hook enhancement - can we supply the whole plist (including tags and key-values) to the task-start hooks, so users can have even smarter hook functions?
* That would mean ensuring that -kv-read runs before other hooks.
* Actually, it should be trivial to access the last expression in the file, so maybe this is unnecessary.
2. [X] Revisit 'no reason' commands - maybe we should ask for tags and key-values with the regular commands, and skip them with the 'no reason' variants?
*** Key-values [66%]
1. [X] bug - value-history appears in chronological rather than reverse chronological order
2. [X] generate history hash table from chronometrist-file.
@ -70,6 +74,7 @@
15. [X] bug - tag history starts at the beginning (wtf?), value history is empty (wtf?)
* Does not occur on master, only on dev
* Does not occur if you disable lexical binding (introduced in 4e89836)
**** Values
What forms can they take?
1. Integers, floating point numbers - easy to identify via regexp
@ -109,6 +114,7 @@
** Maybe
1. Add a new kind of plist - =(:name "NAME" :time "TIME" ...)=
To record events for which the time interval is not relevant. These won't be shown in =chronometrist= - perhaps in a different buffer.
* Optimization
** DONE Don't update task list from data when user has set their own
:PROPERTIES:
@ -135,6 +141,7 @@ Defer (tag/key/value) history generation from file-change-time to prompt-time, a
+ Reduce memory use by allowing user to restrict number of s-expressions read.
+ Per-task history generation will create problems - e.g. values for a given key for one task won't be suggested for values for the same key in another 🤦
* Tags and keys are already task-sensitive; just don't make values task-sensitive.
** DONE Hash file contents to optimize for common changes
Compare partial hashes of file to know what has changed - only update memory when necessary.
@ -143,20 +150,24 @@ Don't store entire file into memory; instead, split midnight-spanning intervals
+ Will increase load time for each forward/backward command in =chronometrist-report= and =chronometrist-statistics=
+ Will reduce memory used by =chronometrist-events=.
* Further reductions can take place, if we automatically discard cache entries past a certain limit. (perhaps excluding data for the current day, week, or month)
** Split and save midnight-spanning intervals to disk
It will remove the need for an in-memory version of data with split midnight-spanning intervals.
+ Least memory use?
+ Might make the file harder for a user to edit.
** Save timestamps as UNIX epoch time
+ Will (probably) greatly speed up time parsing and interval splitting.
+ Will greatly impede human editing of the file, too. 🤔
* An editing UI could help - pretty sure every timestamp edit I've ever made has been for the last interval, or at most an interval in today's data.
- The editing UI could have commands for next/previous interval; one could also have a command which, in the file, opens the plist at point for editing.
** Use an SQL database instead of a text file
That is, assuming SQL can
1. find the difference between ISO-8601 timestamps
2. compare ISO-8601 timestamps, and
3. do 1 and 2 faster than Elisp.
** Change data structure
Instead of storing each plist as-is, split each into two, one with the =:start= and one with the =:end=. Now we have the elegance of the one-plist-is-a-complete-interval schema in the file, and the ease and speed of detection of midnight spanning intervals in memory.
@ -168,6 +179,7 @@ Instead of storing each plist as-is, split each into two, one with the =:start=
(:stop "<timestamp>")
...)
#+END_SRC
** Change file timestamp format to =("<iso-date>" "<iso-time>")=
** Change file schema
Pros
@ -225,6 +237,7 @@ Cons
+ Anything requiring split intervals will first look in =chronometrist-events=, and if not found, will read from the file and update =chronometrist-events=.
+ When the file changes, use the file byte length and hash strategy described below to know whether to keep the cache.
+ Save cache to a file, so that event splitting is avoided by reading from that.
** (old) ideas to make -refresh-file faster
1. Support multiple files, so we read and process lesser data when one of them changes.
2. Make file writing async
@ -236,12 +249,14 @@ Cons
* Or even the first expression of the current date. That way, we just re-read the intervals for today. Because chronometrist-events uses dates as keys, it's easy to work on the basis of dates.
6. [ ] Don't generate tag/keyword/value history from the entire log, just from the last N days (where N is user-customizable).
7. [ ] Just why are we reading the whole file? ~chronometrist~ should not read more than a day; ~chronometrist-report~ should not read more than a week at a time, and so on. Make a branch which works on this logic, see if it is faster.
* Certain
1. [ ] statistics UI for arbitrary queries
* user provides a predicate
* we show buffer with
+ matched unique tag groups, and sparklines for time spent on each
+ matched key-values, and sparklines for time spent on each
** plist-pp [66%]
1. [X] plist-pp - work recursively for plist/alist values
2. [ ] Fix alignment of alist dots
@ -458,6 +473,7 @@ Cons
- untouched project with target defined - red
- target ±5 minutes - green
- target*2 and above - red
* viewing/editing frontends
:PROPERTIES:
:CUSTOM_ID: viewing-editing-frontends
@ -477,8 +493,7 @@ A way to represent s-expressions as Markdown, Org, etc, so the entire plist can
* see [[info:elisp#Swapping Text][swapping text]]
*** A binding in the key-value buffer, which will insert the string at point in a buffer of a certain mode.
* documentation discoverability :doc:
* documentation discoverability :doc:
Ensure that the user manual is easily discoverable.
#+BEGIN_QUOTE
@ -497,7 +512,7 @@ Ensure that the user manual is easily discoverable.
2. provide =:link= argument to =defgroup=
3. open changelog on update
* macro for extensions :code:extension:
* macro for extensions :code:extension:
:PROPERTIES:
:CREATED: 2021-06-07T16:33:54+0530
:END:
@ -527,7 +542,7 @@ Current uses -
1. =chronometrist-goal=
2. =chronometrist-spark=
* macro for frontends :code:
* macro for frontends :code:
:PROPERTIES:
:CREATED: 2021-06-26T08:49:25+0530
:END:
@ -545,7 +560,7 @@ Macro creates -
2. function =frontend-rows=
3. custom variables =frontend-schema=, =frontend-row-transformers=, =frontend-schema-transformers=, =frontend-buffer-name=
* unified format-duration function :code:customization:
* unified format-duration function :code:customization:
:PROPERTIES:
:CREATED: 2021-06-08T11:17:54+0530
:END:
@ -584,7 +599,7 @@ Alternative syntax
+ to display only values, use ="%<code>"=
+ to display long units, use ="~[<separator>]<code>"=
* DONE error - =min= called with nil :spark:bug:
* DONE error - =min= called with nil :spark:bug:
:PROPERTIES:
:CREATED: 2021-06-11T03:44:17+0530
:END:
@ -640,7 +655,7 @@ Debugger entered--Lisp error: (wrong-number-of-arguments #<subr min> 0)
command-execute(file-notify-handle-event nil [(file-notify ((1 . 1) (modify) "chronometrist.sexp" 0) file-notify--callback-inotify)] t)
#+END_SRC
* STARTED discoverability and mouse-accessibility of commands [33%] :ux:
* STARTED discoverability and mouse-accessibility of commands [33%] :ux:
:PROPERTIES:
:CREATED: 2021-06-15T16:18:49+0530
:END:
@ -658,7 +673,7 @@ Strategies
* Perhaps I needn't worry too much. =menu-bar-mode= is enabled by default, and it makes #1 and #2 easy. I think a significant amount of the userbase disables =menu-bar-mode=, but they also have things like =counsel-M-x=, =describe-=.
+ The menu does not make the behavior of the numeric argument discoverable. Doesn't make sense to put it there, either.
* querying the file buffer and editing the results :feature:
* querying the file buffer and editing the results :feature:
:PROPERTIES:
:CREATED: 2021-06-16T07:50:21+0530
:END:
@ -667,7 +682,7 @@ Strategies
2. The result data may be something which corresponds to the input data, in which case we could jump to the corresponding plist.
3. The result data may be impossible to trace back to the input data (e.g. a sum of intervals from many plists), in which case we cannot provide direct editing.
* error in change type detection :core:bug:plist_backend:
* error in change type detection :core:bug:plist_backend:
:PROPERTIES:
:CREATED: 2021-06-16T18:40:18+0530
:END:
@ -678,7 +693,7 @@ Steps
Might have to do with there being 2 empty lines between the last-but-one plist and the new last plist.
* STARTED New frontends and enhancements :feature:
* STARTED New frontends and enhancements :feature:
** Existing frontends
*** chronometrist (overview for a day)
list of tasks, one day, durations and graphs
@ -687,10 +702,13 @@ list of tasks, one day, durations and graphs
2. [ ] set day (blank input to reset to today's date)
3. [ ] set duration format
4. [ ] display sparkline for total time
*** report
list of tasks, one week, durations only
*** statistics
list of tasks, one week/month/year [fn:1]
*** details (intervals for a day) [16%]
list of intervals, one day [fn:1]
+ [-] commands [50%]
@ -707,12 +725,14 @@ list of intervals, one day [fn:1]
Bugs
1. [ ] day-crossing events have the wrong duration and time. e.g. ~1 Exercise walking 30 minutes from 23:37 to 00:07~
** New frontends I want
*** task-key-values
list of unique key-values for a task, one day [fn:1], durations and graphs
+ commands [0%]
1. [ ] set task
2. [ ] set range
*** task-graph [0%]
list of days, weeks, or months [fn:1], one task (with optional key-value filter [fn:2]), horizontal graph (and durations/stats?)
+ columns
@ -736,7 +756,7 @@ between (<date>|<datetime>) and (<date>|<datetime>)|
for <predicate>)"
#+END_SRC
* STARTED customizable duration output :feature:
* STARTED customizable duration output :feature:
:PROPERTIES:
:branch: format-seconds
:END:
@ -744,13 +764,13 @@ for <predicate>)"
2. [ ] define customization type for =chronometrist-duration-formats= to create user-friendly Custom interface
3. [ ] make non-tabular parts of Chronometrist buffers adapt to column widths, to accomodate changes in duration formats
* Incorrect time displayed when midnight crossed with a task active :bug:
* Incorrect time displayed when midnight crossed with a task active :bug:
Midnight-spanning intervals are split in the hash table when the file is read, but not when a task is started and not stopped before midnight. Run a function to check at midnight?
* Duplication - accessing the latest-record :code:
* Duplication - accessing the latest-record :code:
Many functions use the latest record. A =(chronometrist-with-latest-record var &rest body)= might help...
* STARTED Plist-group backend [71%] :feature:optimization:
* STARTED Plist-group backend [71%] :feature:optimization:
See branch =grouped-plist-format=
1. [X] Migrate backend code to use CLOS
2. [-] Implement plist-group backend methods
@ -798,7 +818,7 @@ protocol implementation progress
19. [ ] chronometrist-verify
20. [ ] [superclass] chronometrist-timer
* New backends :feature:
* New backends :feature:
** Org time tracking
** timeclock
See docstring of =timeclock-log-data=
@ -807,6 +827,7 @@ See docstring of =timeclock-log-data=
:PROPERTIES:
:CUSTOM_ID: new-backends-klog
:END:
** SQLite [0%]
Chronometrist allows the user to add arbitrary key-values. There are a few ways to do this in SQL, but [[https://mariadb.com/kb/en/entity-attribute-value-implementation/][this solution]] seems well-suited to Chronometrist - put commonly-queried keys (name, start, stop) into SQL columns, and put user key-values in a JSON/s-exp TEXT column. The latter can be queried separately. SQLite has support for [[https://www.sqlite.org/json1.html][JSON columns]].
@ -820,18 +841,18 @@ Chronometrist allows the user to add arbitrary key-values. There are a few ways
https://github.com/projecthamster/hamster
+ https://github.com/projecthamster/hamster/wiki/Our-datamodel
* Functions doing similar things :code:
* Functions doing similar things :code:
1. =task-time-one-day=
2. =interval= (unused)
3. =events-to-durations=
* Use ISO date for functions operating on dates :time:format:
* STARTED customizable task list [33%] :feature:
* Use ISO date for functions operating on dates :time:format:
* STARTED customizable task list [33%] :feature:
1. [X] Make =chronometrist-task-list= customizable
2. [ ] Interactive, buffer-local modification of task list, with completion (=completing-read-multiple=)
3. [ ] Adding a task? We can modify the task list, but how to persist it?
* Extend time range prompt :feature:
* Extend time range prompt :feature:
Support inputs like "today", "yesterday", "5 days ago", etc.
* =date(1)= does something like this, right? Maybe we could shell out to it.
@ -840,7 +861,7 @@ A general library for this could convert between (both to and from) such strings
* N <seconds/minutes/hours/days/weeks/months/years> (ago|from now/today)
+ plus multiples of those units e.g. "1 year, 5 months, ... from now"
* DONE Kill/discard command :feature:
* DONE Kill/discard command :feature:
Command to delete the interval currently being recorded. (bind to 'k')
+ Most conservative option - it will only operate on the project at point, and will only kill for a clocked-in project.
+ Somewhat less conservative option - it will operate on the currently clocked-in project, no matter where point is.
@ -849,10 +870,9 @@ Command to delete the interval currently being recorded. (bind to 'k')
- Undo and redo seem like the best bets.
* pretty printer
** fix handling of tagged alist group values :bug:
** fix handling of tagged alist group values :bug:
** put each list element on its own line if length of list exceeds fill-column :feature:
* verify command [20%] :feature:
* verify command [20%] :feature:
:PROPERTIES:
:CUSTOM_ID: verify-command
:END:
@ -865,7 +885,7 @@ With different checks, for different levels of speed/thoroughness -
6. [ ] (plist group) Check that plist timestamps have the correct date. Only applicable [[dates-in-timestamps][as long as they have a date.]]
* format changes
** multiple intervals per record :feature:
** multiple intervals per record :feature:
:PROPERTIES:
:CUSTOM_ID: multiple-intervals-per-record
:END:
@ -883,7 +903,7 @@ It doesn't do anything not already possible in the current formats. At best, it
- It will probably complicate all data consuming code, though...think of =chronometrist-details= 🤔
- An alternative idea could be to define a custom variable to hold the user's key values. If this variable is defined, it would be used instead of generating suggestions from past key-values. That way, such situations will not affect key-value suggestions.
** "event records" - records with only a timestamp :feature:
** "event records" - records with only a timestamp :feature:
:PROPERTIES:
:CREATED: 2021-12-18T11:48:53+0530
:END:
@ -893,7 +913,7 @@ Records not used for time tracking, but to store data associated with a date or
[<keyword-value pair>] ...)
#+END_SRC
** STARTED tagging dates with key-values [0%] :feature:
** STARTED tagging dates with key-values [0%] :feature:
:PROPERTIES:
:CREATED: 2022-02-10T22:59:45+0530
:CUSTOM_ID: date-key-values
@ -922,7 +942,7 @@ The plist backend could theoretically be extended to support it, but I'd rather
2. [ ] Update code accessing the hash table
3. [ ] Update =plist-pp= to handle
* undesired file watcher created after literate-elisp-load :bug:
* undesired file watcher created after literate-elisp-load :bug:
:PROPERTIES:
:CREATED: 2021-12-18T15:13:35+0530
:END:
@ -951,7 +971,7 @@ The plist backend could theoretically be extended to support it, but I'd rather
2. [ ] display message on start and completion
3. [ ] test the INPUT-FILE argument
* STARTED Support for the Third Time System [57%] :extension:
* STARTED Support for the Third Time System [57%] :extension:
:PROPERTIES:
:CREATED: 2022-02-10T14:12:12+0530
:branch: third-time
@ -1000,7 +1020,7 @@ Additional rules:
2. edit stop time to add +20 minutes of work (30m total)
* compare with old data in hash table - decrement break time added by old plist, increment break time added by new plist
* STARTED Customizable alerts [50%] :feature:code:
* STARTED Customizable alerts [50%] :feature:code:
:PROPERTIES:
:CREATED: 2022-02-14T08:22:36+0530
:CUSTOM_ID: customizable-alerts
@ -1036,7 +1056,7 @@ Benefits
...)
#+END_SRC
* no-goal-alert - "You have spent on <task>" [0%] :bug:
* no-goal-alert - "You have spent on <task>" [0%] :bug:
:PROPERTIES:
:CREATED: 2022-02-25T00:19:49+0530
:END:
@ -1065,6 +1085,7 @@ A task can be part of any number of categories. A category may also contain othe
:PROPERTIES:
:CREATED: 2022-04-03T09:36:53+0530
:END:
** DONE Replace =add-variable-watcher=
Common Lisp does not have Elisp's =add-variable-watcher=. The Elisp code used that to automatically update the =file= slot of the active backend object when the user changes the value of =chronometrist-file=. The new solution is to remove direct accesses to the =file= slot, define a =backend-file= method which returns the value of =file= if non-nil (it's usually nil), or derives a file path from =*user-data-file*= and the backend's extension.
@ -1078,22 +1099,18 @@ It seems that the =file= slot is now unnecessary - =backend-file= can check =pat
:END:
** CLIM UI design
:PROPERTIES:
:CREATED: 2022-04-08T18:19:58+0530
:END:
*** Task list + duration table
:PROPERTIES:
:CREATED: 2022-04-10T08:09:18+0530
:END:
1. task name
* Right-click on task to toggle inline display of either key-value breakdown, or interval breakdown.
2. task duration today
3. task activity indicator
* click to clock in/out (hover to show icon)
4. graph of daily task durations
* Click on a point in the graph to view that day in the Details View.
click on row to show taller graph
click on row to show taller graph and inline display of either key-value breakdown, or interval breakdown.
*** Categories table
:PROPERTIES:
@ -1114,32 +1131,24 @@ click on row to show taller graph
:PROPERTIES:
:CREATED: 2022-04-08T18:23:42+0530
:END:
Displays (day-grouped) tables of intervals and events in date range (default - today)
Displays tables of intervals and events, grouped by day, in date/week/month/year range (default - today)
can select interval to edit its data
*** Task-over-time table
:PROPERTIES:
:CREATED: 2022-04-10T08:14:15+0530
:END:
List of days, weeks, or months (can vary), one task (with optional key-value filter [fn:2]), horizontal graph (and durations/stats?)
+ columns
1. range (ISO date, week, or year-month)
2. per-range graph (vertical/interals if days, horizontal/days if weeks or months)
+ [ ] commands [0%]
1. [ ] toggle week/month
2. [ ] set [task/key-value] filter
3. [ ] set range
+ [ ] non-tabular text [fn:3]
buttons to add new intervals, events, and day properties
*** Task key-value table
list of unique key-values for a task, one day [fn:1], durations and graphs
+ commands [0%]
1. [ ] set task
2. [ ] set range
When there are no intervals/events/day properties, show placeholder text - "No intervals recorded on this day." / "No events recorded on this day." / "No properties for this day."
*** Graph views
:PROPERTIES:
:CREATED: [2022-04-10 Sun 14:35]
:END:
Graph of one or more tasks/task-property combinations/numeric property value/function with numeric output (adjustable) over time range (adjustable), with points for days/weeks/months/years (adjustable)
User can switch between single graph vs separate graphs, different types of graphs, etc.
*** Default view (overview for a day)
Default view is a composition of the task-list/duration table, day-grouped log, and the categories table.
Default view is a tiled composition of the task-list/duration table, day-grouped log, and the categories table.
*** commands
:PROPERTIES: