Optional support for automatically expiring old tasks

[msnspk]

As of version 3.0, Taskwarrior apparently removes deleted tasks "automatically after they expire". This feature is not documented anywhere, and the user is only made aware of it if they try to run the purge command (which has been deprecated and no longer works).

This new behavior appears to be silent, automatic, undocumented, and irreversible.

The meaning of the "deleted" status in Taskwarrior 2 was confusing, because deleted tasks would remain in the user's task list and data files unless the user explicitly used the purge command to remove the data permanently. However, while confusing, this behavior was the way Taskwarrior worked for years (decades?). As a result, due to deleted tasks being preserved (unless purged) and despite the weird semantics, I (and I suspect many other users) began "deleting" tasks as a way of marking them as cancelled (a semantic Taskwarrior has no command for).

Now, with Taskwarrior 3, all records of those tasks are apparently at risk of being automatically purged? That is not desirable behavior.

Taskwarrior should not automatically or silently purge user data unless explicitly configured to do so by the user.

After updating to Taskwarrior 3, I checked the information for my old deleted tasks and tasks that I created and deleted after the update. Nowhere in the information for those tasks did I find an Expiry field. The tasks have their usual Entered, End, Last modified fields, etc, but nothing telling me when the task will expire and be automatically purged.

1. When and under what conditions a deleted task will be considered "expired" and automatically purged needs to be presented to the user in that task's information.

2. The new behavior of automatic task purging needs to be documented properly and (at least on first run) explicitly enabled by the user.

3. Taskwarrior 3 should implement a "cancelled" status and respective commands, so that users can track not only the tasks they have completed, but the tasks they are not going to complete.

[djmitche]

Interesting that this is sort of the opposite of #3399!

Expiration support was added in GothenburgBitFactory/taskchampion#26. Basically, expiration happens 180 days after the last modification of a task with status:deleted. That was done when Taskchampion was still a separate application, before we decided to merge it with Taskwarrior. So technically it is documented here but that's not where you'd go looking for it!

In the context of building a new application, it seemed reasonable for "delete" to really mean "delete" (similar to how most email clients will delete mail from your trash after 30 days or something). We missed this decision when merging into Taskwarrior, and you're right -- it's not something we should change without warning!

Looking a little more deeply, we also missed a connection: Taskchampion only actually performs expiration when someone calls Replica::expire_tasks, and nothing in Taskwarrior calls that. So, tasks actually don't expire at all in current releases.

But, we should make this work as best we can.

Since Taskwarrior has a task purge, those who want to keep their task DB relatively slim can then opt-in, setting an expiration horizon that makes sense for their use-case.

This will need a little more design here from the Taskwarrior perspective, and then probably a bit more work on the Taskchampion API -- although GothenburgBitFactory/taskchampion#383 is a start.

[djmitche]

Oh, can you break the "cancelled" idea out into a new issue? I'd like that, too, but I want to think carefully about adding a new status -- the "waiting" status, for example, turned out to be problematic.

[huantrinh1802]

Would you mind sharing the reason why the "waiting" status is problematic?

[djmitche]

It was redundant with other information in the task. For example, what does a task with status:waiting but no wait property mean? Or a task with a wait property and status:completed? Such things can happen due to operations sync'd from multiple repositories. In the end, we switched to +WAITING meaning only that there was a wait property, and removed the superfluous status value.

[v0lker]

the current behaviour (silently deleting data) completely breaks my use case, where 'deleted' == abandoned/won't fix/too late.

i run some reports that take the deleted tasks into account and so does taskwarrior itself (e. g. history.monthly), did the change not trigger any of the tests???

e. g.:

Year Month     Added Completed Deleted Net
1970 January         1    11         0       0  11
2022 August         31     0         0       0   0
     October        15    11         0       0  11
     November        1     1         0       0   1
                     4     0         0       0   0
                    14     0         0       0   0
                    15     3         0       0   3
                    20     1         0       0   1
[...]

also, something seems to maybe delete time stamps (first line of the table)?

in case it wasn't obvious: making an unannounced and un-configurable change like that behind the user's back is not great. now i can choose between losing the data i added since migrating to 3.0.0 and losing part of the older data??

[djmitche]

It looks like task history gets the date information from the entry or end properties, and those should persist. I suppose by "deleted" you're referring to the 1970 dates, epoch=0? It's hard to say where that might have come from. Perhaps scan task export and look for rows with no entry or end properties? If you can find a reproduction for that, please open a new issue.

I don't know if you read the above comments, but Taskwarrior does not actually delete (expire) tasks.

[Andonome]

It'd certainly be good to return to the old style, where delete = delete (but remain tracked), while 'delete + purge' removes all record of the task.

[djmitche]

Making task purge work is #3399.