Replies: 5 comments 4 replies
-
I just found hiSHtory also by default uses a cloud backend, which i am not stoked about actually. |
Beta Was this translation helpful? Give feedback.
-
testing atuin now and first impressions are superior to hiSHtory. easy to get into the inspector and it's displaying a lot of interesting data in an interesting way. very impressed. let's see how perf holds up after import once i figure out how. |
Beta Was this translation helpful? Give feedback.
-
conclusion: aawwwwhhh, I really like so many of atuin's little features. There are SO MANY nice touches.
The drawbacks leaving me unsatisfied: general performance is lacking for huge datasetsI have around 200k history entries accumulated across the two main machines I currently drive (m1 max macbook and 5950x workstation), since both computers chips were not released more than 3 years ago this means it may be safe to assume my terminal history output is something like 60k commands per year give or take, that means I need to budget for single digit million entries if i want to be able to instantly retrieve the earliest commands. The commands each are usually well under 1KB but sometimes I end up crafting stuff that belongs in a shell script, and iterating on them for between 3 and 50 trials. These are rare enough though that a 2KB average entry size would more than account for them. That would put an estimate of the working history corpus size to 10M*2KB which is 20GB, a conservative overestimate for certain. fzf can handle this but it won't be snappy. I'm pretty sure this will choke and kill sqlite though, i'm sure it will work and not crash, but impatient me wouldn't use that. I'll play with atuin more on my macbook for sure though. I think I can daily drive this especially if i dont load it down too much with old imported commands, and i still have to work out how to import them preserving proper metadata first anyway. I'm only testing with 200k entries improperly imported. additional metadataI would like the ability to specify more fields to store in the history to help get more context when browsing later. one of these is the aforementioned git hash or branch name if it is present at the time of history entry creation. Another piece of metadata that is important is the hostname. I have the same username on multiple different machines, so just recording the user/author name is insufficient. I don't know, but I think this is a missing feature in atuin. I think atuin is sufficiently better UX that even with this missing here and present in hiSHtory i will stick to atuin. path forward from here? (for me)I will plan to daily drive atuin but the need for a global history search hitting all of my records remains. And fzf off a static file must remain the core engine for it. At the 20GB file range, it may make sense to store the file in zstd and use parallel decompression (or even nvcomp for GPU accelerated decompression) to get faster than disk read speed out of it. ingestion should be the bottleneck at nvme speeds however. And there are likely other technologies I can deploy, but I can say fzf already scales well and has customization features making it suitable for whipping up tooling. I don't think any sql engine can be the answer. Might do some profiling though. There are challenges designing something that will keep all the weird stuff I'm doing sane. So this is just going to be a note to self kind of a thing then:
|
Beta Was this translation helpful? Give feedback.
-
I do need a way to clear the history though. I did find #1022 (comment) let me see if i can find the sqlite file and this should be quick |
Beta Was this translation helpful? Give feedback.
-
It's possible the database is just missing some indexes that would help speed it up. Once I setup a self hosted server I'll likely take a peek. |
Beta Was this translation helpful? Give feedback.
-
I have been trying to solve in my own half assed ways the distributed shell history problem for something like 12 years. I took a look today and found already a wide ecosystem, which is wonderful.
I'll probably try to test both of these but I do want to try to work out which one is the most suitable for me and stick with it since I know from experience I want to put my history content through as little thrash as possible.
I typically use fzf ctrl+r to search through history but this wont leverage the special history i make with a preexec based history recorder i made in my oh-my-zsh config nearly 10 years ago. I fall back on referencing that a lot. I even made a git based sync layer so all my history is replicated on my machines now, but it's a truly horrible system.
I'm curious if atuin has fzf search. I also think sqlite is an obvious choice and both offer that but it's not clear to me why atuin's server uses postgres if it also uses sqlite. this makes me want to try hiSHtory first. I'm posting here because this project is much more popular and visible.
Hoping to hear from folks who have tested both atuin and hiSHtory and can compare their capabilities and behavior. Another one I just found on HN is custom column, the example given is perfect, i do in fact record the git branch in my custom enhanced zsh history, and it is useful, and I will come up with more of those.
Thank you.
Beta Was this translation helpful? Give feedback.
All reactions