Direct intergration with CBL-ReadingLists repo

[DieselTech]

I know this would be a big ask on your part, but I want to see what it would take for Komga to directly fetch the reading lists stored here on github at https://github.com/DieselTech/CBL-ReadingLists

I'm in the process of revamping the repo and attempting to clean up the lists. If there is a structure or set of guidelines you would like to see I'm all ears. My goal is to have the main branch be "very stable" and changes be vetted through a different branch. That way any integrations with apps won't break. I feel like this would bring value to a lot of people by allowing them to read issues in the order the series is intended to be. Also, once it's setup as a fetch type system in Komga there wouldn't need to be any commits or changes on Komga's end to update and maintain lists.

If there is requirements on the CBL repo for this to happen I'm happy to work through. Anything from the structure of the folders / files to the contents of the .cbl files itself.

[gotson]

Before talking about any technical integration, you would need to define the use cases and what you would expect Komga to offer.

What would a user interaction like? Is the import a one time thing, or do you have some other ideas in mind?

[DieselTech]

I agree the current .cbl xml situation isn't ideal. We are just keeping a dead format on life support at this point mainly because it's the only thing that is accepted across the different reading programs right now.

This might be a narrow opinion, but between all the software out there I think the only ones that have a foothold to do something like this are the server types of software, like Komga, Kavita, Codex and Stump. Standalone desktop software like comicrack and YAC aren't really setup for this type of thing.

Within the reading list creation group we've talked a few times about trying to come up with a new format that would allow for more options. We already have some custom scripts made for converting the lists to different formats. Given a spec we could get the old lists converted over relatively quickly.

[gotson]

Going forward it's probably best to maintain the target format, and generate the legacy cbl automatically, for example using github actions.

[gotson]

On top of my previous remarks, I think one of the main issue is the lack of book identifier. I don't believe cbl import in Komga (or others) currently works 100% for most users.

That means the automatic sync would not be automatic, as there would be some matching needed.

If a common identifier existed, it would need to be both in the reading order centralized data (but cbl does not support this, so better move away from this format), and in Komga (which currently only has isbn as a potential candidate).

[DieselTech]

I think one of the main issue is the lack of book identifier

I agree, but don't see a straight forward way around that for the time being. Books don't come with metadata in them, so being able to associate an ID to a random arbitrary file is something I think is out of scope for what we want to do. Even if we list the CV / Metron book ID in the reading list, there is no guarantee the users file on disk has that ID in it to properly match. For now it just comes down to them having a folder structure that is sane and accepted by the community if they want to use these lists.

[themadman0980]

One potential issue with integrating the repo into Komga is that there would almost certainly need to be some sort of folder structure established (matching our repo structure). From what I understand, Komga doesn't have this sort of structuring built in, so I imagine it would take a bit of work. The other alternative would be some sort of tagging & filtering system for reading lists, however we don't currently have a way to include tags with lists (this could easily be fixed with new target format I imagine).

The repo we're building would introduce several hundred reading lists, so some sort of organisation will be a necessity.

[themadman0980]

RE: Versioning & Unique ID's
Would unique ID's and versioning be essential? Could some file properties such as file hash be used to establish whether the local list is up to date? If this is possible, you could delete any local lists that aren't located on the remote source and create new lists for any remote lists not found locally.

RE: New format
Totally agree that a new format is needed. Unfortunately I can't see a replacement for CBL happening any time in the immediate future, so we're currently building everything in CBL with the intention to migrate to whatever new format is established if/when that happens (shouldn't be too hard to script). Given the functionality to import CBL files currently exists in Komga & Kavita, I imagine integrating our current repo using CBL would be able to use the same basic code, and you could simply replace the CBL import with a new import option once a new format is established? Totally understand if you'd prefer to wait until new format is finalised, I just think it might be a long wait.

[gotson]

Would unique ID's and versioning be essential? Could some file properties such as file hash be used to establish whether the local list is up to date? If this is possible, you could delete any local lists that aren't located on the remote source and create new lists for any remote lists not found locally.

Unique ID would be required, it could be in the form of a URL, but in that case moving the repo would not be easily possible.

File hash implies files, but if retrieving data over the wire, files are not necessarily a thing. File hash also implies exact file, meaning any reformating would change the hash, even if the content did not change.

Unfortunately I can't see a replacement for CBL happening any time in the immediate future

Why is that ?

[barrelltitor]

Would be nice to be able to add github repositories with CBL files and have a simple file browser to see which read list you want to add.

The read list itself could store a URL to the file on git and the timestamp it was last updated at. If the file's timestamp in the repo is newer, then an update is available and an automatic update could be attempted. If there are any matching issues, this could be added to the media management section with an option to click on it to manually fix the matches

I have my own opinionated read lists, so this would allow me to use mine and manage them via github, which would be awesome!

This does not solve the comic uniqueness issue, but I believe comic matching and the CBL format are two entirely separate issues. Komga already supports CBL, so I don't think that needs to change for this particular feature

When a new standard appears that makes sense, support for it alongside CBL could be added, and like people said above the existing CBLs could easily be translated to the new format.