Duplicates?

[ThePowerTool]

Hi,

I see feeds where Title and "Created on" fields match. As you likely surmised, I'm no expert.

Can you please give me a few clues on how to troubleshoot or
is there an option to have feed items where the 2 fields match only display a single line?

Thank you very much for your time and assistance!

[martinrotter]

Hi.

What do you mean "match"? You mean you have two articles which have same values in those 2 fields? For example you have 2 articles with title "Title" and created on "2022-06-24 08:00:00"?

Are those two instances of same article or are those in fact two different articles? Can you perhaps share feed containing those articles?

You can eliminate these duplicates with "Article filtering" feature.

[ThePowerTool]

Hi,

Yes, as best I can tell the 2 fields match exactly. Caveats include 1) I don't see the "seconds" display on time; 2) I don't know if there's some way to compare entries to see if they are a repeat (or what type of repeat--meaning legitimate 2nd issue of the same data pushed by a user vs an automatically generated duplicate); 3) something else I don't know to look for given my inexperience.

Is there a way I can privately send you feed details? I don't see any built-in option, here, to pm.

[martinrotter]

Yes, there is discord page for rssguard where you can contact me, see readme for discord address.

[martinrotter]

And to add to this: either way this is easily fixable with custom article filter which will automatically allow only 1 of those duplicate articles and drop other ones, even for newly incoming articles. If you are interested I can write the filter for you.

[ThePowerTool]

Hi martinrotter! Yes, I am interested. Would this only filter to the display--meaning if I turn it off I still see everything? If yes, I would sincerely appreciate your assistance. Thank you very much.

[martinrotter]

"article filters" feature could be used in many ways, it can totally remove unwanted (in your case "duplicate") articles, mark them as read, mark them as starred etc.

If you want just to "hide" duplicate articles, it can be done, but you would have to decide how "hide" you want them, RSS Guard could for example move those to recycle bin, or assign special label to them so that you can organize articles etc.

Possibilities are literally endless.

[ThePowerTool]

My only concern would be if the fields were duplicates but the article was not. In this specific case I would want to retain the article.

If the entire article is a duplicate I am happy to have it deleted.

[martinrotter]

fields were duplicates but the article was not

You mean the situation where author/title/date is same but the actual contents of message are not? This can be easily checked too.

[ThePowerTool]

Yes! That's the only concern I had regarding deleting articles that I hadn't had a chance to view.

[martinrotter]

Yes, for your exact use case (you want to keep the duplicate entries, and somehow differentiate among them), there is a bit tricky stuff.

Do you want the filtering logic to be applied only for new messages when they arrive or for all messages, even for already existing ones you have now?

[ThePowerTool]

I am OK with deleting duplicates where (as you described) the title, author, date, and content are duplicated.

All existing messages, please :)

Once it's done all existing messages a single time--could it be only applied to new messages? I'm just thinking the logic and saving CPU cycles. You're the expert and perhaps you already considered this.

[martinrotter]

All existing messages, please :)

That is the catch. I will think about that, and add relevant support into "article filtering" feature. It already does support manual execution of article filters on existing messages but there is bit lacking support for identifiyng article as "duplicate" for this particular use case.

Once it's done all existing messages a single time--could it be only applied to new messages?

Yes, this is defacto default behavior. Article filters are run automatically for new incoming messages but you have to run it manually for existing messages.

[martinrotter]

Give me some days as I am busy, will try to come with enhancements into the feature to allow for your use case.