[SOLVED] Question marks missing from feed article titles

[andrewpros]

Hi.

I have some issues with feed article titles but im not sure it is a bug or something on my side as it is quite noticeable, but i haven't found any discussion about it.

Titles do not have question marks, im using the latest 4.0.4, the issue was also in previous versions.

I checked the rss source, it is ok, the question marks are there with no special characters, funny thing is, only question marks are missing, exclamations are fine, changing font doesn't do anything, the feeds are rss 2.0 with utf-8.

They are missing in every feed so this is not something related to specific feed. I opened the sqlite db and there are also no marks saved in the titles.

[akinokonomi]

the feeds are rss 2.0 with utf-8.

I can see them just fine on my end, either RSS 2.0 or Atom.

so this is not something related to specific feed.

I wonder.

Here you can test an Atom feed that has some exclamation question marks right now: https://bbs.archlinux.org/extern.php?action=feed&fid=44&type=atom

And I have just tested a custom RSS 2.0 feed (edited existing one), and they are displayed there, either:

[martinrotter]

Provide url of such a feed and related screenshot and application log.

[andrewpros]

I will provide more data soon, for now i can confirm that the provided sample atom bbs.archlinux feed also does not display question marks, so this feed can be used for testing too.

At the time of writing i did see missing marks in titles
[Solved] How does pacman cache work? or [SOLVED] Blacklisting non-system-related packages (paccache)?

Im using non English region windows 10 21H1, i even changed the display lang to ENG, still no luck, but maybe its something more deep related to this.

[martinrotter]

I checked multiple feeds and cannot actually reproduce the issue, sadly. There are some "sanitizations" of titles made by rssguard upon fetching articles but non of those are connected to "?" characters.

Also, what "encoding" do you have set in feed's properties? Does the encoding seem correct?

[andrewpros]

Cant say anything for sure yet, but based on the current symptoms it seems it could be something with the compatibility of the app and windows regional/local settings, the titles are already saved without ? in the db, so either its happening before in the code or the db strips it for whatever reason or regional windows setting/font issues. Will try to check.

It would be helpful if someone could provide me a link to code where the titles are downloaded/"sanitized".

The provided arch linux feed is great for testing, i can add and remove the feed over an over to check things.

[andrewpros]

Ok, i would put an app log but it is useless, haven't had much time, but for whatever reason this line is the issue

https://github.com/martinrotter/rssguard/blob/4.0.4/src/librssguard/core/message.cpp#L81

It is also consistent with my finding in the db as it turns out the ? is missing, but it is replaced by space character and not trimmed, every title missing ? has space at the end in the db.

So, im not a qt nor c++ dev, i dont know why it is happening, maybe u have some ideas.

[martinrotter]

I see. So you compiled the application with line commented out?

[andrewpros]

Yes, i have a debug build, sadly, it was the only way to diagnose it for sure. But yeah, by the looks of it, it doesn't seem to have anything to do with "?" char judging by code and yet, here we are.

[martinrotter]

I see, that is just weird because the line you describe only replaces non-breaking spaces, not question marks.

[andrewpros]

Yeah i know, moreover if i test it on some other external regex tools with the same titles it is fine, they only thing that comes to my mind is some kind of incompatibility in qt and somehow '?' is mistakenly detected as non breaking space and replaced, maybe even a bug, but i dont know how it is possible.

What would be an alternative way in qt code to remove non-breaking spaces without using QRegularExpression?

[andrewpros]

Ok, i did a simple new empty project test and this is insane.

It treats the u202F as ? that is 3F on a string level, i have no idea why.

[martinrotter]

Thanks for tests, will try to repeat tomorrow and maybe will report bug to qt

[martinrotter]

@andrewpros Btw, does

QString::fromUtf8("[\xE2\x80\xAF]")

work?

[andrewpros]

Yes, this works too.

[martinrotter]

OK, I pushed the fix: 206686d

Can you please test once dev. build is compiled?

[andrewpros]

I can confirm it works, thx.