[SOLVED] Raw JSON (not JSON Feed) Parsing

[ImportTaste]

RSS is sadly becoming more rare while JSON-based API is becoming more common. I've looked around, and I haven't yet come across a feed reader that can just read raw JSON. While this is already possible through the use of post-process scripts, being able to do it within the app would be a big improvement.

jq might be a good inclusion (Creative Commons license) for parsing the retrieved JSON data. It also has a lot of documentation on creating filters to extract specific parts of JSON data that could be used to define individual articles and their contents. While this would be another form of post-process scripting, it wouldn't have to rely on an external script file.

[ImportTaste]

While browsing the rss subreddit, I saw someone mention there were some scripts that parse JSON located here. I missed this in the documentation because I was looking around for 'post-process' and not 'scrapers'.

I'll be revising this issue slightly in light of this.

[martinrotter]

Hello @ImportTaste.

1. This defines how the data for the feed are obtained.
2. This defines how the obtained data are transformed.

You can completely avoid 2 and have the script download data and parse it in the same time.

For example from the example scrapers, you can test with one which uses NVidia JSON API to get information about latest driver releases.

1. It downloads very raw specific JSON data from the API.
2. RSS Guard then automatically feeds the data via stdin to the "post-process" script.

This particular script is written in Python and it reads API JSON data with line json_data = json.loads(sys.stdin.read()) and then processes it and outputs transformed JSON FEED data which can subsequently be read by RSS Guard.

Thing is you don't have to use Python to write your scripts. You can use any scripting or compiled language but Python is just quite suitable for these tasks. You can literally write everything you want, you can call "jq" inside your script to do some JSON magic. So basically.

1. You enter URL of your JSON API service to the "URL" field in RSS Guard.
2. RSS Guard downloads the data.
3. You write some script in your preferred scripting language (python, bash, powershell, ...) which parses the specific data and outpus either JSON feed or ATOM/RSS feed.
4. RSS Guard then automatically grabs this output and processes it just like any other feed.

If you are really not sure how to start, maybe provide me URL of your JSON API you want to scrape.

[martinrotter]

Just discovered https://github.com/public-apis/public-apis and RSS Guard can process each and everyone of them with proper scripting.

[ImportTaste]

@martinrotter I'd like to thank you for the help, even if it's a few months after the fact. I've got a couple of questions.

• Does this support relative paths? My hope is I could do something like .\_Scripts\nvidia-driver-articles.py, which would go into the _Scripts folder within the same folder RSS Guard's binary is located.
• Is there a way for me to see what the return of a post-process script looks like? As in, that final line, print(json_feed), I'd like to view it in the console so I know what RSS Guard expects. Or, if RSS Guard has a way to view the raw return data, that would be even more ideal.

Nevermind that second question, it was simpler than I thought. I just had to run: curl -s "https://www.nvidia.com/bin/nvidiaGDC/servlet/article.json?locale=en_US&region=us&type=both&tag=drivers&offset=0" | python nvidia-driver-articles.py | jq
I do still wonder if there's a way to look at the raw output within RSS Guard, though.