@Relequestual Hello, I am currently working to prepare the consolidated new data source for the tooling page. The current data model stores tooling by categories of tooling type and sub categorises them by programming language (if applicable). I like the approach to store each tooling as a separate entity which matches this schema. This allows us to have more control over filters, sorting and other data transformations if ever required. I wanted to propose that we add some field that the tooling is language dependent (to sub categorise toolings such as validators for different languages as default view such as the current view) and language agnostic (for tooling such as editors and more). This way we can consistently maintain the file and the page.

  "required": [
    "name",
    "repositoryURL"
  ],

Regarding the required fields, name, description and toolingType should be mandatory. In addition repositoryURL should not be mandatory as some of the tooling is not opensource.

   "toolingType": {
       "description": "The category of tooling of the project",
       "type": "string",
       "enum": [
       ...
       ]
     }

I believe that the tooling type should be an array instead of a string as some toolings provide JSON Schema framework including validators, schema generators etc. Case: https://www.newtonsoft.com/jsonschema

  "required": [
    "name",
    "repositoryURL"
  ],

Regarding the required fields, name, description and toolingType should be mandatory. In addition repositoryURL should not be mandatory as some of the tooling is not opensource.
-@benjagm

The self reporting tooling will only be tooling which has an associated repository.

We can and should track non-open source tooling here also, but we can do that later.

As discussed, we should define a roadmap of changes we will make for tooling data, and then share it publicly.

I'm not sure that we should be leaving some of this information to the creators of the projects. I think such a system can be easily abused, and we would need to have checks for that.

Agreed. We can make it so the automation creates a PR for us to review, rather than blanket acceptance.

For example, the compliance field: can we really trust maintainers to admit that their libraries aren't 100% compliant by default? We already know of at least one that doesn't.

Not always, no. But, I count SIX implementations where we said that they document what must be done to be fully to the specification compliant. Some libraries are written in specific contexts where they don't want that by default, I guess.
If we include the field, some more may add such details if they exist.

I'm not sure that I agree that this is the right direction. I like the automation aspect, but I still think that the tooling list is something that needs to be curated by a trusted source (someone directly associated with our org).

Agreed. We can make it so the automation creates a PR. We need to do this anyway really, because someone could put ANYTHING there. If an implementation repo was compromised, they could add undesierable content.

I'm okay with this as a data mining effort, but not as a relay directly to our sites. This data should be curated before it's published in any form. - @gregsdennis

The readme states that the process will create a PR into this repo and not just mine the data. So it will be curated.

Data that's collected when it meets the above stated criteria will be used to create a Pull Request into the ecosystem repo to add or update the information.

Is this enough? Did you maybe miss this?

If we don't use this data for the website and landscape, we're loosing a LOT of the value from doing this work in the first place.

Yeah, posting that comment was more to make it public rather than something we just discussed in private. I'm aware that we'll have an opportunity to review a PR before anything goes in.

I was thinking data ingestion would be purely automatic. Then separately we'd have a curation step where the curated data is a new file (and included any overrides/changes we wanted).

Instead, you've chosen to have ingestion and curation together, but we also have overrides as a separate file. It's just a different way to do it. Also, I suppose the curation part would allow us to just decline to add something, but we'd need some way of tracking that we declined it or the bot will keep finding it.

You know, I can easily think of several reasons why we would actually want to have a seperate ingestion file vs curated file. So, let me amend the readme.

For example, it would be helpful to keep ingestion events logged in a file which can then be used to more easily make time series graphs.

...we'd need some way of tracking that we declined it or the bot will keep finding it.
Even with ingestion being automated and copying to the curated data being manual, it could be beneficial to know if an updated entry was rejected before, and why.

🙏 Huge thanks for all your feedback on this work!
I'm convinced this is going to be the start of elevating the ecosystem to the next level.