NVD slowdown - 2,546 (42%) CVEs were not processed by the NVD (since February 12)

[Nath31570]

Is your feature request related to a problem?
Yes. Website notice from the NVD : "NIST is currently working to establish a consortium to address challenges in the NVD program and develop improved tools and methods. You will temporarily see delays in analysis efforts during this transition. We apologize for the inconvenience and ask for your patience as we work to improve the NVD program."

Do you have a solution in mind, or a suggestion to improve OpenCVE?
Considering that the primary role of the NVD is to add data to vulnerabilities that have been assigned a CVE ID. They include additional metadata such as severity levels via Common Vulnerability Scoring System (CVSS) and affected data via Common Platform Enumeration (CPE). Today OpenCVE is based on the data provided by the NVD, how do you envisage the uncertain future and this sudden slowdown in the analyzes carried out by the NVD ?

(Even if MITRE support is added in V2, this will not solve the problem of CPE / CVSS data linked to each CVE.)

Additional comment

[ncrocfer]

Hi @Nath31570 ,

This is a very interesting discussion ! As you said the NVD analysis is currently impacted by their process transition.

Concerning the CVSS problem the V2 solves it as it also integrates the CVSS scores given by the MITRE. So even if the NVD doesn't provide a score, OpenCVE will use the one given in the MITRE data:

Concerning the CPE unfortunately OpenCVE relies on this data to extract vendors and products. The CPEs is a well-known standard used by the cyber community, we difficulty imagine using another configuration dictionnary. The NVD worked hard in the past years to provide a professional platform, we think they're telling the truth about the temporary status of this analysis delay.

In case of it's not temporary (which will be a very bad news for all the tools based on CPE ^^), what do you recommend to replace it? Parsing vendors / products from the description of the CVE ? What about the product versions?

Another question, what is the tool you use for your graph? It seems very cool !

[Nath31570]

Hi @ncrocfer,

Regarding the CVSS score, I didn't know that MITRE could provide it too, that's a good thing ^^

Concerning CPE, indeed in the event that it was not temporary, it would be very bad news. The CVE descriptions are not really explicit enough in the majority of cases for it to be possible to go through them in order to extract information on the impacted sellers/products. Perhaps parsing the vendors' sites could be a solution but it would seem a bit technically complex to me.

(Graph comes from: https://anchore.com/blog/national-vulnerability-database-opaque-changes-and-unanswered-questions/)

[ncrocfer]

This is a very interesting article, thanks to share it.

[sudesh0sudesh]

Can we try extracting the vendor name based on the summary ?

[ncrocfer]

IMHO it's not really accurate. Let's take this CVE as an example: https://www.opencve.io/cve/CVE-2012-2975

The product extracted from CPE is application_security_manager_appliance , and the summary is:

Cross-site scripting (XSS) vulnerability in the traffic overview page on the F5 ASM appliance 10.0.0 through 11.2.0 HF2 allows remote attackers to inject arbitrary web script or HTML via crafted requests that are later listed on a summary page.

I will not be able to find a correct name for this one. And this is just an example.

Even if some vendor and product has simple names (outlook, firefox, wordpress...), how to find the good keyword in a complete sentence?

he Shopkeeper Extender plugin for WordPress is vulnerable to Stored Cross-Site Scripting via the plugin's 'image_slide' shortcode in all versions up to, and including, 3.5 due to insufficient input sanitization and output escaping on user supplied attributes. This makes it possible for authenticated attackers, with contributor-level access and above, to inject arbitrary web scripts in pages that will execute whenever a user accesses an injected page.

Using a regex? Or some AI? Too random, we need to have a standard for that. And the most known standard is the CPE one.

[sudesh0sudesh]

The use of AI is working fine; when an official change occurs it can be pushed to CVE and replace the existing cpe data with a new ones. GPT3.5 is pretty much accurate; I have seen it missing a couple of times but it is working great most of the time

[sudesh0sudesh]

I have forked and integrated it with AI in my GitHub

[ncrocfer]

As I said I prefer to continue working on well-known standards like the CPE one or maybe SBOM tools (SPDX or CyclonDX maybe).

OpenCVE v2 will not introduce AI code used to "discover" the impacted vendors and products.

[Viajaz]

It should be noted that NIST NVD stopped publishing any CVEs (with or without analysis) for about a week because they have not completed their work to support the Version 5.1 of the CVE Record Format.

https://web.archive.org/web/20240514040612/https://services.nvd.nist.gov/rest/json/cves/2.0/?lastModStartDate=2024-05-08T13:15:08-04:00&lastModEndDate=2024-05-14T00:00:00-04:00&pubStartDate=2024-05-08T13:15:08-04:00&pubEndDate=2024-05-14T00:00:00-04:00

[ncrocfer]

@Viajaz This is really sad as if no CPEs are associated to a CVE, we couldn't link it to vendors & product.

The MITRE recently announced they will check to support the CPEs, hopping it will be soon !

[Viajaz]

NIST NVD has finally published the backlog of CVEs but I think this event is indicative of systemic issues within NIST NVD but I mention it here to highlight the increasingly unreliability of NIST NVD.

[ncrocfer]

@Viajaz what would you recommend to solve this problem?

[Viajaz]

@ncrocfer
Diversification of available authoritative sources for CVEs to reduce the overreliance on NIST NVD is needed; unfortunately, there is a lack of such sources. I know of several organizations that use VulnCheck NVD++, but their source is still NIST NVD (they also had a gap in published CVEs during the same period). The CVE Program does publish the official CVE List at https://github.com/CVEProject/cvelistV5 (Example CVE 5.1 JSON doc).

As for CPEs, the two primary issues I've identified are the lack of timely analysis (by NIST NVD and other participating organizations) and poor quality CPEs (specifically the vendor and product fields not being consistent), which makes them very unhelpful in completely automated scenarios. We aim to do a preliminary applicability analysis on every published CVE by the next business day, same day if possible.

It seems that I, and others, have basically given up on CPEs altogether and are using machine-learning-based solutions (such as LLMs) to identify affected products based on CVE descriptions and other available metadata. However, even that is limited when CNAs publish CVEs with little specific detail inside the actual CVE itself (e.g., CVE-2024-23354). In practice, I find we're writing custom parsers for each manufacturer's/vendor's security advisory feeds, but this doesn't scale, and not every vendor has a machine-readable feed or even a feed at all.

I believe OpenCve's philosophy is only to reflect its authoritative source (NIST NVD) and provide the best self-service experience for the data it has. However, due to systemic failures of the NIST NVD and the CVE Program itself, this doesn't seem sufficient in practice when we often only have a few days (and sometimes only hours) to respond and remediate CVEs.

Assuming one had the resources, the development of additional OpenCve features to allow security analysts to submit their own analysis (CVSS, CWEs, CPEs, etc.) to an OpenCve instance would be useful. This could include UI elements and additional API endpoints to support automated workflows. We've even had the need, in the past, to communicate the existence of yet-to-be-published CVEs from vendors to internal stakeholders that are not yet made public by a respective CNA. As a result, we have had to bypass automated or self-service workflows, such as what OpenCve provides, in order to push information out.

Hopefully this gives you a better idea of how we're interacting with CVEs and working around some of the issues.