Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First import-data not working #330

Open
crypto-answer opened this issue Dec 20, 2023 · 1 comment
Open

First import-data not working #330

crypto-answer opened this issue Dec 20, 2023 · 1 comment

Comments

@crypto-answer
Copy link

Expected Behavior

When launching the first import-data i suppose to have about 250k CVE

Actual Behavior

This is what happens when i launch the import-data command:

root@xxxxxxxxx:/opt/opencve-docker# docker exec -it webserver opencve import-data
This command will import initial data in your database. Do you want to continue ? [y/N]: y
#####################
Importing CWE list...
#####################
[] Downloading https://cwe.mitre.org/data/xml/cwec_latest.xml.zip... (done in 5.006s).
[
] Parsing cwes... (done in 2.438s).
[] Creating mappings... (done in 0.022s).
[
] Inserting CWE... (done in 0.213s).
[] 1368 CWE imported.
[
] Downloading https://services.nvd.nist.gov/rest/json/cves/2.0?startIndex=0 (done in 2.791s).
[] Creating model objects (done in 0.604s).
[
] Waiting 6 seconds
[] Downloading https://services.nvd.nist.gov/rest/json/cves/2.0?startIndex=2000 (done in 3.601s).
[
] Creating model objects (done in 0.664s).
[] Waiting 6 seconds
[
] Downloading https://services.nvd.nist.gov/rest/json/cves/2.0?startIndex=4000 (done in 5.325s).
[] Creating model objects (done in 0.679s).
[
] Waiting 6 seconds
[] Downloading https://services.nvd.nist.gov/rest/json/cves/2.0?startIndex=6000 (done in 5.961s).
[
] Creating model objects (done in 0.836s).
[] Waiting 6 seconds
[
] Downloading https://services.nvd.nist.gov/rest/json/cves/2.0?startIndex=8000

it is freezed here and it's not moving forward. I tried to modify the script to import more (or less) than 2.000 CVE per time and insert in the DB but even if the startindex is moving on (it freezes at 98000), the CVE added to the DB are always not more than 80.000.

I have also tried to use the swap file (5G) but the issue remain the same.

Specifications

  • OpenCVE version: 1.5.1
  • Python version: 3.8
  • Platform: Ubuntu 22.04.3 LTS
@ncrocfer
Copy link
Member

How long did you wait during the freeze?
Sometimes the NVD API can take a long time to return the JSON reuslts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants