Reduce image size and increase build frequency to cut high/medium vulnerabilities in tritonserver docker images

[mirekphd]

Is your feature request related to a problem? Please describe.
Container vulnerability scans block tritonserver deployment in our corporate servers because the containers have several high vulnerabilities:

Listing most important vulnerabilities in mirekphd-ml-infer:20230219-clair-cve-scan-20230219-163907.txt:

Unapproved vulnerabilities:
| �[1;31mUnapproved�[0m | High CVE-2022-29500       | slurm-llnl   | 19.05.5-1                | SchedMD Slurm 21.08.x through 20.11.x has Incorrect            |
| �[1;31mUnapproved�[0m | High CVE-2023-0286        | openssl      | 1.1.1f-1ubuntu2.16       | There is a type confusion vulnerability relating to X.400      |

Defcon1 vulnerabilities:

Critical vulnerabilities:

High vulnerabilities:
| �[1;31mUnapproved�[0m | High CVE-2022-29500       | slurm-llnl   | 19.05.5-1                | SchedMD Slurm 21.08.x through 20.11.x has Incorrect            |
| �[1;31mUnapproved�[0m | High CVE-2023-0286        | openssl      | 1.1.1f-1ubuntu2.16       | There is a type confusion vulnerability relating to X.400      |

Medium vulnerabilities:
| �[1;36mApproved�[0m   | Medium CVE-2022-4450      | openssl      | 1.1.1f-1ubuntu2.16       | The function PEM_read_bio_ex() reads a PEM file from a BIO     |
| �[1;36mApproved�[0m   | Medium CVE-2020-27745     | slurm-llnl   | 19.05.5-1                | Slurm before 19.05.8 and 20.x before 20.02.6 has               |
| �[1;36mApproved�[0m   | Medium CVE-2020-12693     | slurm-llnl   | 19.05.5-1                | Slurm 19.05.x before 19.05.7 and 20.02.x before                |
| �[1;36mApproved�[0m   | Medium CVE-2022-0261      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2020-13844     | gcc-9        | 9.4.0-1ubuntu1~20.04.1   | Arm Armv8-A core implementations utilizing speculative         |
| �[1;36mApproved�[0m   | Medium CVE-2020-27746     | slurm-llnl   | 19.05.5-1                | Slurm before 19.05.8 and 20.x before 20.02.6 exposes           |
| �[1;36mApproved�[0m   | Medium CVE-2021-4192      | vim          | 2:8.1.2269-1ubuntu5.11   | vim is vulnerable to Use After Free                            |
| �[1;36mApproved�[0m   | Medium CVE-2023-22490     | git          | 1:2.25.1-1ubuntu3.8      | Git is a revision control system. Using a specially-crafted    |
| �[1;36mApproved�[0m   | Medium CVE-2022-2980      | vim          | 2:8.1.2269-1ubuntu5.11   | NULL Pointer Dereference in GitHub                             |
| �[1;36mApproved�[0m   | Medium CVE-2022-0368      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Read in GitHub                                   |
| �[1;36mApproved�[0m   | Medium CVE-2023-23946     | git          | 1:2.25.1-1ubuntu3.8      | Git, a revision control system, is vulnerable to path          |
| �[1;36mApproved�[0m   | Medium CVE-2023-0054      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Write in GitHub                                  |
| �[1;36mApproved�[0m   | Medium CVE-2022-0361      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2021-4166      | vim          | 2:8.1.2269-1ubuntu5.11   | vim is vulnerable to Out-of-bounds Read                        |
| �[1;36mApproved�[0m   | Medium CVE-2022-42898     | krb5         | 1.17-6ubuntu4.1          | PAC parsing in MIT Kerberos 5 (aka krb5) before 1.19.4         |
| �[1;36mApproved�[0m   | Medium CVE-2022-1927      | vim          | 2:8.1.2269-1ubuntu5.11   | Buffer Over-read in GitHub repository vim/vim prior to 8.2.    |
| �[1;36mApproved�[0m   | Medium CVE-2022-0351      | vim          | 2:8.1.2269-1ubuntu5.11   | Access of Memory Location Before Start of Buffer               |
| �[1;36mApproved�[0m   | Medium CVE-2022-1720      | vim          | 2:8.1.2269-1ubuntu5.11   | Buffer Over-read in function grab_file_name in                 |
| �[1;36mApproved�[0m   | Medium CVE-2022-0685      | vim          | 2:8.1.2269-1ubuntu5.11   | Use of Out-of-range Pointer Offset in GitHub                   |
| �[1;36mApproved�[0m   | Medium CVE-2022-0319      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Read in vim/vim prior to 8.2.                    |
| �[1;36mApproved�[0m   | Medium CVE-2023-0049      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Read in GitHub                                   |
| �[1;36mApproved�[0m   | Medium CVE-2022-2304      | vim          | 2:8.1.2269-1ubuntu5.11   | Stack-based Buffer Overflow in GitHub                          |
| �[1;36mApproved�[0m   | Medium CVE-2021-36222     | krb5         | 1.17-6ubuntu4.1          | ec_verify in kdc/kdc_preauth_ec.c in the Key                   |
| �[1;36mApproved�[0m   | Medium CVE-2022-2946      | vim          | 2:8.1.2269-1ubuntu5.11   | Use After Free in GitHub repository                            |
| �[1;36mApproved�[0m   | Medium CVE-2022-0572      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-0714      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2021-37750     | krb5         | 1.17-6ubuntu4.1          | The Key Distribution Center (KDC) in MIT Kerberos              |
| �[1;36mApproved�[0m   | Medium CVE-2022-0554      | vim          | 2:8.1.2269-1ubuntu5.11   | Use of Out-of-range Pointer Offset in                          |
| �[1;36mApproved�[0m   | Medium CVE-2022-0359      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-2345      | vim          | 2:8.1.2269-1ubuntu5.11   | Use After Free in GitHub repository                            |
| �[1;36mApproved�[0m   | Medium CVE-2022-1968      | vim          | 2:8.1.2269-1ubuntu5.11   | Use After Free in GitHub repository vim/vim prior to 8.2.      |
| �[1;36mApproved�[0m   | Medium CVE-2022-47024     | vim          | 2:8.1.2269-1ubuntu5.11   | A null pointer dereference issue was discovered in             |
| �[1;36mApproved�[0m   | Medium CVE-2022-2923      | vim          | 2:8.1.2269-1ubuntu5.11   | NULL Pointer Dereference in GitHub                             |
| �[1;36mApproved�[0m   | Medium CVE-2022-1674      | vim          | 2:8.1.2269-1ubuntu5.11   | NULL Pointer Dereference in function vim_regexec_string        |
| �[1;36mApproved�[0m   | Medium CVE-2022-1851      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Read in GitHub                                   |
| �[1;36mApproved�[0m   | Medium CVE-2023-0433      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-0413      | vim          | 2:8.1.2269-1ubuntu5.11   | Use After Free in GitHub repository vim/vim prior to 8.2.      |
| �[1;36mApproved�[0m   | Medium CVE-2022-2344      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2023-0288      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-0213      | vim          | 2:8.1.2269-1ubuntu5.11   | vim is vulnerable to Heap-based Buffer Overflow                |
| �[1;36mApproved�[0m   | Medium CVE-2021-31879     | wget         | 1.20.3-1ubuntu2          | GNU Wget through 1.21.1 does not omit the                      |
| �[1;36mApproved�[0m   | Medium CVE-2022-1942      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-0318      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in vim/vim prior to 8.2.            |
| �[1;36mApproved�[0m   | Medium CVE-2022-0629      | vim          | 2:8.1.2269-1ubuntu5.11   | Stack-based Buffer Overflow in GitHub                          |
| �[1;36mApproved�[0m   | Medium CVE-2022-3821      | systemd      | 245.4-4ubuntu3.19        | An off-by-one Error issue was discovered in Systemd            |
| �[1;36mApproved�[0m   | Medium CVE-2023-23916     | curl         | 7.68.0-1ubuntu2.15       | HTTP multi-header compression denial of service                |
| �[1;36mApproved�[0m   | Medium CVE-2023-0215      | openssl      | 1.1.1f-1ubuntu2.16       | The public API function BIO_new_NDEF is a helper function      |
| �[1;36mApproved�[0m   | Medium CVE-2022-45142     | heimdal      | 7.7.0+dfsg-1ubuntu1.3    | signature validation failure                                   |
| �[1;36mApproved�[0m   | Medium CVE-2022-0408      | vim          | 2:8.1.2269-1ubuntu5.11   | Stack-based Buffer Overflow in GitHub                          |
| �[1;36mApproved�[0m   | Medium CVE-2022-2571      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-2183      | vim          | 2:8.1.2269-1ubuntu5.11   | Out-of-bounds Read in GitHub                                   |
| �[1;36mApproved�[0m   | Medium CVE-2023-0361      | gnutls28     | 3.6.13-2ubuntu1.7        | timing sidechannel in RSA decryption                           |
| �[1;36mApproved�[0m   | Medium CVE-2022-2175      | vim          | 2:8.1.2269-1ubuntu5.11   | Buffer Over-read in GitHub repository vim/vim prior to 8.2.    |
| �[1;36mApproved�[0m   | Medium CVE-2022-1629      | vim          | 2:8.1.2269-1ubuntu5.11   | Buffer Over-read in function find_next_quote in                |
| �[1;36mApproved�[0m   | Medium CVE-2022-2343      | vim          | 2:8.1.2269-1ubuntu5.11   | Heap-based Buffer Overflow in GitHub                           |
| �[1;36mApproved�[0m   | Medium CVE-2021-31215     | slurm-llnl   | 19.05.5-1                | SchedMD Slurm before 20.02.7 and 20.03.x through               |
| �[1;36mApproved�[0m   | Medium CVE-2016-1585      | apparmor     | 2.13.3-7ubuntu5.1        | In all versions of AppArmor mount rules                        |
| �[1;36mApproved�[0m   | Medium CVE-2022-4304      | openssl      | 1.1.1f-1ubuntu2.16       | A timing based side channel exists in the OpenSSL RSA          |
| �[1;36mApproved�[0m   | Medium CVE-2020-13844     | gcc-defaults | 1.185.1ubuntu2           | Arm Armv8-A core implementations utilizing speculative         |

Describe the solution you'd like
Slim down the containers, as they are extremely bloated (16 GB), orders of magnitude bigger than our in-house model hosting apps. Some ideas:

• upgrade Ubuntu from Focal to Jammy
• get rid of unnecessary tooling (e.g. nano is safer/cleaner than vim)
• do not ship any models and other data inside
• introduce multi-stage builds
• produce also CPU-only versions, without the heavy CUDA libraries (if your corporate policy permits:)

Describe alternatives you've considered
I've tried to upgrade the packages installed in the container and did manage to remove the high vulnerability in openssl, but not in slurm* (the compilation from source (see docs) did not help remove / overwrite the original package and the vulnerability). So I may end up maintaining tritonserver containers on our own build servers, with daily and hourly builds.

Additional context
I could not make the smaller (but still very bloated) "min" and "sdk" versions to work. The main "py3" containers are as large as our data science and ML modeling development containers used by data scientists, with hundreds of installed python packages (including some CUDA-enabled ones):

$ docker images | grep triton
nvcr.io/nvidia/tritonserver                                     23.01-tf2-python-py3           fea617c492fd   3 weeks ago          10.2GB
nvcr.io/nvidia/tritonserver                                     23.01-py3-min                  631158f7d37d   3 weeks ago          8.27GB
nvcr.io/nvidia/tritonserver                                     23.01-py3                      dd4c2a4267e1   3 weeks ago          16GB
nvcr.io/nvidia/tritonserver                                     22.12-py3                      20d3e6634cd3   2 months ago         14GB
nvcr.io/nvidia/tritonserver                                     22.12-py3-sdk                  2d12690ae145   2 months ago         9.63GB

vs.

$ docker images | grep gpu-py39-cuda112-cust
mirekphd/ml-gpu-py39-cuda112-cust                               20230219                       c33bec4735b8   About an hour ago    16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               latest                         c33bec4735b8   About an hour ago    16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230219                       c33bec4735b8   About an hour ago    16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       latest                         c33bec4735b8   About an hour ago    16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230218                       765cdc3e13be   17 hours ago         16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230218                       765cdc3e13be   17 hours ago         16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230217                       4706c07a6065   41 hours ago         16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230217                       4706c07a6065   41 hours ago         16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230216                       bed2c6f3fba5   2 days ago           16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230216                       bed2c6f3fba5   2 days ago           16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230215                       20d20b613a29   3 days ago           16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230215                       20d20b613a29   3 days ago           16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230214                       ec0d47a3f5cc   4 days ago           16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230214                       ec0d47a3f5cc   4 days ago           16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230213                       99706a5e61f0   5 days ago           16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230213                       99706a5e61f0   5 days ago           16.2GB
mirekphd/ml-gpu-py39-cuda112-cust                               20230212                       57b1ac002db9   6 days ago           16.2GB
registry.hub.docker.com/mirekphd/ml-gpu-py39-cuda112-cust       20230212                       57b1ac002db9   6 days ago           16.2GB

[dyastremsky]

CC: @Christina-Young-NVIDIA

[dzier]

The 23.01 and 23.02 containers will be larger than normally unfortunately. There was a need to support both CUDA 12 and CUDA 11.x, so there are parts of the library CUDA 11.8 SDK in the container. Eventually, we will only need CUDA 12 and the container size will reduce.

The overall size of the container is already a huge concern for our team and many other Triton customers. The min container is currently the smallest we can make the container as it contains the bare minimum requirements and libraries needed for Triton to function. The bulk of the container size come from dependent libraries, such as CUDA SDK and cuDNN.

To your other request, it is not possible to release container more frequently than we do already.

[arunodhayan]

This new pull i update here has no vulnaberities and patched all
docker pull arunodhayan/triton-updated-24.01-py3:latest