Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

elyra-pipeline validate CLI command fails for pipelines with parameters and default properties #3190

Closed
mamurak opened this issue Oct 13, 2023 · 1 comment · Fixed by #3195

Comments

@mamurak
Copy link

mamurak commented Oct 13, 2023

Describe the issue
Running elyra-pipeline validate and by extension elyra-pipeline export fails silently with pipelines that contain both pipeline parameters and default properties.

To Reproduce
Steps to reproduce the behavior:

  1. Create a simple Elyra pipeline with a single node.
  2. Add a pipeline parameter in the Pipeline Parameters configuration tab.
  3. Open the Pipeline Properties configuration tab.
  4. Save the pipeline, e.g. as test.pipeline.
  5. Validate the pipeline through the CLI: elyra-pipeline validate test.pipeline.

Screenshots or log output
The following error message without details is printed after running the above validation command:

Log Output
(app-root) (app-root) elyra-pipeline validate test.pipeline 

────────────────────────────────────────────────────────────────
Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
Error: Pipeline validation FAILED.

Expected behavior
Given that pipelines with parameters and default properties are valid in general, the validation command should resolve without any errors.
In case of actual issues within the pipeline file, there should be detailed information about these issues.

Deployment information
Describe what you've deployed and how:

  • Elyra version: 3.15.0
  • Installation source: quay.io/opendatahub/workbench-images:cuda-jupyter-tensorflow-ubi9-python-3.9
  • Deployment type: Red Hat OpenShift Data Science 1.33.0
  • Operating system: linux
@mamurak
Copy link
Author

mamurak commented Oct 13, 2023

Example pipeline:

{
  "doc_type": "pipeline",
  "version": "3.0",
  "json_schema": "http://api.dataplatform.ibm.com/schemas/common-pipeline/pipeline-flow/pipeline-flow-v3-schema.json",
  "id": "elyra-auto-generated-pipeline",
  "primary_pipeline": "primary",
  "pipelines": [
    {
      "id": "primary",
      "nodes": [
        {
          "id": "54d57acc-4ad0-410e-8d79-268cd52a45e9",
          "type": "execution_node",
          "op": "execute-python-node",
          "app_data": {
            "component_parameters": {
              "pipeline_parameters": [
                "param"
              ],
              "dependencies": [],
              "include_subdirectories": false,
              "outputs": [],
              "env_vars": [],
              "kubernetes_pod_annotations": [],
              "kubernetes_pod_labels": [],
              "kubernetes_secrets": [],
              "kubernetes_shared_mem_size": {},
              "kubernetes_tolerations": [],
              "mounted_volumes": [],
              "filename": "untitled.py",
              "runtime_image": "quay.io/modh/runtime-images@sha256:7dd23e58291cad7a0ab4a8e04bda06492f2c027eb33b226358380db58dcdd60b"
            },
            "label": "",
            "ui_data": {
              "label": "untitled.py",
              "image": "/notebook/fraud-detection/tensorflow/static/elyra/python.svg",
              "x_pos": 505,
              "y_pos": 210,
              "description": "Run Python script"
            }
          },
          "inputs": [
            {
              "id": "inPort",
              "app_data": {
                "ui_data": {
                  "cardinality": {
                    "min": 0,
                    "max": -1
                  },
                  "label": "Input Port"
                }
              }
            }
          ],
          "outputs": [
            {
              "id": "outPort",
              "app_data": {
                "ui_data": {
                  "cardinality": {
                    "min": 0,
                    "max": -1
                  },
                  "label": "Output Port"
                }
              }
            }
          ]
        }
      ],
      "app_data": {
        "ui_data": {
          "comments": []
        },
        "version": 8,
        "runtime_type": "KUBEFLOW_PIPELINES",
        "properties": {
          "name": "untitled",
          "runtime": "Data Science Pipelines",
          "pipeline_parameters": [
            {
              "name": "param",
              "description": "",
              "default_value": {
                "type": "String",
                "value": "val"
              },
              "required": false
            }
          ],
          "pipeline_defaults": {
            "kubernetes_shared_mem_size": {},
            "kubernetes_pod_labels": [],
            "mounted_volumes": [],
            "kubernetes_pod_annotations": [],
            "kubernetes_tolerations": [],
            "kubernetes_secrets": [],
            "env_vars": []
          }
        }
      },
      "runtime_ref": ""
    }
  ],
  "schemas": []
}

Validation through the CLI fails. Remove either pipeline_parameters or pipeline_defaults, and validation succeeds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant