Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't export kubeflow pipelines if I use parameters #3172

Open
MartinKlefas opened this issue Jul 13, 2023 · 1 comment
Open

Can't export kubeflow pipelines if I use parameters #3172

MartinKlefas opened this issue Jul 13, 2023 · 1 comment
Labels
component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines kind:bug Something isn't working

Comments

@MartinKlefas
Copy link

Describe the issue
I've downloaded your notebook server image and successfully submitted a number of pipelines based on your examples and the early stages of my work. I'd like to add parameters to my notebooks, but at present have made a very simple test notebook to try to get my first parametric pipeline created.

To Reproduce
I've made a simple python notebook which just prints "hello {person}". I've then followed this guide https://medium.com/codait/build-versatile-pipelines-in-elyra-using-pipeline-parameters-5634f8d5ae8d#:~:text=Parameterizing%20Pipelines%20in%20Elyra,Node%20Properties%20%2C%20all%20shown%20below. to make a kubeflow pipeline of sorts with single node.
I can save it with no parameters, and can export it to YAML. If I add a parameter, I can still export it to YAML. If I add that parameter to the single node, I'm no longer able to export the pipeline to YAML and get the below error from "argo".

Screenshots or log output
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/kfp/processor_kfp.py", line 676, in _compile_pipeline_dsl
kfp_argo_compiler.Compiler().compile(pipeline_function, output_file, pipeline_conf=pipeline_conf)
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1175, in compile
self._create_and_write_workflow(
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1227, in _create_and_write_workflow
workflow = self._create_workflow(pipeline_func, pipeline_name,
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1005, in _create_workflow
pipeline_func(*args_list, **kwargs_dict)
File "/tmp/tmpdwpcjjxg/generated_dsl.py", line 44, in generated_pipeline
TypeError: Run a file() got an unexpected keyword argument 'toWhom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/tornado/web.py", line 1713, in _execute
result = await result
File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/handlers.py", line 108, in post
pipeline_exported_path = await PipelineProcessorManager.instance().export(
File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor.py", line 111, in export
res = await asyncio.get_event_loop().run_in_executor(
File "/opt/conda/lib/python3.8/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/kfp/processor_kfp.py", line 523, in export
self._compile_pipeline_dsl(pipeline_dsl, workflow_engine, absolute_pipeline_export_path, pipeline_conf)
File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/kfp/processor_kfp.py", line 678, in _compile_pipeline_dsl
raise RuntimeError(
RuntimeError: Failed to compile pipeline with workflow_engine 'argo' to '/home/jovyan/untitled1.yaml'

Expected behavior
exported YAML file

Deployment information
Describe what you've deployed and how:

  • Elyra Version: 3.15.0
  • Operating system: Description: Ubuntu 20.04.2 LTS
  • Installation source:, official container image
  • Deployment type: Kubeflow [notebook server]
    Parameters.zip

Pipeline runtime environment

  • Kubeflow Pipelines - latest version

attachment
I've attached the notebook and the pipeline file as those are probably what I've done wrong

@MartinKlefas MartinKlefas added component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines status:Needs Triage labels Jul 13, 2023
@MartinKlefas
Copy link
Author

It's also worth mentioning that saving as a dsl file produces the python file as expected, but when I run that from the command line I get:
Traceback (most recent call last):
File "untitled1.py", line 116, in
kfp.compiler.Compiler().compile(
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1175, in compile
self._create_and_write_workflow(
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1227, in _create_and_write_workflow
workflow = self._create_workflow(pipeline_func, pipeline_name,
File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1005, in _create_workflow
pipeline_func(*args_list, **kwargs_dict)
File "untitled1.py", line 44, in generated_pipeline
factory_31cf8eea6110c297eb6e7ce7400159bd041c724a42334c025116d58a337198cf(
TypeError: Run a file() got an unexpected keyword argument 'toWhom'

where toWhom is my parameter name.

@lresende lresende added kind:bug Something isn't working and removed status:Needs Triage labels Jul 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines kind:bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants