Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pipeline editor doesn't connect to insecured pipeline server #3149

Open
harshad16 opened this issue May 8, 2023 · 2 comments · May be fixed by #3150
Open

Pipeline editor doesn't connect to insecured pipeline server #3149

harshad16 opened this issue May 8, 2023 · 2 comments · May be fixed by #3150
Labels
component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines status:Needs Triage

Comments

@harshad16
Copy link
Contributor

Describe the issue

The pipeline editor looks for a secured pipeline server for connection and doesn't allow execution of the pipeline with insecure pipeline server.
When tried to connect pipeline editor with insecured pipeline server, the editor throws following error:

HTTPServerRequest(protocol='http', host='jupyter-nb-kube-3aadmin-rhods-notebooks.apps.hnalla.dev.datahub.redhat.com', method='POST', uri='/notebook/rhods-notebooks/jupyter-nb-kube-3aadmin/elyra/pipeline/schedule?1683579505300', version='HTTP/1.1', remote_ip='127.0.0.1')
Traceback (most recent call last):
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
conn.connect()
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/connection.py", line 419, in connect
self.sock = ssl_wrap_socket(
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "/opt/app-root/lib64/python3.8/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.8/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/usr/lib64/python3.8/ssl.py", line 1040, in _create
self.do_handshake()
File "/usr/lib64/python3.8/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1131)

To Reproduce
Steps to reproduce the behavior:

  1. Setup ODH or kubeflow pipeline on an insecure cluster
  2. Execute a notebook with elyra installed
  3. Try to execute a pipeline example
  4. See error

Screenshots or log output
Screenshot from 2023-05-08 16-58-49

Expected behavior
Pipeline executes with any issue.

Deployment information
Describe what you've deployed and how:

  • Elyra version: 3.15.0
  • Operating system: linux
  • Installation source: custom container image
  • Deployment type: Open Data Hub

Pipeline runtime environment
If the issue is related to pipeline execution, identify the environment where the pipeline is executed

Additional details

Simple snippet like this can be executed from the notebook to test this.

from kfp import Client
import sys

host = "https://localhost:8888/"
namespace = "data-science-pipelines"
token = "sha~..."
  
c = Client(host=host, namespace=namespace, existing_token=token)
e = c.list_experiments(namespace=namespace)
print(e)
@harshad16 harshad16 added component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines status:Needs Triage labels May 8, 2023
@harshad16
Copy link
Contributor Author

By providing the ssl certs, user can execute the pipeline. It can be added to snippet:

cert = "/run/secrets/kubernetes.io/serviceaccount/ca.crt"
client = kfp_tekton.TektonClient(host=route, existing_token=token, ssl_ca_cert=cert)

however similar method is not available for pipeline editor.

@harshad16 harshad16 linked a pull request May 8, 2023 that will close this issue
Open
@shalberd
Copy link

Same can be applied at some point for ssl Airflow Openshift svc as well (optional in Airflow Helm chart, I think), good pointer here, this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component:pipeline-editor pipeline editor component:pipeline-runtime issues related to pipeline runtimes e.g. kubeflow pipelines status:Needs Triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Allow users to include ssl_sa_certs via env vars for kfp_tekton client harshad16/elyra
2 participants
@harshad16 @shalberd