Upgrade to Python 3.12 #2072
Upgrade to Python 3.12 #2072
Conversation
|
Haven't tested this yet, but it would be nice if we could start looking at moving the base image to being 3.12. |
|
It seems we depend on this PR: conda-forge/pycurl-feedstock#27 |
|
Waiting for the upstream packages to be fixed. I marked this as a draft and subscribed to the relevant PR, will re-run, when the issue resolves. |
|
Issue for pycurl resolved upstream! |
|
@consideRatio thanks! @max-muoto could you please resolve conflicts? Unfortunately, GitHub is not able to do it in the web browser. |
|
Something is wrong, as the change should have stayed small, just a few lines of code. |
Currently rebasing, should be fixed soon. |
max-muoto
force-pushed
the
python-3.12
branch
3 times, most recently
from
2455425
to
9035f65
Compare
Should be good to take a look at now. |
|
Seems we also need to upgrade Pandas, just went ahead and did that. |
| @@ -63,7 +63,7 @@ USER ${NB_UID} | |||
| RUN mamba install --yes \ | |||
| 'grpcio-status' \ | |||
| 'grpcio' \ | |||
| 'pandas=2.0.3' \ | |||
| 'pandas=2.2.1' \ | |||
There was a problem hiding this comment.
Have you checked the comment above to make sure you’re using the proper version?
There was a problem hiding this comment.
They have pandas<=2.2.1 so we should be good here.
There was a problem hiding this comment.
There was a problem hiding this comment.
You need to check the latest stable tag, not the current main branch.
There was a problem hiding this comment.
Ah sorry there. Looks like we'll need for their next release, as this commit isn't included in the latest stable tag: ericm-db/spark@98ca3ea
There was a problem hiding this comment.
I think we'll need to wait on this as well, since we need at least Pandas 2.1.1 to ensure compatibility with 3.12.
There was a problem hiding this comment.
@bjornjorgensen could you please tell us when the Spark release will include this commit? (at least approximately)
There was a problem hiding this comment.
hmm.. well we are waiting for hadoop 3.4.0 and a new hive release. We haven't started any RC release yet. I build and test my own jupyterlab https://github.com/bjornjorgensen/jupyter-spark-master-docker and I did try python 3.12 but it breake so match so I'm using python 3.11 as debian testing are using.
And Spark 3.5.1 don't support python 3.12 have a look at apache/spark#43922
|
[VOTE] Release Plan for Apache Spark 4.0.0 (June 2024) Apache Spark 4.0.0 Release Plan
|
|
there will be a test soon Re: [DISCUSS] Spark 4.0.0 release |
Describe your changes
Upgrade base image to 3.12