Conda project #363
Conda project #363
Conversation
An anaconda-project.yml file is no longer required.
* If environment.yml or environment.yaml file exists
Create the env as envs/{name} where name comes from the file.
This works for pip entries, too.
* If requirements.txt file exists
Create the env as env/default with the latest version of python.
Add packages from requirements.txt as pip installs
the --python=x.x flag will ensure that the base conda environment has your requested version of python before running pip installs * also stop anaconda-project run from writing the file to disk
* also list pip packages with list-packages command
with out a name field the env-spec becomes "default"
This reverts commit fdd723b7e8e68671217f42dc4b866ecf0905fb6c. Need to think about this more carefully
* anaconda-project entry point renamed to conda-project
* conda searches for conda-* executables in the PATH
to enable "conda project"
* conda-project.yml is preferred but anaconda-project.yml
is still supported
* cli help updated to reflect conda-project
* conda recipe updated to build conda-project
(and project-lock.yml)
|
It will be good to rebase against #359 once it's merged. It should help with adding env_specs to the project.yml file that can extend the default env from environment.yml. Also we could use env_specs to install non-pypi packages before applying the supplied requirements.txt. |
AlbertDeFusco
force-pushed
the
conda-project
branch
from
c4a2b4f
to
2e367bb
Compare
|
This looks great! Here are some general questions/thoughts/comments:
|
|
Some great things in there for us to dig into. Here's another suggestion from Matt |
|
@jlstevens , what's the difference between the two files you're suggesting in 3. to convert? I can't think of any required difference, and would argue for a single type of file, with commands ignored unless one does a |
commands: {<name>: {command: <str>}}with unix-command/win-command when neededsupports_http_optionsin favor of explicit jinjanotebook,bokehcommands in favor docs with jinja docsThis PR combines #284 and #275 along with rebasing against latest commits to rename
anaconda-projecttoconda-projectwith no other change in the commands (at this time).The default project file is now
project.yml, but may also be calledconda-project.ymloranaconda-project.yml.The largest change is that
environment.ymlorrequirements.txtfiles can be used directly without the need to create aproject.yml(nor will the file be created for you).Enabled use cases
conda-project prepareconda-project run <executable> [<arg1>, <arg2>, ...]Note The run command can execute any executable in the environment and pass arguments to it. Commands need not be specified in an
project.ymlfile to be able to run.In the two use cases shown below there is no
project.ymlfile and it will not be created with the commands shown. For both cases you can useconda-project prepareto create the file and import the required packages from eitherenvironmnent.ymlorrequirements.txt.environment.yml
Here's a typical environment specification file.
Create the environment within the
envsdirectory of the project. The name of the env_spec will match thename:key in the environment.yml.Now we can run a command available in the PATH for the conda environment.
Finally, after adding packages to the the
environment.ymlfile they can be installed. (use the--refreshto completely rebuild the env, prepare will not remove packages)locking dependencies
To write the lock file (fully specified cross-platform environments)
project-lock.ymlthe lock file also included all pip packages (i.e., pip freeze)
If the environment.yml file differs from its version when the
project-lock.ymlfile was created thenconda-projectcommands will print a warning.to remove the warning
updatewill re-lock the packages and install the missing pacakgesenvironment.yml and project.yml
To extend the environment.yml file with specific project features like environment variables, supported platforms, commands, and data sets. For example
Beginning by running prepare we see that the environment is created and the dataset downloaded
Now running the default command verifies that the env variable is set
And finally, running lock will only lock for the provided platforms
requirements.txt
If there is a
requirements.txtin the project directory (and noenvironment.yml) all packages listed will be installed as pip packages.Running the prepare will first create a Conda environment with the most recent version of Python (3.8) and pip and then add the packages in the
requirements.txtfile.And confirm the pip packages were installed
If you require a different version of Python it can be supplied during prepare
Again, you can run any executable in the environment
Again, you can add packages to
requirements.txtand install them with prepare