Skip to content

Latest commit

 

History

History

example

README.md

Examples

Tip

Try out unidep in this folder by running:

  • unidep install ./setup_py_project ./hatch_project to install the setup_py_project and hatch_project packages and its dependencies with conda, then the remaining dependencies with pip, and finally the local packages with pip
  • unidep install-all -e to install all packages (setup_py_project, hatch_project, setuptools_project, etc.) in editable mode
  • unidep conda-lock to generate a global conda-lock.yml file and consistent per package conda-lock.yml files
  • unidep merge to merge all requirements.yaml files into a single environment.yaml file
  • unidep pip-compile to generate a locked requirements.txt file

Explore these example projects to understand how unidep integrates with different build tools and configurations:

Project Build Tool pyproject.toml requirements.yaml setup.py Description
setup_py_project setuptools ✅ ✅ ✅ Traditional setuptools project with requirements.yaml.
setuptools_project setuptools ✅ ✅ ❌ Modern setuptools usage with both pyproject.toml and requirements.yaml.
pyproject_toml_project setuptools ✅ ❌ ❌ Pure pyproject.toml setup, showcasing comprehensive dependency management.
hatch_project hatch ✅ ✅ ❌ Demonstrates unidep integration in a Hatchling project with requirements.yaml.
hatch2_project hatch ✅ ❌ ❌ Pure pyproject.toml Hatchling project.

Exploring unidep Through Practical Examples

Combine one or multiple requirements.yaml/pyproject.toml files into a single environment.yaml file

Combine requirements.yaml files in subdirectories and into an environment.yaml file that can be installed with conda.

Here we can just run unidep merge with no arguments, since the defaults are the same as what we want.

This would be the same as running unidep merge --name myenv --verbose:

🔍 Scanning in `.` at depth 0
🔍 Scanning in `hatch2_project` at depth 1
🔍 Found `"pyproject.toml"` with dependencies at `hatch2_project/pyproject.toml`
🔍 Scanning in `hatch_project` at depth 1
🔍 Found `"requirements.yaml"` at `hatch_project/requirements.yaml`
🔍 Scanning in `pyproject_toml_project` at depth 1
🔍 Found `"pyproject.toml"` with dependencies at `pyproject_toml_project/pyproject.toml`
🔍 Scanning in `setup_py_project` at depth 1
🔍 Found `"requirements.yaml"` at `setup_py_project/requirements.yaml`
🔍 Scanning in `setuptools_project` at depth 1
🔍 Found `"requirements.yaml"` at `setuptools_project/requirements.yaml`
📄 Parsing `hatch2_project/pyproject.toml`
📄 Parsing `hatch_project/requirements.yaml`
📄 Parsing `pyproject_toml_project/pyproject.toml`
📄 Parsing `../hatch_project[test]` from `local_dependencies`
📄 Parsing `pyproject_toml_project/../hatch_project/requirements.yaml[test]`
📄 Moving `test` optional dependencies to main dependencies for `pyproject_toml_project/../hatch_project/requirements.yaml[test]`
📄 Parsing `setup_py_project/requirements.yaml`
📄 Parsing `../setuptools_project` from `local_dependencies`
📄 Parsing `setup_py_project/../setuptools_project/requirements.yaml`
📄 Parsing `setuptools_project/requirements.yaml`
📝 Generating environment file at `environment.yaml`
📝 Environment file generated successfully.
✅ Generated environment file at `environment.yaml` from `hatch2_project/pyproject.toml`, `hatch_project/requirements.yaml`, `pyproject_toml_project/pyproject.toml`, `setup_py_project/requirements.yaml`, `setuptools_project/requirements.yaml`

See the resulting environment.yaml file which is installable with mamba. This file is using sel(linux|osx|win) to specify platform specific dependencies. Alternatively, use unidep merge --selector comment to generate a file that uses comments to specify platform specific dependencies, which can be read by conda-lock.

Using pip install

This method allows you to install packages defined in a requirements.yaml file using pip. It focuses on installing only those dependencies that are pip-installable, followed by the local project package.

How to Use:

  • Run pip install ./setup_py_project.
  • This command will process the requirements.yaml in the specified directory (./setup_py_project/), installing all pip-installable dependencies, including the local project itself.

Using unidep install

Using unidep for installation offers a more comprehensive approach. It handles both Conda and Pip dependencies specified in the requirements.yaml file, ensuring all necessary packages are installed, including those not available through pip.

How to Use:

  • To perform a standard installation, run unidep install ./setup_py_project.
  • For an editable installation (useful during development), use unidep install -e ./setup_py_project.
  • The unidep install command first installs any Conda-specific dependencies from the requirements.yaml file, then proceeds to install pip-specific dependencies. Finally, it installs the local project package.
$ unidep install --dry-run -e ./setup_py_project
📦 Installing conda dependencies with `conda install --yes --override-channels --channel conda-forge pandas">=1,<3" adaptive">=0.15.0, <2.0.0" pfapack packaging adaptive-scheduler numpy">=1.21" hpc05 pexpect pytest pytest-cov`

📦 Installing pip dependencies with `/opt/hostedtoolcache/Python/3.12.7/x64/bin/python -m pip install yaml2bib rsync-time-machine slurm-usage pyyaml aiokef markdown-code-runner numthreads unidep`

📝 Found local dependencies: {'setup_py_project': ['hatch_project', 'setuptools_project']}

📦 Installing project with `/opt/hostedtoolcache/Python/3.12.7/x64/bin/python -m pip install --no-dependencies -e /home/runner/work/unidep/unidep/example/hatch_project -e /home/runner/work/unidep/unidep/example/setuptools_project -e ./setup_py_project`

Using unidep install-all for installation across multiple projects

The unidep install-all command provides a convenient way to install all dependencies across multiple projects or packages within a given directory. This command is especially useful in monorepos or when managing several related projects with their own requirements.yaml files.

How unidep install-all Works:

  • This command scans a specified directory (or the current directory if none is specified) for requirements.yaml files.
  • It then installs dependencies for each found project, handling both Conda and Pip dependencies.
  • The local packages are also installed, making this command a one-stop solution for setting up your entire workspace.

Usage Examples:

  • Run unidep install-all to install all dependencies in the current directory.
  • Use unidep install-all -e for an editable install, which is useful during development. This flag ensures that local packages are installed in a way that allows changes to be reflected immediately without needing reinstallation.

Example Command:

# To install all projects in the current directory in editable mode
unidep install-all -e

Output Example:

$ unidep install-all -e --dry-run
📦 Installing conda dependencies with `conda install --yes --override-channels --channel conda-forge adaptive-scheduler numpy">=1.21" hpc05 pandas">=1,<3" pexpect adaptive">=0.15.0, <2.0.0" pfapack packaging pytest pytest-cov`

📦 Installing pip dependencies with `/opt/hostedtoolcache/Python/3.12.7/x64/bin/python -m pip install unidep markdown-code-runner numthreads yaml2bib rsync-time-machine slurm-usage pyyaml aiokef`

📝 Found local dependencies: {'pyproject_toml_project': ['hatch_project'], 'setup_py_project': ['hatch_project', 'setuptools_project'], 'setuptools_project': ['hatch_project']}

📦 Installing project with `/opt/hostedtoolcache/Python/3.12.7/x64/bin/python -m pip install --no-dependencies -e ./hatch2_project -e ./hatch_project -e ./pyproject_toml_project -e ./setup_py_project -e ./setuptools_project`

This command streamlines the process of getting a development environment up and running, particularly in complex setups with multiple interdependent projects.