cluv¶

A powerful and lightweight CLI tool to sync and submit UV-based Python projects across HPC clusters.

This is a quick overview. For more information, check out the introduction.

Installation¶

Cluv can be installed on a local machine or on a remote cluster.

Add the package to your project with uv add:

uv add cluster-uv

Install as a command-line tool in an isolated environment:

uv tool install cluster-uv

If you want the bleeding edge version from GitHub, use:

uv add git+https://github.com/mila-iqia/cluv

Update¶

Run this once in a while to get the latest features and bugfixes.

uv tool update cluster-uv

Usage¶

To view all available commands, use the --help flag:

cluv --help

Here are some common workflows:

Setup cluv in an existing project¶

cd ~/my-project  # Cluv requires your project to be located somewhere under your home directory (`$HOME`).
cluv init

Establish SSH connections to all clusters¶

cluv login

Sync your project on all clusters¶

cluv sync

Need to set up dataset replication with cluv sync? See the "Syncing datasets" guide.

Launch a Hydra sweep on a remote cluster¶

python main.py -m launcher=cluv lr=0.01,0.001 seed=1,2,3

See the Hydra launcher page for setup and usage.

Sync your project on a specific cluster¶

cluv sync rorqual

Clean up old run results from the clusters¶

cluv clean

See the "Cleaning runs" guide for details on how this decides what's safe to delete.

Submit a job to a specific cluster¶

cluv submit rorqual scripts/job.sh --time=00:10:00 -- python main.py

See the "Writing a job script" guide for what job.sh should contain, and "Configuring job submission" for how to use the config to submit jobs.

Run a command in the synced project a specific cluster¶

cluv run mila -- ls logs

Check the status of your clusters and jobs¶

cluv status

Run a raw shell command on every connected cluster¶

cluv sh -- squeue --me

How the commands are used together¶

    graph LR
        init(<b>cluv init</b> <br> Init project)
        sync(<b>cluv sync</b> <br> Sync project on clusters)
        login(<b>cluv login</b> <br> Connect to clusters)
        submit(<b>cluv submit</b> <br> Submit jobs to clusters)
        status(<b>cluv status</b> <br> See clusters and jobs status)
        disable(<b>cluv disable</b> <br> Disable access to clusters)
        enable(<b>cluv enable</b> <br> Enable access to clusters)
        clean(<b>cluv clean</b> <br> Clean old logs on clusters)
        run(<b>cluv run</b> <br> Run commands on clusters)
        sh(<b>cluv sh</b> <br> Run raw shell commands on clusters)

        init ===> login
        init ==> disable
        init ==> enable

        login ===> sync
        login ===> submit
        login ===> status
        login ===> clean
        login ===> run
        login ===> sh

        click init "commands/#cluv-init"
        click login "commands/#cluv-login"
        click sync "commands/#cluv-sync"
        click submit "commands/#cluv-submit"
        click status "commands/#cluv-status"
        click disable "commands/#cluv-disable"
        click enable "commands/#cluv-enable"
        click clean "commands/#cluv-clean"
        click run "commands/#cluv-run"
        click sh "commands/#cluv-sh"