Skip to content

Config

Lightweight config system based on a section in the pyproject.toml file.

PartialClusterConfig

Per-cluster configuration options.

Returned by:

env 

env: dict[str, str] = field(default_factory=dict)

Environment variables to set when running Slurm commands on this cluster.

sbatch_args 

sbatch_args: dict[str, str | int | float | bool] = field(
    default_factory=dict
)

Per-cluster sbatch flags, overriding the global sbatch_args.

results_path 

results_path: str | None = None

Path to the results directory for a specific cluster.

datasets_path 

datasets_path: str | None = None

Different path where the datasets should be replicated on this cluster.

When None, this defaults to the top-level config's datasets_path.

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: str | None = None

Path to the job script to use by default on this cluster.

project_dir 

project_dir: str | None = None

Path where the project should be cloned on this cluster.

ignore 

ignore: bool = False

Whether to ignore this cluster when running commands on all clusters.

ClusterConfig

Per-cluster configuration options.

The path fields in this class are by default 'pure' posix paths, to make it explicit that they can't be used on the local filesystem, and to avoid errors like trying to call .open/.mkdir/etc. On the current cluster, when calling current_cluster_config, they are actual Paths.

Returned by:

env 

env: dict[str, str]

Environment variables to set when running Slurm commands on this cluster.

sbatch_args 

sbatch_args: dict[str, str | int | float | bool]

Merged sbatch flags (global defaults overridden by per-cluster values).

results_path 

results_path: PathType

Path to the results directory for a specific cluster.

datasets_path 

datasets_path: PathType | None

Different path where the datasets should be replicated on this cluster.

When None, this defaults to the top-level config's datasets_path.

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: PathType | None

Path to the job script to use by default on this cluster.

project_dir 

project_dir: PathType | None

Path where the project should be cloned on this cluster.

ignore 

ignore: bool

Whether to ignore this cluster when running commands on all clusters.

CluvConfig

Bases: BaseModel

Configuration options for Cluv, loaded from the pyproject.toml file.

Returned by: Used by:

env 

env: dict[str, str] = {}

Global environment variables set on all clusters when running Slurm commands.

sbatch_args 

sbatch_args: dict[str, str | int | float | bool] = {}

Global sbatch flags applied on all clusters.

These are passed directly to sbatch and complement env (which sets SBATCH_* env vars). See [tool.cluv.clusters.<name>.sbatch_args] for per-cluster overrides.

results_path 

results_path: str

Default path to the results directory for all clusters (may contain env vars like $SCRATCH).

results_symlink: str = 'logs'

Name of the symlink created in the project directory pointing to results_path.

data_source 

data_source: str | None = None

hostname:/path of where to get the data from.

datasets_path 

datasets_path: str | None = None

Path to a dataset directory, for example, '$SCRATCH/my_dataset'

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: str | None = None

Default path to the job script to submit when one is not passed explicitly to cluv submit.

This can be overridden for specific clusters in the clusters section, and can also be overridden on the fly by passing a different job script to cluv submit.

project_dir 

project_dir: str | None = None

Default path where the project should be cloned on clusters.

clusters 

clusters: dict[str, PartialClusterConfig] = {}

Configuration options for each cluster.

The keys are cluster names, and values are configs that override options for that cluster.

get_cluster_config 

get_cluster_config(cluster: str) -> ClusterConfig

Returns the cluster config for a specific cluster.

The environment variables as part of paths will not be resolved.

get_cluv_config 

get_cluv_config() -> CluvConfig

Get the cluv config, loading it from the pyproject.toml if needed.

has_cluv_config 

has_cluv_config(pyproject_path: Path) -> bool

Check if the pyproject.toml contains a cluv config

current_cluster_config 

current_cluster_config() -> ClusterConfig[Path] | None

Returns the ClusterConfig of the current cluster, or None if not currently on a cluster.