Skip to content

Config

Lightweight config system based on a section in the pyproject.toml file.

PartialClusterConfig

Per-cluster configuration options.

Returned by:

env 

env: dict[str, str] = field(default_factory=dict)

Environment variables to set when running Slurm commands on this cluster.

results_path 

results_path: str | None = None

Path to the results directory for a specific cluster.

datasets_path 

datasets_path: str | None = None

Different path where the datasets should be replicated on this cluster.

When None, this defaults to the top-level config's datasets_path.

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: str | None = None

Path to the job script to use by default on this cluster.

ClusterConfig

Per-cluster configuration options.

Returned by:

env 

env: dict[str, str]

Environment variables to set when running Slurm commands on this cluster.

results_path 

results_path: Path

Path to the results directory for a specific cluster.

datasets_path 

datasets_path: Path | None

Different path where the datasets should be replicated on this cluster.

When None, this defaults to the top-level config's datasets_path.

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: Path | None

Path to the job script to use by default on this cluster.

CluvConfig

Bases: BaseModel

Configuration options for Cluv, loaded from the pyproject.toml file.

Returned by: Used by:

env 

env: dict[str, str] = {}

Global environment variables set on all clusters when running Slurm commands.

results_path 

results_path: str

Default path to the results directory for all clusters (may contain env vars like $SCRATCH).

results_symlink: str = 'logs'

Name of the symlink created in the project directory pointing to results_path.

data_source 

data_source: str | None = None

hostname:/path of where to get the data from.

datasets_path 

datasets_path: str | None = None

Path to a dataset directory, for example, '$SCRATCH/my_dataset'

This folder will be synced from the current cluster to all other clusters at their respective dataset_path.

job_script_path 

job_script_path: str | None = None

Default path to the job script to submit when one is not passed explicitly to cluv submit.

This can be overridden for specific clusters in the clusters section, and can also be overridden on the fly by passing a different job script to cluv submit.

clusters 

clusters: dict[str, PartialClusterConfig] = {}

Configuration options for each cluster.

The keys are cluster names, and values are configs that override options for that cluster.

get_cluster_config 

get_cluster_config(cluster: str) -> ClusterConfig

Returns the cluster config for a specific cluster.

The environment variables as part of paths will not be resolved.

get_cluv_config 

get_cluv_config() -> CluvConfig

Get the cluv config, loading it from the pyproject.toml if needed.

has_cluv_config 

has_cluv_config(pyproject_path: Path) -> bool

Check if the pyproject.toml contains a cluv config

get_cluster_choices 

get_cluster_choices() -> list[str]

Return configured clusters or the defaults when config is missing/invalid.

current_cluster_config 

current_cluster_config() -> ClusterConfig | None

Returns the ClusterConfig of the current cluster, or None if not currently on a cluster.