Experiment Design

pydantic model doespy.design.exp_design.Suite

A suite is a collection of experiments, denoted as <EXP1>, <EXP2>, etc. Each experiment has its own set of config variables. It is also possible to define variables that are shared by all experiments in the suite, referred to as SUITE_VARS. In addition to the experiments, a suite can also define an ETL (extract, transform, load) pipeline, which outlines the steps for processing the resulting files.

field $SUITE_VARS$: SuiteVarsConfigDict = {}

Shared variables belonging to every experiment of the suite.

field <EXP1>: Experiment [Required]

A suite needs to contain at least one experiment. Choose a descriptive experiment name for the placeholder <EXP1>.

field <EXP2>: Optional[Experiment] = None

Further experiments are optional. Choose a descriptive experiment name for the placeholder <EXP2>, <EXP3>, etc.

field $ETL$: Dict[str, ETLPipeline] = {}

ETL Pipeline to process the result files.

pydantic model doespy.design.exp_design.SuiteVarsConfigDict

validator include_vars  »  all fields

At any depth of the config dict, we can include variables from another file.

$INCLUDE_VARS$: Optional[Union[str, List[str]]]

Where str corresponds to the file name, e.g., test.yml, in doe-suite-config/designs/design_vars

All the variables in the external file, are included at the level of where $INCLUDE_VARS$ is located. If a variable is already present, then the variable is skipped.

Validates:

• all fields

resolve_include_vars()

Experiment

pydantic model doespy.design.exp_design.Experiment

An experiment is composed of a set of runs, each with its own unique configuration. The configurations of the runs vary different experiment factors, e.g., number of clients. Additionally, an experiment also specifies the hosts responsible for executing the runs.

field n_repetitions: int = 1

Number of repetitions with the same experiment run configuration.

field common_roles: Union[SetupRoleId, List[SetupRoleId]] = []

Ansible roles executed during the setup of all host types.

field host_types: Dict[HostTypeId, HostType] [Required]

The different :ref`host types<Host Type>` involved in the experiment.

field base_experiment: BaseExperimentConfigDict [Required]

Defines constants and factors of an experiment.

field factor_levels: List[Dict] = [{}]

For the factors of an experiment, lists the different levels. For example, n_clients can be a factor with two levels: 1 and 100.

field except_filters: List[Dict] = []

A list of filters that can be used to exclude certain runs from the experiment.

validator context  »  all fields

Validates:

• all fields

validator convert_common_roles  »  common_roles

Validates:

• common_roles

validator check_factor_levels  »  all fields

The base_experiment defines a set of $FACTOR$s that use the level list syntax. (i.e., $FACTOR$ is value). This validator checks that this set of $FACTOR$s matches each list entry of factor_levels.

Validates:

• all fields

validator check_except_filters  »  all fields

Every entry in except_filters must be a subset of the actual factors.

Validates:

• all fields

class doespy.design.exp_design.BaseExperimentConfigDict

__init__(*args, **kwargs)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

Host Type

pydantic model doespy.design.exp_design.HostType

field n: int = 1

field check_status: bool = True

field init_roles: Union[SetupRoleId, List[SetupRoleId]] = []

field $CMD$: Union[Cmd, Dict[str, Cmd], List[Cmd], List[Dict[str, Cmd]]] [Required]

enum doespy.design.exp_design.HostTypeId(value)

Name of a host type and corresponds to folder in doe-suite-config/group_vars.

Valid values are as follows:

server = <HostTypeId.server: 'server'>

small_v2 = <HostTypeId.small_v2: 'small_v2'>

client = <HostTypeId.client: 'client'>

small = <HostTypeId.small: 'small'>

enum doespy.design.exp_design.SetupRoleId(value)

Name of an Ansible role to setup a host. The role is located in folder: doe-suite-config/roles.

Valid values are as follows:

setup_common = <SetupRoleId.setup_common: 'setup-common'>

setup_client = <SetupRoleId.setup_client: 'setup-client'>

setup_small = <SetupRoleId.setup_small: 'setup-small'>

setup_server = <SetupRoleId.setup_server: 'setup-server'>

ETL Pipeline

pydantic model doespy.design.etl_design.ETLPipeline

field experiments: ExperimentNames [Required]

field $ETL_VARS$: Optional[Dict[str, Any]] = None

Extractor

pydantic model doespy.design.etl_design.Extractor

field file_regex: Optional[Union[str, List[str]]] = None

class doespy.design.etl_design.ExtractorId

An enumeration.

CsvExtractor = 'CsvExtractor'

ErrorExtractor = 'ErrorExtractor'

IgnoreExtractor = 'IgnoreExtractor'

JsonExtractor = 'JsonExtractor'

YamlExtractor = 'YamlExtractor'

INCLUDE_STEPS = '$INCLUDE_STEPS$'

Transformer

pydantic model doespy.design.etl_design.Transformer

Loader

pydantic model doespy.design.etl_design.Loader

field $INCLUDE_STEPS$: Optional[List[IncludeEtlSource]] = None

field extra: Dict[str, Any] [Required]