Welcome to the DoE-Suite documentation!

User’s Guide

• Installation
  • Prerequisites
    • AWS - Specific Prerequisites
    • Manual Inventory - Specific Prerequisites
    • ETHZ Euler - Specific Prerequisites
    • Docker - Specific Prerequisites
  • Base Installation
    • Configuring SSH-Agent
    • General Environment Variables
  • AWS-Specific
    • AWS CLI
    • SSH Config (AWS)
    • Environment Variables (AWS)
    • Check Installation (AWS)
  • Manuel Inventory - Specific
    • SSH Config (Inventory)
    • Setting up Inventory
    • Check Installation (Inventory)
  • ETHZ Euler - Specific
    • SSH Config (Euler)
    • Environment Variables (Euler)
    • Check Installation (Euler)
  • Docker-Specific
    • SSH Keys (Docker)
    • SSH Config (Docker)
    • Environment Variables (Docker)
    • Check Installation (Docker)
• Quickstart
  • A Minimal Example
  • Quick Command Reference
• Tutorial
  • Project Layout
    • Suite Design
    • (Add) Host Type
    • (Add) Setup Role
    • (Add) ETL Step
  • Execution
  • Keep Experimenting
• Commands
  • Running an Experiment Suite
  • Cleaning up Cloud
  • ETL Results Processing
  • Status and Info
  • Developing Suite Designs
  • Custom Commands
• Suite Design
  • Suite Design By Example
    • example01-minimal
    • example02-single
    • example03-format
    • example04-multi
    • example05-complex
    • example06-vars
    • example07-etl
    • example08-superetl
  • Experiment Design
    • Suite
    • SuiteVarsConfigDict
    • Experiment
    • ETL Pipeline
  • Special Experiment Design Variables
    • to_ipv4()
    • to_public_dns_name()
    • to_private_dns_name()
    • at_runtime()
• ETL Design
  • Extractors
    • Yaml Files
    • Json Files
    • Csv Files
    • Raising Attention to Errors
    • Ignoring Result Files
  • Transformers
    • Pandas DF Transformers
    • Conditional Replacement
    • Group By Aggregates
  • Loaders
    • Save as Csv
    • Save as Pickle
    • Save as Latex Table
    • Advanced Plots: Column Cross
• Running Experiments
  • Experiment Hosts
    • Host Type
    • Setup Roles
  • Experiment Jobs
    • Design DSL
    • Job Execution
  • Experiment Results
    • Result Directory
    • ETL
    • Super ETL

Developer’s Guide

• Clouds
  • Supported clouds
  • Cloud multiplexing
  • Schedulers
• ETL Processing
  • Dev Transformers
• Dev Super ETL
  • Custom output location
  • Config changes
• Experiment Configuration
  • General Syntax
• Developer Ansible Guide
  • Adding A Variable to the Experiment State
  • Important Data Structures
  • Roles
  • Custom JINJA2 Filters
  • Job scheduling
  • Inventory Host Groups

Indices and tables

• Index

• Module Index

• Search Page