Skip to content

Commit a3e2a46

Browse files
committed
update wfe docs introduction
1 parent 8c1f350 commit a3e2a46

File tree

2 files changed

+42
-2
lines changed

2 files changed

+42
-2
lines changed
Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,13 @@
11
# WFE concept
22

3-
to be filled
3+
## Workflow Engine as the skeleton
4+
The workflow engine functions as the skeletal framework that holds the entire system together. Much like a human skeleton provides structure and support for bodily functions, the workflow engine organizes and orchestrates the various components and configuration involved in prforming and anaylsing the simulation. It ensures that tasks, tools, and set-ups are not only available but interconnected in a coherent, reproduceable and maintainable structure. By providing this underlying architecture, the workflow engine enables a convenient and reproduceable way to perform simulations, making it central to the platform’s operation.
5+
6+
## ... is not functional in itself
7+
While the workflow engine is essential to TSMP2, it is not independently functional. On its own, it neither performs computations nor manages software artifacts. Instead, it acts as an enabler, delegating actual tasks to specialized modules and external services. Think of it as a conductor of an orchestra—it doesn't produce music itself but ensures each instrument plays its part at the right time. The engine relies entirely on the surrounding source code, namelist, static fields aso fulfill real work.
8+
9+
## “Knows” where to set the parts
10+
What makes the workflow engine in TSMP2 particularly powerful is its embedded knowledge of the state of all . This is not a matter of static configuration; it involves dynamic, rule-based decisions that map incoming tasks to appropriate handlers, resources, and services based on context. Whether it’s orchestrating a build pipeline, triggering an automated test suite, or managing artifact promotion, the engine determines the correct placement and timing of each activity. This contextual awareness transforms a potentially chaotic set of tools into a streamlined, coordinated process.
11+
12+
## Git submodules make it complete
13+
In the context of the TSMP2 workflow engine, Git submodules play a crucial role in making the system complete by enabling modular integration of external or shared components directly into the execution environment. Since the workflow engine itself acts as an orchestrator rather than a functional unit, it depends heavily on well-defined, version-controlled tools and configurations. By using Git submodules, TSMP2 can include specific toolchains, templates, or environment definitions as part of a workflow without duplicating code or compromising maintainability. This ensures that each workflow step uses the correct, consistent version of its dependencies, improving reproducibility and reducing integration overhead. The engine “knows” where and how to place these submodules within the runtime, aligning them with the correct parts of the execution graph, thereby completing the workflow setup with precision and modular clarity.
Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,33 @@
11
# Workflow engine overview
22

3-
to be filled
3+
## Overview of TSMP2-WFE
4+
We are utilizing a WorkFlow Engine (WFE) for our simulations, designed to streamline and manage the complex processes involved in running and analyzing them. Its primary purpose is to automate and oversee the various steps required to execute a simulation, including data preparation, model configuration, simulation execution, and result processing.
5+
6+
A key objective of the WFE is to enhance reproducibility, which it achieves more easily through the use of version control with git.
7+
8+
The TSMP2 workflow engine also allows for flexible model component combinations. You can specify the model combination by providing the MODEL_ID, similar to the TSMP2 building system. The WFE itself is lightweight, functioning as a framework, while individual experiments are configured using git submodules to manage components such as namelists, static fields, and other necessary elements.
9+
10+
11+
## Structure of TSMP2-WFE
12+
13+
The directory structure of the WFE is the following:
14+
```
15+
TSMP2_wfe/
16+
|
17+
|---- ctl -> for managing the simulation
18+
| |---- {pre,sim,pos,vis}_ctl -> for controlling the steps
19+
| |---- logs -> for job log during the run
20+
|---- src -> for source code (TSMP2, pre/post-processing tools , ...)
21+
|---- nml -> for component namelists
22+
|---- run -> for number crunching
23+
|---- dta -> for data
24+
```
25+
26+
Two configure files to manage configuration / setup
27+
```
28+
ctl/
29+
|---- master.conf -> general settings
30+
|---- expid.conf -> model specifications
31+
```
32+
33+

0 commit comments

Comments
 (0)