workflow-orchestration
Created: 10/5/2022, 12:00:00 AM
At Meta, we have a few internal workflow orchestration engines. The idea is to allow long-running jobs, or things that can be split into tasks and need to have safe "checkpoints" behind them be defined in a system that can safely execute them and make sure they're executed in a durable, reliable and scalable way.
As I stumble upon things that fit paradigms within Facebook externally, this page is to document them.
Core-Workflow-Services, Task-Oriented tools
These typically have the concept of a "worker" program to define the flow of the workflow, and distributed "workers" to execute the steps.
State machine driven approach
These differ from the above by having very fixed, simple states that we transition to. Typically better to build a nicer UI, but less flexible unless creating complex states (sub-jobs, etc)