This document is a Work In Progress

Processing Pipeline

The processing pipeline handles all of the logic to download, index, query, and persist data for a single ExecutionResult. See Result Processing for protocol design.

State Machine

The pipeline will be implemented as a state machine, with the following states:

1. ready → This is the initial state after instantiation and before downloading has begun.
2. downloading → BlockExecutionData download is in progress.
3. indexing → BlockExecutionData is being indexed.
4. waiting_persist → All data is indexed, but conditions to persist are not met.
5. persisting → Persist conditions are all met, and the indexed data is being persisted to storage.
6. complete → All prior steps completed successfully.
7. canceled → Processing was aborted. This is generally because it no longer descends from the latest persisted sealed result.

The following conditions must be met before transitioning to the next state:

• ready → downloading
  1. ExecutionResult descends from the last persisted sealed result.
  2. The parent pipeline must be downloading, indexing, waiting_persist, persisting, or complete.
• downloading → indexing
  1. Download step is complete.
  2. ExecutionResult descends from the last persisted sealed result.
  3. The parent pipeline must not be canceled.
• indexing → waiting_persist
  1. Indexing step is complete
  2. ExecutionResult descends from the last persisted sealed result.
  3. The parent pipeline must not be canceled.
• waiting_persist → persisting
  1. ExecutionResult is sealed
  2. Parent pipeline is complete