<aside> 🚧
This document is a Work In Progress
</aside>
The Results Forest will use the forest.LevelledForest module, and work somewhat similarly to the Consensus nodes’ ExecutionTree. Unlike the ExecutionTree, the Results Forest does not need to track receipts. However, it does have some additional responsibilities.
During startup, the ingestion engine will load unhandled results from the protocol db into the forest. As the ingestion engine discovers new ExecutionResults, they are added to the forest. They are added in an inactive state, and are started by the Pipeline manager.
Since there may be many thousands of vertices in the forest, we need to manage how many are actively running at a time to limit the memory usage. To solve this, there is a pipeline manager process that periodically checks how many of the pipelines are running and how many have completed. When the number of running falls below the max threshold, new pipelines are started.
This process iterates the tree starting from the latest persisted sealed result, counting the number of active pipelines and starting up to the configured max. It uses a breadth first search algorithm to ensure that pipelines with lower heights are started first.
Pipelines are started by calling the Run(ctx) method.
As results are sealed, the ingestion engine will notify the forest of new sealed result. The forest will then call the SetSealed() method on the pipeline associated with the result, which allows persisting once the previous result’s pipeline is complete.
The forest then looks up the other children of the sealed result’s parent, and calls
OnParentStateUpdated(state) passing canceled to indicate that they no longer descend from the latest sealed result. This is then propagated to their descendants by the pipelines. This causes them to halt processing and free memory used for their caches. These pipelines are kept in the forest to allow differentiating abandoned forks from new disconnected branches.
Pipelines communicate state updates by calling their state update publish callback. The Results Forest provides the implementation, which OnParentStateUpdated(state) on all while vertices. The forest uses these callbacks to identify pipelines that have completed. As pipelines complete, the forest will notify the Pipeline Manager that there may be more room to start new pipelines and kick off pruning.
As processing pipelines enter the completed state, the forest will be pruned to remove completed and abandoned pipelines. Vertices are pruned by level (view), which leaves nodes on abandoned forks in the forest until the fork is eventually fully pruned.
The general logic for this can be borrowed from the ExecutionTree's PruneUpToHeight.
The forest should also allow efficient querying by ExecutionResult.ID, as well as traversal from a pipeline to its ancestors. This will be used by the storage layer to query data from the in-memory databases.