<aside> 🚧

This document is a Work In Progress

</aside>

The goal of the storage layer is to provide a unified interface to all data persisted within storage or unpersisted within the Results Forest. This will be used to serve the Access API.

Context

This storage layer sits in front of the persisted storage (protocol db), and the ResultsForest, providing a unified view of all indexed data held within the system. In general, data in the persisted db is for sealed blocks, and data in the in-memory dbs is for unsealed blocks. However, it’s possible that data for sealed blocks is not yet persisted, and only exists in memory.

The ResultsForest is a fork aware mempool, holding data for all known ExecutionResults. For the purposes of the storage layer, it can be thought of as a tree of ExecutionResults forming from the latest persisted sealed result.

By convention, the latest persisted sealed result always exists in the tree.

The forest may contain results for unfinalized blocks, so any usage of block height is only relevant within a given fork.

The caller into the storage layer must first determine the ExecutionResult to query based on the block. This will be handled in the API and use criteria such as number of executors, preferred executors, etc. These may be specified by the node operator or user. This ExecutionResult determines the fork for which all data will be extracted.

Summary

From the API perspective, the workflow will be something like:

  1. Identify the block(s) the query references
  2. Identify the ExecutionResult that best matches the client’s query criteria for the block
  3. Get the Snapshot for the selected ExecutionResult
  4. Use the Snapshot to get the storage reader instance
  5. Query the reader for execution data required to serve the request
  6. Return the result (including some metadata about the ExecutionResult)

The complexity of querying data within the ResultsForest is absorbed into the caching layer, which exposes a simple set of functions for the API. Most of the complexity is managed within steps 2, 3 and 5.