<aside> 🚧

This document is a Work In Progress

</aside>

The goal of the storage layer is to provide a unified interface to all data persisted within storage or unpersisted within the Results Forest. This will be used to serve the Access API.

Context

This storage layer sits in front of the persisted storage (protocol db), and the ResultsForest, providing a unified view of all indexed data held within the system. In general, data in the persisted db is for sealed blocks, and data in the in-memory dbs is for unsealed blocks. However, it’s possible that data for sealed blocks is not yet persisted, and only exists in memory.

The ResultsForest is a fork aware mempool, holding data for all known ExecutionResults. For the purposes of the storage layer, it can be thought of as a tree of ExecutionResults forming from the latest persisted sealed result.

By convention, the latest persisted sealed result always exists in the tree.

The forest may contain results for unfinalized blocks, so any usage of block height is only relevant within a given fork.

The caller into the storage layer must first determine the ExecutionResult to query based on the block. This will be handled in the API and use criteria such as number of executors, preferred executors, etc. These may be specified by the node operator or user. This ExecutionResult determines the fork for which all data will be extracted.

Snapshots

A snapshot is view of the data within the execution fork ending on the requested ExecutionResult and starting from the root block. Unpersisted data is queried from the execution result processing pipeline’s in-memory storage objects. Persisted data is queried from the persisted database. Only data from results in the execution fork is queryable.

The Snapshot method must check:

• that the result is available in the tree, otherwise return storage.ErrNotFound
• the result is in the waiting_persist, persisting , or completed state, otherwise return storage.ErrNotFound
• the result descends from the latest sealed result

The snapshot itself will have methods that return implementations of the corresponding data type’s Reader interfaces. The reader implementations allow querying all data within the fork ending on the snapshot’s result, including persisted sealed data.

Interfaces

type ExecutionStateCache interface {
	Snapshot(executionResultID flow.Identifier) Snapshot
}