Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stateful Storage Pallet Documentation Improvement #1989

Merged
merged 8 commits into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions pallets/stateful-storage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Stateful Storage Pallet

The Stateful Storage Pallet provides per-MSA functionality for reading and writing stateful data where only the latest state is relevant.

## Summary

When a Schema uses `Paginated` or `Itemized`, the payload data is stored in this pallet.
Data stored is user-centric instead of time-centric as with the Messages Pallet.
claireolmstead marked this conversation as resolved.
Show resolved Hide resolved
The pallet storage uses [child storage](https://paritytech.github.io/polkadot-sdk/master/frame_support/storage/child/index.html) making direct query access complex.
Custom RPCs are provided for easy access to data.

### Paginated Data (`PayloadLocation:Paginated`)

Data is stored in multiple pages, each `1_024` bytes in size (as defined by `constants::MaxPaginatedPageSizeBytes`).
Each page contains a single item of the associated schema.
Page count is limited to `33` per Schema Id, though there may be holes in that range (limit defined by `constants::MaxPaginatedPageId`).
This is most useful for schemas with a larger per-item size and smaller potential item count.

### Itemized Data (`PayloadLocation:Itemized`)

Data is stored in a single page with a max size of `10_240` bytes (defined by `constants::MaxItemizedPageSizeBytes`).
The page contains multiple items of the associated schema.
The maximum size of each items is `1_024` bytes (defined by `constants::MaxItemizedBlobSizeBytes`) .
This is most useful for schemas with a relatively small item size and higher potential item count.
The read and write complexity is O(n) when n is the number of bytes for all items.



### Actions

The Stateful Storage pallet provides for:

- Per MSA and Schema storage of stateful data
- Read/write access storage cost limited to a single MSA Id and Schema Id pair
- A high write throughput
- A high read throughput
- Data race condition protection


## Interactions

### Extrinsics

| Name/Description | Caller | Payment | Key Events | Runtime Added |
| -------------------------------- | ------------- | ------- | ------------------------------------------------------------------------------------------------------------- | ------------- |
| `apply_item_actions`<br />Applies a set of actions to an itemized storage array | Provider or MSA Owner | Capacity or Tokens | [`ItemizedPageUpdated`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.ItemizedPageUpdated)<br />[`ItemizedPageDeleted`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.ItemizedPageDeleted) | 22 |
| `apply_item_actions_with_signature_v2`<br />Applies a set of actions to an itemized storage array with a signature authorization | Provider or MSA Owner | Capacity or Tokens | [`ItemizedPageUpdated`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.ItemizedPageUpdated)<br />[`ItemizedPageDeleted`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.ItemizedPageDeleted) | 45 |
| `upsert_page`<br />Sets the data for a specific page index | Provider or MSA Owner | Capacity or Tokens | [`PaginatedPageUpdated`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.PaginatedPageUpdated) | 22 |
| `delete_page`<br />Deletes a specific page index | Provider or MSA Owner | Capacity or Tokens | [`PaginatedPageDeleted`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.PaginatedPageDeleted)| 22 |
| `upsert_page_with_signature_v2`<br />Sets the data for a specific page index with a signature authorization | Provider or MSA Owner | Capacity or Tokens | [`PaginatedPageUpdated`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.PaginatedPageUpdated) | 46 |
| `delete_page_with_signature_v2`<br />Deletes a specific page index with a signature authorization | Provider or MSA Owner | Capacity or Tokens | [`PaginatedPageDeleted`](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/enum.Event.html#variant.PaginatedPageDeleted)| 46 |

See [Rust Docs](https://frequency-chain.github.io/frequency/{pallet_name}/pallet/struct.Pallet.html) for more details.

### RPCs

Note: May be restricted based on node settings and configuration.

| Name | Description | Call | Node Version |
| ------- | ----------------- | ---------------------------------------------------------------------------------------------------- | ------------ |
| Get Paginated Storage | Retrieves the paginated storage for the given MSA Id and Schema Id | [`getPaginatedStorage`](https://frequency-chain.github.io/frequency/pallet_stateful_storage_rpc/trait.StatefulStorageApiServer.html#tymethod.get_paginated_storage) | v1.4.0+ |
| Get Itemized Storage | Retrieves the itemized storage for the given MSA Id and Schema Id | [`getItemizedStorage`](https://frequency-chain.github.io/frequency/pallet_stateful_storage_rpc/trait.StatefulStorageApiServer.html#tymethod.get_itemized_storage) | v1.4.0+ |

See [Rust Docs](https://frequency-chain.github.io/frequency/pallet_stateful_storage_rpc/trait.StatefulStorageApiServer.html) for more details.
31 changes: 3 additions & 28 deletions pallets/stateful-storage/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,38 +1,13 @@
//! # Stateful Storage Pallet
//! The Stateful Storage pallet provides functionality for reading and writing stateful data
//! representing stateful data for which we are only ever interested in the latest state.
//! MSA-centric Storage for [`PayloadLocation::Paginated`] and [`PayloadLocation::Itemized`]
//!
//! ## Quick Links
//! - [Configuration: `Config`](Config)
//! - [Extrinsics: `Call`](Call)
//! - [Runtime API: `StatefulStorageRuntimeApi`](../pallet_stateful_storage_runtime_api/trait.StatefulStorageRuntimeApi.html)
//! - [Custom RPC API: `StatefulStorageApiServer`](../pallet_stateful_storage_rpc/trait.StatefulStorageApiServer.html)
//! - [Event Enum: `Event`](Event)
//! - [Error Enum: `Error`](Error)
//!
//! ## Overview
//! For state transitions for which we only care about the latest state, Stateful Storage provides a way to store and retrieve such data
//! outside of the existing Announcement mechanism, which would require the latest state to be tracked using some kind of 3rd-party indexer.
//!
//! This pallet supports two models for storing stateful data:
//! 1. **Itemized:** Data is stored in a single **page** (max size: [`Config::MaxItemizedPageSizeBytes`]) containing multiple items (max item size `MaxItemizedBlobSizeBytes`) of the associated schema.
//! Useful for schemas with a relative small item size and higher potential item count. The read and write complexity is O(n) when n is the number of bytes for all items.
//! 2. **Paginated:** Data is stored in multiple **pages** of size [`Config::MaxPaginatedPageSizeBytes`], each containing a single item of the associated schema.
//! Page IDs range from 0 .. `MaxPaginatedPageId` (implying there may be at most `MaxPaginatedPageId` + 1 pages per MSA+Schema at any given time, though
//! there may be holes in that range). Useful for schemas with a larger item size and smaller potential item count.
//!
//! ## Features
//! * Provide for storage of stateful data with flexible schemas on-chain.
//! * Data stored for one MSA does not have impact read/write access times or storage costs for any data stored for another MSA.
//! * High write throughput.
//! * High read throughput.
//! * Data race condition protection
//!
//! The Stateful Storage pallet provides functions for:
//!
//! - Appending items in an **itemized** model
//! - Removing items in an **itemized** model
//! - Upserting items in a **paginated** model
//! - Removing pages in a **paginated** model
#![doc = include_str!("../README.md")]
//!
//! ## Terminology
//! - [`Page`](../pallet_stateful_storage/types/struct.Page.html): Block of on-chain data of a fixed size, which is the underlying type for Itemized and Paginated storage.
Expand Down