Storage in substrate -Part2

I will be discussing about each of the below storage methods in detail in this article.

• StorageValue to store any single value, such as a u64.
• StorageMap to store a single key to value mapping, such as a specific account key to a specific balance value.
• StorageDoubleMap to store values in a storage map with two keys as an optimization to efficiently remove all entries that have a common first key.
• StorageNMap to store values in a map with any arbitrary number of keys.

What Are Simple StorageValue?

Simple storage values in Substrate are a way to store individual data units efficiently on the blockchain. They are suitable for a wide range of use cases but require careful consideration of data size and performance to ensure smooth blockchain operation, especially in parachain projects.

Simple storage values are a way to store data on the blockchain. They are designed for values that are considered a single unit by the runtime. These values are commonly used for various purposes, including:

1. Single Primitive Values: This refers to storing basic data types like integers, booleans, or simple strings. For example, you might store a single number representing a user's balance.
2. Single Struct Data Type Objects: Substrate allows you to define custom data structures known as structs. You can use simple storage values to store instances of these custom structs when you need to represent a single unit of data with multiple attributes.
3. Single Collection of Related Items: If you have a group of related data items that logically belong together, you can use a simple storage value to store them as a single unit. For instance, a list of user accounts or a set of configuration parameters.

Considerations for Simple Storage Values:

When working with simple storage values, there are a few important considerations to keep in mind:

1. Size of Lists: If you intend to use simple storage values for lists of items, be cautious about the size of these lists. Large lists and complex structs can lead to increased storage costs on the blockchain.
2. Performance Impact: Iterating over large lists or structs in the blockchain's runtime can have a significant impact on network performance. If the time it takes to iterate over storage exceeds the block production time, it can disrupt the blockchain's operation.
3. Parachain Consideration: If your project is a parachain (a connected blockchain in the Polkadot ecosystem), exceeding block production time due to storage operations can result in the blockchain temporarily stopping its block production.

StorageValue Documentation:

To explore the full capabilities and methods of StorageValue, Lets refer StorageValue in frame_support::storage - Rust (paritytech.github.io)

Below define 3 different ways of storage items within a Substrate runtime module, each with its own type and accessibility which allows us to store and retrieve data on the blockchain in an organized and efficient manner, catering to various data types and runtime configurations.

1. SomePrivateValue:

#[pallet::storage] 
type SomePrivateValue<T> = StorageValue<_, u32, ValueQuery>;        

• SomePrivateValue is a storage item that stores a single value of type u32 in the blockchain's storage.
• The <T> is a generic type parameter, which means this storage item can work with various types of runtime configurations (T).
• StorageValue is a helper type provided by FRAME pallets. It's used to define a simple storage item that stores a single value. In this case, it's used to store a 32-bit unsigned integer (u32).
• ValueQuery is used to specify that the value can be queried (read) from storage.

2. SomePrimitiveValue:

#[pallet::storage] 
#[pallet::getter(fn some_primitive_value)] 
pub(super) type SomePrimitiveValue<T> = StorageValue<_, u32, ValueQuery>;        

SomePrimitiveValue is another storage item, similar to SomePrivateValue, but with a few differences:

It has a getter function specified using #[pallet::getter(fn some_primitive_value)]. This means that other parts of the code can access this value using some_primitive_value as a function call.

pub(super) indicates that this storage item is publicly accessible within the module (pub) but not outside of it (super limits access to the module itself).

Like SomePrivateValue, it stores a single value of type u32 and can be queried (ValueQuery).

3. SomeComplexValue:

#[pallet::storage] 
pub(super) type SomeComplexValue<T: Config> = StorageValue<_, T::AccountId, ValueQuery>;        

• SomeComplexValue is a more complex storage item compared to the previous ones.
• It's also publicly accessible within the module (pub(super)).
• The type parameter <T: Config> indicates that this storage item can be customized based on the runtime configuration (Config) of the blockchain.
• It stores values of type T::AccountId, which is typically an alias for user account identifiers in the runtime.
• Just like the previous storage items, it can be queried (ValueQuery).

Single Key Storage Maps in Substrate:

Single key storage maps in Substrate are a way to efficiently manage and access data where each element has a unique key. You can choose the hashing algorithm that best suits your needs, and Substrate provides documentation to help you understand how to use these maps effectively.

Single key storage maps are data structures used to manage sets of items where elements need to be accessed randomly, rather than sequentially. These maps work similarly to traditional hash maps, where each element has a unique key associated with it, allowing for efficient random lookups.

Key-to-Value Mapping:

• In a single key storage map, each item is associated with a unique key, and you can quickly find an item by specifying its key. It's like looking up information in a dictionary where each word has a unique definition.

Customizable Hashing Algorithm:

• Substrate provides flexibility by allowing you to choose the hashing algorithm you want to use for generating map keys. This means you can decide how the keys are calculated.
• For example, if you're storing sensitive data, you might opt for a hashing algorithm with stronger encryption properties, even if it has slightly lower performance, to ensure the security of your data.

Selecting the Right Hashing Algorithm:

• Choosing the right hashing algorithm is crucial, especially when dealing with sensitive or important data. You should consider factors like security and performance when making this choice.
• Substrate offers options for selecting the most suitable hashing algorithm for your specific use case.

StorageMap Methods:

• To work with single key storage maps in Substrate, Lets refer StorageMap in frame_support::storage - Rust (paritytech.github.io)

Example :

Below code defines a storage map named SomeMap within a Substrate runtime module. It's a map that associates user/account IDs (of type T::AccountId) with 32-bit unsigned integers. Users can use the some_map getter function to access and query the values stored in this map within the runtime. The data is stored using the Blake2_128Concat hashing algorithm for efficient key storage.

#[pallet::storage]
#[pallet::getter(fn some_map)]
pub(super) type SomeMap<T: Config> = StorageMap<
    _,
    Blake2_128Concat, T::AccountId,
    u32,
    ValueQuery
>;
        

1. #[pallet::storage]: This is an attribute macro in Substrate that marks the following item as storage for a runtime module. It's used to define a storage item within the pallet.
2. #[pallet::getter(fn some_map)]: This is another attribute macro that generates a "getter" function for the storage item. In this case, it generates a function named some_map that can be used to retrieve data from this storage map.
3. pub(super) type SomeMap<T: Config>: This line defines a new type named SomeMap. It's a generic type that takes a type T that implements the Config trait as a parameter. Config is a trait used in Substrate to configure the runtime.
4. = StorageMap<_, Blake2_128Concat, T::AccountId, u32, ValueQuery>: Here, you're specifying the actual storage type and its configuration:StorageMap: This is a storage item type provided by Substrate. It represents a key-value map that can be used to store data in the runtime's storage._, Blake2_128Concat, T::AccountId, u32, ValueQuery: These are type parameters for the StorageMap. Let's break them down:_: This is a placeholder for the prefix, which is automatically generated by Substrate.Blake2_128Concat: This is the hashing function used to hash the keys in the map. It's using the Blake2 hashing algorithm with 128-bit output and concatenation mode.T::AccountId: This is the type of keys in the map. T::AccountId is a type parameter provided by the runtime's configuration (Config), and it typically represents user or account identifiers.u32: This is the type of values stored in the map. In this case, it's a 32-bit unsigned integer.ValueQuery: This specifies the query type for the storage item, indicating that it can be queried using the some_map getter function to retrieve values.

Double Key Storage Maps:

In blockchain development, storage is crucial for keeping track of data. A "DoubleStorageMap" is a specialized data structure that is used to store information in a way that involves two keys instead of just one.

DoubleStorageMap is a data structure that stores information using two keys, making it particularly useful when you need to retrieve data based on two pieces of related information. It's a way to efficiently handle common queries in a blockchain application.

Example:

The below code defines a Double Map storage item named SomeDoubleMap. It allows you to store values associated with a combination of two keys: a u32 and an account ID (represented by T::AccountId). The hashing algorithms Blake2_128Concat are used to efficiently manage these keys in storage. This storage item is available within the module and its submodules and can be used to store and query data in the Substrate runtime.

#[pallet::storage]
pub(super) type SomeDoubleMap<T: Config> = StorageDoubleMap<
    _,
    Blake2_128Concat, u32,
    Blake2_128Concat, T::AccountId,
    u32,
    ValueQuery
>;        

Here StorageDoubleMap is a special type provided by Substrate for defining a storage item that uses a double map data structure.

Similar to Single Key Storage Maps:

DoubleStorageMap works similarly to a "Single Key Storage Map," which is a common way to store data on a blockchain. In a Single Key Storage Map, you have a key-value pair, where you can retrieve a value by providing a specific key.

Two Keys for Common Queries:

The key feature of a DoubleStorageMap is that it uses two keys. This is especially useful when you need to query or retrieve values based on two pieces of information that are commonly used together.

Example:

Imagine you're building a blockchain application for a library. In a Single Key Storage Map, you might store books by their ISBN (International Standard Book Number), and you can easily find a book by its ISBN.

However, if you want to add the ability to search for books by both ISBN and author's name, a DoubleStorageMap is handy. You can use one key for ISBN and another for the author's name. This way, you can efficiently query the database for books using either the ISBN or the author's name, or even both together.

Why It's Useful:

Using a DoubleStorageMap makes sense when you have data that is commonly looked up or queried using a combination of two attributes or keys. It helps optimize the way data is stored and retrieved, enhancing the efficiency of your blockchain application.

Link: StorageDoubleMap in frame_support::storage - Rust (paritytech.github.io)

Multi-Key Storage Maps:

In blockchain development, storage is a fundamental concept. It's where you store data that needs to be maintained on the blockchain. Storage maps are a way to organize and store data efficiently. They allow you to associate a key with a value, making it easy to retrieve and update data.

StorageNMap:

StorageNMap is a type of storage structure in Substrate (a blockchain development framework). It's similar to single-key and double-key storage maps, but with a significant difference: it enables you to define and use any number of keys for each entry in the map.

Defining Keys in StorageNMap:

To use StorageNMap, you need to specify the keys you want to use. These keys are defined as a tuple containing the NMapKey struct. The number of keys you specify in the tuple determines how many keys you can use for each entry in the map.StorageNMap in frame_support::storage - Rust (paritytech.github.io)

Example :

The below code defines a multi-key storage map named SomeNMap that can store u32 values. It uses three different key types with their corresponding hashing algorithms for indexing values in the storage map. This storage map is associated with a Substrate pallet and can be accessed within the current module and its submodules. The actual behavior and usage of this storage map would depend on the context and functionality of the Substrate runtime module in which it is defined.

#[pallet::storage]
#[pallet::getter(fn some_nmap)]
pub(super) type SomeNMap<T: Config> = StorageNMap<
    _,
    (
        NMapKey<Blake2_128Concat, u32>,
        NMapKey<Blake2_128Concat, T::AccountId>,
        NMapKey<Twox64Concat, u32>,
    ),
    u32,
    ValueQuery,
>;        

1. StorageNMap is the actual type of the storage item. It represents a storage map that can hold multiple values indexed by a combination of keys.
2. The next part defines the key types and their hashing algorithms used for indexing values in the storage map:NMapKey<Blake2_128Concat, u32>: This is the first key type. It uses the Blake2_128Concat hashing algorithm and is associated with u32 values.NMapKey<Blake2_128Concat, T::AccountId>: This is the second key type. It also uses the Blake2_128Concat hashing algorithm but is associated with T::AccountId values. T::AccountId represents the account identifiers used in your runtime, and it is parameterized by the runtime's Config.NMapKey<Twox64Concat, u32>: This is the third key type. It uses the Twox64Concat hashing algorithm and is associated with u32 values.

Handling Query Return Values in Storage:

In a blockchain's storage system, we can declare how queries should behave when they try to retrieve a value, but there's no value stored for the specified key. This is important because it allows us to handle cases where data might or might not exist in storage.

Three Query Options:

1. OptionQuery:When you use an OptionQuery, it's like asking for data from storage, but you're okay with the possibility that there might be no value stored at the specified location.If storage contains a value for the specified key, the query returns Some(value), where value is the stored data.If there's no value stored at that key, the query returns None.
2. ResultQuery:With a ResultQuery, you expect there to be a value stored at the specified key, and you want an error if there isn't one.If storage contains a value for the key, the query returns Ok(value), where value is the stored data.If there's no value stored at that key, the query returns an error.
3. ValueQuery:ValueQuery is used when you want to retrieve the data without any extra conditions or handling. You expect a value to be present.If storage contains a value for the key, the query simply returns value, where value is the stored data.You can also configure ValueQuery to return a default value if there's no data stored at that key. This can be useful if you've specified a default value for a storage item or if you've set up a value with the OnEmpty generic (a way to specify what to return when data is empty).

Practical Use:

• Let's say you have a blockchain storage system where users' account balances are stored. When querying a user's balance, you could use an OptionQuery if you want to gracefully handle the case where the user's balance hasn't been set yet (returning None). You could use a ResultQuery if you want to treat a missing balance as an error. Or, you could use a ValueQuery with a default value if you want to return a predetermined amount (e.g., zero) if the balance is missing.

These three query options (OptionQuery, ResultQuery, and ValueQuery) allow developers to handle different scenarios when querying data from a blockchain's storage system. They provide flexibility in dealing with situations where data may or may not exist, and they allow for various error-handling and default value strategies.

When we declare a storage item in a blockchain's runtime, we can choose how queries should respond to it. We can use OptionQuery if we are okay with the possibility of no data, ResultQuery if we expect data and want an error otherwise, or ValueQuery for straightforward retrieval with optional defaults. These choices allow developers to handle data retrieval and errors in a way that makes sense for their specific use cases.

Let me end this article here , will write a separate article on below topics which are demonstrated in the above examples :

1. Visibility of storage items
2. Getter methods provided by storage
3. Default Values
4. Accessing the storage items
5. Hashing algorithms .

Thanks for reading till end . Please comment of you have anything :

Referance : Runtime storage structures | Substrate_ Docs