This article was originally posted by me on November 28, 2021 at thiscoindaily
This is the first part of a series where I take a deep dive into some rust techniques that you’ll need to understand in order to write a more elegant and hassle-free substrate code.
I’ve received numerous messages from aspiring (and new) substrate developers who are often confused with rust and how its logic is structured in substrate. This confusion comes mainly from developers who haven’t worked with a low-level programming language like C/C++ or who haven’t worked with any programming language at all.
Well, freight not!
In this series, I’ll be introducing you to some important rust techniques and how they’re used in substrate. Understanding these techniques and working with some examples of how they’re used on substrate will go a long way in helping you understand how the various components work together.
For this article, we’ll be focusing on macros. Yeah! those defogging lines of code you often find above things like structs/impl, functions, etc.
To be fair, macros have been a big nightmare for new substrate developers as a lot of them have complained of a poor understanding of what’s going on with their codes.
At the end of this article, you should have understood what macros are and how substrate uses them to help you write less code. You should also be able to decide on which macros to use while writing on substrate, as well as how to avoid substrate errors that can result from inappropriate use of macros.
Note: It’s advisable that you have some knowledge of substrate and rust before you undertake this series. This series will contain some advanced techniques which you might have a hard time grasping if you don’t know some basic rust syntax and general knowledge of substrate.
If this is your first exposure to substrate and rust, I’d suggest you undertake the first two steps of this tutorial. But as always, you can continue with this tutorial if you feel comfortable.
For this tutorial, we’ll be using substrate playground for demonstrations. hence, ensure you set up a node on the playground to fully follow along.
Are you ready? let’s fire on!
First of all, what exactly are macros?
Before proceeding with how substrate uses macros, Let’s dive into a general understanding of what macros are.
Basically, macros are lines of code that help generate code from existing code (that’s a bit redundant haha). The new codes that macros generate could range from simple variables to more complex things like functions, structs, or even other macro attributes.
It’s quite common to refer to a macro as being similar to a function, but there are a lot of differences between them. Of course, declarative and function-like macros are called like function calls, but the logic of how Macros and functions behave, are two entirely different things.
There are two broad types of macros, each with its own suitable use-cases. these include
- Declarative macros and
- procedural macros ( which include custom-derived macros, attribute-like macros, and function-like macros)
Substrate uses macros heavily for runtime development. This is not to say that every developer must use the macros while writing code (as a matter of fact, developers can ditch some macros and write their own rust code from scratch). However, substrate macros help you write way lesser code while focusing on the stuff that matters.
About declarative macros
Declarative macros are the most commonly used types of macros. Basically, declarative macros use matching to compare written codes with patterns specified in the macro definition. if the structure of parts of your codes matches any of these patterns, that part of your code will be replaced with the codes associated with that pattern. a good example of declarative macros native to rust is the Vec! macro.
let’s take a look at an example. Run the code below on your rust environment (you can use rustplayground if you don’t have an environment set up.
#[macro_export]
macro_rules! substrate_printer {
($x:expr) => (println! ("{}", $x));
}
fn main() {
substrate_printer!("substrate_printer macro helped print this");
}Let’s break it down.
The first line ( #[macro_export] ) ensures that anytime you bring the crate that contains this macro definition into scope, the macro should be made available for use.
The second line contains the macro_rules! construct (which must be written at the beginning of the macro definition), and the name of the macro (without the exclamation mark)
The third line contains the actual pattern (to the left of the =>) and the code it should use for replacement when matched (to the right of the =>) in this case, we’re specifying that if the argument passed unto the macro is an expression (expr), that expression should be printed. Note that multiple patterns can be specified.
Note: substrate rarely makes use of declarative macros. Rather, it makes heavy use of function-like macros which are kind of similar to declarative macros but allow for a higher level of flexibility. see below.
About procedural Macros
whereas declarative macros simply use matching to replace your code with other codes when patterns are found, procedural macros do things differently. they do not perform matching against code inputs. Rather, they take in a code input (of type TokenStream), perform some operations on the code, and return another TokenStream as output.
Also, unlike declarative macros, procedural macro definitions must be defined inside their own crate
Before writing a procedural macro, you must
- Import the crate that contains the TokenStream type (proc_macro).
- Add an attribute that specifies the type of procedural macro we’re creating.
The code below shows a general skeleton of how procedural macros are structured.
use proc_macro; //to bring the TokenStream type into scope
#[some_attribute] // specify the type of procedural macro here
pub fn func_name(input: TokenStream) -> TokenStream {
//-- snip --
}Let’s now take a look at individual types of procedural macros
Custom derive macros
There are a lot of times when we have to implement a certain trait for various types. to implement traits for each type, the trait implementations have to be specified individually. This is quite redundant (don’t you think?). Custom derive macros help avoid this kind of redundancy by allowing you to create a macro that applies default implementations on multiple types at once.
Note that custom derive macros only work with structs and enums.
All you have to do is annotate your types with the macro that contains the trait you’d like to use. the annotation will look like below
#[derive (TraitName)]
// define your struct or enum types herewhere TraitName is the name of the trait which will be implemented on the types by default.
Note that for any procedural macro, the crate that contains the procedural macro must be different from the crate that contains the actual trait definitions. This is definitely not the best design choice but was made that way for a reason.
In general, to create and use a custom derive macro, you’ll have to
- Create a new library (let’s call it traits_library).
- Define the trait and its functions inside the projects, lib.rs file.
- Define your procedural macro within the directory of the new library you created (remember, it has to be in its own crate. hence, you’ll have to create a new library for your macro. conventionally, the name of your macro takes the name of the traits library appended with “_derive”.
- Declare your macro library as a procedural macro. this is done within your macro library’s cargo.toml file, by changing proc-macro to true.
- define your procedural macro.
- use your macro. To use your macro, you must add the two crates created for your macro into your project dependencies. you then have to bring them into scope in your projects lib.rs file as well as import some necessary crates. in general, the usage of a custom derive macro will have the structure below.
use traits_crate::Trait_Name;
use traits_crate_derive::Trait_Name;
#[derive(Trait_Name)]
//specify your types here
fn main() {
Type::Trait_Name();
}Examples of derive macros used in Substrate are the Deserialize and Serialize in sp_runtime crate and the DefaultNoBound macro in substrate’s frame-support library
Attribute-like Macros
Attribute-like Macros allow you to create new attributes. You can then use these new attributes to generate other codes. These new attributes will follow principles similar to the custom derive macro. However, unlike the customer derive macro which only works with struct and enum types, attribute-like macros can work with other types, and even with functions.
This is very useful as you have the autonomy to name your attributes depending on their intended use.
A good example of substrate’s implementation of Attribute-like Macros is the pallet macro. we’ll be diving deep into the pallet macro in the next section. for now, let’s have a quick look at function-like macros.
function-like macros
As the name implies, function-like macros are called as you’d call a function (the only difference being the exclamation mark in front of the macro name). They are similar to declarative macros, but they don’t use matching. Instead, they act like the two previous procedural macros we discussed; they take in a TokenStream (your rust code as input), perform some operations, and pass out another TokenStream (expanded rust code) as output.
To use a function-like macro, you add an exclamation mark in front of the macro name and input your parameters inside the parenthesis (just as you’d do with functions). A common macro that’s used every day by rustaceans is the println! Macro.
The substrate systems library makes heavy use of function-like macros, especially the serde, sp_api, and sp_std crates. In general, function-like and custom derive macros are the most commonly used macros in the substrate systems library.
How to use use Substrate macros
Function-like and custom derive macros are currently the most commonly used macros in substrate (Although we may see some heavy use of attribute-like macros in the future).
To demonstrate how macros are used on substrate, we are going to use the frame-support library’s pallet macro and the pallets template as an example. To follow along, set up a new substrate node here. If you already have a local node set up, feel free to use that too.
As with the convention, to use any substrate macro, you first have to add its crate (or the library containing its crate) as a dependency on the project’s cargo.toml file (pallets/template/cargo.toml )
[dependencies.frame-support]
default-features = false
git = 'https://github.com/paritytech/substrate.git'
tag = 'monthly-2021-09+1'
version = '4.0.0-dev'Notice that in this case, the entire frame-support crate is added as a dependency. However, individual macros could be added as dependencies if required.
then you’ll have to bring the pallet’s macro into scope in your lib.rs file. (pallets/template/src/lib.rs in this case)
// -- snip --
pub use pallet::*;
// --snip--
#[frame_support::pallet]
pub mode pallet {
// --snip--
#[pallet::Config]
// --snip--pub use pallet::* ensures that all the attributes created by the pallet macro are made available for use.
Once your macro has been brought into scope, you can then use individual attributes by using #[pallet::Attribute_Name]
There is a whole list of attributes that can be used with the pallet macro. We shall talk about how to use the right attributes in a later section.
Here’s a quick exercise for you: Go to the runtime’s lib.rs file (runtime/src/lib.rs). try to recognize the implementation of any of the substrate macros we highlighted earlier. what were the frame support library and system library macros that were brought into scope?
The process that was followed above when trying to use the pallet macro is the same process that’ll be followed for any other substrate macro;
- Add the crate containing the macro to the project’s cargo.toml file
- Bring the macro into scope in your project’s lib.rs file
- Implement the macro, depending on what it’s meant to be used for.
Deciding on which macros to use
Honestly, this only improves with practice and the recognition of patterns. In the beginning, I’d highly suggest you save up some patterns in a separate file. that way, you could just copy and paste those macros when needed (although, you might need to make some customizations in some cases).
For example, to create a struct while creating a custom pallet, you’ll need the #[pallet::pallet] , #[pallet::generate_storage_info] and #[pallet::generate_store(pub(super) trait Store)] (optional) macros
#[pallet::pallet]
#[pallet::generate_store(pub(super) trait Store)]
#[pallet::generate_storage_info]
//your struct hereHence, you could save the above somewhere and use it whenever you need to create a new struct.
As you become more advanced, a lot of these implementations become muscle memory. You could always check out substrate docs to learn more about individual macros, as well as situations in which various macros are applicable.
How to avoid Macro errors
In my experience, the three most common reasons for Macro errors are the usage of the wrong macros, syntax error when passing arguments into macros, and failure to bring required macros into scope.
To avoid macro errors, always remember to
- Add the crates containing your macro as a dependency in the project’s cargo.toml file.
- Bring the macro to scope.
- Use the right macros (when in doubt, check for previous patterns, google for answers or check the substrate docs )
- Always cross-check the arguments you pass into the macros.
Conclusion
In this article, we gave a general overview of substrate and rust macros. By now, you should have had a better idea of how substrate macros help you write fewer lines of code, how to go about implementing these macros in your projects, as well as how to decide on which macros to use. to learn more about Substrate macros, check out substrate docs.
Happy coding!