Token Metadata extension to form standard

[afa7789]

Unified metadata: One place for account/faucet metadata: token (symbol, decimals, max_supply), owner, name, and content URI. Slot names live under miden::standards::metadata::* (and ownable for owner).

Layout: Token metadata and owner in slots 0–1; name in 2 words (name_0, name_1); content URI in 6 words (content_uri_0..5). Same layout in Rust and MASM.

Faucets: Basic and network fungible faucets support optional name and content URI; both re-export metadata getters (get_name, get_content_uri, get_token_metadata, get_max_supply, get_decimals, get_token_symbol; network also get_owner).

Standalone Info: Non-faucet accounts can use the metadata Info component (name + content URI) for future use (e.g. NFTs).

Testing: Unit tests in miden-standards (metadata storage, recovery); integration tests in miden-testing (MASM getters, faucet + metadata).

[onurinanc]

Why do we have both logo_uri & content_uri? we can rename it later for NFTs

[afa7789]

yeah, renaming it, u right.

[onurinanc]

Thank you. contract_uri would be a better option.

[onurinanc]

@afa7789 In the context of token metadata discussion #2423 : Let's keep the contractURI as optional.

Basically, instead of:

 name: Option<TokenName>,
 contract_uri: Option<TokenContractURI>

We need to have:

 name: TokenName,
 contract_uri: Option<TokenContractURI>

[onurinanc]

We should also consider adding the corresponding metadata and the new constructor to the network fungible faucets: https://github.com/afa7789/miden-base/blob/f7426116833b1f76da3195738ccb838a52880f80/crates/miden-standards/src/account/faucets/network_fungible.rs#L93-L101

[onurinanc]

@afa7789 we should also add a flag and procedure to change max_supply. It's basically similar to have we have done in optional_set_contract_uri

[afa7789]

@afa7789 we should also add a flag and procedure to change max_supply. It's basically similar to have we have done in optional_set_contract_uri

In this branch ? pr ?

[onurinanc]

@afa7789 we should also add a flag and procedure to change max_supply. It's basically similar to have we have done in optional_set_contract_uri

In this branch ? pr ?

Yes, it would be better if you can have this in this PR.

[afa7789]

@bobbinth @mmagician this is ready for review :)

[bobbinth]

Not a review - but I left a couple of comments inline. The main one is about how we can handle returning large amounts of data from account interface procedures.

[onurinanc]

@afa7789 Additionally, as this discussion #2423 (comment) has been concluded, you can update the PR with the following:

• name (mandatory): up to 2 words, ~64 bytes (UTF-8)
• description (optional): up to 6 words, ~192 bytes (UTF-8)
• logo_uri (optional): up to 6 words, ~192 bytes (UTF-8)
• external_link (optional): up to 6 words, ~192 bytes (UTF-8)

For description, logo_uri, and external_link:

• configuration flags will be introduced to indicate whether the account was initialized with these fields.
• mutable/immutable flags will also be added for these fields. (similar to how you did in content_uri)

[PhilippGackstatter]

This looks quite unwieldy because of all the optional fields. I would introduce a builder at this point, e.g.:

TokenMetadataBuilder::new(name, symbol, max_supply)
  .token_supply(...)
  .description(...)
  .logo_uri(...)
  .external_link(...)
  .build()?;

Token supply can default to 0 if not explicitly set as it does now, I think. Since symbol and name are so closely related, I would consider combining them (maybe in a separate PR) to further reduce the number of fields needed here. Though maybe this isn't really possible if these end up being part of different types (TokenMetadata and Info).

The builder as a whole could be done as a follow-up.

[PhilippGackstatter]

Is it correct to have max_supply_mutable in both of these? It seems like it should not be present in initialized_config.

I'm also wondering if we definitely need those. If we do need them, I would combine this data into a single slot, e.g. by using a word with the following layout:

felt 0: [62 zero bits | is_desc_mutable (1 bit) | is_desc_initialized (1 bit)],
felt 1: [62 zero bits | is_logo_mutable (1 bit) | is_logo_initialized (1 bit)],
...

But maybe we can do without the _initialized ones in any case. For instance, for token symbols we encode the length into the felt, and so we know that a Felt::ZERO never encodes a valid token symbol, because valid symbols must have length > 0. If we could do something similar for description, logo, extlink, could we not say that an encoded [Word::empty(); 6] means they are absent? Then we wouldn't need at least the _initialized slot.

[PhilippGackstatter]

Nit: I think this module is quite packed and the relation of each function to the respective type is not as easy to understand. I think it could be clearer if these were methods on the respective types, e.g. TokenName::to_words(&self) -> [Word; 2] and TokenName::from_words([Word; 2]) -> Result<Self, _>. Similarly for the other encoding/decoding functions. Wdyt?

[PhilippGackstatter]

Why is this not part of TokenMetadata directly?

It feels very closely related, so I would consider including it. Making it a separate component means users always need to remember to include it in their account next to TokenMetadata and need to decode both TokenMetadata and Info to get all the related data, so this pushes some complexity up the stack.

It would also be nice if we could set mutability flags directly via the mentioned TokenMetadataBuilder, e.g. TokenMetadataBuilder::new(...).description("abc").mutable_description().build().

Related, I think Info does not be an AccountComponent, since it does not have any code. This suggests it is a set of standard storage slots but not a full account component (a combination of functionality / code and storage). So in the same way as TokenMetadata is not an account component (but more like a standardized storage slot), we could make Info a reusable set of storage slots. I would then include it in TokenMetadata, which in turn is included in BasicFungibleFaucet (a proper account component). Notably, this does not prevent reusing Info for other purposes in the future (such as for NFTs).

Naming: I think this is more aptly described as TokenMetadata. This is more generic metadata than what is currently called TokenMetadata which is specific to fungible assets. So maybe it is better to rename the current TokenMetadata to FungibleTokenMetadata to free up that name for this.

[afa7789]

@PhilippGackstatter I did most of the mentioned things.
The exceptions are the Builder pattern and Generalized encoding, which will be left for the future.

[PhilippGackstatter]

Left a few comments. I'm primarily wondering if we need the on-chain metadata getters and I think we should turn TokenMetadata into a proper AccountComponent with code.

Separately, it would be awesome to split this PR into multiple ones given it adds 4500 lines of code in one go (even though 2100+ is test code).

Maybe we can try to pull the following changes into a separate PR X:

• Rename TokenMetadata to FungibleTokenMetadata unless the diff is very small).
• Add TokenName, ExternalLink, and related types and deal with the string encoding.

This PR (2439) can then be rebased on top of X.

[PhilippGackstatter]

I think we should apply the same pattern here as in #2292, where we initially had NetworkFungibleFaucet contain owner: AccountId and then refactored Ownable2Step into its own account component (in #2572). Here, we would move TokenMetadata(Info) out of BasicFungibleFaucet.

So we would have:

pub struct BasicFungibleFaucet {
    metadata: FungibleTokenMetadata,
}

When building an account, we'd pass BasicFungibleFaucet and TokenMetadata as two components. I think these don't depend on each other, right? So the faucet doesn't use code from the metadata and vice versa?

The only dependency is BasicFungibleFaucet needing token_supply, which is part of FungibleTokenMetadata, so it makes sense for that to stay where it is.

If that's true, then I think we can remove FungibleTokenMetadata::to_token_metadata_info and remove the new fields added in FungibleTokenMetadata (these are duplicates of the fields in TokenMetadata.

[PhilippGackstatter]

Related to a previous comment about TokenMetadata being its own account component: We can change this to a library that basically does pub use ::miden::standards::metadata::fungible::* (what crates/miden-standards/asm/account_components/faucets/network_fungible_faucet.masm currently re-exports) and use it as the library for TokenMetadata.

When we do this, we can remove the corresponding re-exports from basic and network fungible faucet, because these procedures would be provided by the TokenMetadata component instead.

The downside is that if we create an account with BasicFungibleFaucet + TokenMetadata then the account would also export the setters. If I understand correctly, these would fail at runtime because mutability would not be configured and so that's probably fine, albeit not ideal.

[PhilippGackstatter]

Question: Are there use cases for accessing this data on-chain? I'm wondering what a contract would do with a logo URI or an external link. The TokenSymbol we've had before doesn't have a getter either because it is not acessed on-chain. So, from what I can tell, this metadata is only accessed off-chain, and so we wouldn't need these getters.

The setters on the other hand are required for updating metadata, so these make sense to me.

[onurinanc]

@PhilippGackstatter as a part of the token standard, the on-chain data is a source of canonical trust, and we can not trust the off-chain data for some variables, especially this variables should be on-chain:

• name
• decimals
• symbol
• token supply

For the logo_uri  and external_link and description might be off-chain as long as you could provide one of them as a reference, such as we could just have contract_uri  would contain other metadata, and then we need to provide a scheme as we have discussed here: as we have discussed here: #2423

Then, we come to the point that instead providing a scheme for a contract_uri  we would provide logo_uri, external_link and description as optional slots.

CC: @bobbinth for this.

Additionally, I believe the naming that we use here as get_ is so much repetitive and although it defines exactly what the procedures does, we don't generally use get_ as part of the standards.

A part of an example ERC20 token standard in Ethereum:

    function name() public view virtual returns (string memory) {
        return _name;
    }

    function symbol() public view virtual returns (string memory) {
        return _symbol;
    }

    function decimals() public view virtual returns (uint8) {
        return 18;
    }

    function totalSupply() public view virtual returns (uint256) {
        return _totalSupply;
    }

A part of an ERC20 Standard in StarkNet for Cairo

impl ERC20Metadata<
        TContractState,
        +HasComponent<TContractState>,
        impl Immutable: ImmutableConfig,
        +ERC20HooksTrait<TContractState>,
    > of interface::IERC20Metadata<ComponentState<TContractState>> {
        /// Returns the name of the token.
        fn name(self: @ComponentState<TContractState>) -> ByteArray {
            self.ERC20_name.read()
        }

        /// Returns the ticker symbol of the token, usually a shorter version of the name.
        fn symbol(self: @ComponentState<TContractState>) -> ByteArray {
            self.ERC20_symbol.read()
        }

        /// Returns the number of decimals used to get its user representation.
        fn decimals(self: @ComponentState<TContractState>) -> u8 {
            Immutable::DECIMALS
        }
    }

I believe you should also consider using a general naming convention of OpenZeppelin in the naming by removing get_

[PhilippGackstatter]

This is related to the limitation of returning 16 elements through a call: For instance, you get the commitment via call.get_description_commitment which also insertes the preimage into the advice map. Then you pipe the the data from the advice map into memory and check that it matches COMMITMENT.

I think the approach here using get_description_commitment and get_description wouldn't quite work like that because get_description must be exec-uted, and so executes in the context of the caller, not the account, but internally it calls procedures of the account (get_description_chunk_0).

My question is whether we need any of this at all if there is no use to access this data on-chain.

[onurinanc]

My question is whether we need any of this at all if there is no use to access this data on-chain.

@PhilippGackstatter I think this should answer your question: #2439 (comment)

[PhilippGackstatter]

the on-chain data is a source of canonical trust, and we can not trust the off-chain data for some variables

That makes sense! To be clear: It makes sense to me that this data must be stored on-chain. What I'm asking is if the data must also be accessed on-chain. For access, we need the getters, but for storage we don't.

Off-chain (say in the Miden explorer to display metadata), you'd access the data by syncing the faucet account (and its storage) via the client and then use something like TokenMetdadata::try_from_storage(faucet_account.storage())?, e.g. like here:

protocol/crates/miden-testing/tests/scripts/faucet.rs

Lines 288 to 292 in 69f0c9e

	let token_metadata = TokenMetadata::try_from(faucet.storage())?;
	// Check that max_supply at the word's index 0 is 200. The remainder of the word is initialized
	// with the metadata of the faucet which we don't need to check.
	assert_eq!(token_metadata.max_supply(), Felt::from(max_supply));

So the MASM getters would not be involved in this at all.

I believe you should also consider using a general naming convention of OpenZeppelin in the naming by removing get_

I'm not sure about this. In Rust, I like avoiding the get_ prefix (as is convention), e.g. token_metadata.token_supply(). In MASM, we don't have methods, only procedures. fungible::get_token_supply is a bit clearer than fungible::token_supply, though the latter is fine.

We currently use get_ consistently (afaict), but if we change this it would be great to do this across all APIs. Then I'm not sure if active_account::get_item can be replaced by active_account::item without loosing clarity.

[onurinanc]

Off-chain (say in the Miden explorer to display metadata), you'd access the data by syncing the faucet account (and its storage) via the client and then use something like TokenMetdadata::try_from_storage(faucet_account.storage())?, e.g. like here:

I got your point! It's a nice point! For, description, logo_uri, and external_link, I don't think there are much use cases to access the data on-chain. So, I think removing the getters would be better.

I'm not sure about this. In Rust, I like avoiding the get_ prefix (as is convention), e.g. token_metadata.token_supply(). In MASM, we don't have methods, only procedures. fungible::get_token_supply is a bit clearer than fungible::token_supply, though the latter is fine.

We currently use get_ consistently (afaict), but if we change this it would be great to do this across all APIs. Then I'm not sure if active_account::get_item can be replaced by active_account::item without loosing clarity.

That's totally clear to me now.

[onurinanc]

Related to this: #2585, I think we need name, symbol, and symbol to be accessed on-chain to verify the correct hash, or is it still satisfied by an off-chain access? @PhilippGackstatter @mmagician

[PhilippGackstatter]

Good catch, it looks like that is a use case where we'd need to access name, symbol and decimals on-chain.

Then I think we should drop the description, logo_uri and external_link getters.

[onurinanc]

@afa7789 could you remove the getters for description, logo_uri and external_link when possible?

[PhilippGackstatter]

I think overall we can proceed with two approaches for the metadata structure. In general, I believe the goal was to have TokenMetadata contain token metadata that is not specific to fungible or non-fungible assets, so we can reuse it easily in the future.

Approach 1

Keep fungible metadata in BasicFungibleFaucet and extend it with TokenMetadata.

So we'd have:

• FungibleTokenMetadata: Helper struct
• TokenMetadata: Account component
• BasicFungibleFaucet: Account component containing FungibleTokenMetadata

Metadata functionality would be split in two:

#! Re-exported from module miden::standards::components::token_metadata
pub use ::miden::standards::metadata::fungible::get_name
pub use ::miden::standards::metadata::fungible::get_mutability_config
pub use ::miden::standards::metadata::fungible::is_max_supply_mutable
pub use ::miden::standards::metadata::fungible::is_description_mutable
pub use ::miden::standards::metadata::fungible::is_logo_uri_mutable
pub use ::miden::standards::metadata::fungible::is_external_link_mutable
pub use ::miden::standards::metadata::fungible::optional_set_description
pub use ::miden::standards::metadata::fungible::optional_set_logo_uri
pub use ::miden::standards::metadata::fungible::optional_set_external_link
pub use ::miden::standards::metadata::fungible::optional_set_max_supply

#! Re-exported from module miden::standards::components::faucets::basic_fungible_faucet
pub use ::miden::standards::metadata::fungible::get_token_metadata
pub use ::miden::standards::metadata::fungible::get_max_supply
pub use ::miden::standards::metadata::fungible::get_decimals
pub use ::miden::standards::metadata::fungible::get_token_symbol
pub use ::miden::standards::metadata::fungible::get_token_supply

Right now we export all of these from the faucet components, but since fungible metadata is defined by the faucets rather than TokenMetadata, it makes sense to re-export it from there.

Approach 2

Unify generic and fungible metadata in FungibleTokenMetadata.

So we'd have:

• TokenMetadata: Helper struct
• FungibleTokenMetadata: Account component that wraps TokenMetadata and re-exports all its functionality and storage slots
• BasicFungibleFaucet: Account component without storage slots (pub struct BasicFungibleFaucet;). It would depend on FungibleTokenMetadata being present in the account.

Metadata functionality would be unified:

#! Re-exported from module miden::standards::components::fungible_token_metadata
# All of the above mentioned metadata-related procedures are re-exported here.
pub use ::miden::standards::metadata::fungible::*

The second approach results in a slightly more cohesive DevX, since all metadata is accessible via one Rust type and via one component's procedures, so I think my preference is slightly on that approach.

[PhilippGackstatter]

Should we drop the optional prefix? I think we don't need to indicate in the name that this only works if the mutability flag is set.

[PhilippGackstatter]

This module is pub and contains many pub free functions/statics/consts. I think we should try to reduce the things that are public to the necessary ones:

• The statics that define slot names should be exported by the account components that have these slots, e.g. TOKEN_METADATA_SLOT as FungibleToken::token_metadata_slot. So we can make miden_standards::account::metadata::token_metadata_slot and TOKEN_METADATA_SLOT private or at most pub(crate). The same may apply to the other slot names.
• A few functions are unused (but do not get flagged as unused because the module is public), e.g. field_from_bytes, LOGO_URI_DATA_KEY, name_from_utf8.

I'd also suggest splitting the TokenMetadata into its own module since this module is large now and contains the unrelated AccountSchemaCommitment.

[afa7789]

@PhilippGackstatter , I have coded the suggestions you made, and reviewed here on my side, you mind taking a look?

[bobbinth]

Once #2603 is merged, this part of the diff should go away.

[afa7789]

ok

[bobbinth]

How is this related to the FungibleTokenMetadata component? It seems like there is quite a bit of duplication across these two.

[bobbinth]

Instead of (or maybe in addition to) having these individual types, we could create a generic type for representing "fixed-size" strings. Something like: FixedWidthString. It could look something like:

pub struct FixedWidthString<const N: usize>(Box<str>);

impl<const N: usize> FixedWidthString<N> {

    pub fn new(value: &str) -> Result<Self, FixedWidthStringError> {
        ...
    } 

    pub fn as_str(&self) -> &str {
        ...
    }
    
    pub fn to_words(&self) -> Vec<Word> {
        ...
    }

    pub fn try_from_words(words: &[Word]) -> Result<Self, FixedWidthStringError> {

    }
}

Then, we could define the metadata as something like:

pub struct FungibleTokenMetadata {
    ...
    name: FixedWidthString<2>,
    description: Option<FixedWidthString<6>>,
    logo_uri: Option<FixedWidthString<6>>,
    external_link: Option<FixedWidthString<6>>,
    ...
}

Or, if we wanted to enforce similar type safety as now, we could do:

pub struct TokenName(FixedWidthString<2>);

[bobbinth]

I don't think this is the correct approach for how we should set description (and other similar data). What we want is to pass in the hash of the data we want to set and then inside the procedure "unhash" this data by using the advice provider. Specifically, the inputs/outputs for this procedure could look something like:

#! Inputs:
#!    Stack: [DESCRIPTON_HASH, pad(12)]
#!    Advice map: {
#!        [DESCRIPTON_HASH] |-> [description_elements]
#!    }
#! Outputs:
#!    Stack: [pad(16)]

[afa7789]

I will change this + new generic type for representing "fixed-size" strings.
Also, fix the mistakes on the error approach I made when merging the fungible and metadata, and will review more extensively.