Skip to content

feat: migrate storage-client and provider-node to typed subxt bindings via storage-subxt#239

Open
danielbui12 wants to merge 27 commits into
devfrom
subxt_runtime_metadata
Open

feat: migrate storage-client and provider-node to typed subxt bindings via storage-subxt#239
danielbui12 wants to merge 27 commits into
devfrom
subxt_runtime_metadata

Conversation

@danielbui12

@danielbui12 danielbui12 commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Changes

  • New crate storage-subxt: generated subxt typed bindings for the runtime, re-exporting subxt and subxt_signer as the single source of truth for those deps across the workspace
  • storage-client
    • Replaced all subxt::dynamic::tx/storage/constant calls with typed runtime::tx() / runtime::storage() / runtime::constants() builders
    • Added runtime_convert.rs: byte-exact conversion layer between domain types (AccountId32, H256, MultiSignature, AgreementTerms, Role, EndAction, proof structs) and generated runtime_types::*
    • Replaced per-file manual scale_value decoders with direct typed-struct field access
    • Migrated event_subscription.rs to typed as_event::<ev::*>() decoding; deleted scale_decode.rs
    • Removed direct subxt/subxt-signer deps from client/Cargo.toml; all imports now flow through storage_subxt::subxt / storage_subxt::subxt_signer
  • provider-node
    • SubxtChainClient rewritten: holds a SubstrateClient (from storage_client) instead of raw OnlineClient + Keypair; all coordinator trait impls use typed storage::* / extrinsics::*
    • ChainMembershipResolver in auth.rs replaced dynamic Buckets query + scale_value decoding with typed storage::bucket_info() + rt::storage_primitives::Role matching
    • Added three new functions to client/src/substrate.rs: extrinsics::update_provider_multiaddr, extrinsics::provider_checkpoint, storage::checkpoint_config

Issue

@danielbui12 danielbui12 self-assigned this Jun 25, 2026
@danielbui12 danielbui12 marked this pull request as ready for review June 25, 2026 10:58
@danielbui12 danielbui12 requested a review from bkontur June 25, 2026 10:59
@bkontur

bkontur commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

@danielbui12 please, make some TODO to check also using storage queries in TS SDK e.g.: 48d7560#diff-9c9ec00799f06105758439222211e7ef2fc782460416fb49c8234c3c1a8305efR174

@danielbui12

danielbui12 commented Jun 26, 2026

Copy link
Copy Markdown
Member Author

@danielbui12 please, make some TODO to check also using storage queries in TS SDK e.g.: 48d7560#diff-9c9ec00799f06105758439222211e7ef2fc782460416fb49c8234c3c1a8305efR174

I created an issue #243

@danielbui12

danielbui12 commented Jun 29, 2026

Copy link
Copy Markdown
Member Author

/cmd fmt

@danielbui12 danielbui12 mentioned this pull request Jun 29, 2026
Closed
@danielbui12
feat(runtime-api): version StorageProviderApi v2 with bucket_id, chal…
7d38ac4 
…lenge index, and challenge queries

Bump StorageProviderApi to #[api_version(2)] and extend its read surface so
clients can recover double-map keys and enumerate challenges via the typed
runtime API instead of hand-parsing raw storage-key bytes.

- AgreementResponse gains `bucket_id`; ChallengeResponse gains `index`
  (position within a deadline block's Vec<Challenge>), populated in
  query_agreement_info/query_bucket_agreements/query_provider_agreements and
  query_challenges_at.
- Add bucket_challenges(bucket_id) and provider_challenges(provider) following
  the existing bucket_agreements/provider_agreements pattern; wired into both
  the local and paseo runtime impl blocks.
- Cover the new fields and queries with pallet unit tests.
@danielbui12
feat(runtime-api): add challenger_challenges to StorageProviderApi v2
fc7aca5 
Add a symmetric challenger_challenges(challenger) -> Vec<ChallengeResponse>
method alongside the existing bucket_challenges/provider_challenges, completing
the key-scoped challenge query surface. This serves list_my_challenges which
filters by the challenger field and has no other clean runtime-API mapping.

Implements query_challenger_challenges in the pallet (identical shape to
query_provider_challenges, filtering on challenge.challenger). Wired into both
the local and paseo runtime impl blocks. Test added: 194 pallet tests pass.
@danielbui12
refactor(client): query providers via runtime API in discovery.rs
0f19c56 
Replace the hand-rolled Providers storage-key byte parsing in
DiscoveryClient with typed StorageProviderApi calls, removing the
account_ss58_from_key helper that decoded the SS58 account out of raw
key bytes at fixed offsets.

- list_providers now calls providers(offset, limit).
- providers_with_capacity now calls providers_with_capacity(bytes_needed,
  offset, limit) (server-side filter also enforces stake sufficiency).
- Return type changes from (String, ProviderInfo) to
  (String, ProviderInfoResponse); integration tests updated to the flat
  response fields.

Includes the regenerated storage-subxt bindings (api_version 2) the
client now compiles against.
@danielbui12
refactor: replace manual storage-key parsing with StorageProviderApi
396a8ba 
Migrate all client and provider-node sites that hand-parsed storage keys
to the versioned StorageProviderApi runtime API:

- discovery.rs: enumerate providers via providers(offset,limit)
- admin.rs: list_bucket_agreements via bucket_agreements; return
  AgreementResponse directly (drop AgreementInfo wrapper)
- provider.rs: provider_agreements / provider_challenges; drop redundant
  ChallengeInfo.deadline (already in ChallengeResponse)
- challenger.rs: challenger_challenges; paginated provider enumeration
- subxt_client.rs: provider_agreements for replica discovery; delete
  extract_bucket_if_provider
- storage-subxt: AgreementResponse::is_primary helper
@danielbui12
Merge branch 'dev' into subxt_runtime_metadata
d1e37b0
bkontur
bkontur reviewed Jun 29, 2026
View reviewed changes
Comment thread pallet/src/lib.rs Outdated
.collect()
}

/// Query all challenges for a specific bucket.

@bkontur bkontur Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielbui12 ok, let's push this PR, I suggest couple of things:

  • create impls.rs and move all this functions impl<T: Config> Pallet<T> { there
  • create separate PR just for this changes touching pallet + runtimes (looks like just introducing new runtime apis)

And we continue with review there :)

@danielbui12 danielbui12 Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkontur I created a follow-up PR here #256

bkontur
bkontur reviewed Jun 29, 2026
View reviewed changes
Comment thread pallet/src/runtime_api.rs
bkontur
bkontur reviewed Jun 29, 2026
View reviewed changes
Comment thread pallet/src/lib.rs Outdated
bkontur
bkontur reviewed Jun 29, 2026
View reviewed changes
Comment thread pallet/src/lib.rs Outdated
bkontur
bkontur reviewed Jun 29, 2026
View reviewed changes
Comment thread pallet/src/runtime_api.rs
pub leaf_index: u64,
pub chunk_index: u64,
pub deadline: u32,
pub index: u16,

@bkontur bkontur Jun 29, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielbui12 hmm, not sure if we need this index it is just random index which is generated from enumerate - I would remove that, but if we need this for some client processing, we should have some unique dedicated hash/id - for example if you delete challange (are they even deleted?) the index will change or something. But let's continue with separate PR mentioned above for pallet/runtimes changes

@danielbui12 danielbui12 Jun 30, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    pub type Challenges<T: Config> =
        StorageMap<_, Blake2_128Concat, BlockNumberFor<T>, Vec<Challenge<T>>>;

concretely, Challenges is a mapping from block number to a vector of Challenge.
hence index is necessary to identify which Challenge are we want to interact with.

@danielbui12 danielbui12 Jul 2, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: #125 already migrated Challanges to

  pub type Challenges<T: Config> = StorageDoubleMap<
        _,
        Blake2_128Concat,
        BlockNumberFor<T>,
        Twox64Concat,
        u16,
        Challenge<T>,
        OptionQuery,
    >;

now it is more intuitive that index: u16 is essential

@danielbui12 danielbui12 mentioned this pull request Jun 30, 2026
Closed
@bkontur bkontur mentioned this pull request Jul 1, 2026
Open
@danielbui12 danielbui12 mentioned this pull request Jul 1, 2026
Merged
@RafalMirowski1 RafalMirowski1 mentioned this pull request Jul 2, 2026
Open
@danielbui12 danielbui12 requested a review from bkontur July 3, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bkontur bkontur Awaiting requested review from bkontur

At least 1 approving review is required to merge this pull request.

Assignees

@danielbui12 danielbui12

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Migration: subxt::dynamic → typed storage_subxt bindings Expose interfaces via a separate RuntimeApi

2 participants

@danielbui12 @bkontur