How to subscribe to the balance change of multiple wallets

[realblackbot]

Hey py-substrate-interface community!

Polkadot.js clearly states it's possible (and even encouraged) to subscribe to events of multiple subjects:
https://polkadot.js.org/docs/api/start/api.query.multi/

Let's say I would like to track the balance change of two wallets, how could I do that using SubstrateInterface, as the following code for a single wallet doesn't work:
self._substrate.query( 'System', 'Account', [[walletaddress1, walletaddress2]], None, self.__on_balance_changed, None )
and query_map() doesn't have subscription support.

Could somebody point me to the right direction on how can I do such a Multi Query via SubstrateInterface please?
Thanks for any help in advance!

[arjanz]

It's possible, but because this library is blocking by nature (unlike PolkadotJS), you'll need to use threads with separate instances. Here's an example of how you can do it:

from concurrent.futures import ThreadPoolExecutor
from substrateinterface import SubstrateInterface


accounts = [
    "5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY",
    "5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty"
]


def track_account(account: str):
    
    print(f"Tracking {account} ..")

    with SubstrateInterface(url='ws://127.0.0.1:9944/', ss58_format=42) as substrate:

        def subscription_handler(account_info_obj, update_nr, subscription_id):
            print("Balance change:", account, account_info_obj.value)

        substrate.query("System", "Account", [account], subscription_handler=subscription_handler)


with ThreadPoolExecutor(4) as executor:
    executor.map(track_account, accounts)

[arjanz]

Sidenote after doing some research about subscription in Substrate: it is technically possible to subscribe to multiple 'storage keys' in one subscription, like PolkadotJs does, but this hasn't been implemented yet in py-substrate-interface. I will open a PR for this soon

[realblackbot]

Dear @arjanz, first of all: thank you very much for looking into it, I highly appreciate your help! 🙏
Am I right that the (really elegant) example code of your first reply would execute as many threads as many wallets I want to track, so it might work well with a few wallets, but probably not with thousands?

Opening a PR for a proper solution sounds amazing sir, thank you very much! 🙏

[arjanz]

@gergelykrasznai yes this wouldn't scale well for thousands of wallets, as this will need thousands of instances with each its own memory overhead and separate websocket connection (also, public endpoints probably won't accept this).

Implementing the multiple storage key support is the way to go, because this will need only one instance, one connection and even one subscription. This will take some refactoring of existing code, and won't be trivial, but it is on the short term todolist for sure.

[realblackbot]

@arjanz thank you very much once again! 🙏

[realblackbot]

(created and issue about this:
#327)

[arjanz]

Since v1.6.0 is now possible to subscribe to multiple storage keys in one instance:

def subscription_handler(storage_key, updated_obj, update_nr, subscription_id):
    print(f"Update for {storage_key.params[0]}: {updated_obj.value}")

# Accounts to track
storage_keys = [
    substrate.create_storage_key(
        "System", "Account", ["5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY"]
    ),
    substrate.create_storage_key(
        "System", "Account", ["5FHneW46xGXgs5mUiveU4sbTyGBzmstUspZC92UhjJM694ty"]
    )
]

result = substrate.subscribe_storage(
    storage_keys=storage_keys, subscription_handler=subscription_handler
)