Lazy payouts

[gavofyork]

Closes: #4407

TODO: status of PR updated just below

[gavofyork]

CC @thiolliere

[gui1117]

Let's write the current status of the PR here:

Context

This PR address:

• Lazy payouts #4407
• In staking rewards go to next validator set #4121
• Validator with apparently zero backing chosen #4331
• Figure out why Phragmen is taking so long for staking #4313
• ValidatorPrefs Updates Should Apply at Next Era #4220
  In the current implementation validators can change their preference at the same time as nominators are nominating. Thus if the validator change its preference just before the staking is fixed for the era then it can "steal" all the reward for one era.
  But nominators will probably not nominate him anymore.
  We can introduce a delay for validators to change their preference but then we need to copy all validators preferences in a storage map "buffer". which is fine.
• Superseed: Fix staking reward issue #4346

as explained in #4407 the architecture of this module has changed.
for the 84 last eras we keep information of:

• exposures (what is staked by nominators and validators) in ErasStakers
• exposures clipped (staked information used for rewards (used to limit i/o for nominator reward)) in ErasStakersClipped
• validator commissions in ErasValidatorPrefs
• era total payout in ErasValidatorReward
• era total stake in ErasTotalStake
• era start session index in ErasStartSessionIndex
• eras reward points (which validator have how many points) in ErasRewardPoints

We also keep track of the latest era planned and the currently rewarded era in CurrentEra and ActiveEra respectively.

When the active era is finished we can compute the total payout of this era (as we have its duration) and stakers can claims their reward.

To claim a reward stakers have two new call: payout_nominator and payout_validator, those calls allow to claim the reward for a specific era. A staker can't claim the reward for an era lower that a previously claimed one (claim of era must be made in the order of the era).

Done

• implementation
• upgrade of test
• migration
• removal of slot stake storage as it is not used anywhere
• (a few removal related to the removal of feature equalize)
• rewards_by_indices function has been removed. The reward points structure now use the validator id so this optimisation doesn't make sense anymore. But we could keep it and document that rewards_by_ids is more efficient.
• the reward is limited to the T::MaxNominatorRewardedPerValidator biggest nominators, this is to limit the i/o cost of computing nominators payout.

TODO

• test from kusama dump as stated in Lazy payouts #4474 (comment)
  those are made in the branch https://github.com/paritytech/polkadot/compare/gui-lazy-reward-test
  specifically in the commit paritytech/polkadot@435409c
  with substrate branch gav-lazy-rewards...gui-gav-lazy-rewards-test-kusama-upgrade
• add another test as stated in Lazy payouts #4474 (comment)

Note for reviewer

The logic is all implemented and is ready for review.

[gui1117]

(It should explicitly report so that the number of sessions that the validator set lags behind continues to be immaterial for the Staking module.)

note: this is difficult to do because we have to compute the duration of the era and tell when the era is fully rewarded and we can make payments for it.
but it is ok.

[gui1117]

this is a draft that should make the API for session more explicit #4609

[shawntabrizi]

This looks good. I think we should still combine the one storage item I pointed out.

[gnunicorn]

I think we should still combine the one storage item I pointed out.

is this intended to happen before the merge? adding grumble, please remove once done/if not to be done.

[shawntabrizi]

@gnunicorn all is good now!