From 81825fd29fbf976ca19edaadc8f9445fdd2d833d Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Mon, 31 Oct 2022 22:51:20 +0100 Subject: [PATCH 01/14] Add `engine_newPayloadHeaderV1` While `engine_forkchoiceUpdatedV1` specifies that passing an unknown `headBlockHash` MAY trigger a sync, EL implementations practically require `engine_newPayloadV1` to be called as well. This is prohibitive for applications where the CL is a light client that does not have the capabilities to obtain full beacon block data. To better support that scenario, a new endpoint is proposed to provide the EL client software with just the `ExecutionPayloadHeader`, therefore simplifying integration with existing sync logic. Furthermore, this unlocks additional experiences: 1. A CL that is syncing optimistically after a prolonged outage can run CL light client sync to obtain the latest `ExecutionPayloadHeader`. This allows the EL to start syncing to a state close to wall time without having to wait for the CL optimistic sync to catch up. 2. EL light client implementations that wish to sync only data relevant to a certain use case (e.g., transactions for a specific wallet) can use the `ExecutionPayloadHeader` to decide what blocks are relevant. The new endpoint enables combined CL/EL deployments. --- src/engine/specification.md | 53 +++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/src/engine/specification.md b/src/engine/specification.md index 2ef315bc..ca610d42 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -16,6 +16,7 @@ This document specifies the Engine API methods that the Consensus Layer uses to - [Timeouts](#timeouts) - [Structures](#structures) - [ExecutionPayloadV1](#executionpayloadv1) + - [ExecutionPayloadHeaderV1](#executionpayloadheaderv1) - [ForkchoiceStateV1](#forkchoicestatev1) - [PayloadAttributesV1](#payloadattributesv1) - [PayloadStatusV1](#payloadstatusv1) @@ -41,6 +42,11 @@ This document specifies the Engine API methods that the Consensus Layer uses to - [Request](#request-3) - [Response](#response-3) - [Specification](#specification-3) +- [Optional endpoints](#optional-endpoints) + - [engine_newPayloadHeaderV1](#engine_newpayloadheaderv1) + - [Request](#request-4) + - [Response](#response-4) + - [Specification](#specification-4) @@ -174,6 +180,25 @@ This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/cons - `blockHash`: `DATA`, 32 Bytes - `transactions`: `Array of DATA` - Array of transaction objects, each object is a byte list (`DATA`) representing `TransactionType || TransactionPayload` or `LegacyTransaction` as defined in [EIP-2718](https://eips.ethereum.org/EIPS/eip-2718) +### ExecutionPayloadHeaderV1 + +This structure maps on the [`ExecutionPayloadHeader`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#executionpayloadheader) structure of the beacon chain spec. The fields are encoded as follows: + +- `parentHash`: `DATA`, 32 Bytes +- `feeRecipient`: `DATA`, 20 Bytes +- `stateRoot`: `DATA`, 32 Bytes +- `receiptsRoot`: `DATA`, 32 Bytes +- `logsBloom`: `DATA`, 256 Bytes +- `prevRandao`: `DATA`, 32 Bytes +- `blockNumber`: `QUANTITY`, 64 Bits +- `gasLimit`: `QUANTITY`, 64 Bits +- `gasUsed`: `QUANTITY`, 64 Bits +- `timestamp`: `QUANTITY`, 64 Bits +- `extraData`: `DATA`, 0 to 32 Bytes +- `baseFeePerGas`: `QUANTITY`, 256 Bits +- `blockHash`: `DATA`, 32 Bytes +- `transactionsRoot`: `DATA`, 32 Bytes - `hash_tree_root(transactions)`, not the keccak256 hash but instead the Consensus Layer [SSZ merkle root](https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md) + ### ForkchoiceStateV1 This structure encapsulates the fork choice state. The fields are encoded as follows: @@ -396,3 +421,31 @@ The payload build process is specified as follows: 7. Considering the absence of the `TERMINAL_TOTAL_DIFFICULTY` value (i.e. when a value has not been decided), Consensus Layer and Execution Layer client software **MUST** use `115792089237316195423570985008687907853269984665640564039457584007913129638912` value (equal to`2**256-2**10`) for the `terminalTotalDifficulty` input parameter of this call. [json-rpc-spec]: https://playground.open-rpc.org/?schemaUrl=https://raw.githubusercontent.com/ethereum/execution-apis/assembled-spec/openrpc.json&uiSchema[appBar][ui:splitView]=false&uiSchema[appBar][ui:input]=false&uiSchema[appBar][ui:examplesDropdown]=false + +## Optional endpoints + +### engine_newPayloadHeaderV1 + +#### Request + +* method: `engine_newPayloadHeaderV1` +* params: + 1. [`ExecutionPayloadHeaderV1`](#ExecutionPayloadHeaderV1) +* timeout: 8s + +#### Response + +* result: [`PayloadStatusV1`](#PayloadStatusV1) +* error: code and message set in case an exception happens while processing the payload. + +#### Specification + +1. Execution Layer client software **MUST** handle calls to this endpoint the same as [`engine_newPayloadV1`](#engine_getpayloadv1) and provide compatible responses. When needed, Execution Layer client software **MUST** obtain the block's transactions autonomously. + +2. Consensus Layer client software **SHOULD NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV1`](#engine_getpayloadv1) endpoint **SHOULD** be used to reduce sync latency and maximize validator rewards. + +3. Consensus Layer client software **MAY** use this endpoint during [optimistic sync](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md) to inform Execution Layer client software about blocks far in the future. Execution Layer client software **MUST** support switching to this future block if requested to do so with [`engine_forkchoiceUpdatedV1`](#engine_forkchoiceupdatedv1). This allows the Execution Layer client software to sync close to current wall time without having to wait for optimistic sync to catch up. + +4. Consensus Layer [light clients](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md) **MAY** use this endpoint together with [`engine_forkchoiceUpdatedV1`](#engine_forkchoiceupdatedv1) to sync Execution Layer client software. Execution Layer client software **MUST** support syncing with only those two endpoints. Notably, syncing **MUST NOT** require [`engine_newPayloadV1`](#engine_getpayloadv1) calls. Furthermore, Execution Layer client software **MAY** also support syncing with solely `engine_forkchoiceUpdatedV1` calls. + +5. Client software **MAY** offer configuration options to limit the sync scope to use case dependent data (e.g., only sync transactions relating to a certain wallet). This enables combined Consensus Layer / Execution Layer light client experiences. From 1d4252580c1508fa18400b622008b403473b31a8 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Fri, 4 Nov 2022 18:21:14 +0100 Subject: [PATCH 02/14] Satisfy spellcheck --- wordlist.txt | 3 +++ 1 file changed, 3 insertions(+) diff --git a/wordlist.txt b/wordlist.txt index a38207e4..2cc1feaa 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -8,10 +8,13 @@ endian enum eth ethereum +executionpayloadheaderv interop json +keccak mempool merkle +newpayloadheaderv npm ommers openrpc From d794b90c2dfc5887141bde8a267bbfb5d17f0b09 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:00:02 +0100 Subject: [PATCH 03/14] Change from SSZ to RLP hash --- src/engine/specification.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 8e521e2d..9de4f968 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -178,7 +178,7 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string ### ExecutionPayloadV1 -This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#ExecutionPayload) structure of the beacon chain spec. The fields are encoded as follows: +This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#ExecutionPayload) structure of the Bellatrix beacon chain spec. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes - `feeRecipient`: `DATA`, 20 Bytes @@ -211,6 +211,8 @@ The fields are encoded as follows: This structure has the syntax of `ExecutionPayloadV1` and appends a single field: `withdrawals`. +This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the `ExecutionPayloadV1` with the fields: `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: + - `parentHash`: `DATA`, 32 Bytes - `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes @@ -225,11 +227,13 @@ This structure has the syntax of `ExecutionPayloadV1` and appends a single field - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes - `transactions`: `Array of DATA` - Array of transaction objects, each object is a byte list (`DATA`) representing `TransactionType || TransactionPayload` or `LegacyTransaction` as defined in [EIP-2718](https://eips.ethereum.org/EIPS/eip-2718) +- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash (`transactions_hash` in beacon chain spec) - `withdrawals`: `Array of WithdrawalV1` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. +- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash (`withdrawals_hash` in beacon chain spec) ### ExecutionPayloadHeaderV1 -This structure maps on the [`ExecutionPayloadHeader`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#executionpayloadheader) structure of the Capella beacon chain spec. The fields are encoded as follows: +This structure matches [`ExecutionPayloadV2`](#executionpayloadv2) but omits the `transactions` and `withdrawals` arrays. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes - `feeRecipient`: `DATA`, 20 Bytes @@ -244,8 +248,8 @@ This structure maps on the [`ExecutionPayloadHeader`](https://github.com/ethereu - `extraData`: `DATA`, 0 to 32 Bytes - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes -- `transactionsRoot`: `DATA`, 32 Bytes - `hash_tree_root(transactions)`, not the keccak256 hash but instead the Consensus Layer [SSZ merkle root](https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md) -- `withdrawalsRoot`: `DATA`, 32 Bytes - `hash_tree_root(withdrawals)`, not the keccak256 hash but instead the Consensus Layer [SSZ merkle root](https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md) +- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash (`transactions_hash` in beacon chain spec) +- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash (`withdrawals_hash` in beacon chain spec) ### ForkchoiceStateV1 From 08c00309dd142fb7c42ee7c1956fc43f15c4ab79 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:02:51 +0100 Subject: [PATCH 04/14] Cleanup --- src/engine/specification.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 9de4f968..1b47ceb1 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -209,8 +209,6 @@ The fields are encoded as follows: ### ExecutionPayloadV2 -This structure has the syntax of `ExecutionPayloadV1` and appends a single field: `withdrawals`. - This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the `ExecutionPayloadV1` with the fields: `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes @@ -227,9 +225,9 @@ This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/cons - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes - `transactions`: `Array of DATA` - Array of transaction objects, each object is a byte list (`DATA`) representing `TransactionType || TransactionPayload` or `LegacyTransaction` as defined in [EIP-2718](https://eips.ethereum.org/EIPS/eip-2718) -- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash (`transactions_hash` in beacon chain spec) +- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec - `withdrawals`: `Array of WithdrawalV1` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. -- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash (`withdrawals_hash` in beacon chain spec) +- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec ### ExecutionPayloadHeaderV1 From 834b36b5ec069e8ab5dd3bc77bfc0fdcbee95004 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:03:50 +0100 Subject: [PATCH 05/14] Improve wording --- src/engine/specification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 1b47ceb1..f4609efd 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -209,7 +209,7 @@ The fields are encoded as follows: ### ExecutionPayloadV2 -This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the `ExecutionPayloadV1` with the fields: `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: +This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the [`ExecutionPayloadV1`](#executionpayloadv1) structure with the fields `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes - `feeRecipient`: `DATA`, 20 Bytes From 6054dfb9257d449da899c9d601757d4fc6368d87 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:05:07 +0100 Subject: [PATCH 06/14] Synchronize layout --- src/engine/specification.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index f4609efd..a8a89ddf 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -181,7 +181,7 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#ExecutionPayload) structure of the Bellatrix beacon chain spec. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes +- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes - `receiptsRoot`: `DATA`, 32 Bytes - `logsBloom`: `DATA`, 256 Bytes @@ -212,7 +212,7 @@ The fields are encoded as follows: This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the [`ExecutionPayloadV1`](#executionpayloadv1) structure with the fields `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes +- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes - `receiptsRoot`: `DATA`, 32 Bytes - `logsBloom`: `DATA`, 256 Bytes @@ -234,7 +234,7 @@ This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/cons This structure matches [`ExecutionPayloadV2`](#executionpayloadv2) but omits the `transactions` and `withdrawals` arrays. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes +- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes - `receiptsRoot`: `DATA`, 32 Bytes - `logsBloom`: `DATA`, 256 Bytes @@ -246,8 +246,8 @@ This structure matches [`ExecutionPayloadV2`](#executionpayloadv2) but omits the - `extraData`: `DATA`, 0 to 32 Bytes - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes -- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash (`transactions_hash` in beacon chain spec) -- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash (`withdrawals_hash` in beacon chain spec) +- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec +- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec ### ForkchoiceStateV1 From c86db034279c33a6b69f6c676849c92cf8f8bae1 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:13:40 +0100 Subject: [PATCH 07/14] Consistent format to refer to newPayload response --- src/engine/specification.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index a8a89ddf..2eb33cb5 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -553,8 +553,7 @@ Refer to the specification for [`engine_getPayloadV1`](#engine_getpayloadv1). #### Response -* result: [`PayloadStatusV1`](#PayloadStatusV1) -* error: code and message set in case an exception happens while processing the payload. +Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). #### Specification From 53beebe126f3e428ea00a0a9f746c5c8a676368d Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:54:29 +0100 Subject: [PATCH 08/14] Add new fields to nullability check --- src/engine/specification.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 2eb33cb5..be01cf26 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -225,9 +225,9 @@ This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/cons - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes - `transactions`: `Array of DATA` - Array of transaction objects, each object is a byte list (`DATA`) representing `TransactionType || TransactionPayload` or `LegacyTransaction` as defined in [EIP-2718](https://eips.ethereum.org/EIPS/eip-2718) -- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec -- `withdrawals`: `Array of WithdrawalV1` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. -- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec +- `transactionsRoot`: `DATA|null`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec +- `withdrawals`: `Array of WithdrawalV1|null` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. +- `withdrawalsRoot`: `DATA|null`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec ### ExecutionPayloadHeaderV1 @@ -392,10 +392,11 @@ Refer to the response for [`engine_newPayloadV1`](#engine_newpayloadv1). This method follows the same specification as [`engine_newPayloadV1`](#engine_newpayloadv1) with the exception of the following: -1. If withdrawal functionality is activated, client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if `payload.withdrawals` is `null`. - Similarly, if the functionality is not activated, client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if `payloadAttributes.withdrawals` is not `null`. - Blocks without withdrawals **MUST** be expressed with an explicit empty list `[]` value. +1. The fields `payload.transactionsRoot`, `payload.withdrawals`, and `payload.withdrawalsRoot` **MUST NOT** be `null` iff withdrawal functionality is activated. + Likewise, these fields **MUST** be `null` if withdrawal functionality is not activated. + Client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if fields are inappropriately specified/omitted. Refer to the validity conditions for [`engine_newPayloadV1`](#engine_newpayloadv1) to specification of the appropriate `latestValidHash` value. + Blocks without withdrawals **MUST** be expressed with an explicit empty list `[]` value. ### engine_forkchoiceUpdatedV1 From baa849ca9ad8d406625aaac43b681bf07de1f496 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Sat, 5 Nov 2022 00:56:19 +0100 Subject: [PATCH 09/14] Extend requirement for withdrawals --- src/engine/specification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index be01cf26..e3060d0a 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -558,7 +558,7 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). #### Specification -1. Execution Layer client software **MUST** handle calls to this endpoint the same as [`engine_newPayloadV2`](#engine_getpayloadv2) and provide compatible responses. When needed, Execution Layer client software **MUST** obtain the block's transactions autonomously. +1. Execution Layer client software **MUST** handle calls to this endpoint the same as [`engine_newPayloadV2`](#engine_getpayloadv2) and provide compatible responses. When needed, Execution Layer client software **MUST** obtain the block's transactions and withdrawals autonomously. 2. Consensus Layer client software **SHOULD NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **SHOULD** be used to reduce sync latency and maximize validator rewards. From c206bc8170b7e8feb1a0eb2f6b10004e7892cab8 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Mon, 28 Nov 2022 20:45:33 +0100 Subject: [PATCH 10/14] Only pass minimal sync context information --- src/engine/specification.md | 57 +++++++++++++++---------------------- wordlist.txt | 3 -- 2 files changed, 23 insertions(+), 37 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index e3060d0a..6d728bb4 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -18,7 +18,7 @@ This document specifies the Engine API methods that the Consensus Layer uses to - [ExecutionPayloadV1](#executionpayloadv1) - [WithdrawalV1](#withdrawalv1) - [ExecutionPayloadV2](#executionpayloadv2) - - [ExecutionPayloadHeaderV1](#executionpayloadheaderv1) + - [ExecutionPayloadSyncContextV1](#executionpayloadsynccontextv1) - [ForkchoiceStateV1](#forkchoicestatev1) - [PayloadAttributesV1](#payloadattributesv1) - [PayloadAttributesV2](#payloadattributesv2) @@ -58,7 +58,7 @@ This document specifies the Engine API methods that the Consensus Layer uses to - [Response](#response-6) - [Specification](#specification-6) - [Optional endpoints](#optional-endpoints) - - [engine_newPayloadHeaderV1](#engine_newpayloadheaderv1) + - [engine_newPayloadSyncContextV1](#engine_newpayloadsynccontextv1) - [Request](#request-7) - [Response](#response-7) - [Specification](#specification-7) @@ -178,10 +178,10 @@ Values of a field of `QUANTITY` type **MUST** be encoded as a hexadecimal string ### ExecutionPayloadV1 -This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#ExecutionPayload) structure of the Bellatrix beacon chain spec. The fields are encoded as follows: +This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/bellatrix/beacon-chain.md#ExecutionPayload) structure of the beacon chain spec. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes +- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes - `receiptsRoot`: `DATA`, 32 Bytes - `logsBloom`: `DATA`, 256 Bytes @@ -209,10 +209,10 @@ The fields are encoded as follows: ### ExecutionPayloadV2 -This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/consensus-specs/blob/dev/specs/capella/beacon-chain.md#ExecutionPayload) structure of the Capella beacon chain spec, extending the [`ExecutionPayloadV1`](#executionpayloadv1) structure with the fields `transactionsRoot`, `withdrawals`, and `withdrawalsRoot`. The fields are encoded as follows: +This structure has the syntax of `ExecutionPayloadV1` and appends a single field: `withdrawals`. - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes +- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes - `receiptsRoot`: `DATA`, 32 Bytes - `logsBloom`: `DATA`, 256 Bytes @@ -225,29 +225,16 @@ This structure maps on the [`ExecutionPayload`](https://github.com/ethereum/cons - `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes - `transactions`: `Array of DATA` - Array of transaction objects, each object is a byte list (`DATA`) representing `TransactionType || TransactionPayload` or `LegacyTransaction` as defined in [EIP-2718](https://eips.ethereum.org/EIPS/eip-2718) -- `transactionsRoot`: `DATA|null`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec -- `withdrawals`: `Array of WithdrawalV1|null` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. -- `withdrawalsRoot`: `DATA|null`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec +- `withdrawals`: `Array of WithdrawalV1` - Array of withdrawals, each object is an `OBJECT` containing the fields of a `WithdrawalV1` structure. -### ExecutionPayloadHeaderV1 +### ExecutionPayloadSyncContextV1 -This structure matches [`ExecutionPayloadV2`](#executionpayloadv2) but omits the `transactions` and `withdrawals` arrays. The fields are encoded as follows: +This structure provides a subset of `ExecutionPayload` fields to allow triggering sync without supplying the full `ExecutionPayload`. The fields are encoded as follows: - `parentHash`: `DATA`, 32 Bytes -- `feeRecipient`: `DATA`, 20 Bytes - `stateRoot`: `DATA`, 32 Bytes -- `receiptsRoot`: `DATA`, 32 Bytes -- `logsBloom`: `DATA`, 256 Bytes -- `prevRandao`: `DATA`, 32 Bytes - `blockNumber`: `QUANTITY`, 64 Bits -- `gasLimit`: `QUANTITY`, 64 Bits -- `gasUsed`: `QUANTITY`, 64 Bits -- `timestamp`: `QUANTITY`, 64 Bits -- `extraData`: `DATA`, 0 to 32 Bytes -- `baseFeePerGas`: `QUANTITY`, 256 Bits - `blockHash`: `DATA`, 32 Bytes -- `transactionsRoot`: `DATA`, 32 Bytes - RLP hash; `transactions_hash` in beacon chain spec -- `withdrawalsRoot`: `DATA`, 32 Bytes - RLP hash; `withdrawals_hash` in beacon chain spec ### ForkchoiceStateV1 @@ -392,11 +379,10 @@ Refer to the response for [`engine_newPayloadV1`](#engine_newpayloadv1). This method follows the same specification as [`engine_newPayloadV1`](#engine_newpayloadv1) with the exception of the following: -1. The fields `payload.transactionsRoot`, `payload.withdrawals`, and `payload.withdrawalsRoot` **MUST NOT** be `null` iff withdrawal functionality is activated. - Likewise, these fields **MUST** be `null` if withdrawal functionality is not activated. - Client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if fields are inappropriately specified/omitted. - Refer to the validity conditions for [`engine_newPayloadV1`](#engine_newpayloadv1) to specification of the appropriate `latestValidHash` value. +1. If withdrawal functionality is activated, client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if `payload.withdrawals` is `null`. + Similarly, if the functionality is not activated, client software **MUST** return an `INVALID` status with the appropriate `latestValidHash` if `payloadAttributes.withdrawals` is not `null`. Blocks without withdrawals **MUST** be expressed with an explicit empty list `[]` value. + Refer to the validity conditions for [`engine_newPayloadV1`](#engine_newpayloadv1) to specification of the appropriate `latestValidHash` value. ### engine_forkchoiceUpdatedV1 @@ -543,13 +529,13 @@ Refer to the specification for [`engine_getPayloadV1`](#engine_getpayloadv1). ## Optional endpoints -### engine_newPayloadHeaderV1 +### engine_newPayloadSyncContextV1 #### Request -* method: `engine_newPayloadHeaderV1` +* method: `engine_newPayloadSyncContextV1` * params: - 1. [`ExecutionPayloadHeaderV1`](#ExecutionPayloadHeaderV1) + 1. [`ExecutionPayloadSyncContextV1`](#executionpayloadsynccontextv1) * timeout: 8s #### Response @@ -558,12 +544,15 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). #### Specification -1. Execution Layer client software **MUST** handle calls to this endpoint the same as [`engine_newPayloadV2`](#engine_getpayloadv2) and provide compatible responses. When needed, Execution Layer client software **MUST** obtain the block's transactions and withdrawals autonomously. +1. Execution Layer client software **MUST** handle calls to this endpoint the same as if the corresponding full `ExecutionPayload` were passed to the `engine_newPayload` endpoint, with the exception of the following: + * `blockHash` validation is skipped + * If the block referred to by the provided `blockHash` is not locally available, Execution Layer client software **MUST** respond to this method call with `{status: SYNCING, latestValidHash: null, validationError: null}` + * If the provided data does not match the corresponding data from the locally available block with provided `blockHash`, Execution Layer client software **MUST** respond to this method call with `{status: INVALID_BLOCK_HASH, latestValidHash: null, validationError: errorMessage | null}` -2. Consensus Layer client software **SHOULD NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **SHOULD** be used to reduce sync latency and maximize validator rewards. +2. Execution Layer client **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST NOT** require `engine_newPayload` calls. -3. Consensus Layer client software **MAY** use this endpoint during [optimistic sync](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md) to inform Execution Layer client software about blocks far in the future. Execution Layer client software **MUST** support switching to this future block if requested to do so with `engine_forkchoiceUpdated`. This allows the Execution Layer client software to sync close to current wall time without having to wait for optimistic sync to catch up. +3. Consensus Layer client software **MUST NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **MUST** be used to validate the full `ExecutionPayload` structure. -4. Consensus Layer [light clients](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md) **MAY** use this endpoint together with `engine_forkchoiceUpdated` to sync Execution Layer client software. Execution Layer client software **MUST** support syncing with only those two endpoints. Notably, syncing **MUST NOT** require `engine_newPayload` calls. Furthermore, Execution Layer client software **MAY** also support syncing with solely `engine_forkchoiceUpdated` calls. +4. Consensus layer client software **MAY** use this endpoint during an ongoing sync to inform Execution Layer client software about blocks far in the future (e.g., by running a [light client](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md) during [optimistic sync](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md)). Execution Layer client software **SHOULD** consider interrupting the ongoing sync when requested to switch to the provided `blockHash` using `engine_forkchoiceUpdated`. -5. Client software **MAY** offer configuration options to limit the sync scope to use case dependent data (e.g., only sync transactions relating to a certain wallet). This enables combined Consensus Layer / Execution Layer light client experiences. +5. Client software **MAY** offer configuration options to limit the sync scope to use case dependent data (e.g., only sync data related to a certain wallet). diff --git a/wordlist.txt b/wordlist.txt index 2cc1feaa..a38207e4 100644 --- a/wordlist.txt +++ b/wordlist.txt @@ -8,13 +8,10 @@ endian enum eth ethereum -executionpayloadheaderv interop json -keccak mempool merkle -newpayloadheaderv npm ommers openrpc From d7be255c18542b50b4051bbe6189cea55a3b8bc5 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Mon, 28 Nov 2022 20:49:43 +0100 Subject: [PATCH 11/14] Typo --- src/engine/specification.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 6d728bb4..0ee68eea 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -549,7 +549,7 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). * If the block referred to by the provided `blockHash` is not locally available, Execution Layer client software **MUST** respond to this method call with `{status: SYNCING, latestValidHash: null, validationError: null}` * If the provided data does not match the corresponding data from the locally available block with provided `blockHash`, Execution Layer client software **MUST** respond to this method call with `{status: INVALID_BLOCK_HASH, latestValidHash: null, validationError: errorMessage | null}` -2. Execution Layer client **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST NOT** require `engine_newPayload` calls. +2. Execution Layer client software **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST NOT** require `engine_newPayload` calls. 3. Consensus Layer client software **MUST NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **MUST** be used to validate the full `ExecutionPayload` structure. From 17b42224db05410b54ef4a97206d957f43c3d782 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Wed, 7 Dec 2022 13:19:02 +0100 Subject: [PATCH 12/14] Always return `SYNCING` for `newPayloadSyncContext` --- src/engine/specification.md | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 0ee68eea..095cc810 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -544,12 +544,10 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). #### Specification -1. Execution Layer client software **MUST** handle calls to this endpoint the same as if the corresponding full `ExecutionPayload` were passed to the `engine_newPayload` endpoint, with the exception of the following: - * `blockHash` validation is skipped - * If the block referred to by the provided `blockHash` is not locally available, Execution Layer client software **MUST** respond to this method call with `{status: SYNCING, latestValidHash: null, validationError: null}` - * If the provided data does not match the corresponding data from the locally available block with provided `blockHash`, Execution Layer client software **MUST** respond to this method call with `{status: INVALID_BLOCK_HASH, latestValidHash: null, validationError: errorMessage | null}` +1. Execution Layer client software **MAY** initiate a sync process if the described block is not locally available. Sync process is specified in the [Sync](#sync) section. Execution Layer client software **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST** be possible without `engine_newPayload` calls. -2. Execution Layer client software **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST NOT** require `engine_newPayload` calls. +2. Execution Layer client software **MUST** respond to this method call in the following way: + * `{status: SYNCING, latestValidHash: null, validationError: null}` in all cases. 3. Consensus Layer client software **MUST NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **MUST** be used to validate the full `ExecutionPayload` structure. From 4a09624baea569fbd5390381a773b78f99c52577 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Wed, 7 Dec 2022 14:02:40 +0100 Subject: [PATCH 13/14] Document error object --- src/engine/specification.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index 095cc810..c56b39e4 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -547,7 +547,8 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). 1. Execution Layer client software **MAY** initiate a sync process if the described block is not locally available. Sync process is specified in the [Sync](#sync) section. Execution Layer client software **MUST** support syncing solely based on calls to this endpoint and `engine_forkchoiceUpdated`. Notably, syncing **MUST** be possible without `engine_newPayload` calls. 2. Execution Layer client software **MUST** respond to this method call in the following way: - * `{status: SYNCING, latestValidHash: null, validationError: null}` in all cases. + * `{status: SYNCING, latestValidHash: null, validationError: null}` if the request was processed + * With an error object in any error conditions unrelated to the normal processing flow of the method 3. Consensus Layer client software **MUST NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **MUST** be used to validate the full `ExecutionPayload` structure. From b05cb7486858beea41670c3673dccfefd8f52441 Mon Sep 17 00:00:00 2001 From: Etan Kissling Date: Wed, 7 Dec 2022 21:17:48 +0100 Subject: [PATCH 14/14] Improve grammar --- src/engine/specification.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/engine/specification.md b/src/engine/specification.md index c56b39e4..a4c05907 100644 --- a/src/engine/specification.md +++ b/src/engine/specification.md @@ -548,10 +548,10 @@ Refer to the response for [`engine_newPayloadV2`](#engine_newpayloadv2). 2. Execution Layer client software **MUST** respond to this method call in the following way: * `{status: SYNCING, latestValidHash: null, validationError: null}` if the request was processed - * With an error object in any error conditions unrelated to the normal processing flow of the method + * An error object in any error conditions unrelated to the normal processing flow of the method 3. Consensus Layer client software **MUST NOT** use this endpoint for validator duties. Instead, the [`engine_newPayloadV2`](#engine_getpayloadv2) endpoint **MUST** be used to validate the full `ExecutionPayload` structure. -4. Consensus layer client software **MAY** use this endpoint during an ongoing sync to inform Execution Layer client software about blocks far in the future (e.g., by running a [light client](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md) during [optimistic sync](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md)). Execution Layer client software **SHOULD** consider interrupting the ongoing sync when requested to switch to the provided `blockHash` using `engine_forkchoiceUpdated`. +4. Consensus Layer client software **MAY** use this endpoint during an ongoing sync to inform Execution Layer client software about blocks far in the future (e.g., by running a [light client](https://github.com/ethereum/consensus-specs/blob/dev/specs/altair/light-client/sync-protocol.md) during [optimistic sync](https://github.com/ethereum/consensus-specs/blob/dev/sync/optimistic.md)). Execution Layer client software **SHOULD** consider interrupting the ongoing sync when requested to switch to the provided `blockHash` using `engine_forkchoiceUpdated`. 5. Client software **MAY** offer configuration options to limit the sync scope to use case dependent data (e.g., only sync data related to a certain wallet).