diff --git a/book/src/developers/protocols/discovery.md b/book/src/developers/protocols/discovery.md deleted file mode 100644 index 728e00ca9..000000000 --- a/book/src/developers/protocols/discovery.md +++ /dev/null @@ -1,34 +0,0 @@ -# Discovery - -## Node Discovery Protocol v5 (Discv5) - -A protocol ([spec](https://github.com/ethereum/devp2p/blob/master/discv5/discv5.md)) for nodes to identify each other. There are three main capabilities: -- Sample (walk the network to find nodes) -- Search (locate nodes interested in a specific topic) -- Update (navigate when a peer updates their details, such as IP address) - -Discovery is a high level protocol that is further defined with the Discovery wire protocol. - -## Discovery (Discv5) wire protocol - -An application-level protocol ([spec](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)) for nodes using Discv5. It describes the structure and logic of different -messages sent between nodes. - -Some important properties of the protocol are: -- UDP based scheme (tolerant to packet loss compared with TCP) - - The Portal Network uses a variant of UDP called uTP that is more friendly for larger - packets. -- Message encryption (ENRs are used to encrypt data with the recipients public key) -- Protocol differentiation (allow nodes to avoid unrelated networks) -- Message types (for finding peers, establishing connections, requesting specific things) -- Flexible request/types types (for sub-protocols to use custom messages) - -## Ethereum Node Records (ENR) - -A data format ([spec](https://github.com/ethereum/devp2p/blob/master/enr.md)) that allows nodes to know the identity and important information about -peers. This includes data like IP address and ports. - -Nodes generate a private key for the purpose of node discovery. This is used to sign -the ENR to prevent impersonation. Peers can encrypt messages for each other using the ENR. - -The private key is unrelated to private keys used to sign Ethereum transactions. diff --git a/book/src/developers/protocols/portal_wire.md b/book/src/developers/protocols/portal_wire.md deleted file mode 100644 index 864554db8..000000000 --- a/book/src/developers/protocols/portal_wire.md +++ /dev/null @@ -1,69 +0,0 @@ -# Portal wire protocol - -The Portal Wire protocol ([spec](https://github.com/ethereum/portal-network-specs/blob/master/portal-wire-protocol.md)) is a variant of the discovery (Discv5) wire protocol ([spec](https://github.com/ethereum/devp2p/blob/master/discv5/discv5-wire.md)). - -This means that the basic protocol is the same, but there are custom portal-specific messages. - - -## Protocol ID -Nodes identify each other by a protocol ID. - -- `0x50..` Portal Network - - `0x500A` portal state sub-protocol. - - `0x500B` portal history sub-protocol. - - Etc. -- `0x....` Other networks (Ethereum execution, Ethreum consensus chain, others.) - -## Messages -Messages the define the portal sub-protocols are: -- `TALKREQ` (talk requests). These have a request ID. -- `TALKRESP` (talk responses). These refer to the reqest ID being responded to. - -Messages contain data that is an SSZ union. This means that the message contains -one of the possible message content types, and that the type will be specified. -```py -# Python -message = Union[ping, pong, find_nodes, nodes, find_content, content, offer, accept] -``` -## Message encoding - -The SSZ Union encoding means that each component has a selector (`PING 0x00, PONG 0x01, FIND_NODES 0x02, ...`). -That way, different clients on the network can listen to messages on the right protocol -and correctly decode them. - -For example, receiving a message and seeing that the first byte is `0x02` indicates that the -message contains a `FIND_NODES` type of content. - -## Message data - -Each message has specific data that is sent. For example, a `find_content` message component will have the content that is being sought. The details of these can be found in the spec. - -## Additional API exposure - -The above message definitions are sufficient for a Trin node to participate in the network. - -However, as Trin is a JSON-RPC server (serving Ethereum-related requests like `eth_getBlockNumber`) -it also exposes the wire methods. This is not strictly required by the Portal Network specification, -but is very useful. - -Messages that are wire responses are not exposed, as they are not requests. - -## Relationship to JSON-RPC - -The following table shows the wire definition, purpose and how it is exposed for querying via -the JSON-RPC server. - -Recall that each Trin can serve multiple sub-protocols simultaneously. Hence, the -following table is for the History sub-protocol (`0x500A` in Discovery-terms), and -JSON-RPC methods start with `portal_history*`. - -|message|SSZ union message selector|purpose|JSON-RPC| -|-|-|-|-| -|ping|`0x01`|"Are you alive?"|`portal_historyPing`| -|pong|`0x02`|"I'm alive"|None| -|find_nodes|`0x03`|"Give me peers at specific distances x, y & z"|`portal_historyFindNodes`| -|nodes|`0x04`|"Response to findnodes"|None| -|find_content|`0x05`|"I want content x, or peers who might have it"|`portal_historyFindContent`| -|content|`0x06`|"Here is content x, or peers who might have it."|None| -|offer|`0x07`|"I have content x, y & z, would you like any of them?"|`portal_historyOffer`| -|accept|`0x08`|"Yes please, I would like x, y & z"|None| diff --git a/book/src/developers/protocols/ssz.md b/book/src/developers/protocols/ssz.md deleted file mode 100644 index 75b3ddc3d..000000000 --- a/book/src/developers/protocols/ssz.md +++ /dev/null @@ -1,66 +0,0 @@ -# SSZ - -The Simple Serialize (SSZ) protocol ([spec](https://github.com/ethereum/consensus-specs/blob/dev/ssz/simple-serialize.md)) is used to ensure that data sent to peers is interpreted unambiguously. - -It is used in two main ways: -- Encoding: from rich data (struct, enum) to bytes (`Vec`). -- Decoding: from bytes (`Vec`) to rich data (struct, enum). - -The encoded data is not self-describing, so you have to know what sort of data you -are expecting. Hence, the data type descriptions in the Portal Network spec. - -Encoded data can only be interpreted in one way. Additionally, encoded data -can also be used in Merkle proofs efficiently. - -## Types - -The following is a quick overview of major composite types used. See the spec for -basic types (e.g., bits, bools, unsigned integers). - -|Type|Description|Note| -|-|-|-| -|List|Holds variable number of a specified item|Specify max number| -|Vector|Holds specific number of a specified item|Specify number| -|Container|Holds many different specified items| Items are spaced into 32 byte partitions| -|Union|Holds one of many different specified items| Item kind is specified using a prepended selector byte| - -Each type can hold any of the other types, so a Containter can hold a List -and a Union, and the Union can hold another Container, etc. -Anything that is put into one of the types above must itself be SSZ-able - -## Implementations - -The `ssz` crate does a lot of the work by providing `Encode` and `Decode` methods that -can be derived. -```rs,no_run -#[derive(Clone, Debug, Decode, Encode, PartialEq)] -#[ssz(enum_behaviour = "union")] -pub enum HistoryContentKey { - BlockHeaderWithProof(BlockHeader), - BlockBody(BlockBody), - BlockReceipts(BlockReceipts), - EpochAccumulator(EpochAccumulator), -} -``` -The spec defines a content key for the History sub-protocol as: -```py -block_header_key = Container(block_hash: Bytes32) -selector = 0x00 -content_key = selector + SSZ.serialize(block_header_key) -``` -The inclusion of the selector is handled by implementing `serde::Serialize` for -the enum, and including the appending the appropriate selector byte. - -That is, the hex string ready to be serialized into bytes for a block header would be -```sh -# Header, body or receipts -"0x" -# Header specifically -"0x00" -# Serialize to bytes -[0x00, ...] -``` -## Merkle proofs - -The Epoch Accumulator uses SSZ encoding. This allows for Merkle proofs to be made -for arbitrary historical blocks against the accumulator. \ No newline at end of file diff --git a/book/src/developers/protocols/utp.md b/book/src/developers/protocols/utp.md deleted file mode 100644 index a0cd5c402..000000000 --- a/book/src/developers/protocols/utp.md +++ /dev/null @@ -1,25 +0,0 @@ -# uTP - -The Dicv5 protocol normally uses UDP packets, however this has some limitations -in packet size. The Portal Network uses Micro Transport Protocol (uTP) -([spec](https://github.com/ethereum/portal-network-specs/blob/master/discv5-utp.md)) -to avoid this problem. - -uTP is similar to the BitTorrent protocol and provides a way to send ordered packets. - -Once two peers have agreed to send data via messages in the Portal Wire protocol (E.g., via -an `OFFER` and `ACCEPT`) sequence, the peers can then open communication on the uTP -sub-protocol. Inside this protocol they can send the data to each other, following the -uTP protocol until the data transfer is complete. - -First, a sub-protocol is used: -- History sub-protocol (arrange data transfer, but don't start sending) -- State sub-protocol (arrange data transfer, but don't start sending) -- ... - -Second, once data transfer is arranged, peers switch to the uTP protocol to start the -data transfer. uTP uses the relevant sub-protocol (E.g., History Discv5 overlay, -State Discv5 overlay) as transport. - -By providing an ID for the content that they are transferring, the two peers -can easily switch from one protocol to the uTP protocol and complete the specified transfer.