add swip-27: strictly typed wire-level messages

SWIPs/assets/swip-27/README.md

@@ -0,0 +1,146 @@
+# Protocol Improvements Report: Old vs. New Swarm Protobufs
+
+## Table of Contents
+
+| Protocol | Description |
+|----------|-------------|
+| [common.proto](./common.proto) | Common types used across multiple protocols |
+| [handshake.proto](./handshake.proto) | Node connection establishment protocol |
+| [headers.proto](./headers.proto) | Protocol for transmitting metadata headers |
+| [hive.proto](./hive.proto) | Peer discovery and sharing protocol |
+| [pingpong.proto](./pingpong.proto) | Simple connectivity testing protocol |
+| [pricing.proto](./pricing.proto) | Payment threshold announcement protocol |
+| [pseudosettle.proto](./pseudosettle.proto) | Lightweight settlement protocol |
+| [pullsync.proto](./pullsync.proto) | Protocol for syncing chunks between nodes |
+| [pushsync.proto](./pushsync.proto) | Protocol for distributing new chunks |
+| [retrieval.proto](./retrieval.proto) | Protocol for retrieving chunks |
+| [status.proto](./status.proto) | Node status reporting protocol |
+| [swap.proto](./swap.proto) | Token-based settlement protocol |
+
+## Executive Summary
+The new Swarm protocol definitions represent a significant evolution from the older versions, with improvements in organization, consistency, error handling, and documentation. These changes enhance maintainability, interoperability, and developer experience.
+
+## Key Improvements
+
+### 1. Unified Message Type System
+**Before**: Protocol messages were defined independently with inconsistent structures.
+**After**: Introduction of a common `Chunk` message type in `common.proto` that standardizes chunk representation across all protocols.
+
+```diff
+- // Different chunk representations in each protocol
+- message Delivery {
+-   bytes Address = 1;
+-   bytes Data = 2;
+-   bytes Stamp = 3;
+- }
+
++ // Standardized chunk representation
++ message Chunk {
++   ChunkType chunk_type = 1;
++   uint32 version = 2;
++   bytes header = 3;
++   bytes payload = 4;
++   bytes proof = 5;
++ }
+```
+
+### 2. Consistent Error Handling
+**Before**: Error handling varied across protocols with string-based errors (`string Err`).
+**After**: Standardized error model with a common `Error` type that includes both error codes and messages.
+
+```diff
+- // String-based error handling
+- message Receipt {
+-   // ...
+-   string Err = 4;
+- }
+
++ // Structured error handling
++ message Error {
++   uint32 code = 1;
++   string message = 2;
++ }
+```
+
+### 3. Message Type Hierarchies
+**Before**: Flat message structures without protocol organization.
+**After**: Introduction of message type enums and wrapper messages, creating clear protocol hierarchies.
+
+```diff
+- // Flat message structures
+- message Payment { /* ... */ }
+- message PaymentAck { /* ... */ }
+
++ // Organized message hierarchies
++ enum PseudoSettleMessageType {
++   PAYMENT = 0;
++   PAYMENT_ACK = 1;
++ }
++
++ message PseudoSettleMessage {
++   PseudoSettleMessageType type = 1;
++   oneof message {
++     Payment payment = 2;
++     PaymentAck payment_ack = 3;
++   }
++ }
+```
+
+### 4. Field Naming Standardization
+**Before**: Inconsistent field naming with mixed case (e.g., `bytes Address`).
+**After**: Standardized lowercase snake_case field names following protobuf best practices (e.g., `bytes chunk_addr`).
+
+### 5. Field Separation and Specialization
+**Before**: Mixed functionality in single messages.
+**After**: Clear separation of concerns with specialized subtypes.
+
+```diff
+- // Mixed success/error in one message
+- message Receipt {
+-   bytes Address = 1;
+-   bytes Signature = 2;
+-   bytes Nonce = 3;
+-   string Err = 4;
+- }
+
++ // Clear separation with oneof
++ message Receipt {
++   bytes chunk_addr = 1;
++   oneof result {
++     ReceiptSuccess success = 2;
++     swarm.common.Error error = 3;
++   }
++ }
+```
+
+### 6. Documentation and Comments
+**Before**: Minimal documentation beyond copyright notices.
+**After**: Enhanced message and field documentation with explanatory comments.
+
+```diff
++ // ChunkType enum defines the different chunk formats
++ enum ChunkType {
++   CAC = 0;  // Content-addressed chunk
++   SOC = 1;  // Single-owner chunk
++ }
+```
+
+### 7. Result Handling with 'oneof'
+**Before**: Results and errors mixed in a single message structure.
+**After**: Clear use of `oneof` for mutually exclusive fields like success/error responses.
+
+### 8. Domain Model Clarity
+**Before**: Some domain concepts were unclear from the protocol definition.
+**After**: Better expression of domain concepts with dedicated types and enums.
+
+## Conclusion
+
+The new protocol definitions represent a substantial improvement in design quality. Key benefits include:
+
+1. **Improved maintainability** through consistent naming and structure
+2. **Better error handling** with standardized error types and codes
+3. **Enhanced clarity** through proper message hierarchies and documentation
+4. **Reduced duplication** by centralizing common types like `Chunk` and `Error`
+5. **Proper use of protobuf features** like enums, oneofs, and nested messages
+
+These improvements will lead to more reliable implementations, easier debugging, and a better developer experience across different Swarm clients.

SWIPs/assets/swip-27/common.proto

@@ -0,0 +1,76 @@
+syntax = "proto3";
+
+package swarm.common;
+
+import "google/protobuf/any.proto";
+
+option go_package = "pb";
+
+// ChunkType enum defines the different chunk formats
+// This follows the type information preservation principle
+// by explicitly encoding the chunk type for protocol handlers
+enum ChunkType {
+    CAC = 0;  // Content-addressed chunk
+    SOC = 1;  // Single-owner chunk
+}
+
+// Chunk represents a data chunk in the Swarm network
+// This follows the type information preservation principle by
+// including explicit type and version fields that inform handlers
+// how to interpret the remaining fields
+message Chunk {
+    // Chunk type identifier
+    ChunkType chunk_type = 1;
+
+    // Chunk format version
+    uint32 version = 2;
+
+    // Chunk type-specific header
+    bytes header = 3;
+
+    // Chunk data payload (serialized based on chunk_type)
+    bytes payload = 4;
+
+    // Optional proof of chunk authenticity (from a system perspective)
+    bytes proof = 5;
+}
+
+// PostageStamp represents a stamp proving chunk postage payment
+message PostageStamp {
+    // The batch ID of the stamp
+    bytes batch_id = 1;
+
+    // The index of the stamp within its batch
+    uint32 index = 2;
+
+    // Timestamp of the stamp
+    uint64 timestamp = 3;
+
+    // Signature validating the stamp
+    bytes signature = 4;
+}
+
+// BzzAddress contains addressing information for a Swarm node
+message BzzAddress {
+    // Underlay network address bytes
+    bytes underlay = 1;
+
+    // Cryptographic signature of handshake
+    bytes signature = 2;
+
+    // Nonce used for overlay address generation
+    bytes nonce = 3;
+
+    // Overlay address bytes (optional)
+    bytes overlay = 4;
+}
+
+// Error represents a standardized error response
+// This implements the explicit error handling principle
+message Error {
+    // The error code
+    uint32 code = 1;
+
+    // An optional error message
+    string message = 2;
+}

SWIPs/assets/swip-27/handshake.proto

@@ -0,0 +1,57 @@
+syntax = "proto3";
+
+package swarm.handshake;
+
+import "common.proto";
+
+option go_package = "pb";
+
+// HandshakeMessageType indicates the type of handshake message
+enum HandshakeMessageType {
+    SYN = 0;
+    ACK = 1;
+    SYN_ACK = 2;
+}
+
+// HandshakeMessage is the wrapper for all handshake protocol messages
+message HandshakeMessage {
+    // The type of handshake message
+    HandshakeMessageType type = 1;
+
+    // Only one of the following messages should be set based on the type
+    oneof message {
+        Syn syn = 2;
+        Ack ack = 3;
+        SynAck syn_ack = 4;
+    }
+}
+
+// Syn message is sent to initiate a handshake
+message Syn {
+    // The observed underlay address of the peer
+    bytes observed_underlay = 1;
+}
+
+// Ack message is sent in response to a Syn
+message Ack {
+    // The sender's Swarm address
+    swarm.common.BzzAddress address = 1;
+
+    // Network ID to ensure peers are on the same network
+    uint64 network_id = 2;
+
+    // Indicates if the node is a full node
+    bool full_node = 3;
+
+    // Optional welcome message (max 140 chars)
+    string welcome_message = 99;
+}
+
+// SynAck message combines Syn and Ack for efficiency
+message SynAck {
+    // The Syn part of the message
+    Syn syn = 1;
+
+    // The Ack part of the message
+    Ack ack = 2;
+}

SWIPs/assets/swip-27/headers.proto

@@ -0,0 +1,26 @@
+syntax = "proto3";
+
+package swarm.headers;
+
+import "common.proto";
+
+option go_package = "pb";
+
+// Headers contains a collection of protocol headers
+// This implements the strongly typed messages principle by using
+// explicit message types rather than generic byte arrays
+// This is also the message that is sent in response to a Headers request
+// and is infallible.
+message Headers {
+    // List of header entries
+    repeated Header headers = 1;
+}
+
+// Header represents a single key-value header
+message Header {
+    // The header key
+    string key = 1;
+
+    // The header value as bytes
+    bytes value = 2;
+}

SWIPs/assets/swip-27/hive.proto

@@ -0,0 +1,15 @@
+syntax = "proto3";
+
+package swarm.hive;
+
+import "common.proto";
+
+option go_package = "pb";
+
+// Peers message contains a list of peer addresses to be shared with other nodes
+// This is an infallible response as the protocol can return an empty list
+// when no peers are available
+message Peers {
+    // List of peer addresses
+    repeated swarm.common.BzzAddress peers = 1;
+}

SWIPs/assets/swip-27/pingpong.proto

@@ -0,0 +1,19 @@
+syntax = "proto3";
+
+package swarm.pingpong;
+
+option go_package = "pb";
+
+// Ping message sent to test connectivity
+// This is an infallible request that cannot fail to be constructed
+message Ping {
+    // Optional greeting text
+    string greeting = 1;
+}
+
+// Pong message sent in response to a Ping
+// This is an infallible response as PingPong is a basic connectivity test
+message Pong {
+    // Response text
+    string response = 1;
+}

SWIPs/assets/swip-27/pricing.proto

@@ -0,0 +1,12 @@
+syntax = "proto3";
+
+package swarm.pricing;
+
+option go_package = "pb";
+
+// AnnouncePaymentThreshold message announces the payment threshold to peers
+// This is an infallible message as it simply declares a value
+message AnnouncePaymentThreshold {
+    // The payment threshold value in bytes as this may be up to u256.
+    bytes payment_threshold = 1;
+}

SWIPs/assets/swip-27/pseudosettle.proto

@@ -0,0 +1,39 @@
+syntax = "proto3";
+
+package swarm.pseudosettle;
+
+option go_package = "pb";
+
+// PseudoSettleMessageType indicates the type of pseudosettle message
+enum PseudoSettleMessageType {
+    PAYMENT = 0;
+    PAYMENT_ACK = 1;
+}
+
+// PseudoSettleMessage is the wrapper for all pseudosettle protocol messages
+// This implements the stateful protocol pattern with explicit message types
+message PseudoSettleMessage {
+    // The type of pseudosettle message
+    PseudoSettleMessageType type = 1;
+
+    // Only one of the following messages should be set based on the type
+    oneof message {
+        Payment payment = 2;
+        PaymentAck payment_ack = 3;
+    }
+}
+
+// Payment message represents a settlement payment
+message Payment {
+    // The payment amount in bytes (big.Int)
+    bytes amount = 1;
+}
+
+// PaymentAck acknowledges receipt of a payment
+message PaymentAck {
+    // The accepted payment amount in bytes (big.Int)
+    bytes amount = 1;
+
+    // The timestamp of the payment
+    int64 timestamp = 2;
+}