Critical: Network Lockup Caused by Elliptic Curve Mismatch in Peer Discovery

[marcosio]

When configuring a private Besu network with mixed cryptographic curves (bootnode using secp256r1, validators using default secp256k1), the network becomes completely inoperable. Validators fail to mine blocks due to incompatible peer discovery protocols, resulting in a permanent network lock.

Severity

Critical - This issue causes 100% network failure with no recovery possible.

Environment

• Besu Version: 23.10.0
• Java Version: OpenJDK 11.0.20
• OS: Ubuntu 22.04 LTS
• Docker: 24.0.7 (if applicable)

Steps to Reproduce

1. Set up a local Besu network using isbe-local-besu-deployer
2. Configure bootnode with secp256r1 curve in genesis:

   "config": {
     "chainId": 1337,
     "ethash": {},
     "crypto": {
       "curve": "secp256r1"
     }
   }

3. Configure validator nodes with the default secp256k1 curve
4. Start the network and attempt a peer connection
5. Result: Network becomes permanently locked

Expected Behaviour

• The network should detect curve mismatch during the initial handshake
• Incompatible peers should be rejected gracefully with clear error logging
• Validators should continue mining with compatible peers
• No network lockup should occur

Actual Behaviour

• 100% network failure: Validators never mine blocks
• Complete network lock: No peer discovery or communication occurs
• Repeated errors: "Could not recover public key" exceptions in logs
• No graceful rejection: IllegalArgumentException instead of proper peer filtering

Error Logs

2025-12-23 13:11:16.443+0000 | vert.x-eventloop-thread-3 | ERROR | VertxPeerDiscoveryAgent | Encountered error while handling packet
java.lang.IllegalArgumentException: Could not recover public key
at org.hyperledger.besu.crypto.SECP256K1.recoverPublicKeyFromSignature(SECP256K1.java:139)
at org.hyperledger.besu.ethereum.p2p.discovery.internal.packet.PacketFactory.create(PacketFactory.java:126)
at org.hyperledger.besu.ethereum.p2p.discovery.internal.packet.PacketDeserializer.decode(PacketDeserializer.java:104)
at org.hyperledger.besu.ethereum.p2p.discovery.VertxPeerDiscoveryAgent.lambda$handlePacket$9(VertxPeerDiscoveryAgent.java:331)
at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$5(ContextImpl.java:205)

Root Cause Analysis

1. Discovery Protocol Failure: SECP256K1 signature verification attempts to process secp256r1 signatures, causing exceptions
2. No Handshake Validation: Crypto algorithm information is not exchanged during the initial handshake
3. Exception Propagation: IllegalArgumentException instead of proper peer rejection
4. Cascading Failure: Discovery failures prevent block production entirely

Suggested Fixes (Priority Order)

1. Add crypto algorithm exchange during handshake - Include curve information in initial discovery packets
2. Implement graceful peer rejection - Validate curve compatibility before signature verification
3. Improve error logging - Provide clear messages about curve incompatibility
4. Add network validation - Warn if mixed curves are detected during network setup

Additional Information

• Reproduction Repository: https://github.com/red-isbe/isbe-besu-local-deployer
• Key observations:
  • Error occurs in SECP256K1.recoverPublicKeyFromSignature() during signature verification
  • Incompatibility should be detected before the signature verification attempt
  • Current implementation lacks a proper peer filtering mechanism
  • The issue is 100% reproducible with a mixed curve configuration

Code References

• org.hyperledger.besu.crypto.SECP256K1.recoverPublicKeyFromSignature (line 139)
• org.hyperledger.besu.ethereum.p2p.discovery.internal.packet.PacketFactory.create (line 126)
• org.hyperledger.besu.ethereum.p2p.discovery.VertxPeerDiscoveryAgent.handlePacket (line 331)