feat: Add HTTP proxy support for OTLP/OTAP gRPC exporters

[lalitb]

fixes: #1678

Summary

Implements HTTP CONNECT proxy support for OTLP/OTAP gRPC exporters:

• HTTP/1.1 CONNECT tunneling to the proxy for both http:// targets (gRPC over h2c inside the tunnel) and https:// targets (TLS inside the tunnel).
• Environment variable support (HTTP_PROXY, HTTPS_PROXY, NO_PROXY, ALL_PROXY)
• NO_PROXY with CIDR notation (e.g., 192.168.0.0/16, 10.0.0.0/8)
• Configures TCP socket options (TCP_NODELAY + keepalive). When using a proxy, these are applied to the TCP connection to the proxy (the same connection that carries the CONNECT tunnel).
• Explicit rejection of https:// proxy URLs with helpful error messages

Why OTLP/OTAP exporters need custom proxy implementation

Azure Monitor and Geneva exporters use the reqwest HTTP client, which provides built-in proxy support via reqwest::Proxy::all() - no custom code needed.

OTLP/OTAP exporters use gRPC (tonic), which:

• Has no built-in proxy support
• Requires custom TCP connectors with tower::service_fn
• Needs manual HTTP CONNECT tunnel implementation for proxy traversal

This PR implements the missing proxy infrastructure for gRPC-based exporters.

Changes

• proxy.rs: HTTP/1.1 CONNECT tunnel implementation with socket option handling
• client_settings.rs: Integration with tonic via custom connector
• Dependencies: Added socket2 and ipnet crates

How HTTP CONNECT Tunneling Works

Step 1: THE HANDSHAKE
The Exporter creates a TCP connection to the Proxy and sends a plaintext request. The Proxy establishes a TCP leg to the Backend.

┌───────────┐  TCP + HTTP/1.1 CONNECT backend:PORT       ┌───────────┐
│ Exporter  │ ─────────────────────────────────────────>.│   Proxy   │
└───────────┘ <─────── 200 Connection Established ────── └───────────┘

Step 2: THE DATA TUNNEL

Once the 200 is received, the Exporter uses the socket for the actual OTLP data. The Proxy merely moves bytes back and forth.

┌───────────┐       ┌───────────┐       ┌───────────────┐
│ OTLP/OTAP │  TCP  │   Proxy   │  TCP  │    Backend    │
│ Exporter  │══════>│ (relays)  │══════>│               │
└───────────┘       └───────────┘       └───────────────┘
          ║                                     ║
          ╚═════════════════════════════════════╝
     On-the-wire protocol inside the tunnel (opaque to proxy):
     - Case 1 (TLS): TLS + gRPC over HTTP/2 (HTTP/2 negotiated via ALPN)
     - Case 2 (h2c): gRPC over HTTP/2 cleartext (HTTP/2 prior knowledge)

Test Setup:

The implementation was verified via a manual end-to-end test setup (not included in the PR). The architecture below was used to validate proxy traversal for both h2c and TLS traffic:

┌─────────────────┐
│ Fake Data Gen   │ Generates test telemetry (logs/traces/metrics)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ OTAP Exporter   │ gRPC client with proxy configuration
│ (df_engine)     │ • HTTP_PROXY=http://localhost:8080
│                 │ • grpc_endpoint=http://remote:4317
│                 │ • admin_port: 8081 (changed to avoid proxy conflict)
└────────┬────────┘
         │
         │ HTTP CONNECT tunneling
         ▼
┌─────────────────┐
│   mitmproxy     │ Intercepts and logs proxy traffic
│   :8080         │ (validates CONNECT requests are working)
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ OTel Collector  │ Receives telemetry via gRPC
│   :4317         │
└────────┬────────┘
         │
         ▼
┌─────────────────┐
│ Debug Exporter  │ Displays received data
└─────────────────┘

[codecov]

Codecov Report

❌ Patch coverage is 88.98734% with 87 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.09%. Comparing base (32a6fbb) to head (3e40a04).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #1679    +/-   ##
========================================
  Coverage   84.08%   84.09%            
========================================
  Files         469      470     +1     
  Lines      137651   138441   +790     
========================================
+ Hits       115746   116422   +676     
- Misses      21371    21485   +114     
  Partials      534      534            

Components	Coverage Δ
otap-dataflow	85.35% <88.98%> (+<0.01%)	⬆️
query_abstraction	80.61% <ø> (ø)
query_engine	90.39% <ø> (ø)
syslog_cef_receivers	∅ <ø> (∅)
otel-arrow-go	53.50% <ø> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lalitb]

We use Arc because tonic/tower requires Send + 'static for connectors.

[lalitb]

This closure runs per TCP connection, not per RPC. With HTTP/2 multiplexing, connections are long-lived (startup + reconnects only), so the cost here is negligible. The cost here involved are:

• Arc::clone for proxy object,
• get_proxy_for_uri lookup
• string operations in should_bypass (have TODO in code to optimize this later).

Also this code path is only invoked when a proxy is configured. Without a proxy, tonic uses its default connector directly, so above overhead is not there.

[lalitb]

this is ready for review now.

[lquerel]

It's really great to add this support to both our OTLP and OTAP exporters. I made a few suggestions, mainly to make this functionality more secure and more robust with certain proxies.

I think it would be a good idea to add complete documentation in a markdown file describing this tunneling capability. For now, we have a very detailed PR description, but unfortunately it won't be seen by future users.

[lquerel]

You could replace this chain of conversions tokio -> std -> socket2 -> std -> tokio by creating a socket2 first, configuring it, and then converting it to a Tokio TcpStream. This would simplify the code and would most likely be more efficient.

[lalitb]

Good suggestion! You're right that creating the socket2 first, configuring it, and then converting to tokio would eliminate the conversion chain.

The main reason I didn't go that route is the async connect handling - since socket2 is synchronous, we'd need to properly handle non-blocking connect (dealing with EINPROGRESS/WouldBlock, waiting for writability, checking SO_ERROR, etc.). The current approach lets Tokio handle the async connect, and we just use socket2 to apply the keepalive options on an already-established socket.

Since this only runs on connection establishment (not per-RPC), I felt the simpler approach is fine for now. Happy to track it as a follow-up issue if you think it's worth revisiting.

[lalitb]

It's really great to add this support to both our OTLP and OTAP exporters. I made a few suggestions, mainly to make this functionality more secure and more robust with certain proxies.

I think it would be a good idea to add complete documentation in a markdown file describing this tunneling capability. For now, we have a very detailed PR description, but unfortunately it won't be seen by future users.

Thanks for the thorough review and great suggestions! I have tried incorporating them in code and with tracking issues. Also copied the architecture details from PR desc to ./docs/proxy-support.md.

[lquerel]

LGTM