Skip to content

Add payload size metrics to OTLP receiver #14287

@prasanthi-k

Description

@prasanthi-k

Component(s)

receiver/otlp
exporter/otlp

Is your feature request related to a problem? Please describe.

The OTLP receiver currently lacks metrics to track the size of incoming payloads. This makes it difficult to:

  1. Monitor bandwidth usage per client
  2. Detect unusually large payloads that might impact performance
  3. Monitor Bytes processed

Describe the solution you'd like

Add new metrics to the OTLP receiver and exporter for OTLP payloads

For HTTP and gRPC protocol:

Receivers:
otelcol_receiver_request_size - Histogram measuring incoming OTLP request payload size in bytes (uncompressed)
otelcol_receiver_response_size - Histogram measuring OTLP response size in bytes

Exporters:
otelcol_exporter_request_size - Histogram measuring outgoing OTLP request payload size in bytes (before compression)
otelcol_exporter_response_size - Histogram measuring backend response size in bytes

These metrics should include attributes like:

receiver/exporter - component name (e.g., "otlp", "otlp/2")
transport - protocol used ("grpc" or "http")
data_type - type of signal ("traces", "metrics", "logs")

Describe alternatives you've considered

  1. Using a custom processor - but this adds complexity and doesn't capture data at the ingestion point
  2. Using transform processor - cannot measure original payload size accurately, what ever comes to transform processor will be only considered.
  3. For HTTP and rpc transport there are .server.request.body.size and .server.response.size
    While rpc.server.request.size and http.server.request.body.size metrics exist at the detailed level, they lack OTLP-specific context and don't distinguish between different receivers. We need OTLP receiver-specific payload size metrics similar to how we have receiver-specific otelcol_receiver_accepted_
    and otelcol_receiver_refused_
    metrics. (for exporter I couldn't find how to calculate bytes it is exporting)

Additional context

Real-world evidence: When monitoring http.server.request.body.size at the detailed level, I observe request sizes of ~7.7KB with response sizes of ~128 bytes (expected for acknowledgments). However:

Cannot distinguish this from which HTTP receiver
No way to track exporter payload sizes

Related issue: #13685 describes how oversized messages are rejected before metrics can capture them, making it impossible to observe when size limits are being hit.

No response

Tip

React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions