Skip to main content

OpenMetrics

OvenMediaEngine Enterprise exposes its runtime metrics at /v2/metrics in the OpenMetrics 1.0.0 exposition format. It is a vendor-neutral, pull-based text format that any OpenMetrics-compatible monitoring system can read.

The response is rendered on demand from the live monitoring model and is read-only: scraping never changes collection state.

Endpoint

GET /v2/metrics is served by the API server, on the same host and port as the REST API.

The path narrows what a scrape returns. Labels are always complete (vhost, app, stream, and so on) regardless of the path; the path only decides which series are emitted.

PathCovers
/v2/metricsWhole server
/v2/metrics/vhosts/{vhost}One virtual host
/v2/metrics/vhosts/{vhost}/apps/{app}One application
/v2/metrics/vhosts/{vhost}/apps/{app}/streams/{stream}One stream
/v2/metrics/vhosts/{vhost}/apps/{app}/streams/{stream}/sessionsOne stream, including its per-session metrics

Only the .../sessions path includes per-session metrics by default; at any other path, request them explicitly with ?collect[]=session. See Selecting collectors below.

Authentication

/v2/metrics uses the same HTTP Basic authentication as the rest of the API server, keyed on the configured <AccessToken>.

Set <AccessToken> to a username:password pair, then authenticate with that username and password from any standard Basic-auth client:

curl -u 'user:secret' http://<ome-host>:<api-port>/v2/metrics

The username and password are simply the two sides of the colon in <AccessToken>. A leading-colon token such as :secret means an empty username (curl -u ':secret').

You can also use a single opaque token with no colon. In that case send it as the Base64-encoded Basic credential directly:

curl -H "Authorization: Basic $(printf '%s' 'mytoken' | base64)" http://<ome-host>:<api-port>/v2/metrics

Configuration

The endpoint is available whenever the API manager is enabled. An optional <OpenMetrics> block tunes it:

<Server>
<Managers>
<API>
<AccessToken>user:password</AccessToken>
<OpenMetrics>
<Cache>
<Enable>true</Enable>
<DurationMs>3000</DurationMs>
</Cache>
</OpenMetrics>
</API>
</Managers>
</Server>
  • <Cache> (optional): scrape-response caching.
    • <Enable> (default true): set to false to render every request fresh and bypass the cache.
    • <DurationMs> (default 3000): how long a cached response stays valid, in milliseconds. Keep it well under your scrape interval.

Keep caching on when several collectors scrape the same endpoint, or the scrape interval is short on a deployment with many streams and sessions: overlapping scrapes within DurationMs reuse one render instead of walking the whole model each time. Turn it off when you need exact point-in-time values on every request (for example, manual debugging with curl), or when a single, infrequent scraper makes caching pointless.

Selecting collectors

Metrics are grouped into collectors. Select a subset with repeated query keys (the node_exporter convention). collect[] and exclude[] are mutually exclusive.

  • ?collect[]=traffic&collect[]=stream: emit only these collectors.
  • ?exclude[]=queue: emit everything except these.

Collector names are lowercase. An unknown name returns 400 with the list of valid names. A request with no selection emits all default collectors.

CollectorEmitsIn default scrape
coreBuild info, process start timeAlways on
trafficReceived/transmitted byte countersYes
connectionCurrent connection gaugesYes
streamMedia/track characteristics, source timing, ingest RTT, and SRT ingest packet lossYes
queueManaged-queue gauges and countersYes
pushPush/record byte counters and stateYes
sessionPer-session byte counter, throughput, RTT, and SRT egress packet lossNo (opt-in)

core is always emitted regardless of the selection.

session is not part of the default scrape: per-session series are high-cardinality and churn quickly. Request it explicitly with ?collect[]=session, or scrape the .../streams/{stream}/sessions path.

Per-session series are populated for WebRTC, LLHLS, and HLS playback sessions, and for SRT subscriber sessions. Other publishers (OVT, Thumbnail, File, and so on) are reflected at the stream level (via traffic/connection) but do not create per-session entries.

An SRT subscriber session populates only the SRT egress packet-loss counters. Its bytes and throughput are accounted at the stream level rather than per session, so ome_session_transmit_bytes_total and ome_session_transmit_bps read 0 for SRT sessions (use the stream-level ome_transmit_bytes_total{publisher="srt"} for SRT egress bytes). SRT has no per-session RTT, so ome_session_rtt_seconds is not emitted for SRT sessions at all (no method applies), the same way RTT methods are omitted rather than zeroed for any protocol that does not support them.

Metrics reference

Every metric emitted, grouped by collector. Each carries the entity labels of its scope (vhost/app/stream, plus output_stream, publisher, session/session_id, track/media/codec, method, protocol where applicable). All values are raw (no timestamps).

core (always on)

MetricTypeDescription
ome_build_infoinfoBuild/version identification (value is always 1; data in labels).
process_start_time_secondsgaugeProcess start time, in seconds since the Unix epoch.

traffic

MetricTypeDescription
ome_receive_bytes_totalcounterMedia bytes received from the provider for this stream (excludes protocol/socket overhead).
ome_transmit_bytes_totalcounterMedia bytes transmitted to subscribers, by publisher.

connection

MetricTypeDescription
ome_connectionsgaugeCurrent number of connected sessions, by publisher.
ome_connections_maxgaugePeak concurrent connections observed for this stream.

stream

MetricTypeDescription
ome_stream_source_connect_secondsgaugeTime taken to connect to the origin (0 for non-pull streams).
ome_stream_rtt_secondsgaugeIngest round-trip time, split by method. See Round-trip time.
ome_stream_srt_lost_packets_totalcounterSRT ingest: packets the receiver detected as lost. SRT inputs only. See SRT packet loss.
ome_stream_srt_dropped_packets_totalcounterSRT ingest: packets dropped too-late-to-play (TLPKTDROP). SRT inputs only.
ome_track_bitrate_bpsgaugeMeasured track bitrate (video and audio), by track/media/codec.
ome_track_framerate_fpsgaugeMeasured video frame rate.
ome_track_keyframe_interval_secondsgaugeMeasured video keyframe interval.
ome_track_widthgaugeVideo frame width, in pixels.
ome_track_heightgaugeVideo frame height, in pixels.
ome_track_has_bframesgauge1 if the video track carries B-frames, 0 otherwise.
ome_track_samplerate_hertzgaugeAudio sample rate, in hertz.
ome_track_channelsgaugeNumber of audio channels.

queue

MetricTypeDescription
ome_queue_sizegaugeCurrent number of messages in the managed queue.
ome_queue_peakgaugePeak number of messages observed.
ome_queue_thresholdgaugeConfigured message-count threshold.
ome_queue_dropped_totalcounterTotal messages dropped by the queue.
ome_queue_input_messages_per_secondgaugeMessages enqueued per second.
ome_queue_output_messages_per_secondgaugeMessages dequeued per second.
ome_queue_waiting_secondsgaugeAverage time a message waits in the queue.

push

MetricTypeDescription
ome_push_transmit_bytes_totalcounterMedia bytes transmitted to the push/record target.
ome_push_statestatesetPush lifecycle state: one series per candidate state, value 1 for the active state.

session (opt-in)

MetricTypeDescription
ome_session_transmit_bytes_totalcounterMedia bytes transmitted to this subscriber session.
ome_session_transmit_bpsgaugeCurrent transmit throughput to the session, in bits per second.
ome_session_rtt_secondsgaugePer-session round-trip time, split by protocol and method. See Round-trip time.
ome_session_srt_lost_packets_totalcounterSRT egress: packets the subscriber reported lost via NAK. SRT sessions only. See SRT packet loss.
ome_session_srt_retransmitted_packets_totalcounterSRT egress: packets retransmitted to the subscriber. SRT sessions only.
ome_session_srt_dropped_packets_totalcounterSRT egress: packets dropped too-late-to-send (TLPKTDROP). SRT sessions only.

Round-trip time

Two RTT gauges are exposed in seconds, each carrying a method label that says how the value was measured:

  • ome_session_rtt_seconds (in session): RTT to a playback subscriber. Per-session metrics are produced for WebRTC, LLHLS, and HLS sessions, and every session series also carries a protocol label (webrtc, llhls, hlsv3, ...) so RTT can be split by delivery protocol as well as by method. The methods available depend on the protocol:
    • method="rtcp": derived from RTCP Receiver Reports (the media path). WebRTC only.
    • method="stun": derived from ICE STUN binding request/response timing. WebRTC only.
    • method="tcp": the kernel's smoothed TCP round-trip time (TCP_INFO.tcpi_rtt, SRTT) of the session's socket. Produced for HLS/LLHLS sessions (the HTTP connection that most recently served the session) and for WebRTC sessions carried over ICE-over-TCP; WebRTC sessions on UDP (the common case) emit no tcp series.
    • method="tcp_min": the kernel's minimum observed RTT (TCP_INFO.tcpi_min_rtt) of the same socket - the best-case path floor, filtering out the queuing/retransmit/delayed-ACK inflation carried by tcp (SRTT). Same TCP applicability as tcp.
  • ome_stream_rtt_seconds (in stream): ingest RTT of an input stream, by method:
    • method="stun": from STUN binding request/response timing (WebRTC/WHIP input).
    • method="tcp": the ingest socket's smoothed TCP round-trip time (TCP_INFO.tcpi_rtt, SRTT). Produced only for TCP-based inputs such as RTMP and RTSP-over-TCP; UDP-based inputs emit no tcp series.
    • method="tcp_min": the ingest socket's minimum observed RTT (TCP_INFO.tcpi_min_rtt), the best-case path floor. Same TCP applicability as tcp.

Because these methods measure the same round-trip at different layers, they will not agree exactly: rtcp reflects the media path, stun the ICE application exchange, and tcp the kernel transport (with tcp_min its best-case floor). Comparing them is useful - for example, a stun/rtcp value that spikes while tcp stays flat points to jitter or queuing above the transport rather than a network problem; a large tcp vs tcp_min gap points to queuing/retransmit within the TCP connection itself.

Do not expect tcp/tcp_min to match a raw ping. TCP_INFO derives RTT from the connection's own data/ACK timing, not from an echo probe. tcp (SRTT) is a smoothed estimate, and delayed ACKs, retransmits, and send-buffer queuing under the media load push it well above the true path latency - so on even a fast LAN it can read milliseconds while ping shows microseconds. tcp_min (tcpi_min_rtt) strips that inflation out, so it is the closest of the TCP values to the raw path RTT, and the most directly comparable to stun/rtcp (which are lightweight single-packet probes). Use tcp_min when you want the path floor and tcp when you want what the connection is actually experiencing right now.

Only the methods that apply to a session's/stream's protocol are exported - there are no always-zero series for methods that do not apply (e.g. an HLS/LLHLS session emits only tcp/tcp_min, never rtcp/stun; an RTMP input emits only tcp/tcp_min, never stun). For a method that does apply, a value of 0 means it is supported but has not measured an RTT yet.

The rtcp and stun series are refreshed only when the peer drives new traffic: rtcp on each incoming RTCP Receiver Report, and stun on each STUN binding response (the peer's ICE connectivity/consent-freshness checks, typically every few seconds). If the peer goes silent, these gauges hold their last value until the session or stream is torn down and the series disappears - they are not reset to 0, so treat a non-zero value as "last measured RTT", not necessarily "current". The tcp and tcp_min series are instead sampled at scrape time by reading the socket directly, so they always reflect the current transport state. Each stun value (session, and the stun series of ome_stream_rtt_seconds) reflects the candidate pair that produced the most recent binding response, which is normally the nominated pair but may differ while the peer is still probing multiple pairs.

SRT packet loss

For SRT connections, OvenMediaEngine exposes libsrt's packet-loss counters (from srt_bstats) split by whether the connection is an ingest (SRT provider) or an egress (SRT publisher). Each is a counter reported as a cumulative total since the SRT connection was established; the value only grows, and the series disappears (resetting the count) when that connection reconnects. The counters are sampled directly from the socket at scrape time, so they always reflect the current transport state.

Ingest side, on the stream collector (receiver statistics of the SRT input):

  • ome_stream_srt_lost_packets_total: SRT data packets the receiver detected as lost, before retransmission recovery (pktRcvLossTotal).
  • ome_stream_srt_dropped_packets_total: SRT data packets dropped too-late-to-play by the TLPKTDROP mechanism (pktRcvDropTotal).

Egress side, on the session collector (sender statistics towards one SRT subscriber):

  • ome_session_srt_lost_packets_total: SRT data packets the subscriber reported lost via NAK (pktSndLossTotal).
  • ome_session_srt_retransmitted_packets_total: SRT data packets retransmitted to the subscriber in response (pktRetransTotal).
  • ome_session_srt_dropped_packets_total: SRT data packets dropped too-late-to-send by the TLPKTDROP mechanism (pktSndDropTotal).

These families are emitted only when at least one in-scope stream or session is carried over SRT; non-SRT streams and sessions produce no SRT series at all (rather than always-zero samples). These counters are causally ordered: detected loss (lost) is what triggers a retransmission (retransmitted), and a packet is counted as dropped (TLPKTDROP) only when neither the original nor its retransmission arrives within the latency window. So on a lossy path lost moves first, retransmitted tracks the recovery attempts it triggers, and dropped rises only for the packets that recovery could not deliver in time.

Example queries

A few PromQL starting points (replace the label matchers with your own vhost/app/stream):

# Current viewers of a stream, by publisher
sum by (publisher) (ome_connections{stream="my_stream", output_stream=""})

# Ingest bitrate, bits per second
rate(ome_receive_bytes_total{stream="my_stream", output_stream=""}[1m]) * 8

# Egress bitrate by publisher, bits per second
sum by (publisher) (rate(ome_transmit_bytes_total{stream="my_stream", output_stream=""}[1m])) * 8

# Per-session RTT (all protocols/methods) for an application
ome_session_rtt_seconds{app="my_app"}

# Transport-vs-media RTT divergence for WebRTC sessions
# (large positive = TCP transport is slower than the media-path RTT)
ome_session_rtt_seconds{method="tcp"} - ignoring(method) ome_session_rtt_seconds{method="rtcp"}

Stream-level series (ome_connections, ome_receive_bytes_total, ome_transmit_bytes_total, ome_stream_rtt_seconds) are emitted both for the input stream and for each output rendition. Match output_stream="" to select the input-side aggregate once and avoid double-counting across renditions.

Compression

If the scraper sends Accept-Encoding: gzip, the response is gzip-compressed and returned with Content-Encoding: gzip.

Format notes

The output conforms to OpenMetrics 1.0.0:

  • One # HELP and # TYPE line per metric family.
  • Counters end with _total; values use base units (_bytes, _seconds).
  • No per-sample timestamps.
  • The document is terminated by a single # EOF line.

A minimal scrape looks like:

# HELP ome_build_info OvenMediaEngine build information.
# TYPE ome_build_info info
ome_build_info{version="...",git_version="..."} 1
# HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1783445283.19
# EOF