Metrics

FluxaORM exposes Prometheus metrics for MySQL queries, Redis commands, ClickHouse queries, NATS operations and stream consumers. Metrics are disabled by default and cost nothing until you enable them.

Enabling Metrics

Pass a promauto.Factory to the registry with EnableMetrics before calling Validate(). The metric vectors are created inside Validate(), so enabling metrics afterwards has no effect.

import (
    "net/http"

    "github.com/prometheus/client_golang/prometheus"
    "github.com/prometheus/client_golang/prometheus/promauto"
    "github.com/prometheus/client_golang/prometheus/promhttp"

    "github.com/latolukasz/fluxaorm/v2"
)

func main() {
    registry := fluxaorm.NewRegistry()
    registry.RegisterMySQL("user:password@tcp(localhost:3306)/db", fluxaorm.DefaultPoolCode, &fluxaorm.MySQLOptions{})
    registry.RegisterRedis("localhost:6379", 0, fluxaorm.DefaultPoolCode, nil)

    factory := promauto.With(prometheus.DefaultRegisterer)
    registry.EnableMetrics(factory)

    engine, err := registry.Validate()
    if err != nil {
        panic(err)
    }

    // expose everything registered in prometheus.DefaultRegisterer
    http.Handle("/metrics", promhttp.Handler())
    _ = engine
}

The signature is:

EnableMetrics(factory promauto.Factory)

Use promauto.With(registerer) to target a custom registerer, e.g. prometheus.WrapRegistererWith(prometheus.Labels{"app": "api"}, prometheus.DefaultRegisterer) to add constant labels to every FluxaORM metric.

The source Label

Every metric except the consumer ones carries a source label. It defaults to "default" and is read from the context metadata key fluxaorm.MetricsMetaKey:

ctx.SetMetaData(fluxaorm.MetricsMetaKey, "checkout_api")

All queries executed through ctx (and through contexts created with ctx.Clone() / ctx.CloneWithContext(), which copy the metadata) are then tagged source="checkout_api".

Task consumers set the tag automatically: before a task handler runs, the consumer calls SetMetaData(MetricsMetaKey, taskName) on the handler's context, so every MySQL, Redis, ClickHouse and NATS metric emitted while handling a task carries source="<task name>". See Tasks.

MySQL Metrics

fluxaorm_db_queries_seconds (Histogram)

Help: Total number of DB queries executed. Default Prometheus buckets.

fluxaorm_db_queries_errors (Counter)

Help: Total number of DB queries errors. Incremented when a MySQL query returns an error.

Redis Metrics

fluxaorm_redis_queries_seconds (Histogram)

Help: Total number of Redis queries executed. Default Prometheus buckets. One observation per command executed through the FluxaORM Redis client, including commands issued by generated code (Redis cache, Redis Search) and by the distributed lock.

Operation categories and which methods report set="1":

miss="1" is reported by Get when the key does not exist, by HGet when the field does not exist, and by MGet / HMGet when at least one requested value is nil. All other reads report miss="0".

GetSet is recorded as a Get plus, on a miss, a Set. IncrWithExpire is recorded once as key / set="1". XReadGroup with Block >= 0 is not observed here at all (see fluxaorm_redis_queries_block).

Pipelines

When a Redis pipeline is executed, every command in it gets one observation with pipeline="1" and a duration equal to the whole Exec time divided by the number of commands. The category is derived from the command name:

fluxaorm_redis_queries_block (Counter)

Help: Total number of Redis blocking queries executed. Incremented only by XReadGroup when XReadGroupArgs.Block >= 0 (note that the zero value of Block counts as blocking). Such calls are not timed and do not appear in fluxaorm_redis_queries_seconds.

fluxaorm_redis_queries_errors (Counter)

Help: Total number of Redis queries errors. Incremented when a Redis command (including a pipeline Exec or a lock operation) returns an error.

ClickHouse Metrics

fluxaorm_clickhouse_queries_seconds (Histogram)

Help: Total number of ClickHouse queries executed. Default Prometheus buckets.

fluxaorm_clickhouse_queries_errors (Counter)

Help: Total number of ClickHouse queries errors.

NATS Metrics

See NATS for the publisher and consumer API.

fluxaorm_nats_operations_seconds (Histogram)

Help: Total number of NATS operations executed. Default Prometheus buckets.

fluxaorm_nats_operations_errors (Counter)

Help: Total number of NATS operation errors.

fluxaorm_nats_publish_batch_size (Histogram)

Help: Number of messages per batched NATS publish. Buckets: prometheus.ExponentialBuckets(1, 2, 12) (1, 2, 4, ... 2048). Observed once per PublishBatch call with the number of messages in the batch. Use it to check whether batching actually amortises round trips.

Consumer Metrics

These are emitted by consumers reading entity events from JetStream. They have no source label.

fluxaorm_cdc_messages_total (Counter)

Help: Total number of entity change events fetched from JetStream. Incremented once per entity-change message fetched by a consumer, before the handler runs.

fluxaorm_stream_consume_lag_seconds (Histogram)

Help: Time from publish (JetStream broker timestamp) to consume, per consumer. Buckets: prometheus.ExponentialBuckets(0.001, 4, 10) (1ms to about 262s). Observed per fetched entity-change message as time.Since(msg.Timestamp), skipped when the broker timestamp is zero.

Summary