Bedrock — Go Observability Library

Overview

Bedrock is built on the principle that observability should be automatic, consistent, and non-intrusive. Rather than requiring developers to manually instrument every function with metrics, traces, and logs, Bedrock provides high-level primitives that handle this automatically.

The core design philosophy is context-based flow: all observability state is carried through context.Context, eliminating global state and making it trivial to test, compose, and reason about instrumented code.

Automatic Metrics

Every operation automatically records count, success, failure, and duration. No manual instrumentation required.

Controlled Cardinality

Define metric labels upfront with MetricLabels() to prevent label explosion in production.

W3C Trace Context

Standards-compliant distributed tracing with automatic header propagation across service boundaries.

HTTP Instrumentation

Built-in middleware for handlers and instrumented clients that automatically propagate trace context.

Observability Server

Prometheus metrics, pprof profiling, and health check endpoints ready out of the box.

Production Defaults

Security-hardened timeouts and DoS protections like Slowloris mitigation built in.

Runtime Metrics

Automatic Go runtime metrics collection including goroutine counts, memory stats, and GC metrics.

Design Principle

Operations are success by default. They only count as failures when you explicitly register an error via attr.Error(err). This eliminates false positives and ensures your metrics accurately reflect actual failures.

Installation

Install Bedrock using Go modules. Requires Go 1.21 or later.

go get github.com/kzs0/bedrock

Quick Start

Initialize Bedrock at application startup. The Init function returns a context carrying all observability state and a cleanup function that should be deferred.

                package main

import (
    "context"
    "net/http"

    "github.com/kzs0/bedrock"
    "github.com/kzs0/bedrock/attr"
)

func main() {
    // Initialize bedrock - returns enriched context and cleanup function
    ctx, close := bedrock.Init(context.Background(),
        bedrock.WithStaticAttrs(attr.String("env", "production")),
    )
    defer close()

    // Create HTTP handler
    mux := http.NewServeMux()
    mux.HandleFunc("/users", handleUsers)

    // Wrap with bedrock middleware for automatic instrumentation
    handler := bedrock.HTTPMiddleware(ctx, mux)

    http.ListenAndServe(":8080", handler)
}

func handleUsers(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()

    // Start an operation - metrics are recorded automatically
    op, ctx := bedrock.Operation(ctx, "fetch_users",
        bedrock.MetricLabels("status"),
    )
    defer op.Done()

    users, err := getUsers(ctx)
    if err != nil {
        // Register error - marks operation as failed
        op.Register(ctx, attr.Error(err))
        op.Register(ctx, attr.String("status", "error"))
        http.Error(w, "internal error", 500)
        return
    }

    op.Register(ctx, attr.String("status", "success"))
    op.Register(ctx, attr.Int("user_count", len(users)))
    // ... send response
}
            

Operations

Operations are the primary unit of work in Bedrock. Each operation represents a discrete task that you want to observe, like an API request handler, a database query, or a message processor.

When you create an operation, Bedrock automatically:

Creates a trace span for distributed tracing
Starts a timer for duration measurement
Prepares to record success/failure metrics
Sets up structured logging context

Automatic Metrics

Every operation generates four metrics automatically, using the operation name as the base:

{name}_count

Total number of operations started

{name}_successes

Operations completed without error

{name}_failures

Operations that registered an error

{name}_duration_ms

Histogram of operation durations

Creating Operations

                // Basic operation
op, ctx := bedrock.Operation(ctx, "process_order")
defer op.Done()

// With initial attributes (added to trace span)
op, ctx := bedrock.Operation(ctx, "process_order",
    bedrock.Attrs(
        attr.String("order_id", orderID),
        attr.String("customer_id", customerID),
    ),
)

// With metric labels (controls cardinality)
op, ctx := bedrock.Operation(ctx, "process_order",
    bedrock.MetricLabels("order_type", "region"),
)

// Register attributes during execution
op.Register(ctx, attr.String("order_type", "subscription"))
op.Register(ctx, attr.String("region", "us-east"))
op.Register(ctx, attr.Int("item_count", 5))
            

Recording Failures

Operations succeed by default. To record a failure, register an error attribute:

                result, err := processOrder(ctx, order)
if err != nil {
    // This marks the operation as failed
    op.Register(ctx, attr.Error(err))
    return err
}

// Operation completes as success (no error registered)
            

Nested Operations

Operations can be nested. Child operations automatically inherit the parent's trace context and create child spans:

                func handleRequest(ctx context.Context) {
    op, ctx := bedrock.Operation(ctx, "handle_request")
    defer op.Done()

    // Child operation - creates nested span
    validateInput(ctx)
    processData(ctx)
}

func validateInput(ctx context.Context) {
    op, ctx := bedrock.Operation(ctx, "validate_input")
    defer op.Done()
    // Automatically a child of handle_request
}
            

NoTrace Option

For high-frequency operations where tracing overhead is a concern, use the NoTrace option to record metrics without creating trace spans. This is useful for operations that run thousands of times per second where full tracing would be prohibitively expensive.

                // Operation with metrics only - no trace span created
op, ctx := bedrock.Operation(ctx, "cache_lookup",
    bedrock.NoTrace(),
    bedrock.MetricLabels("cache_name"),
)
defer op.Done()

// Still records metrics: cache_lookup_count, cache_lookup_successes, etc.
// But no trace span is created, reducing overhead
            

Sources

Sources represent long-running processes that spawn multiple operations over time: background workers, queue consumers, WebSocket handlers, or any persistent goroutine that processes work continuously.

Unlike operations which track individual units of work, sources track aggregate metrics across their entire lifetime and automatically prefix child operations with their name.

                func runWorker(ctx context.Context) {
    // Create a source for this long-running worker
    source, ctx := bedrock.Source(ctx, "order_processor",
        bedrock.SourceAttrs(attr.String("worker_id", workerID)),
        bedrock.SourceMetricLabels("queue"),
    )
    defer source.Done()

    for {
        select {
        case <-ctx.Done():
            return
        case order := <-orderCh:
            // Child operation: "order_processor.process"
            processOrder(ctx, order)

            // Track aggregate metrics on the source
            source.Aggregate(ctx,
                attr.Sum("orders_processed", 1),
                attr.Gauge("queue_depth", len(orderCh)),
            )
        }
    }
}

func processOrder(ctx context.Context, order Order) {
    // Creates "order_processor.process" operation
    op, ctx := bedrock.Operation(ctx, "process")
    defer op.Done()
    // ...
}
            

Aggregate Metrics

Sources support three types of aggregate metrics:

attr.Sum(name, value) - Cumulative counter (e.g., total items processed)
attr.Gauge(name, value) - Point-in-time value (e.g., queue depth)
attr.Histogram(name, value) - Distribution (e.g., batch sizes)

Steps

Steps are lightweight tracing spans that add granularity to traces without generating separate metrics. Use steps when you want to see timing breakdown within an operation but don't need individual success/failure tracking or metric cardinality.

                func handleRequest(ctx context.Context) {
    op, ctx := bedrock.Operation(ctx, "handle_request")
    defer op.Done()

    // Steps add trace detail without metric overhead
    {
        step := bedrock.Step(ctx, "parse_input")
        input := parseInput(request)
        step.Register(ctx, attr.Int("input_size", len(input)))
        step.Done()
    }

    {
        step := bedrock.Step(ctx, "validate")
        validate(input)
        step.Done()
    }

    {
        step := bedrock.Step(ctx, "transform")
        result := transform(input)
        step.Done()
    }
}
            

In the trace, you'll see the parent handle_request span with three child spans (parse_input, validate, transform) showing exactly where time was spent.

HTTP Instrumentation

Server Middleware

Wrap your HTTP handlers to automatically create operations for each request with standardized attributes:

                mux := http.NewServeMux()
mux.HandleFunc("/api/users", handleUsers)
mux.HandleFunc("/api/orders", handleOrders)

handler := bedrock.HTTPMiddleware(ctx, mux,
    // Customize operation name (default: "http.request")
    bedrock.WithOperationName("api.request"),

    // Add extra labels to metrics
    bedrock.WithAdditionalLabels("user_agent"),

    // Custom attribute extraction
    bedrock.WithAdditionalAttrs(func(r *http.Request) []attr.Attr {
        return []attr.Attr{
            attr.String("user_id", r.Header.Get("X-User-ID")),
        }
    }),

    // Define which status codes count as success
    bedrock.WithSuccessCodes(200, 201, 204, 304),
)

http.ListenAndServe(":8080", handler)
            

Auto-captured attributes:

http.method - GET, POST, etc.
http.route - The matched route pattern
http.scheme - http or https
http.host - The request host
http.status_code - Response status code

Client Instrumentation

Instrumented HTTP clients automatically inject trace context headers into outgoing requests:

                // Wrap an existing client
client := bedrock.NewClient(&http.Client{
    Timeout: 30 * time.Second,
})
resp, err := client.Get(ctx, "https://api.example.com/users")

// Or use convenience functions
resp, err := bedrock.Get(ctx, "https://api.example.com/users")
resp, err := bedrock.Post(ctx, url, "application/json", body)
resp, err := bedrock.Do(ctx, req)
            

Distributed Tracing

Bedrock implements the W3C Trace Context standard for distributed tracing, ensuring compatibility with any tracing backend that supports the standard (Jaeger, Zipkin, OpenTelemetry, etc.).

Automatic Propagation

Trace context is automatically propagated via HTTP headers:

traceparent - Contains trace ID, span ID, and flags
tracestate - Vendor-specific trace data

                // Header format: traceparent: 00-{trace-id}-{parent-id}-{flags}
// Example:      traceparent: 00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01

// Middleware extracts inbound context automatically
handler := bedrock.HTTPMiddleware(ctx, mux)

// Client injects context into outbound requests automatically
resp, err := bedrock.Get(ctx, "https://downstream-service/api")
            

Manual Context Handling

For non-HTTP transports (message queues, gRPC, etc.), you can manually handle trace context:

                // Extract trace context from incoming message
spanCtx, err := trace.ExtractContext(message.Headers)

// Create operation with remote parent
op, ctx := bedrock.Operation(ctx, "process_message",
    bedrock.WithRemoteParent(spanCtx),
)

// Inject context into outgoing message
b := bedrock.FromContext(ctx)
b.Tracer().Inject(ctx, outgoingMessage.Headers)
            

Logging

Bedrock provides structured logging that automatically includes static attributes and trace context. Logs are correlated with traces, making it easy to find relevant logs for a specific request.

                // Log levels
bedrock.Debug(ctx, "cache lookup", attr.String("key", cacheKey))
bedrock.Info(ctx, "user logged in", attr.String("user_id", userID))
bedrock.Warn(ctx, "rate limit approaching", attr.Int("current", 95))
bedrock.Error(ctx, "database connection failed", attr.Error(err))

// Generic log with custom level
bedrock.Log(ctx, slog.LevelInfo, "custom message", attr.String("key", "value"))

// Access underlying slog.Logger for advanced use
logger := bedrock.FromContext(ctx).Logger()
            

Canonical Logs

When LogCanonical is enabled, Bedrock emits a single comprehensive log line at the end of each operation containing all registered attributes. This is useful for analytics pipelines that process structured logs.

Metrics

Beyond automatic operation metrics, Bedrock provides type-safe APIs for creating custom metrics that automatically include static labels defined at initialization.

Counter

                // Create a counter with labels
requestCounter := bedrock.Counter(ctx,
    "http_requests_total",      // name
    "Total HTTP requests",       // help text
    "method", "status",           // label names
)

// Increment with labels
requestCounter.With(
    attr.String("method", "GET"),
    attr.String("status", "200"),
).Inc()

// Add arbitrary value
requestCounter.With(...).Add(5)

// Without labels (uses static labels only)
simpleCounter := bedrock.Counter(ctx, "events_total", "Total events")
simpleCounter.Inc()
            

Gauge

                queueGauge := bedrock.Gauge(ctx,
    "queue_depth",
    "Current queue depth",
    "queue_name",
)

queueGauge.With(attr.String("queue_name", "orders")).Set(42)
queueGauge.With(...).Inc()
queueGauge.With(...).Dec()
queueGauge.With(...).Add(10)
queueGauge.With(...).Sub(5)
            

Histogram

                // With default buckets
latencyHist := bedrock.Histogram(ctx,
    "request_duration_ms",
    "Request duration in milliseconds",
    nil,  // use default buckets
    "endpoint",
)

// With custom buckets
sizeHist := bedrock.Histogram(ctx,
    "response_size_bytes",
    "Response size in bytes",
    []float64{100, 1000, 10000, 100000},
    "endpoint",
)

latencyHist.With(attr.String("endpoint", "/users")).Observe(123.45)
            

Runtime Metrics

Bedrock automatically collects Go runtime metrics with static attributes. These metrics provide visibility into your application's resource usage and garbage collection behavior.

go_goroutines

Number of active goroutines

go_memstats_alloc_bytes

Bytes allocated and in use

go_memstats_heap_objects

Number of allocated heap objects

go_gc_duration_seconds

GC pause duration histogram

Runtime metrics are automatically exposed on the /metrics endpoint and include your static attributes as labels for consistent filtering across all metrics.

Configuration

Bedrock can be configured programmatically or via environment variables. Environment variables take precedence when using FromEnv().

Environment Variables

Variable	Description	Default
BEDROCK_SERVICE	Service name for identification	`unknown`
BEDROCK_LOG_LEVEL	Minimum log level (DEBUG, INFO, WARN, ERROR)	`INFO`
BEDROCK_LOG_FORMAT	Output format (json or text)	`json`
BEDROCK_LOG_CANONICAL	Enable canonical log lines per operation	`false`
BEDROCK_LOG_ADD_SOURCE	Include source file:line in logs	`false`
BEDROCK_TRACE_URL	OTLP endpoint for trace export	`""`
BEDROCK_TRACE_SAMPLE_RATE	Trace sampling rate (0.0 - 1.0)	`1.0`
BEDROCK_METRIC_PREFIX	Prefix for all metric names	`""`
BEDROCK_SERVER_ENABLED	Enable observability server	`true`
BEDROCK_SERVER_ADDR	Observability server address	`:9090`
BEDROCK_SHUTDOWN_TIMEOUT	Graceful shutdown timeout	`30s`

Programmatic Configuration

                // Using Config struct
cfg := bedrock.Config{
    Service:         "my-service",
    LogLevel:        "info",
    LogFormat:       "json",
    LogCanonical:    true,
    TraceURL:        "http://localhost:4318/v1/traces",
    TraceSampleRate: 0.1,  // 10% sampling
    MetricPrefix:    "myapp",
    ServerEnabled:   true,
    ServerAddr:      ":9090",
    ShutdownTimeout: 30 * time.Second,
}

ctx, close := bedrock.Init(ctx, bedrock.WithConfig(cfg))

// Using init options
ctx, close := bedrock.Init(ctx,
    bedrock.WithLogLevel("debug"),
    bedrock.WithStaticAttrs(
        attr.String("env", "production"),
        attr.String("version", "1.2.3"),
    ),
)

// Load from environment with defaults
cfg, err := bedrock.FromEnv()
// or panic on error
cfg := bedrock.MustFromEnv()
            

Observability Server

Bedrock includes a built-in HTTP server that exposes metrics, profiling, and health check endpoints. This server runs separately from your application server and is typically bound to a different port for security.

Endpoints

Path	Description
/metrics	Prometheus-format metrics
/health	Liveness check (returns 200 if running)
/ready	Readiness check for load balancers
/debug/pprof/	pprof index
/debug/pprof/profile	CPU profile
/debug/pprof/heap	Heap memory profile
/debug/pprof/goroutine	Goroutine stack traces
/debug/pprof/trace	Execution trace

                // Server starts automatically with Init() if enabled
// Access metrics: curl http://localhost:9090/metrics
// Health check:   curl http://localhost:9090/health
// CPU profile:    go tool pprof http://localhost:9090/debug/pprof/profile
            

Security Defaults

Bedrock's observability server includes production-hardened defaults to protect against common denial-of-service attacks. These can be customized via configuration if needed.

Setting	Default	Purpose
ReadHeaderTimeout	`5s`	Slowloris attack protection
ReadTimeout	`10s`	Slow-read attack protection
WriteTimeout	`30s`	Slow-write attack protection
IdleTimeout	`120s`	Connection cleanup for keep-alives
MaxHeaderBytes	`1MB`	Header bomb prevention

API Reference

Complete API documentation is available on pkg.go.dev, including the full reference generated from Go docstrings.

pkg.go.dev/github.com/kzs0/bedrock Browse the complete API documentation, type definitions, and source code on Go's official package registry. View Documentation

Core Functions

func Init(ctx context.Context, opts ...InitOption) (context.Context, func())

Initialize Bedrock and return an enriched context with a cleanup function. The cleanup function should be deferred to ensure proper shutdown.

func Operation(ctx context.Context, name string, opts ...OperationOption) (*Op, context.Context)

Start a new operation. If a parent operation exists in the context, creates a child operation with automatic trace linking.

func Source(ctx context.Context, name string, opts ...SourceOption) (*Src, context.Context)

Register a long-running source. Child operations are automatically prefixed with the source name.

func Step(ctx context.Context, name string, attrs ...attr.Attr) *OpStep

Create a lightweight tracing step within the current operation without generating separate metrics.

func FromContext(ctx context.Context) *Bedrock

Extract the Bedrock instance from context for direct access to Logger, Metrics, and Tracer.

HTTP Functions

func HTTPMiddleware(ctx context.Context, handler http.Handler, opts ...MiddlewareOption) http.Handler

Wrap an HTTP handler with automatic operation creation, trace propagation, and standardized attributes.

func NewClient(base *http.Client) *http.Client

Create an instrumented HTTP client that automatically injects trace context into outgoing requests.

func Get(ctx context.Context, url string) (*http.Response, error)

Convenience function for instrumented GET requests.

func Post(ctx context.Context, url, contentType string, body io.Reader) (*http.Response, error)

Convenience function for instrumented POST requests.

func Do(ctx context.Context, req *http.Request) (*http.Response, error)

Execute an instrumented HTTP request.

Logging Functions

func Debug(ctx context.Context, msg string, attrs ...attr.Attr)

Log at DEBUG level with automatic static attributes and trace context.

func Info(ctx context.Context, msg string, attrs ...attr.Attr)

Log at INFO level with automatic static attributes and trace context.

func Warn(ctx context.Context, msg string, attrs ...attr.Attr)

Log at WARN level with automatic static attributes and trace context.

func Error(ctx context.Context, msg string, attrs ...attr.Attr)

Log at ERROR level with automatic static attributes and trace context.

Metrics Functions

func Counter(ctx context.Context, name, help string, labelNames ...string) *CounterWithStatic

Create or retrieve a counter metric with automatic static labels.

func Gauge(ctx context.Context, name, help string, labelNames ...string) *GaugeWithStatic

Create or retrieve a gauge metric with automatic static labels.

func Histogram(ctx context.Context, name, help string, buckets []float64, labelNames ...string) *HistogramWithStatic

Create or retrieve a histogram metric with automatic static labels. Pass nil for default buckets.

Init Options

func WithConfig(cfg Config) InitOption

Initialize with a complete configuration struct.

func WithStaticAttrs(attrs ...attr.Attr) InitOption

Set static attributes that are automatically added to all metrics, logs, and traces.

func WithLogLevel(level string) InitOption

Override the log level (DEBUG, INFO, WARN, ERROR).

Operation Options

func Attrs(attrs ...attr.Attr) OperationOption

Set initial attributes on the operation's trace span.

func MetricLabels(labelNames ...string) OperationOption

Define which attributes should be used as metric labels. Controls cardinality.

func WithRemoteParent(parent trace.SpanContext) OperationOption

Link operation to a remote parent span for cross-service tracing.

func NoTrace() OperationOption

Disable trace span creation for this operation. Metrics are still recorded, but no tracing overhead is incurred.