dlbuild.net
Home Lab Build Log

Monitoring Stack

Building a production-style monitoring platform using Prometheus, Grafana, Node Exporter, Alertmanager, and cAdvisor.

Overview

  • Containerized monitoring stack
  • Real-time and historical metrics
  • Alerting foundation
  • Portfolio-quality documentation

Technology Stack

  • Prometheus (metrics)
  • Grafana (visualization)
  • Node Exporter (host stats)
  • cAdvisor (container stats)
  • Alertmanager (alerts)
  • Docker Compose (orchestration)

System Architecture

  • Exporters → Prometheus
  • Prometheus → Grafana
  • Prometheus → Alertmanager
  • Docker internal networking

Key Challenges

  • Port conflicts with Cockpit
  • YAML syntax failures
  • Grafana datasource routing
  • Dashboard variable resets
  • Container restart loops

Solutions Implemented

  • Custom port remapping
  • Prometheus config validation
  • Docker DNS-based routing
  • Variable refresh automation
  • Centralized health checks

Operations Checklist

  • docker compose ps
  • curl /-/ready
  • Target health validation
  • Dashboard review
  • Log inspection

Results

  • 24/7 visibility
  • Early issue detection
  • Improved system reliability
  • Enterprise-style monitoring