Loki¶

What Is It?¶

Loki is an open-source log aggregation system developed by Grafana Labs. It stores and indexes logs with a labels-based approach (similar to how Prometheus handles metrics), making it very storage-efficient. Unlike Elasticsearch, Loki does not index the full text of log entries — it only indexes metadata labels, and queries use regex pattern matching against the raw text.

Loki is designed to work closely with Promtail (a log shipping agent) and Grafana (for visualization and searching).

Installation¶

Loki is deployed as a Docker Compose service. No dnf installation is needed.

Container image:

grafana/loki:latest — log aggregation backend

Key Files and Directories¶

loki-config.yaml: Loki configuration file. Defines storage paths, ingestion limits, and schema. Bind-mounted into the container at /etc/loki/local-config.yaml.
/loki (inside container): Storage directory for log chunks and the TSDB index. Backed by a named Docker volume.

Default Ports¶

Port	Protocol	Purpose
3100	TCP	Loki HTTP API (ingestion and queries)

Configuration¶

Minimal Working Configuration¶

loki-config.yaml:

auth_enabled: false

server:
  http_listen_port: 3100

common:
  instance_addr: 127.0.0.1
  path_prefix: /loki
  storage:
    filesystem:
      chunks_directory: /loki/chunks
      rules_directory: /loki/rules
  replication_factor: 1
  ring:
    kvstore:
      store: inmemory

schema_config:
  configs:
    - from: 2020-10-24
      store: tsdb
      object_store: filesystem
      schema: v13
      index:
        prefix: index_
        period: 24h

limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h
  allow_structured_metadata: false

Docker Compose service definition (add to docker-compose.yml):

loki:
  image: grafana/loki:latest
  volumes:
    - ./loki-config.yaml:/etc/loki/local-config.yaml:ro
    - loki-data:/loki
  command: -config.file=/etc/loki/local-config.yaml
  ports:
    - "3100:3100"
  restart: unless-stopped

Open the firewall port so the scoring server can reach Loki:

sudo firewall-cmd --permanent --add-port=3100/tcp
sudo firewall-cmd --reload

Important Directives¶

auth_enabled: false: Disables multi-tenant authentication. Required for single-instance use.
schema_config: Defines which storage backend and schema version to use. tsdb + v13 is the current recommended combination for new installations.
limits_config.reject_old_samples_max_age: Rejects log entries older than this value. 168h (7 days) prevents Loki from refusing logs from Promtail if the system clock drifts or the VM was offline.
allow_structured_metadata: false: Required for Loki 3.x when not using the structured metadata feature. Without this, Loki rejects pushes from older Promtail versions.

Common Commands¶

# Check Loki is ready
curl http://<vm-ip>:3100/ready

# Query logs for the 'varlogs' job (last 10 entries)
curl "http://<vm-ip>:3100/loki/api/v1/query?query={job=\"varlogs\"}&limit=10"

# View Loki logs
sudo docker compose logs loki

# Tail Loki logs
sudo docker compose logs -f loki

Logging and Debugging¶

ready endpoint not responding: Loki takes 30–60 seconds to initialize on first start. Wait and retry.
TSDB head not found errors: normal on first startup; Loki is creating its index files.
Promtail gets 429 Too Many Requests: limits_config is too restrictive. Increase ingestion_rate_mb or check reject_old_samples_max_age.
No logs appear in Grafana: check that Promtail is running and the Loki data source URL in Grafana is http://loki:3100.

Troubleshooting checklist:

sudo docker compose ps — is the loki container running?
curl http://localhost:3100/ready — does it return ready?
sudo firewall-cmd --list-ports — is port 3100 open?
sudo docker compose logs loki — any startup errors?

Security Considerations¶

No authentication: Loki's HTTP API is unauthenticated. Anyone who can reach port 3100 can read and write logs. Restrict access via firewall rules.
Disk usage: logs accumulate over time. Monitor the Docker volume size and configure retention if needed.