Log Collector

Collect nginx and traefik ingress access logs and turn them into request tallies.

The backend chart can optionally run an ingress log collector. It reads your nginx or traefik controller access logs — by default straight from the controller pods via the Kubernetes API, so no sidecar is required — and ships request counts to the backend, where they're stored as tallies.

One lightweight collector Deployment is rendered per entry in collector.sources.

Prerequisites

A LogSource created per controller through the backend API (POST /v1/log-sources). Note the returned sourceId.
An API key with the write:logs scope, stored in a Kubernetes Secret referenced by collector.apiKey.secretName.

Enable the collector

collector:
  enabled: true
  apiKey:
    secretName: taptally-collector # holds key "api-key"
  rbac:
    create: true
    scope: ClusterRole # or "Role" (one Role per source namespace)
  pollInterval: 10 # seconds between polls
  windowSeconds: 60 # per-poll log lookback window
  sources:
    - name: nginx
      ingressType: nginx
      sourceId: 1
      logFormat: combined
      kubernetes:
        namespace: ingress-nginx
        selector: "app.kubernetes.io/name=ingress-nginx"
        container: controller
    - name: traefik
      ingressType: traefik
      sourceId: 2
      logFormat: json
      kubernetes:
        namespace: traefik
        selector: "app.kubernetes.io/name=traefik"

Source fields

Field	Description	Default
`name`	DNS-safe name used in resource names	(required)
`ingressType`	`nginx` or `traefik`	(required)
`sourceId`	The `LogSource` id from the backend API	(required)
`sourceType`	`kubernetes` (default) or `file`	`kubernetes`
`logFormat`	Log format the controller emits (see below)	`auto`
`logPattern`	Custom access-log pattern; used only with `logFormat: custom`	`""`
`timestampFormat`	Optional strptime pattern for the timestamp field	`""`
`kubernetes.namespace`	Namespace of the controller pods	(required)
`kubernetes.selector`	Label selector for the controller pods	(required)
`kubernetes.container`	Container name within the pod (optional)	(auto)

nginx vs traefik log formats

The accepted logFormat presets depend on the ingress type:

`ingressType`	`logFormat` presets	Notes
`nginx`	`auto`, `combined`, `json`, `custom`	`combined` is the ingress-nginx default; use `json` when the controller sets `log-format-escape-json: "true"` / a JSON `log_format`.
`traefik`	`auto`, `common`, `json`, `custom`	`common` is the Common Log Format (CLF); use `json` when Traefik has `accessLog.format: json`.

auto — let the backend parser detect the format. A good default for stock controller configurations.
combined / common / json — pin a known format explicitly; more robust than auto and avoids misdetection.
custom — for a non-default nginx log_format. Set logPattern to the exact log_format directive string so fields map correctly. Only set timestampFormat if your format uses a non-standard time layout.

Custom nginx format example

collector:
  enabled: true
  apiKey:
    secretName: taptally-collector
  sources:
    - name: nginx-custom
      ingressType: nginx
      sourceId: 3
      logFormat: custom
      logPattern: '$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent'
      kubernetes:
        namespace: edge
        selector: "app.kubernetes.io/name=ingress-nginx"

RBAC

When collector.rbac.create is true, the chart grants the collector read access to controller pod logs:

scope: ClusterRole — read pod logs cluster-wide (one ClusterRole).
scope: Role — a Role + RoleBinding per source namespace (least privilege).

How it maps to the pod

Each source's settings become TAPTALLY_COLLECTOR_* environment variables on the collector Deployment — including TAPTALLY_COLLECTOR_INGRESS_TYPE, TAPTALLY_COLLECTOR_LOG_FORMAT, and, when set, TAPTALLY_COLLECTOR_LOG_PATTERN and TAPTALLY_COLLECTOR_TIMESTAMP_FORMAT. See the backend chart README for the full env-var reference.

Found an issue in the docs? Open an issue on GitHub.