auto-syncing kubernetes ingress to uptime monitors

Stub i haven't finished writing this yet. i publish drafts early as part of WFD 17.

i keep forgetting to set up uptime monitors. i'll deploy something, move on, and two weeks later realize it's been unreachable because i never added it to the dashboard. it takes two minutes to add one manually. i just never remember.

so i wrote a cronjob that watches my kubernetes ingress resources, diffs them against existing monitors, and creates whatever's missing. it runs every five minutes. i don't have to think about it anymore.

how it works

a shell script mounted into a kubernetes cronjob via a configmap. it talks to an external uptime API.

deployment

flowchart LR
    tf_values["values.yaml"] -->|"loaded into"| tf_helm["helm_release"]
    tf_secret["kubernetes_secret (api_token)"] -->|"depends_on"| tf_helm
    tf_helm -->|"deploys"| cj["CronJob (*/5 * * * *)"]
    cm["ConfigMap (sync.sh)"] -->|"mounted into"| cj
    secret["Secret (API_TOKEN)"] -->|"env var"| cj
    sa["ServiceAccount (ingresses: get, watch, list)"] -->|"identity for"| cj

sync flow

flowchart TD
    s1["Validate API token"]
    s2["Fetch existing monitors (paginated)"]
    s3{"Count >= limit?"}
    s4["Collect ingress hosts (deduplicate + sort)"]
    s5["For each host"]
    s6{"Already monitored?"}
    s7["Probe /health"]
    s8{"/health returns 200 with keyword?"}
    s9["Create keyword monitor"]
    s10["Create status monitor"]
    s11["Print summary"]

    s1 -->|"token set"| s2
    s1 -->|"missing"| exit0["exit 0"]
    s2 --> s3
    s3 -->|"yes"| exit0
    s3 -->|"no"| s4
    s4 --> s5
    s5 --> s6
    s6 -->|"skip"| s5
    s6 -->|"no"| s7
    s7 --> s8
    s8 -->|"yes"| s9
    s8 -->|"no"| s10
    s9 --> s5
    s10 --> s5
    s5 -->|"done"| s11

keyword vs status monitors

if a host's /health endpoint returns 200 with something recognizable ("status": "ok" or similar), it creates a keyword monitor that checks for that string. otherwise it falls back to a basic status monitor on the root URL. keyword monitors are better because a service can return 200 while being functionally broken.

the script checks both root and /health URLs before creating anything, so it's idempotent. if the API token is missing it exits cleanly instead of erroring. it also rate-limits API calls between creates.

the pod runs non-root with a read-only filesystem and all capabilities dropped. concurrencyPolicy: Forbid prevents overlapping runs.

stakater already did this

stakater/IngressMonitorController is a full kubernetes operator for this exact problem. 715 stars, Apache-2.0, supports eight providers including UptimeRobot, StatusCake, and Pingdom. uses a custom EndpointMonitor CRD.

the key difference is opt-in vs opt-out. their operator requires you to create an EndpointMonitor resource for each service you want monitored. mine monitors everything in the namespace by default. adding new providers to theirs also means writing Go and contributing upstream.

the tradeoff is that mine is less robust. 5-minute polling instead of event-driven, single provider, and it doesn't clean up monitors when i delete an ingress.

arguments against

• stakater's operator already exists and is battle-tested
• a shell script in a cronjob is not serious infrastructure
• adding a monitor manually takes two minutes

arguments for

• i don't want to deploy a full operator for this
• i don't want to create a CRD every time i deploy something new
• i have a handful of services and i keep forgetting

this might grow into something better or it might stay as a script that scratches an itch. haven't decided yet.