← Back to Home

☸️ k3s + Traefik + Gateway API

A hands-on guide to exposing k3s workloads under a wildcard subdomain (*.k3s.ytamang.com) by running Traefik on a NodePort behind your existing Nginx, routing with the Gateway API (HTTPRoute>), and securing it with a wildcard TLS cert via certbot DNS-01.



    
      Contents
      
        The Problem & the Big Picture
        k3s and Its Bundled Traefik
        The Port Conflict: NodePort vs LoadBalancer
        Gateway API: GatewayClass → Gateway → HTTPRoute
        Wildcard TLS: Why DNS-01
        The Full Deployment, Step by Step
        Three Gotchas We Hit (and Fixed)
        Verification


      1. The Problem & the Big Picture

      You have
 a single server running Nginx> that serves several static sites on ports 80> and 443 (ytamang.com

, learn.ytamang.com, softskills.ytamang.com). Now you want to run workloads in k3s — a lightweight Kubernetes — and reach them through a wildcard like *.k3s.ytamang.com, without breaking the Nginx sites.

The challenge is subtle: k3s ships with Traefik, its default ingress controller, and Traefik wants to listen on 80 and 443 too. Those ports are already taken by Nginx. So we can't just "turn Traefik on" — we have to teach it to live behind Nginx.

Here

is the whole request path we are building:

Browser asks for app.k3s.ytamang.com

↓

DNS wildcard A record *.k3s.ytamang.com → 5.75.145.2

↓

Nginx :443 terminates TLS, matches the wildcard

↓

Traefik NodePort :30080 its web entrypoint, reached only by Nginx

↓

Gateway API HTTPRoute matches the host, picks the service

↓

Service → Pod your workload

💡 The one-sentence idea: Nginx keeps owning :443 and does TLS. It reverse-proxies *.k3s.ytamang.com to Traefik on a different port (:30080). Traefik then routes by the Host> header to the right workload using an HTTPRoute.

Walk

the request path step by step:

The browser resolves app.k3s.ytamang.com → the wildcard A record returns 5.75.145.2.
The browser opens TLS to 5.75.145.2:443 — Nginx answers with the *.k3s.ytamang.com certificate.
Nginx decrypt

Host:

Nginx >proxy_passes plain HTTP to 127.0.0.1:30080, forwarding the original Host header.
Traefik receives

web> entrypoint and finds an HTTPRoute

app.k3s

Traefik

Service

Pod

2. k

3s and Its Bundled Traefik

k3s> is a CNCF-certified, lightweight Kubernetes distribution packaged as a single binary. It's the same Kubernetes API you know, but trimmed for small servers and edge nodes: it uses containerd as its runtime and ships a few helpful components by default — including Traefik> as its ingress controller.

"Ingress controller" is the piece that watches for routing rules and turns them into live traffic routing. Traefik is one of several choices (others include ingress-nginx and HAProxy Ingress). Because Traefik is already bundled with k3s, we reuse it instead of installing another controller.

it meansogy that routes external HTTP(S) traffic to services based on rulesk3s's default ingress controller, supports both classic Ingress and the newer Gateway API

Term What Anal

Ingress controller Software The building's front desk that directs visitors to the right office

Traefik A specific, well-equipped front desk that reads many kinds of signs

Gateway API The modern, extensible successor to Ingress> for defining routing A standardized sign language the front desk understands

k
3s can be installed with --disable traefik to skip its bundled controller. That's exactly how this server was first set up (to avoid the port clash), which is why our first task was to re-enable it on a different port.

Term	What	Anal
Ingress controller	Software	The building's front desk that directs visitors to the right office
Traefik		A specific, well-equipped front desk that reads many kinds of signs
Gateway API	The modern, extensible successor to `Ingress> for defining routing`	A standardized sign language the front desk understands

	`Node`Port	`LoadBal`ancer (with klipper)
Port	A high port you choose, e.g. 30080	The service's own ports, e.g. 80/443
Who owns the host port	K	klipper binds
Clash with Nginx
Use case here	Perfect

3. The Port Conflict
: NodePort vs LoadBalancer
By
default, Traefik exposes itself as a Kubernetes LoadBalancer service. In k3s, LoadBalancer services are fulfilled by a tiny built-in load balancer called klipper (or servicelb), which binds the service's ports directly on the host. Traefik's ports are 80> and 443 — the exact ports Nginx already owns.

⚠️ The clash: turning Traefik on with its defaults would make klipper try to grab :80 and :443 from under Nginx. The result: your static sites break, because external traffic to those ports gets redirected into Traefik instead.

The fix is to expose Traefik as a NodePort> service on non-conflicting ports — 30080 (HTTP) and 30443 (HTTPS) — and let Nginx forward to 127.0.0.1:30080.

ubernetes routing (kube-proxy) it directlyNo (different port)Yes (same ports) — Nginx is the only public entrypointConflicts, so we avoid it

NodePort LoadBalancer (with klipper)

Port A high port you choose, e.g. 30080 The service's own ports, e.g. 80/443

Who owns the host port K klipper binds

Clash with Nginx

Use case here Perfect

The redirect trap

There
's a second, sneakier problem. Traefik's HTTP entrypoint is configured to redirect HTTP → HTTPS (redirectTo: websecure). If Nginx terminates TLS and forwards plain HTTP to Traefik, Traefik would answer with a 301 → https, sending the browser back to Nginx's :443 — which forwards again → an infinite redirect loop.

💡 Rule of thumb: exactly one hop should handle the HTTPS redirect. Since Nginx already does HTTPS, we clear Traefik's redirectTo> so it just routes plain HTTP from Nginx.

4. Gateway API: GatewayClass → Gateway → HTTPRoute

The classic Kubernetes Ingress resource is simple but has limits — each controller interprets rules slightly differently, and it can't express modern needs like traffic splitting or cross-namespace routing cleanly. The Gateway API fixes this with a clear, role-based split of responsibilities.

>Who creates itWhat it declares adminWhich controller implements routing (here traefik.io/gateway-controller) admin

Resource
`GatewayClass>`	Cluster
`Gateway>`	Cluster	A listener — e.g. "accept HTTP on the `web` entrypoint"
`HTTPRoute`	App developer	"Route host `app`.k3s.ytamang.com to this service"

This separation

is the real win: the platform team owns the entrypoint (the Gateway), while each app team owns their own routing (an HTTPRoute) without touching cluster-wide settings.

The Gateway and GatewayClass:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: traefik
spec:
  controllerName: traefik.io/gateway-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: k3s
  namespace: kube-system
spec:
  gatewayClassName: traefik
  listeners:
    - name: web
      protocol: HTTP
      port: 8000
      allowedRoutes:
        namespaces:
          from: All

A workload's HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: hello
  namespace: default
spec:
  parentRefs:
    - name: k3s
      namespace: kube-system
      sectionName: web
  hostnames:
    - app.k3s.ytamang.com
  rules:
    - backendRefs:
        - name: hello
          port: 80

💡 Note the port: the Gateway listener uses port: 8000 — Traefik's internal entrypoint port. The external port is the NodePort 30080. They're different layers: container port vs. host port.

⚠️ CRDs are bundled: the Gateway API CRDs (gateway.networking.k8s.io) are shipped inside Traefik's traefik-crd chart. Do not install them separately — we learned this the hard way (see the gotchas).

5. Wildcard TLS: Why DNS-01

We want a single certificate that covers every *.k3s.ytamang.com host. Let's Encrypt offers two challenge types to prove you control a name:

01DNS-01No>No

	HTTP-
How it proves control	Serves a token over HTTP on port `80`	Creates a >TXT record in your DNS
Works for wildcards?		Yes>
Needs port 80 open	Yes
Needs DNS API access	No>	Yes

A wildcard like *.k3s.ytamang.com can only be validated with DNS-01 — there's no single HTTP path that represents "all subdomains." So we use the certbot-dns-route53 plugin, which talks to Route 53 to create the challenge record.

cert

bot needs credentials with Route 53 access. Rather than hand it a full-admin key, we create a least-privilege IAM user with Terraform:

resource "aws_iam_policy" "certbot_dns" {
  name        = "certbot-dns-route53"
  description = "Least-privilege access for certbot DNS-01 challenge"

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "route53:ListHostedZones",
          "route53:GetChange"
        ]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["route53:ChangeResourceRecordSets"]
        Resource = "arn:aws:route53:::hostedzone/${aws_route53_zone.ytamang.zone_id}"
      }
    ]
  })
}

The policy scopes record changes to a single hosted zone (ytamang.com). It can create/delete the challenge TXT record and check the change status — and nothing else.

Then cert

bot obtains the wildcard cert:

certbot certonly \
  --dns-route53 \
  -d '*.k3s.ytamang.com' \
  --email 48yogen+k3snginx@gmail.com \
  --agree-tos \
  --non-interactive

💡 Wildcards match one label: *.k3s.ytamang.com covers app.k3s.ytamang.com but not a.b.k3s.ytamang.com, and not the bare k3s.ytamang.com.

6. The Full Deployment

, Step by Step

a

) Re-enable Traefik

The server was installed with --disable traefik. We remove that flag and restart k3s, which makes k3s deploy its bundled Traefik (plus its CRD chart).

b) Configure Traefik to run on a NodePort

A HelmChartConfig overrides Traefik's values: NodePort service, clear the redirect, enable the Gateway API provider.

apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    service:
      type: NodePort
    ports:
      web:
        redirectTo: ""
        nodePort: 30080
      websecure:
        nodePort: 30443
    providers:
      kubernetesGateway:
        enabled: true

c)

Create the Gateway and GatewayClass

shown in section 4 — one GatewayClass bound to Traefik's controller, and one Gateway> with a web listener.

d) N

ginx wildcard server block

ginx terminates TLS and forwards to Traefik's NodePort, preserving the Host header:

server {
    listen 80;
    server_name .k3s.ytamang.com;
    return 301 https://$host$request_uri;
}

server {
    listen 443 ssl http2;
    server_name .k3s.ytamang.com;

    ssl_certificate     /etc/letsencrypt/live/k3s.ytamang.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/k3s.ytamang.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:30080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

e

) Deploy a workload with an HTTPRoute

Finally, any workload just declares an HTTPRoute (section 4). No Nginx edits are needed per app.

7. Three Gotchas

We Hit (and Fixed)

These are

real bugs from doing this live. Each one cost a few minutes — knowing them up front saves you the same.

1)

The flag was split across lines

k3s's

installer wrote the --disable traefik flag as two separate, single-quoted lines in the systemd unit:

ExecStart=/usr/local/bin/k3s \
    server \
        '--write-kubeconfig-mode' \
        '0644' \
        '--disable' \
        'traefik' \

⚠️ Lesson: a grep/sed/replace looking for the contiguous string --disable traefik never matched — the two words were on different lines. The reliable fix was to rewrite the whole unit with a clean ExecStart.

2) The Gateway API CRD ownership conflict

We first

installed the Gateway API CRDs from the upstream manifest. But Traefik's traefik-crd chart also ships them. When the chart tried to install, Helm refused to adopt CRDs it didn't own:

CustomResourceDefinition "gatewayclasses.gateway.networking.k8s.io"
exists and cannot be imported into the current release:
missing key "meta.helm.sh/release-name"

⚠️ Lesson: if a chart bundles CRDs, let it install them. Delete the hand-installed CRDs and let the traefik-crd chart own them.

3)

LoadBalancer still hijacked :80/:443

Even

with service.type: NodePort in the Helm values, this Traefik chart version still rendered a LoadBalancer service — so klipper added an iptables rule that DNAT'd :80/443 into Traefik, clobbering Nginx. The fix was a direct patch:

kubectl -n kube-system patch svc traefik --type=merge \
  -p '{"spec":{"type":"NodePort"}}'

⚠️ Lesson: verify the actual rendered resource, not just the values you asked for. A chart version can silently ignore a value, so assert the end state.

8. Verification

After everything

is in place, we confirm the full path with a real request:

curl -sk -i https://app.k3s.ytamang.com/

A working setup returns HTTP/2 200 with the workload's content, and DNS resolves correctly:

>Expected resultresolves to 5.75.145.2

Check
`GatewayClass` traefik	`AC`CEPTED=True
Gateway k3s	`PROGRAMMED=True`
`curl https`://app.k3s.ytamang.com/	`HTTP/2` 200
`dig` app.k3s.ytamang.com

💡 Summary: Nginx stays the single public entrypoint and TLS terminator. Traefik lives behind it on a NodePort and routes via the Gateway API. Each new app is just an HTTPRoute — no Nginx changes, no new certificates, no per-app load balancers.

1. The Problem &

Interactive

2. k

3. The Port Conflict

The redirect trap

4. Gateway

6. The Full Deployment

a

b) Configure Traefik to run on a NodePort

c)

d) N

e

7. Three Gotchas

1)

3)

8. Verification