Lab 07: GPU Operator, dcgm-exporter, Prometheus and Grafana
180 minutes suggested, up to 360 in one session. Price: 5 credits per minute. Includes Check my work.
Hosted, this lab covers Parts A, B, C, D, E. The other Parts need your own machine (Home Lab Kit) and are shown collapsed in the lab.
The GPU tier is not available on this site yet.
What you will be able to do
- Name every GPU Operator component on a node and say what each one does.
- Read
dcgm-exportermetrics, and tell the legacy metric names from the DCGM 4.6+ canonical names. - Install kube-prometheus-stack with Helm and make Prometheus scrape
dcgm-exporterwith a ServiceMonitor you wrote. - Import NVIDIA's DCGM dashboard into Grafana and read its panels under load.
- Write PromQL alert rules for GPU temperature, XIDs and a missing exporter, and prove them with
promtool test rules. - Find out why a GPU metric or alert has gone silent.