My Nomad homelab
Go to file
IamTheFij 0f5df9580a Add influxdb provisioning for Grafana
This will provide access to Home Assistant and Proxmox metrics.
2023-08-18 21:20:11 -07:00
acls Add nomad ACLs and roles for use in oidc auth 2023-07-07 00:30:02 -07:00
ansible_galaxy Refactor ansible to clean root dir 2022-11-02 14:20:09 -07:00
ansible_playbooks Upgrade to nomad 1.6.1 2023-07-26 15:29:39 -07:00
core Add influxdb provisioning for Grafana 2023-08-18 21:20:11 -07:00
databases Increase pgsql and lidarr memory to prevent crashes on library 2023-07-31 10:43:51 -07:00
services Try to format time zones from minitor 2023-08-18 12:15:34 -07:00
storage_plugins Update hooks 2022-11-02 12:59:32 -07:00
.gitignore Ignore nomad variables file 2023-04-14 13:54:43 -07:00
.pre-commit-config.yaml Add pre-commit hook to make sure variable sample is up to date 2023-07-25 16:57:44 -07:00
.secrets-baseline Run pre-commit on everything 2023-07-25 16:57:44 -07:00
.terraform.lock.hcl Add nomad ACLs and roles for use in oidc auth 2023-07-07 00:30:02 -07:00
.tflint.hcl Update hooks 2022-11-02 12:59:32 -07:00
ansible.cfg Refactor ansible to clean root dir 2022-11-02 14:20:09 -07:00
core.tf Big refactor to split core and services for better ordering 2022-10-27 14:28:34 -07:00
Makefile Add Nomad var example and remove old examples 2023-07-11 12:46:47 -07:00
nomad_vars.py Add Nomad var example and remove old examples 2023-07-11 12:46:47 -07:00
providers.tf Remove whitespace 2023-07-07 15:56:25 -07:00
README.md Fix incorrect README 2023-08-13 20:54:46 -07:00
requirements.txt Update hooks 2022-11-02 12:59:32 -07:00
root.tf Add nomad ACLs and roles for use in oidc auth 2023-07-07 00:30:02 -07:00
service.nomad Use stunnel for mysql 2023-05-09 13:20:36 -07:00
services.tf A whole lot of incremental fixes for nomad variables and such 2023-03-24 16:32:37 -07:00
vars.tf Remove whitespace 2023-07-07 15:56:25 -07:00

Homelab Nomad

My configuration for creating my home Nomad cluster and deploying services to it.

This repo is not designed as general purpose templates, but rather to fit my specific needs. That said, I have made an effort for things to be as useful as possible for someone wanting to use or modify this.

Running

make all

Design

Both Ansible and Terraform are used as part of this configuration. All hosts must be reachable over SSH prior to running any of this configuration.

To begin, Ansible runs a playbook to setup the cluster. This includes installing Nomad, bootstrapping the cluster and ACLs, setting up NFS shares, creating Nomad Host Volumes, and setting up Wesher as a Wireguard mesh between hosts.

After this is complete, Nomad variables must be set for services to access and configure correctly. This depends on variables to be set based on the sample file.

Finally, the Terraform configuration can be applied setting up all services deployed on the cluster.

The configuration of new services is intended to be as templated as possible and to avoid requiring changes in multiple places. For example, most services are configured with a template that provides reverse proxy, DNS records, database tunnels, database bootstrapping, metrics scraping, and authentication. The only real exception is backups, which requires a distinct job file, for now.

What does it do?

  • Nomad cluster for scheduling and configuring all services
  • Blocky DNS servers with integrated ad blocking. This also provides service discovery
  • Prometheus with autodiscovery of service metrics
  • Loki and Promtail aggregating logs
  • Minitor for service availability checks
  • Grafana providing dashboards, alerting, and log searching
  • Photoprism for photo management
  • Remote and shared volumes over NFS
  • Authelia for OIDC and Proxy based authentication with 2FA
  • Sonarr and Lidarr for multimedia management
  • Automated block based backups using Restic

Step by step

  1. Update hosts in ansible_playbooks/ansible_hosts.yml
  2. Update ansible_playbook/setup-cluster.yml
    1. Update backup DNS server
    2. Update NFS shares from NAS
    3. Update volumes to make sure they are valid paths
  3. Create ansible_playbooks/vars/nomad_vars.yml based on the sample file. TODO: This is quite specific and probably impossible without more documentation
  4. Run make all
  5. Update your network DNS settings to use the new servers IP addresses