Commit Graph

322 Commits

Author SHA1 Message Date
1ef4b988ac Add recovery make targets 2023-01-07 14:09:38 -08:00
bbe5bfaba4 Decode nomad node-ids in recovery playbook 2023-01-07 14:09:19 -08:00
290b8885b7 Fix vault hostnames for recover-consul playbook 2023-01-07 14:08:55 -08:00
acdccbc057 Fix consul recovery and decode node ids 2023-01-07 14:03:31 -08:00
e0c8d1f3c1 Exporters depend on prometheus 2023-01-06 23:07:33 -08:00
976f8f9e4e Change ddclient ip url 2023-01-06 23:06:23 -08:00
66db9fbd58 Update prune settings for backups 2023-01-06 16:08:19 -08:00
54d479ee9f Rollback consul version
This error on 1.14 needs to be resolved https://github.com/hashicorp/consul/issues/15753
2023-01-06 16:07:55 -08:00
91c2ff6345 Update blocklists 2022-12-22 15:13:31 -08:00
7ec6f38c80 Update consul and vault: 2022-12-22 15:11:33 -08:00
fd731971d3 Try to stabilize DNS
Add all cluster nodes to each nodes resolv.conf and update blocky config
template to delay render on update to avoid unnecessary restarts
2022-11-27 22:46:25 -08:00
7bed73b9a7 Disable consul autopilot 2022-11-27 22:46:05 -08:00
b9aec2a3c4 Update make targets
Add all and clean

Also removes prompts during make
2022-11-27 22:44:55 -08:00
fc86b974a7 Update versions 2022-11-27 22:44:26 -08:00
b0ea77a9f7 Update Cloudflare token variables 2022-11-21 14:25:01 -08:00
b046962b8c Disable consul autopilot
Was getting some instablity issues. Seeing if this helps
2022-11-21 14:22:49 -08:00
33be46add4 Fix galaxy make targets 2022-11-20 17:26:33 -08:00
4f5f4e0fe6 Increase priority of backup tasks 2022-11-18 08:58:38 -08:00
162f567c85 Stop Nomad and Vault when recovering Consul 2022-11-18 08:58:20 -08:00
3b9ad36ed0 New playbook to stop cluster gracefully 2022-11-18 08:57:41 -08:00
049d9f0fe0 Make sure grafana points to port bound within it's task group 2022-11-18 08:57:06 -08:00
3a828af690 No longer need to pin the envoy proxy 2022-11-18 08:56:29 -08:00
080cea9637 Fix minitor
Healthcheck was failing due to wrong path and log alert was failing
due to the config file being templated by Nomad. Updated to use a
different delimiter.
2022-11-16 09:04:32 -08:00
f481e7b938 Update blocky dashboards 2022-11-16 08:42:36 -08:00
35403d0219 Update nomad dashboard 2022-11-16 08:37:29 -08:00
416676c9f9 Update minitor dashboard 2022-11-16 08:35:01 -08:00
12b91e9566 Fix env location for lldap 2022-11-15 16:54:37 -08:00
8a21dd7eb4 Bump traefik version 2022-11-15 15:57:23 -08:00
a1def1c69d Increase memory for lldap
Password hashing was causing OOM kills
2022-11-15 15:57:23 -08:00
c7d0fca6e7 Pin lldap verison 2022-11-15 15:57:23 -08:00
86b472435c Use explicit lldap ports so that connect proxy can find them 2022-11-15 15:57:23 -08:00
2db266bda7 Update blocky upstream dns to bootstrap better (hopefully) and forward to consul 2022-11-15 10:26:26 -08:00
cf2779c971 Update lldap to use dynamic ports 2022-11-15 09:43:13 -08:00
b098a325f8 Add backups for photoprism 2022-11-15 09:19:55 -08:00
c5135be4a0 Pin photoprism version to avoid pulling every time 2022-11-15 09:19:32 -08:00
4a68894238 Add some minitor checks 2022-11-15 09:04:18 -08:00
17ab7f637f Add metrics support to service template 2022-11-15 09:04:18 -08:00
e104cbccc4 Use new Photos share for Photoprism 2022-11-15 09:04:18 -08:00
954a878915 Grafana update + renderer + new dashboards 2022-11-15 09:04:18 -08:00
bee9b641cc Fix local dns in docker 2022-11-15 09:04:18 -08:00
19f1f8448d WIP: Add a new playbook for stopping cluster
Hopefully without data loss
2022-11-14 23:11:13 -08:00
bbec244f45 Fix typo in recovery playbooks 2022-11-14 23:10:16 -08:00
0d47a1f8c8 Tune PhotoPrism 2022-11-11 16:21:47 -08:00
bb400a3f1c Add blocky metrics to grafana 2022-11-11 16:21:17 -08:00
da70aa74ca Add expose path for blocky /metrics
Not sure if it's needed
2022-11-11 13:47:27 -08:00
a30749f357 Switch services to module based Terraform template from levant
This ends up with a better experience in dealing with tfstate for some
services. Not sure why.
2022-11-11 13:47:26 -08:00
49c8a73ac9 Store loki data on ephemeral disk 2022-11-11 13:24:54 -08:00
3bfc1f61ac Update levant service template with some defaults 2022-11-11 13:24:28 -08:00
af32c9e2e5 Put grafana bootstrap secrets in secrets location 2022-11-10 13:39:12 -08:00
3077e66e70 Limit all existing services to websecure entrypoint
This will be a bigger issue if exposing a public entrypoint.
2022-11-10 13:37:50 -08:00