Commit Graph

160 Commits

Author SHA1 Message Date
c01d45c7a2 Upgrade grafana to version 10 2024-01-08 10:11:42 -08:00
9be16fef1f Upgrade traefik to 2.10 2024-01-04 13:25:10 -08:00
c26da678b3 Small traefik cleanup
Remove fallback DNS since we only care about internal DNS

Use loopback address for accessing Nomad UI
2024-01-04 13:24:49 -08:00
6b9533ef71 Run traefik on multiple hosts 2024-01-04 13:24:15 -08:00
0bd995ec2b Traefik: Use nomad vars for dynamic certs
Rather than having Traefik handle cert fetching, instead
it is delegated to a separate job so that multiple Traefik
instances can share certs
2024-01-04 10:55:49 -08:00
0d340f3349 Periodic job to renew lego certs and store them in Nomad Variables
This will allow multiple instance of Traefik to serve certs.
2024-01-04 10:53:25 -08:00
cda2842f8f Switch to image containing stunnel
Rather than installing on container startup, using an image with
stunnel pre-installed. This avoids issues with DNS breaking
the container on startup.
2024-01-03 13:50:49 -08:00
ca55209316 Fix blocky redis 2023-12-10 20:37:43 -08:00
1b49f015c5 Update blocky config to v0.22 schema 2023-11-30 14:00:27 -08:00
eb25138675 Remove defunct lists 2023-11-30 13:39:22 -08:00
69a0f760b4 Remove defunct lists 2023-11-30 13:39:01 -08:00
3fcedaddb7 Remove todo from traefik 2023-11-30 13:26:15 -08:00
bb34b434b8 Add custom blocklists hosted on my gitea server 2023-11-30 13:23:54 -08:00
36cdb8f41b Add Gitea
Currently it won't auto bootstrap auth. A command has to be executed one
time to get it to be added to the database.
2023-11-30 13:22:54 -08:00
e21ec11eb5 Fix grafana
Broken template
2023-11-20 10:35:49 -08:00
891cfa7b2d Update blocky dashboard to not use consul tags 2023-11-16 12:21:59 -08:00
c11b8e157b Fix grafana dashboard provisioning
A path mismatch existed after migrating to alloc storage
2023-11-16 12:21:40 -08:00
a2d33ac309 Add proxmox influxdb to Grafana 2023-10-23 13:10:01 -07:00
0c3f98d5c3 Pin Grafana to amd64 since renderer requires it.
This could be mitigated by moving the renderer to another task group.
2023-10-19 12:06:47 -07:00
ad439d48f3 Add waiting for loki and prom dependencies in core 2023-09-27 21:30:22 -07:00
b29f405090 Bump prometheus versiosn and pin blocky 2023-09-18 21:58:43 -07:00
8dd00c1249 authelia and grafana to shared smtp secrets 2023-08-29 15:11:40 -07:00
2bd939e651 Remove deprecated hcl2 enabled 2023-08-29 13:02:04 -07:00
ea8ca478c6 Fix blocky acl 2023-08-29 12:59:14 -07:00
f5898b0283 Add workload ACL management for mysql and postgres access
Allows required jobs to access shared secrets and auto generates psks
for stunnel.

Currently supporting MySQL, Postgres, and LDAP.
2023-08-29 12:48:48 -07:00
013dd8248b Make base_hostname more configurable 2023-08-24 15:03:36 -07:00
f6dd3f4284 Clean up root module and move lldap to databases 2023-08-24 13:52:03 -07:00
4a7bff7611 Move metrics out of a module and into core 2023-08-24 13:00:36 -07:00
d5078b24da Refactor use of wesher to be behind a variable toggle
Occasionally I run into issues with Wesher. This makes it easier to
disable use of Wesher by setting TF_VAR_use_wesher to false.
2023-08-24 12:51:32 -07:00
e2c35a82a9 Fix grafana config loading
For some reason, the env variable method stoped working.
2023-08-24 11:59:10 -07:00
1715b58ca9 Pin image versions for more critical services 2023-08-24 11:39:00 -07:00
ddeb8fffbc Move services to their own tf files for easier locating 2023-08-07 11:37:19 -07:00
fa0da05343 Change authelia port to avoid conflict with prometheus 2023-08-02 21:31:08 -07:00
4b94f66786 Increase Traefik memory 2023-07-31 10:43:03 -07:00
f333031c25 bootstrap blocky with stunnel 2023-07-26 23:23:23 -07:00
744466bf07 Use static port for Authelia so that nomad middleware config is the same for each service 2023-07-07 16:34:50 -07:00
df062000e7 Run two authelia instances now that it's stateless 2023-07-07 15:56:23 -07:00
0a7ad7a9dc Enable redis for authelia
This also splits redis instances by service
2023-07-07 15:50:23 -07:00
b0c1aca497 Increase token time for Nomad OIDC 2023-07-07 15:47:08 -07:00
60a4051988 Enable Authelia OIDC for Nomad 2023-07-07 00:41:44 -07:00
0ceb513216 Switch Grafana to OIDC from proxy auth 2023-07-07 00:40:19 -07:00
9d5aeeec96 Enable Authelia OIDC provider 2023-07-07 00:39:44 -07:00
eae5b201b6 Add two factor for external IPs 2023-07-06 21:25:31 -07:00
532d7f9a4c Use Authelia for Grafana login 2023-07-06 18:00:06 -07:00
88e91e5e5d Deploy authelia
Backed by lldap and mysql and deployed on whoami for now as a forward
proxy example

Would be good to add oidc for Nomad as well as make policies configurable
via Nomad variables.
2023-07-06 18:00:06 -07:00
8650ab973a Add stunnel for ldap as part of service template 2023-07-06 17:25:13 -07:00
acc80868f9 Switch lldap storage to mysql 2023-07-05 17:30:54 -07:00
f606e0a17e Remove blocky client groups because fallback server masks them 2023-07-05 15:45:55 -07:00
2df43584cf Grafana config reloading: Use explicit path and echo
Was running into some issues with this not running. Using an explicit
path seems to help, so I'll try it for now. Also added some echo statements
to make it easier to discern when run.
2023-06-20 09:44:04 -07:00
2c128b25f3 Add additional blocking for wemo 2023-06-20 09:42:33 -07:00
1df5545835 Promtail: use local task dir rather than bind mount 2023-05-12 10:11:30 -07:00
d4cb91d58d Rename metrics job to exporters 2023-05-12 10:11:11 -07:00
48322d9a78 Document what the nomad stalker is for 2023-05-12 10:10:31 -07:00
5169aecc6d Add pushgateway to prometheus 2023-05-09 15:56:20 -07:00
f11fad30a5 Use stunnel for mysql
Doesn't remove wesher or normal mysql service
2023-05-09 13:20:36 -07:00
27fd60d84d Add missing service to Wesher
Promtail, Backups, service module
2023-05-02 21:14:36 -07:00
0a84fd04bc Automatically re-provision grafana when data source addresses change 2023-05-02 21:13:59 -07:00
7d8bc45090 Move blocky custom mappings above catchall 2023-04-04 13:12:34 -07:00
485bc22e78 Add TODO for using nomad api socket 2023-03-27 15:50:15 -07:00
c38ba8589a Clean blocky config for latest version 2023-03-27 15:21:35 -07:00
c7f85bd985 Fix blocky redis stunnel lookup 2023-03-27 15:21:19 -07:00
f17dec7b57 Add nomad services to nomad zone using hosts in blocky 2023-03-27 15:20:50 -07:00
a748adbab0 Store blocky config in local task dir 2023-03-27 15:19:53 -07:00
747d5ef0e7 Remove vault stanza from Grafana 2023-03-27 14:10:10 -07:00
08d0e93638 Clean up and remove some consul and vault stuff 2023-03-24 22:58:44 -07:00
74ce30c3c1 Get nomad client scraping working 2023-03-24 22:22:11 -07:00
98ea2a1ca0 A whole lot of incremental fixes for nomad variables and such
Also adds stunnel between redis and clients
2023-03-24 16:32:37 -07:00
d8307935f5 Refactor everything for nomad vars 2023-03-24 11:24:36 -07:00
5fb0e0841e Blocky do not create read only user to reduce password exposure 2023-03-24 09:56:56 -07:00
00697ebb02 Blocky use wgoverlay for api 2023-03-24 09:56:29 -07:00
f31569ad56 Update cloudflare variable names 2023-03-24 09:56:03 -07:00
46dc44aca4 Simplify mysql for blocky 2023-03-24 08:55:27 -07:00
4430b3570e Fix blocky template 2023-03-24 08:55:27 -07:00
65cb6afaf9 WIP: Moving vars and service discovery to Nomad
Starting with core
2023-03-24 08:55:23 -07:00
ee68310e58 Add Nomad provider and sample using Wesher 2023-03-24 08:50:16 -07:00
3ebb616219 Add nomad labels to docker logs 2023-03-17 11:47:40 -07:00
437b5ce72e Update grafana 2023-03-12 10:22:47 -07:00
19d5321731 Increase memory for promtail
n2 was getting OOM
2023-02-27 11:54:33 -08:00
3a95fb46db Add more conditional checks to Blocky so it is more resiliant
Hopefully this will allow it to deploy if mysql or vault are down
2023-02-27 11:54:33 -08:00
1811a851ab Tighten diun watch expressions 2023-02-14 12:28:41 -08:00
0d9d2c7d21 Update promtail version and version checker 2023-01-13 15:47:48 -08:00
03fd68b4f7 Add diun for monitoring images 2023-01-12 12:11:16 -08:00
b92917329f Use a different ip address host for ddns 2023-01-07 14:10:20 -08:00
e0c8d1f3c1 Exporters depend on prometheus 2023-01-06 23:07:33 -08:00
976f8f9e4e Change ddclient ip url 2023-01-06 23:06:23 -08:00
91c2ff6345 Update blocklists 2022-12-22 15:13:31 -08:00
fd731971d3 Try to stabilize DNS
Add all cluster nodes to each nodes resolv.conf and update blocky config
template to delay render on update to avoid unnecessary restarts
2022-11-27 22:46:25 -08:00
b0ea77a9f7 Update Cloudflare token variables 2022-11-21 14:25:01 -08:00
049d9f0fe0 Make sure grafana points to port bound within it's task group 2022-11-18 08:57:06 -08:00
f481e7b938 Update blocky dashboards 2022-11-16 08:42:36 -08:00
35403d0219 Update nomad dashboard 2022-11-16 08:37:29 -08:00
416676c9f9 Update minitor dashboard 2022-11-16 08:35:01 -08:00
12b91e9566 Fix env location for lldap 2022-11-15 16:54:37 -08:00
8a21dd7eb4 Bump traefik version 2022-11-15 15:57:23 -08:00
a1def1c69d Increase memory for lldap
Password hashing was causing OOM kills
2022-11-15 15:57:23 -08:00
c7d0fca6e7 Pin lldap verison 2022-11-15 15:57:23 -08:00
86b472435c Use explicit lldap ports so that connect proxy can find them 2022-11-15 15:57:23 -08:00
2db266bda7 Update blocky upstream dns to bootstrap better (hopefully) and forward to consul 2022-11-15 10:26:26 -08:00
cf2779c971 Update lldap to use dynamic ports 2022-11-15 09:43:13 -08:00
954a878915 Grafana update + renderer + new dashboards 2022-11-15 09:04:18 -08:00