this post was submitted on 05 Jun 2023
38 points (100.0% liked)
Technology
37705 readers
305 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I've got a small kubernetes cluster set up using Talos with 3 controlplane / 3 workers in VMs on the proxmox nodes. The vultr node is also running Talos and attached to the same cluster. Their KubeSpan feature is pretty neat, automatic full mesh wireguard between all cluster nodes.
Traffic inside the cluster flows seamlessly between all nodes, and I can even use it as sort of a proxy server using Cilium's Egress Gateway function.
Meanwhile my Pi4 is running k3s, to host a few services needed to operate the main cluster, such as the Harbor registry operating as a cache and a zigbee2mqtt instance because I have a raspbee2 for a zigbee adapter.
The main reason I'm using K3S even on the single node Pi is because I very much like using flux to manage the deployments on the servers.
Network wise, I've got a USG-3P, one of the newer compact 16 port POE switch. And a pair of UAP-AC-LITE for APs.
Maybe one day I'll get around to switching the USG for something a little more capable. And maybe capable of doing IPS/IDS on my 500M/100M internet connection. But no idea what kind of specs I'd need for that.
Would also like a NAS but... eh.. Maybe I'll just see if i can add more storage to the proxmox nodes and expand the ceph cluster or something.
Actually. Now that I think of it, I should probably diagram that out hmm. Anyone know any good tools for making that?
draw.io is one, I've started using LucidChart (personally) and https://d2lang.com at work for process diagrams.
And seriously, Talos Linux is really, really, nice. If I ever manage to mess up a kubernetes node (which has happened a few times when I was messing around), I just wipe it, reboot it from the ISO, and reprovision it with the machine configuration.
Talos is a great OS! I just wish there was some way to get the IPs from DHCP via Proxmox so I could automate it with terraform.
This sounds a lot like my old cluster config (I stepped away from the lab for a few months and forgot how it works, so started over lmao), but basically it would spin up a talos cluster on proxmox using terraform, and then bootstrap FluxCD and the rest of the software would be setup using that. It was a pretty slick system.