this post was submitted on 08 Oct 2024
33 points (97.1% liked)

Selfhosted

39921 readers
371 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
Loki@lemmy.world
CannaVet@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz
33
Simpler alternative to prometheus-alertmanager and/or graphana? (lemmy.world)
submitted 3 weeks ago by talkingpumpkin@lemmy.world to c/selfhosted@lemmy.world
11 comments fedilink hide all child comments
 

Prometheus-alertmanager and graphana (especially graphana!) seem a bit too involved for monitoring my homelab (prometheus itself is fine: it does collect a lot of statistics I don't care about, but it doesn't require configuration so it doesn't bother me).

Do you know of simpler alternatives?

My goals are relatively simple:

  1. get a notification when any systemd service fails
  2. get a notification if there is not much space left on a disk
  3. get a notification if one of the above can't be determined (eg. server down, config error, ...)

Seeing graphs with basic system metrics (eg. cpu/ram usage) would be nice, but it's not super-important.

I am a dev so writing a script that checks for whatever I need is way simpler than learning/writing/testing yaml configuration (in fact, I was about to write a script to send heartbeats to something like Uptime Kuma or Tianji before I thought of asking you for a nicer solution).

top 11 comments
sorted by: hot top controversial new old
[–] miau@lemmy.sdf.org 8 points 3 weeks ago

Have you played around with Grafana? It really is quite simple if you have prometheus already working.

For a home lab environment you dont even need to use prometheus-alertmanager. Grafana can handle alerts as well.

Grafana also has hundreds of pre-made dashboards you can import. Node monitoring is quite straightforward.

Assuming you have prometheus good to go, all you need to do is go to Grafana - Datasources, create a new datasource, point to your prometheus instance.

Then you can import the dashboards you want.

Now you can setup your alerts - you can use SMTP, telegram, slack among others for your notifications.

[–] pageflight@lemmy.world 3 points 3 weeks ago

Is there a self hosted OpenTelemetry consumer?

[–] Mora@pawb.social 3 points 3 weeks ago

The easiest solution I found and use is Beszel.

https://github.com/henrygd/beszel

Just a hub with the most important stats and some simple agents on the servers.

[–] themoonisacheese@sh.itjust.works 2 points 3 weeks ago

SNMP does what you want. You just need a good monitoring solution that's not as involved as Prometheus+grafana (I feel you, I've been there)

I really enjoy PRTG, but it's way too expensive for a home lab, still throwing it out there if you feel like you have money to burn.

I hear good word about libreNMS, it's next on my list when my PRTG licence runs out.

Be warned that monitoring is ultimately a fickle thing; what you don't write in yaml config for grafana, you get to dig through obscure SNMP libs to find out (though I find that's easier for me, ymmv) for other tools.

I recommend against: nagios (I like it but if you hate Prometheus it's definitely not for you), checkmk (throw checkmk into the sun please it just fucking sucks), cacti (NO!), solar winds (why?)

if you feel like you want to become a datacenter admin: zabbix scales very very well, both in performance and ease of admin against hundreds of servers, but it's overkill for a home lab, and it can get you lost in configs for hours.

[–] SuperiorOne@lemmy.ml 2 points 3 weeks ago

I'm currently using InfluxDB + Telegraf + Grafana combination to monitor Linux systems and k3s pods. It's basically same as Prometheus, but InfluxDB uses push model, which makes it easier to develop tools for collecting custom time series data.

For alerts and dashboards, I think Grafana is the simplest and most hassle free solution available at the moment.

[–] TheHolm@aussie.zone 2 points 3 weeks ago

ICINGA/NAGIOS? you can even feed data already collected by Prometheus to it if you want.

[–] horse_battery_staple@lemmy.world 1 points 3 weeks ago* (last edited 3 weeks ago) (1 children)

Edit: found better resources

https://linuxhandbook.com/syslog-guide/

https://github.com/linuxserver/docker-syslog-ng

That should be a good place to start. Syslog will do what you want.

[–] themoonisacheese@sh.itjust.works -1 points 3 weeks ago (1 children)

Syslog is considerable overkill for home lab monitoring.

[–] horse_battery_staple@lemmy.world 1 points 3 weeks ago* (last edited 3 weeks ago)

It's as complex as you make it, is linux native, is scriptable, doesn't use YAML, is native to the OS, and is free as in beer. Just like SNMP. however they'll also get logs at a central server they can drill into if needed.

Which I believe fulfills the requirements of OPs post.

Sidenote, self-hosting is absolutely overkill just as a theory and process. I often read responses to suggestions as this or that is overkill, or complicated, or non-trivial effort.

The self hosting community is a broad spectrum of users , from those with home labs on an old dying laptop to those with a full rack setup. People have different needs and interests. Some are learning infra and devops for work or to get into a new job. Some are privacy minded. Some are trying to get the most bang for their buck. Some just want to pay for a cloud hosted solution. Some just want an automated home. Some run a home business.

Edit: to the point of your valid and helpful SNMP post, most syslog servers also will ingest and report on SNMP traffic as well. The container I linked does exactly that. If they find they want to automate processes in the future they can also trigger on the syslog stream as well. But that complexity is only there if they want it. Otherwise it's just a stream they can parse and trigger an alert, just like SNMP. So OP could have an extensible solution if they want to expand. Also Grafana/Prometheus will take in syslog natively with a couple standard YAML configs if they choose that they want to look at that solution again in the future.

/Rant

[–] static09@lemmy.world 1 points 3 weeks ago

Zabbix
https://www.zabbix.com/

Netdata
https://www.netdata.cloud/

Nagios
https://www.nagios.org/

[–] linearchaos@lemmy.world 1 points 3 weeks ago

I mean, you get a lot of advantages from fluffy pretty systems. But extracting data from df and systemctl and curling it into telegram is going to be like a 10 line bash script called from a one-line cron job.

I pump a lot of complicated metrics through Prometheus / grafana to get graphs and history.

Most of my critical stuff is still in Nagios and instead of using nagios standardized plugins I just query the operating system directly in bash.