Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
-
No spam posting.
-
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
-
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
-
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
-
No trolling.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
Here goes my experience.
When I started the self hosted trip, I was against containers and tried to avoid them at all costs. Then I learned about containers, and now I still am against containers but less vividly so. I have used them and still use them.
Containers are good for the self hoster because they deliver fast deploy and easy testing of lots of services quickly. They are good for developers because they can provide one common installation approach that reduces greatly user issues and support requests.
But containers also have downsides as well. First of all they make the user dumber. Instead of learning something new, you blindly "compose pull & up" your way. Easy, but it's dumbifier and that's not a good thing. Second, there is a dangerous trend where projects only release containers, and that's bad for freedom of choice (bare metal install, as complex as it might be, need to always be possible) and while I am aware that you can download an image and extract the files inside, that's more an hack than a solution. Third, with containers you are forced to use whatever deployment the devs have chosen for you. Maybe I don't want 10 postgres instances one for each service, or maybe I already have my nginx reverse proxy or so. I have seen projects release different composer files for different scenarios, but at that point I would prefer to deploy on bare metal.
Said so, containers are not avoidable today, so study and embrace them, you will not be disappointed as its a cool piece of tech. But please stay clear of docker and go podman instead. Podman doesn't rely on a potentially insecure socket and does not require an always running daemon. Podman also by default doesn't force you to run services as root which you should never do. Also, networking feels clearer on podman and podman feels more .modern by using nft instead of iptables. Yes most of this can be fixed on docker, but since podman is a drop in replacement, why bother? Also, podman is truly open source while docker, shockingly, its not.
Here is my wiki page on the subject: https://wiki.gardiol.org/doku.php?id=gentoo:containers feel free to read it.
One last thought: updating containers should not be taken lightly. Its so easy and fast that you might be tempted to setup cron jobs or install watchtower, but you will end sooner or later with a broken service and lost data. So backup, always backup, and keep updating with rationale.
Tldr: containers are unavoidable today and are a cool piece of tech worth investigating. Don't blindly use them as there are security issues involved, and I hope the trend of making containers the only way doesn't take hold, because containers also make self hosters dumber and that's not good.
I don't like this Docker trend because, besides what you've said, it 1) leads you towards a dependence on property repositories and 2) robs you from the experience of learning Linux (more later on) but I it does lower the bar to newcomers and let's you setup something really fast. In my opinion you should be very skeptical about everything that is "sold to the masses", just go with a simple Debian system (command line only) SSH into it and install what you really need, take your time to learn Linux and whatnot.
And the second danger there is that when developers don't have to consider the setup of a their solution the code tends to be worse. Why bother with having single binaries, stuff that is easy to understand and properly document things when you can just pull 100 dependencies and compose files? :) This is the unfortunate reality of modern software.
See? Poorly written software. Not designed to be sane and reasonable and integrate with existing stuff.
Your article said it all and is very well written. Let me expand a bit into the "different issues":
The thing with Docker is that people don’t want to learn how to use Linux and are buying into an overhyped solution that makes their life easier without understanding the long term consequences. Most of the pro-Docker arguments go around security, reproducibility and that’s mostly BS because 1) systemd can provide as much isolation a docker containers and 2) there are other container solutions and nobody cares about them.
Companies such as Microsoft and GitHub are all about re-creating and re-configuring the way people develop software so everyone will be hostage of their platforms - that's why nowadays everything and everyone is pushing for Docker/DockerHub/Kubernetes, GitHub actions and whatnot. We now have a generation that doesn’t understand the basic of their tech stack, about networking, about DNS, about how to deploy a simple thing into a server that doesn’t use some Docker BS or isn’t a 3rd party cloud xyz deploy-from-github service.
Before anyone comments that Docker isn’t totally proprietary and there’s Podman consider the following: It doesn’t really matter if there are truly open-source and open ecosystems of containerization technologies. In the end people/companies will pick the proprietary / closed option just because “it’s easier to use” or some other specific thing that will be good on the short term and very bad on the long term.
Docker may make development and deployment very easy and lowered the bar for newcomers have the dark side of being designed to reconfigure and envelope the way development gets done so someone can profit from it. That is sad and above all set dangerous precedents and creates generations of engineers and developers that don’t have truly open tools like we did. There’s LOT of money into transitioning everyone to the “deploy-from-github-to-cloud-x-with-hooks” model so those companies will keep pushing for it.
At the end of the day technologies like Docker are about commoditizing development and about creating a negative feedback loop around it that never ends. Yes, I say commoditizing development because if you look at it those techs only make it easier for the entry level developer and companies instead of hiring developers for their knowledge and ability to develop they’re just hiring “cheap monkeys” that are able to configure those technologies and cloud platforms to deliver something.
Successful cloud companies are not longer about selling infrastructure, we're past that - the profit is now in transforming developer knowledge into products/services that can be bought with a click.
There is a lot of truth in your words.
Unfortunately, things will not change.
At least let's use podman and I will keep fighting for containers being at least optional.
Well, systemd can also provide as much isolation and security. It's another option... :) as well as LXC.
You can host your own container repository and write your own docker files to control all your own deployments though, it's not like your have to be at the behest of any company to use containerization to make your own life easier with the benefits of reproducibility.
Do you write all the programs you use too or do you rely on the work of others and are drawing an arbitrary line in the sand when it comes to containerising those apps?
Yes, I can, but this not about what I or you can do. This is about what the actually do, the direction technology is taking and the lack of freedoms that follows. Distribution is important.
Do you object to software repositories that install dependencies precompiled?
Your "lines in the sand" seem idiosyncratic and arbitrary. You are happy presumably to use precompiled software or at the very least rely on software written by others which is already ceding some freedom but then claim that using systems that package all the dependencies into a single runnable unit is too much and cedes too much freedom?
I agree that containers are allowing software projects to push release engineering and testing down stream and cut corners a bit but that was ever the case with precomplied releases that were only tested on a single version of a single distro.
Look this isn't even about "drawing lines in the sand", I do understand why use containers and I use them in certain circumstances, usually not Docker but that's more due to the requirements in said circumstances and not about personal decision.
No and I never claimed that. I'm perfectly happy to use a single-binary statically linked applications, in fact I use quite a few such as FileBrowser and Syncthing and they're very good and reasonable software. Docker however isn't one of those cases or, at least, not just that.
Docker is being used and abused for cutting corners and now we've developers that are just unable to deploy any piece of software without it. They've zero understanding of infrastructure and anything related to it and this has a big negative impact on the way they develop software. I'm not just talking about FOSS projects, we see this in the enterprise and bootcamps as well.
Docker is a powerful thing, so powerful it opens the door for poorly put together software to exist and succeed as it abstracts people from having to understand architectures, manually install and configure dependencies and things that anyone sane would be able to do in a lifetime. This is why we sometimes see "solutions" that run 10 instances of some database or some other abnormality.
Besides all that, it adds the half-open repository situation on top. While we can host repositories and use open ones the most common thing is to see everything on Docker Hub and that might turn into a CentOS style situation anytime.
I don't agree with the premise of your comment about containers. I think most of the downsides you listed are misplaced.
I'd argue, that actually using containers properly requires very solid Linux skills. If someone indeed blindly "compose pull & up" their stuff, this is no different than blind
curl | sudo bashwhich is still very common. People are going to muddle through the installation copy pasting stuff no matter what. I don't see why containers and compose files would be any different than pipe to bash or random reddit comment with "step by step instructions". Look at any forum where end users aren't technically strong and you'll see the same (emulation forums, raspberry pi based stuff, home automation,..) - random shell scripts,rm -rf this ; chmod 777 thatContainers are just another piece of software that someone can and will run blindly. But I don't see why you'd single them out here.
As a developer I can't agree here. The docker images (not "containers" to be precise) are not there replacing deb packages. They are there because it's easy to provide image. It's much harder to release a set of debs, rpms and whatnot for distribution the developer isn't even using. The other options wouldn't even be there in the first place, because there's only so many hours in a day and my open source work is not paying my bills most of the time. (patches and continued maintenance is of course welcome) So the alternative would be just the source code, which you still get. No one is limiting your options there. If anything the Dockerfile at least shows exactly how you can build the software yourself even without using docker. It's just bash script with extra isolation.
Yeah please don't do that. It's probably not a good idea. Just build the binary or whatever you're trying to use yourself. The binaries in image often depend on libraries inside said image which can be different from your system.
It might be easier (effort-wise) but you're certainly not forced. At the very least you can clone the repo and just edit the Dockerfile to your liking. With compose file it's the same story, just edit the thing. Or don't use it at all. I frequently use compose file just for reference/documentation and run software as a set of systemd units in Nix. You do you. You don't have to follow a path that someone paved if you don't like the destination. Remember that it's often someone's free time that paid for this path, they are not obliged to provide perfect solution for you. They are not taking anything away from you by providing solution that someone else can use.
I fully agree with you that devs should not release debs&rpms&etc, that's distro responsibility to create and manage from the binaries that the devs should release. No Dev should have to create those distro-bases formats, it's evil and useless.
Let me be more clear: devs are not required to release binaries at all. Bit they should, if they want their work to be widely used. And in this case, providing also a binary release alongside images solves all freedom of choice issues in my opinion. Here you show me my lack of preparedness as I didn't considered docker files as actual build instructions, I will do in the future.
I also fully agree with you that curl+pipe+bash random stuff should be banned as awful practice and that is much worse than containers in general. But posting instructions on forums and websites is not per se dangerous or a bad practice. Following them blindly is, but there is still people not wearing seatbelts in cars or helmets on bikes, so..
I was not single containers out, I was replying to a post about containers. If you read my wiki, every time a curl/pipe/bash approach is proposed, I decompose it and suggest against doing that.
Chmod 777 should be banned in any case, but that steams from containers usage (due to wrongly built images) more than anything else, so I guess you are biting your own cookie here.
Having docker files and composer file is perfectly acceptable. What is not acceptable is having only those and no binary releases. Usually sources are available (in FOSS apps at least) but that can be useless if there are no building instructions provided or the app uses some less common build stack.
On Immich, which is a perfect example of an amazing piece of software fast growing and very polished, I did try to build from sources but I couldn't manage the ML part properly. This is indeed due to my lack of experience with the peculiar stack they are using, but some build instructions would have been appreciated greatly (now I realize I should have started from the docker files). I gave up and pulled the images. No harm done, but little extra fun for me, and while I do understand the devs position, they too keep talking about making a living out of it and that's a totally different point to discuss on a different thread. I would suggest them that public relations and user support is more important than actually releasing an amazing product for making a living out of it. But that's just my real world experience as product manager.
In a world where containers are the only proposed solution, I believe something will be taken from us all. Somebody else explained that concept better then me in this thread. That's all.
Yeah, but that's not there reality of the situation. Docker images is what drives wide adoption. Docker is also great development tool if one needs to test stuff quickly, so the Dockerfile is there from the very beginning and thus providing image is almost for free.
Binaries are more involved because suddenly you have multiple OSes, libc, musl,.. it's not always easy to build statically linked binary (and it's also often bad idea) So it's much less likely to happen. If you tried just running statically linked binary on NixOS, you probably know it's not as simple as
chmod a+x.Exactly what I'm saying. People will do stupid stuff and containers have nothing to do with it.
Most of the time it's not necessary at all. People just have "allow everything, because I have no idea where the problem could be". Containers frequently run as root, so I'd say the chmod is not necessary.
I think you mean images not containers? I don't think anything will be taken, image is just easy to provide, if there is no binary provided, there would likely be no binary even without docker.
In fact IIRC this practice of providing binaries is relatively new trend. (Popularized by Go I think) Back in the days you got source code and perhaps Makefile. If you were lucky a debian/src directory with code to build your package. And there was no lack of freedom.
On one hand you complain about docker images making people dumb on another you complain about absence of pre-compiled binary instead of learning how to build stuff you run. A bit of a double standard.