this post was submitted on 10 Sep 2023
136 points (95.9% liked)
Linux
48153 readers
763 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I am on AMD, as I had too many problems with Nvidia over the years (e.g. driver borked after upgrade and stuck on console). That said, AMD driver ain't of great quality these days either. Playing around with StableDiffusion and running out of VRAM will crash the driver and require a reboot. Some Xorg/kernel/driver versions crash frequently. One of my monitors does not work when connected over HDMI in Linux (flickering image), but works fine in Windows.
AMD is also quite a bit more challenging with AI stuff in general, as everything is using Nvidia's CUDA. The situation is however improving. On the positive side, AMD cards have more VRAM than equally priced Nvidia cards, which is far more important than raw performance in AI workloads (not having VRAM means you just can't do some things).
So not exactly smooth sailing, but AMD mostly works ok. For plain gaming via Proton I didn't have any issues.
How does it work for ML and AI tasks with AMD and their ROCm? I really despise Nvidia as a company but unfortunately when it comes to AI/ML and other productivity tasks AMD is just much inferior to Nvidia's offering. Otherwise I would have definitely been very interested in the AMD 7900XTX, that has everything for pretty much half the price of 4090.
Install the ROCm version of packages and it mostly works (at least on Arch, and I assume Ubuntu since it has official packages from AMD.)
I am also using arch anyway, even though I am thinking of switching to Void now. I am just a bit worried about the "mostly" part. I was toying with the idea of getting 4060Ri with 16Gb of RAM, because of the VRAM. The other two cards are 4080 and 4090 I was considering but both are overpriced, especially the 4080.
I was running a LLM earlier this evening and my 3090 started to make a ticking noise everytime a token was generated. Panicing, I look for the first time in years at AMD's lineup. A 7900 XTX goes for $1400 CDN, but a 4090 is almost $2500.
I also use Daz Studio/Iray to render characters before feeding them to Stable Diffusion, because it's a lot easier to get exactly what you want without spending hours tweaking prompts and seeds and hoping for the best.
An extra grand isn't really that bad when you factor in the lifetime of the card.
If it’s a ZOTAC card it might just click when the fans start and stop. My ZOTAC 3060 makes a click when the fans start and stop. It’s a good way to know when my PC wakes itself up lol
Naw this was a click everytime a token was generated, and they were generating really slowly so I knew something was wrong. I think it was bouncing off the 24GB memory limit and something was being tripped, so I changed the loader from AutoGPQ to ExLlama_HF and everything works fine now.