Arguably what makes AV1 so attractive nevertheless is that it's royalty free.
AV1
Community for AV1 video codec
My major discontent with AV1 has been how the encoder blurs some details completely out
That's the main reason I did not personally switch to AV1 yet as well. (Second reason being that my laptop struggled with playback too much.) Last time I tested it was 2 years back or so, and only using libaom, so I definitely hoped it would be better by now. I was so hyped about Daala (and then AV1) all those years back, so it's a bit disappointing how it turned out to be amazing and "not good enough" at the same time. :)
Losing details seems to be a common problem with "young" encoders. HEVC had similar problems for quite some time; I remember many people preferred x264 for transparent encodings, because HEVC encoders tended to blur fine-grained textures even at high bitrates. It may still be true even today; I didn't really pay attention to the topic for the last few years.
IIRC, it has to do mainly with perceptual optimizations: x264 was tweaked over many years to look good, even if it hurts objective metrics like PSNR or SSIM. On the other hand, new encoders are optimized for those metrics first, because that's how you know if a change you made to the code helped or made things worse.
I suppose only when the encoder reaches maturity and you know it preserves as much real detail as possible, then you can go wild and start adding fake detail or allocating bits to areas more important to subjective quality. I'm sure some (many? most?) of such techniques are already supported and used by AV1 encoders, but getting the most of it may still take some time.
With SVT-AV1 have you been using the --film-grain-denoise 0
flag? If you share what your encode settings are there may be improvements that can be found.
I use that everytime with film grain. Also without grains I always noticed blocky patches around the edges (which can still be found but much less) in many of my encodes.
This is what I usually use -yuv420p10le -preset 3-6 -crf 18-32 -svtav1-params film-grain=8-20:film-grain-denoise=0:enable-overlay=1:scd=1:scm=0
For what it's worth, I personally think svt-av1 also produces a worse looking picture than aomenc. I'm not personally convinced that AV1 is at fault, just that the x265 encoder is much more mature than svt-av1, aomenc, and rav1e. I haven't tried a high quality film source yet, mostly just SD animated things, where svt-av1 usually loses to aomenc in what I'm looking for (best quality picture per bitrate), but svtav1 tends to win out in encode speed.
If you haven't tried it out yet, I would suggest trying av1an (an encoding framework) with aom and at least comparing a few samples with what you get out of ffmpeg and svt-av1. Av1an can do chunked encoding, which helps speed by running multiple encoders in parallel. I don't think that aspect helps much with svt-av1, but it does seem to help with aomenc.
I wrote a bit about it here (https://lemmy.ml/post/2843230), though I didn't include any film grain settings in that post.
For me, they both lose details the same way, AOM is just higher quality and compresses even better, neither of which is feasible for personal use at this time. I have some hope for rav1e but it is not mature enough and also missing many features, so not using that either.
The suggestions in this document may be of interest to you if you are willing to give aomenc one last look. And as for using it with ffmpeg, aomenc benefits from a two pass encode (which av1an will do automatically) https://gist.github.com/shssoichiro/a46ff01db70243c1719479f6518ea34d
It's entirely an encoder issue...
I'm surprised nobody here has recommend the usage of the 2 main aomenc forks (aom-av1-psy and aom-av1-lavish) and my fork of SVT-AV1: https://github.com/BlueSwordM/SVT-AV1/commits/master
It can make quite the difference in terms of fidelity.