Honestly the parts of world building that always intrigues me is the daily life of people. Sometimes the most mundane daily activity can be very telling of the world and characters.

Anime does this so damn well that Disney (do they even make cartoons anymore) and other animation studios don’t get into. Characters cooking, cleaning, getting dressed or even just lounging around on a hot day. They feel like more believable characters because of the nuances of humanity that they reveal through those mundane moments.

I'm gonna say for the the most important aspect is the building part in "worldbuilding".

Good worldbuilding is not simply a matter of stacking buzzwords like "culture", "politics" or "rites of passage" on top of each other and calling it a day. Different aspects of the mechanics and the life of a world have to be joined by tendrils, scaffolding, whatever you want to call them, that connects them, holds them together and gives them a distinct or at least a definite shape. Even if that shape is more or less "Medieval Europe retrofantasy pastiche 14772396293".

Things like the "mundane focus" (that @BroBot9000@lemmy.world mentions) , the in-universe anecdotary or documentary, or the bystander portrayal of the heroes and antagonists's activities, help a lot with that.

The hook.

What is the thing that makes the built details worth caring about? Is there a single big novel concept or a bunch of interesting stories, or just anything worth sinking into?

Pages upon pages of how a magic system works is like disconnected rambling if it isn't in service of something interesting.

I much prefer broad stroke worldbuilding that initially starts by creating a theme or an atmosphere, touches on details in passing as they come up to be of interest, and only after establishing these broad foundations gets into the nitty gritty, and even then only if going into the ultra micro detail level is needed to be explained for some other purpose.