As someone who spent a lot of time learning html/css and wordpress to get a site i'm ok with, but not thrilled.
I'd say skip wordpress. I think it's overkill for a portfolio. Mine is just a personal blog, and i'm about to sign up for the $9 a month Ghost Pro hosting with a free theme. I'm over the plugins and getting nickel and dimed for features.
Like others have mentioned, a static site generator may be good too.
Also, it isn't open source, but i really like blot.im. I use it for Obsidian.md instead of the built-in publish because it's a lot cheaper, and it's close enough in appearance/quality.
EDIT: Actually, it looks like I was wrong. Blot.im appears to be open source too.
It's been almost a year since I checked it out, but I had a lot of issues with jellyfin compared to plex. Mainly transcoding would never work, so streaming outside of my local network never worked well.