Worth noting that you don't necessarily need new hardware, you can install Linux on your currently Windows PC in either a dual boot configuration (both OS installed) or replace Windows. You can also run Linux in a virtual machine to test the waters.
For a decent, x86_64 PC for Linux, the Steam Deck is ironically a pretty good deal for what you get. The Deck will run VSCode just fine, maybe just add a monitor and keyboard/mouse.
Linux can be pretty lightweight, it runs beautifully on my cheapo netbook from a couple years ago: WinBook CW140. I've done a good chunk of professional web work on that thing. The Deck beats it in every way, but it does web dev perfectly fine.