-
For the NUC I'd see if you can stretch slightly more and get an 8th Gen processor one or later. They're quite a bit more powerful for not much of an increase in cost.
My library has something like 8,000 tv shows and 1,000 movies and is perfectly fine. I occasionally run it on my file server when I'm testing stuff and it chugs a bit on there (and can't transcode).
I run my plex server on Ubuntu (and also use it for playback) as (I think) it's a little lower OS overhead and a bit more stable. Saying that I've done a fair bit of cursing whilst mapping network drives and getting shit to work whereas it's much easier to do all of that on Windows (although someone who uses linux a lot may disagree with me there)
format wise the stuff i have tends to be .mkv or .avi (less common)
What qualifies as a big library? I won't be using plex for photos or music so won't encounter the same problem that @Velocio had
So I could use something like this combined with something like this this
If so, that's the hardware taken care of but software?