Keeping in mind that you do mostly portraits. Okay!
For 'environmental' shooting, or to make the most of backgrounds behind sitters, THE ideal is the Nikkor 35mm f/2 OC. Mine is a 'retired' newspaper lens from 1970 and gives the most fantastic mid-tones and exquisite definition.
Standard lenses, the Nikkor 50mm f/2 HC is as much as you will ever want. The best all-round lens Nikon ever made, in my estimation. The 50mm f/1.4 is also okay but more 'boutique' than the slower f/2 which is just as good. If you can find the faster '50' at a good price, go for it. I also have a Series E 50mm f1.8 that is no mean shooter for people images, but it's an AIS from the 1980s, so may not be the era you are looking for.
If you want warts-and-all images, try the 55mm f/3.5 Micro Nikkor. I have an early version, mid- to late 1960, with all the legendary quirks, but needle-sharp definition. For the older sitters, with this lens you may need a Nikon Soft 0 or 1 filter to gloss over those aged skin tones and life creases. Think Steve McCurry.
Shooting the more elusive subjects call for either the Nikkor 105mm f/2.5 or the old classic 135mm f/3.5. Either suits me fine. The former has acquired legend status and often sells at too high prices, the latter will be much cheaper, like less than $100.
Remember that with so many old Nikkors available, 'beater' lenses if the glass is clean and clear and they work well will be your best value. My '35' looks like a bottom-of-the-harbor budget buy, but always delivers the goodies.
The choice is really up to you, depending on your needs. Two or three carefully chosen lenses will suit you, without the need to own the entire Nikkor range.