Since you're just starting out, I'd say it's hard to go too far wrong so long as you stick to fairly ordinary products -- that is, avoid infrared films or developers designed to produce exotic results.
For film, every film available today has its fans. The suggestions so far in this thread are for some of the more popular films. Part of the problem with making recommendations is that the differences in image quality are matters of personal preference, they're hard to describe, and you may not even know what you'll eventually prefer until you've tried a few options. (You should probably stick with just one or two films to start with, though, so that you can learn how to use them.)
For paper, similar comments apply as for film -- there are lots of products available, and every one of them has people who like it. Pick one and stick with it for at least a couple hundred sheets. Variable contrast (VC; aka polycontrast, multi-grade, or other terms) paper is convenient, particularly when you're just starting out, since you don't need to buy several boxes to get a full range of contrasts. Resin-coated (RC) paper is easier to handle than fiber-based (FB) paper, and RC paper also dries wrinkle-free. (AFAIK, all current color papers are RC papers, so you're probably familiar with the RC characteristics.) Some people prefer FB papers for final and/or high-quality prints, so you may want to sample them eventually, but I'd start with a VC RC paper.
For developers, I'd stick with something mainstream to begin with. For film, Kodak D-76 (Ilford's ID-11 is almost identical), Kodak HC-110, and Kodak XTOL all spring to mind, although there are others that qualify. For paper, Kodak Dektol and the Agfa Neutol line (there are several variants) are popular. Ilford and others make popular print developers, too, but I'm not familiar with Ilford's print developer line.
Overall, then, I'd say you should probably buy based on convenience (what local stores or preferred mail-order vendors stock), price, and perhaps any general brand preferences you have.