I echo what Travis said. I check every site / airline that has the route I am taking. Sometimes (a lot actually) the airlines directly have the best value, sometimes not.
To me, I value such things as being able to select seats ahead of time, direct flights, flexibility in selecting times of departure, and since I'm a points whore, I like to maximize my points accumulations so I'd be willing to pay a few bucks extra to get points than waste potential points on a carrier I don't have a long term reward membership program relationship.
Since most of my travel is business related, checking for the cheapest fare isn't up to me. When I go personal, mostly to Canada, I have found directly booking with the airline online, not by phone or at the airport, offers the best value. When it's really last minute, some of the discount sites offer better deals than some of the airlines, but for some sites, you have limited control over your route.
Check them all.
For example I am going to Toronto next weekend (LGA to YYZ). Here are the airlines:
Air Canada - $485, nonstop, 1.2 hr flights both ways, reward points, seat selection, can save extra $12 if I don't check any baggage and commit to flight, I select departure times (10:30am/11:15am), 4 departure time options
USAir - $541, 6am/7am departure, one option only, 1 stop, 4.5 hr flights each way, no rewards
NWA - $542, 14 departure options, 1 stop, min 4 hr flight each way, rewards
Continental - $591, 3 departure options, 1 stop, min 4 hrs each way, rewards
AA - $640, nonstop, seat selection, multiple options, 7:25am/12:50am departure, 1.2 hr flights each way, rewards
United - $719, non stop, same departures as Air Canada, Tango class versus Tango Plus on AC, rewards 50% less than booking directly on AC
And the travel websites:
Travelocity - $487, 1 stop going, nonstop return, 6am departure going, no rewards going, Air Canada return, rewards 50% less, no seat selection
Expedia - $490, same departures as Air Canada, Tango class versus Tango Plus, 50% less rewards than Air Canada directly, $30 extra change fee on top of Air Canada, no seat selection
Orbitz - $492, 1 option only, 6am/7am departure, two different airlines, 1 stop, 4 hr each way
Yahoo - $497, 1 stop, 4 hr flights each way, two different airlines, 1 with no reward program, no seat selection, only one departure option, 6am / 7am departures, Air Canada on one leg, 50% less rewards
Anyway, you get the idea. In general, the travel sites offer the best deals, but upon closer examination, most of the cheapest fares are on Air Canada on those sites anyway, so in this case, I will be buying the ticket directly from Air Canada.
Regards, Art.