Particularly among experienced cyclists, one will often hear advice about "bike fit" or "improving the engine" when people talk about riding more comfortably, faster, or further. I'll be the first person to say that even as an experienced cyclists whose read and thought about the subject, I am no expert. So instead of rattling off detailed nonsense, I have collected a few sources of bike fit information and categorized them below.
My only advice is to be willing to try different things if the fit isn't perfect. A little experimentation will go a long way towards figuring at a comfortable setup for you.
How to Fit a Bicycle ...
Peter White discusses his method in fine detail.
The Competitive Cyclist has a GUI that maps your body measurements to an optimally sized bike. They also have some verbiage that discusses their methodology on that page.
If you're really new at this, Jim Langley's page is a nice place to start.
For a scientific oriented discussion, the pages at slowtwitch are interesting.
What I chose ...
After reading a few articles and seeing some videos, I became a fan of Dr. Andy Pruitt. A google/youtube search on his name will produce a ton of hits which, unfortunately, seem to be somewhat unstable.
His medical guide for cyclists.
His fitting method and "optimized" components/clothing/shoes sold by Specialized.
A summary from a popular randonneuring site.
Some stuff on saddles ...
A summary from Captain Bike.
Jim Langley's discussion on saddles.
Need to change a stem?
A page that shows extension and rise as a function of stem angle and length.
Jim G's that compares two stems to get the change in rise and extension.
Looking at different (road) handlebars?
Two general discussions on handlebars ...
... by Grant Peterson of Rivendell.
... by Tom Deakins of the Harris Cyclery.
Probably more details than you want, but nonetheless, this is an awesome comparison tool. The rest of the blog has other pages with more handlebars, saddles, and so on.