Damn, you guys are all fancypants. *Buying* parts? HA.
http://freecycle.org <-- Good place to get things, usually whoever claims first or can pick up first is given dibs. If you ask for broken stuff, you'll get lots.
Other than that, tell your friends you make things, and make sure they give you all their scrapable garbage. Even if you have no use for anything, get in the habit of doing a complete teardown on anything before you'd throw it out, at least you'll build a mental inventory of where you could find things. Haul things out of the alleys on garbage day, check the dump, try to set up arrangements with thrift stores in your area for all their broken electronic donations once a week or whenever they'd bin them, (that otherwise cost them money to haul away, some might even deliver for you). Appliance and motor repair shops may be helpful. Call up each of the schools in your area and ask if they have obsolete or broken electronics they're getting rid of soon, and if you can have them instead. Sometimes they auction them (where they then end up at scrappers), but sometimes they don't and just junk them. Police lost/seized property divisions often junk massive amounts of things, for example, for every 30,000 people a city has, the police round up about 1 abandoned bicycle a day. That doesn't sound like much, until you note that a city of a million hauls in about 1,000 bikes a month, that are unclaimed and thrown to the dump, or at minimum sold for scrap. If you have scrap you can't use, head to the scrap yard and see if they'll let you pick over recent arrivals and trade it. Some'll let you trade at par, as long as you stay out of their way. Scrap steal is basically not worth your gas money, but copper is $5/pound, alum is $1/pound, transformer or motor iron goes for $0.30/pound if stripped, or half those amounts if you don't take stuff apart. Scrap materials are easy to find. If you need scrap bits of wood for smaller projects, construction sites have bins full (under 4' is considered scrap to a framer), and Big Home Stores will have a cut room with all the end-cuts, they'll usually just give you whatever you want, plywood, 2x4 ends, whatever.
Most appliances have enough components for at least one big project (fridges/freezers being the exception, they're good for just about nothing but sheet metal). Old TVs and VCRs are plentiful these days for free. Tons of basic components inside them. Stereos, old computer power supplies, motherboards, even phones are good. Just on down-cycled items that people keep until they next see me, about once a year I have a scrapathon where I haul out the blowtorch and pliers and salvage everything I've accumulated and not bothered to strip yet. I've never had to buy parts, except a few specialty things, and I build all sorts of stuff in my spare time.