Chances are that most people here would tell you to buy the parts and build it yourself (which I have done). If you're into that kind of thing, there are lots of great online resources that can help you get started. It sounds complicated, but putting a computer together is like playing with Legos that have sharp edges and a few cables. If you're interested in it, have a look at http://www.http://pcpartpicker.com/, it's a great resource for pretty much anything related to PC building. It'll save you a few dollars too if you decide to go that route.
Sorry for the wall of text, I went a bit overboard.
If that isn't really your cup of tea, I'll try my best to give you a decent recommendation for parts and a general overview of the main parts you should think about.
CPU: AMD vs Intel
There used to be a time where there was usually a clear option for what you want to pick, which is still fairly true nowadays, but not quite as definitive as it used to be. Generally, AMD has a large market share on budget parts. Their CPU's are cheap, yet powerful, the perfect bang-for-your-buck. I'm running an AMD FX-8150 in my computer, and it's all that I've really needed. As long as you don't do any extremely CPU intensive tasks like encoding, I would definitely recommend that you get an AMD CPU.
Intel has a lot of fans though, as their chips generally are better performing than their AMD counterparts, but with a higher price tag to boot. A good current-gen i5 though is pretty much always a solid option, they're relatively inexpensive and pack a lot of punch.
GPU (Graphics Card): AMD vs NVIDIA
Now, as with CPU's, there was a time where NVIDIA was undeniably better in all regards. Now that isn't exactly true. As with CPU's, AMD has some great budget graphics cards that are pretty powerful cards. AMD cards are better for bitcoin mining if you're into that kind of thing. Some good manufacturers for AMD cards are Sapphire and XFX. I used to run a Sapphire HD7870 in my computer, but it had faulty memory and I had to have it replaced. (It wasn't AMD nor Sapphire's fault, I bought it in a Hong Kong marketplace and they somehow managed to damage the card)
I'd recommend an NVIDIA card though. They have better drivers, and although they are more expensive, they are really powerful. Even the lower-end cards are pretty dang strong. I'm running a GTX 760 in my computer at the moment, and it runs like a champ. I can play most games at 70-80 FPS at almost maxed graphics, running Diablo 3 at about 120-150 at max pretty consistently. The only time that I drop below 90 FPS is when I'm partying with 3 frozen orb wizards or something. For NVIDIA card manufacturers I would go with EVGA, MSI, or Asus. EVGA has great customer service and reliably sturdy cards.
Anything more than 8GB of RAM is going to be wasted for what you're doing. Just make sure that you get a good speed on your memory, the higher the better (Don't spend too much though, RAM prices are insanely inflated right now)
Hard Disk Drive (HDD) vs Solid State Drive (SSD)
To put it simply, Hard Disk Drives are cheaper and more reliable than SSD's. However, because of the way SSD's store memory they are insanely fast. So installing games on an SSD will make games load faster. A lot of people opt to have a 1TB HDD for main data storage, movies/pictures/documents/non-essentail programs etc, while having a secondary drive, a small SSD, to install games on. SSD's are expensive, so if it isn't in your budget it isn't that big of a deal.
Edited by SideGFX#1780 on 3/20/2014 7:05 PM PDT