For a completely silent PC , with the budget in mind, I can recommend these components. The only constantly "Spinning" device will be the hard drive which won't be heard anyways.
Case: If you are going to hide it anyways then it really doesn't matter as long as it can fit a standard ATX PSU & has decent venting. Personally I prefer a good look so I'd look at a Silverstone model.
Power supply: Silverstone Nightjar Silent (Fanless) PSU
CPU: AMD BE-2350 45W
Heatsink: Scythe Ninja Mini (Fanless)
Motherboard: Gigabyte GA-MA78GM-S2H
Memory: 2gig (2x1gig) Transcend, Patriot or other DDR2. (Great price, warranty & performance)
Hard Drive: Samsund Spinpoint F1 (There are others but for low sound vs performance the F1 Series delivers.)
Samsung's Spinpoint F1 hard drive - The Tech Report - Page 1
This isn't a be all & end all of components just my take on what I believe you are after. This will be an actual Silent System. I may even build one myself.