It depends on the spec of what your computer is. There is no point spending cash adding a rather good graphics card for instance onto a PC that is so slow it wouldn't see the advantage.
If you could post what your PC is ie speed of processor, amount of memory, operating system and hard drive size you would get more reasonable answers.
In general the easiest and most cost effective upgrade is memory. Hard drives are also easy to add as are optical drives CD Roms DVDs etc. Graphics card are fairly simple as are adding in extra cards such as a TV card.
It depends on how fast your PC is and what you want to do.