Before you blame component failure try this.- Shutdown and switch off the computer at the wall. Now press and hold the reboot button for at least 20 seconds. Cross fingers and switch on again as normal. I cured a similar non-start some time ago on this much younger computer.
Just my opinion:.............. Yes, easy to replace the PSU but make quite sure of the physical attributes to match your motherboard and case.
The root problem needs inspired guesswork! In a younger machine I would say the PSU is to blame. It should be giving accurately controlled voltages at all its potential leads. Many computer repair shops are able to check your PSU whereas you can't do this with just a multimeter because the various voltages are only true when under load If you remove the existing PSU and take it in, the shop would doubtless sell you a suitable one if yours is faulty.
Motherboard failure is bad news indeed and probably not worth replacing in such an old model.