You say that the fans are spinning. Is this when the computer fails to start or only when it has started correctly?
The computer starting up involves the power on/off switch, the motherboard and the power supply. You have eliminated the switch, so that leaves the other two. telling which one is almost impossible other than by substitution. When the computer is off there is still 5V standby power applied to the motherboard by the PSU, and this is what is used to start the computer. I suspect that when you "pull the main plug out of the wall and fiddle with the fuse" you are just removing the 5V standby and for some reason this lets the computer restart. I suggest that you buy or borrow another PSU and see if that cures the problem. A PSU isn't terribly expensive and is fairly simple to replace. A new motherboard is another matter entirely.