I had a Power Supply that acted like this once.
It's impossible to deduce what may be the problem, many different parts might cause it - you'll only find out by substitution. Try unplugging the PSU from the motherboard, and firmly replacing the connector first - then try another PSU.
"Even when i disconnect it from the mains the power button will still remain lit as orange for a while."
That's normal - it usually takes a fair few seconds for the power from the PSU to die fully away.