Downloaded software I would move to another folder as you never know when you may need to do a re-install. Anything else that you download can be deleted if you are certain that you won't need them again. Obviously, I presume, any 'paid for' downloads such as music, films etc. you will have already moved to their relevant folders. It is best to treat downloads folder as just a temporary location for files.
Best Android apps 2015/2016: Best apps for your new phone or tablet - what apps should I download?