Downloaded software I would move to another folder as you never know when you may need to do a re-install. Anything else that you download can be deleted if you are certain that you won't need them again. Obviously, I presume, any 'paid for' downloads such as music, films etc. you will have already moved to their relevant folders. It is best to treat downloads folder as just a temporary location for files.
The Legend of Zelda Breath of the Wild review: Five hours with Zelda on the Nintendo Switch