Most people have a view about how they would like their life to be, but for most of us the reality is somewhat different. The real world dictates to us in lots of ways, and the trick is in finding the right compromise between hope and reality.
Your situation is an increasingly common one, and there's no magic answer. The one piece of advice I might offer is that you're going to find all kinds of reasons not to make changes. Your 20 year old son is perfectly capable of looking after himself, but young men are lazy by nature they like having someone around to do the drudgery, and from what you say he is no different.
My mother taught me to wash, iron and cook for myself when I was about 17 years old, and although I grumbled endlessly at the time I was very grateful for the knowledge when I went to university and had to fend for myself. You can do the same with your son, and gently disengage from being his home help. If you don't you'll find it hard to get that life you want for yourself.