Am I being too hard on my son?

no you are not being too hard or mean. you would be doing him an injustice to continue to allow him to be lazy and disrespectful. it sounds like, even though he has a slight disability, you biggest problem is he is milking the situation. God forbid, but if something happened to you, he would need to know how to survive in this world. Time stops for no man!