first you are not wrong for caring for him. babies develop faster when in school as they are exposed. you may want to consider placing him for playgroup at least. in that way you are still the main caregiver at the same time baby goes to school and has friends his age.