and I feel that much worse about one of the hardest decisions I've ever made as a parent.

You made the best decision you could with the information you had at the time. Imagine how much worse off he could be now if you had said, "Nah, we'll just wait for him to outgrow it."