I believe you should sit him down and have a heart to heart talk with him cos in the long run the one who is going to suffer is you.
If need be, do seek marriage counselling. His perception of marriage maybe different from yours. Embarrassment and ego are 2 did issue and it shouldn't be hindering your trusting relationship with him.
Read more