Lets face it that your horoscope matching is not perfect. There is Gana dosha, you can read about Gana dosha on Astrologymag.com, besides other things, which I will not go now. Im reminded of a saying I read long time back and it said “If you love somebody set it free, if it comes back it is yours and if it does not come back it was never meant to be.
So instead of putting barriers on him, let him go and once he is done with them, he will return back to you.
Keep fast on Thursdays to improve your relationship with your husband.