Looking at your horoscope matching I see that Yes you two have good attraction towards each other. However astrologically your charts suggest that your relationship will suffer because of interference of others. I think I have already indicated that earlier.
I can only say that you need to balance every relationship in such a manner that it does not break and at the same time it is not a burden for you to carry. I guess you should interact with his family more so that you know how comfortable you will be with them.
Give your relationship some more time and many times answers to our problems come from within, and you do not need to look any further. I believe if you are open and honest with them you will find a solution to your concerns.