The one that you don't love because you will eventually begin to realize what you have(which is a good relationship with someone who cares about you) and the other person wouldn't matter anymore...
If you love him and his ashamed of you that just shows a sign that u arer just waisting youre time, but in my opinion I think you should just follow youre heart