It happens but u made a mistake when u started dating nd you know you have feelings for ur ex because if ur boyfriend comes to know tht he will get really pissed nd mad, he'll think tht u were using him all this while, anyways if you want to move on, do your best to love ur boyfriend nd any sign which appears from ur ex, just ignore it nd move on :) I hope it helps you what I just said :D
Trencekt