I’ve played around with it some more and it looks like there is no part of the custom icon that remains at the actual location. The whole icon slowly drifts away from the correct location as you zoom out.
It also looks like this problem repeats for any custom icon and for any map location.