The same thing happens to me - quite often I might say.
For example:
I have a Container grid 4x1 ( desktop ) and when I go to mobile view I transformed it to a 1x4 grid but the elements got displaced. When I try to place them in the correct order, they go back to the position they were before, as like they were pinned there.
In another post, someone suggested to change layout setting on the inspector.
From Fixed to Fluid, etc… but it didn’t work.
Any other suggestions?