Scroll view will attempt to move only if the children happen to lie outside of it, and are not taking advantage of the space within. So if your starting position has space on one side of the scroll view, and items extend past the border of another, then it will attempt to fix that. Can you post a pic of what your initial startup looks like in the scene view?