We've been wrestling with very similar issues here. The concept of the panel is just a bit confusing, at least to me. It suggests to me something physical, visual. But actually, although the panel does have physical location in the 3D / 2D space, it's "draw space" is separate from that.
I think I have a basic understanding, technically, of why this is the case.
However! Now you can change the panel to a clipping or constraining panel. And if you turn anchoring on, anchoring gets applied to the constraints / clipping, not to the physical position. This adds one layer of confusion. AND NOW on top of that we have the fact that scroll views use panels as the basis for their scrolling, so the physical position is moving but the clip area... uh... probably isn't?
And again, the technical argument is, well, how do you handle deciding how far the scroll view may scroll? You need some kind of bounds related to some kind of viewable area.
ArenMook, you have a real talent for making technically challenging things simple for users, as demonstrated by the vast majority of NGUI. Perhaps there is a simpler solution here too?