Restrict within panel option works on panels that have clipping set to something other than "None".
If you set up your panel with Invisible clipping, with a UIStretch on that panel that would automatically adjust the clipping to screen's dimensions, and change UIScrollView's Drag Effect to "Momentum" or "None", then you will achieve the effect you're going for.