If your graphics is ordered by Z position and it's in a clipped panel, then you may have only move a few of the materials that need to be in the front forward. If there is an instant where you don't have something from the material that needs to be in the front at a lower Z, then the whole thing will be pushed under the background.
This means, for instance, if you have a background at 0 that is always in the screen, and you scroll a bunch of widgets from your other atlas around - if only a single widget in the panel is set to z = -1, then when that particular widget is clipped and removed the rest of the widgets can fall below the background in draw order.
Does that make sense?