UIDragDropItem was written with 2D in mind, not 3D. The screen space pixel position is simply added to the transform's local position, without any consideration for angle of drag or other 3D-related corrections. You will need to handle dragging differently with 3D. You need to create a plane in OnPress(true), save it, then instead of reacting to OnDrag(delta), do raycasts into that plane in Update(), then update the position of the dragged object by setting the position, rather than by passing the delta.
It's very different from the 2D approach.