If the UICamera is set to "UI" mode, then hit detection is fully dependant on the widget depth, and colliders always use the depth of the widget they're attached to. If the collider is attached to a game object, then it will try its best to figure out what depth it should use, but if there are multiple widget children, it may not choose the right one. Safest thing to do is to use invisible widgets (UIWidget) when you need, or to ensure that you always attach colliders to widgets, not game objects.
If the UICamera is set to "World" mode, then it's all based on the distance from the hit position on the collider to the eye point of the camera.