OnDrag events are missing on some frames, causing jitter.

[calbar]

I have NGUI events driving a camera target based on the OnDrag() mouse delta.

Separately, I have the camera following this target doing some fancy stuff (like tilting) based on a derived delta in LateUpdate().

Unfortunately, my camera movement was very jittery, so I started logging the frame and deltas reported just inside the mouse's OnDrag() method and the camera's LateUpdate() method. Here's what that looks like:


Frame: 88 Mouse:  (-8.2, -7.4)
Frame: 88 Camera: (0.1, 0.0, 0.1)
Frame: 89 Mouse:  (-2.2, -4.4)
Frame: 89 Camera: (0.0, 0.0, 0.1)
Frame: 90 Mouse:  (-1.5, -5.9)
Frame: 90 Camera: (0.0, 0.0, 0.1)
Frame: 91 Mouse:  (0.7, -18.5)
Frame: 91 Camera: (0.0, 0.0, 0.4)
Frame: 92 Camera: (0.0, 0.0, 0.0)
Frame: 93 Mouse:  (5.9, -11.1)
Frame: 93 Camera: (-0.1, 0.0, 0.2)
Frame: 94 Camera: (0.0, 0.0, 0.0)
Frame: 95 Mouse:  (26.0, -25.2)
Frame: 95 Camera: (-0.5, 0.0, 0.5)
Frame: 96 Camera: (0.0, 0.0, 0.0)
Frame: 97 Mouse:  (19.3, -11.9)
Frame: 97 Camera: (-0.3, 0.0, 0.2)
Frame: 98 Mouse:  (22.3, -12.6)
Frame: 98 Camera: (-0.4, 0.0, 0.2)
Frame: 99 Mouse:  (11.9, -4.5)
Frame: 99 Camera: (-0.2, 0.0, 0.1)


I've highlighted what looks like the problem. While the camera's LateUpdate() is being called consistently every frame, you can see that OnDrag() is not. As the camera delta is derived from the OnDrag() mouse delta, when OnDrag() misses a frame, the camera's delta becomes 0, introducing a single frame jitter on things that rely on this delta (such as tilting).

What I'd like to know is if I'm missing something or making a silly mistake that could easily fix this, or if I need to explore a more involved solution, such as keeping a history of deltas for smoothing. If it's the latter, I'd love any better suggestions.

Thanks!

[ArenMook]

OnDrag is only called when you actually drag the mouse, not every update. I'm not sure which camera you are moving, but if it's the UI camera then you're in for some trouble as delta is based off of it as well. Bottom line, you can't use screen space delta. You need to record the starting position, and knowing the current position offset your movement accordingly.

[calbar]

Thanks for your feedback! I appreciate it.

Yeah, I'm familiar OnDrag() and I can assure you the missing events occur during constant mouse motion with a button engaged, between OnPress(true) and OnPress(false) events. I wondered if maybe the mouse signal itself was shoddy so I also tried my MacBook trackpad, moving in smooth, deliberate circles, and both devices are randomly missing OnDrag() events. I'm not manipulating the UI camera, and outside the sporadically missing events, everything is working exactly as intended, so I don't think there's a conflict there.

Finally, I'll definitely note the offset approach you suggest, but I'm not sure this situation could benefit. Let me quickly explain:

The camera target is directly driven by screen space delta, so I could definitely change that to use an offset. But the camera that is watching the target is just setting its position relative to the target position, then determining its velocity by simply comparing its current position to last position.

So in the end, if an OnDrag() event is missed, the target won't move (whether it's using a delta or offset), therefore the camera won't move, making the camera velocity zero on that frame, causing features that rely on that velocity (like tilting) to jitter for a frame.

I hope that makes sense!
So... why would OnDrag() be skipped on a frame? Without a way to fix the missing events, it seems like I'll have to cache at least a single-frame history of deltas in order to smooth over the jitters, but that's a total band-aid. If you have any additional suggestions or even just hunches, I'd love to hear 'em.

Thanks again!

[ArenMook]

OnDrag happens roughly 20 times per second when you are always moving the mouse (or less).

I am guessing your framerate is higher than 20 frames, no?

[calbar]

Ah, yes! That would make sense... I'm testing at 60 and hope keep it around 30 on devices.

Hmm... well this gives me something to think about. Do you have any suggestions?

[ArenMook]

I already gave you one -- save the starting position, calculate the delta since start, not since last OnDrag.

[calbar]

Sorry, I did read your suggestion, but that doesn't solve the issue I'm having as described in my previous post.

I did a bunch more testing. Turns out Unity Input has the exact same issues, and since I'm guessing NGUI is wrapping Unity Input calls, this isn't necessarily NGUI-specific. Here's a log of Time.frameCount, Input.mousePosition, and Input.GetAxis deltas inside an Update method while making smooth circles with the mouse:

Frame: 31 Position: (477.3, 230.8, 0.0) Delta: (1.8, -1.3)
Frame: 32 Position: (477.3, 230.8, 0.0) Delta: (0.8, -0.4)
Frame: 33 Position: (522.5, 217.5, 0.0) Delta: (3.5, -0.6)
Frame: 34 Position: (522.5, 217.5, 0.0) Delta: (0.0, 0.0)
Frame: 35 Position: (565.5, 220.4, 0.0) Delta: (3.7, 0.9)
Frame: 36 Position: (565.5, 220.4, 0.0) Delta: (0.0, 0.0)
Frame: 37 Position: (586.3, 239.0, 0.0) Delta: (1.3, 2.2)
Frame: 38 Position: (589.3, 260.5, 0.0) Delta: (0.2, 1.4)
Frame: 39 Position: (589.3, 272.4, 0.0) Delta: (-0.2, 2.4)

Same behavior as I described with NGUI above. Here is a list of everything I've tried so far:

• Swapping out my mouse and using the MacBook trackpad to rule out hardware polling issues. (same results)
• Caching deltas and reusing the previous when 0.0 is detected, correcting for missed frames after. (feels kinda "swimmy")
• Calculating my own deltas using mousePosition offset (ArenMook's suggestion, same results)
• Lerping the mouse-driven object's position. (degrades response time)

Lerping the object to smooth over the missing deltas is currently the best option, and that kinda sucks. Not sure what else can be done, though. I'm still open to other suggestions, but as this is probably not an NGUI issue, feel free to let this slide and I'll pursue help through the Unity channels.