Tasharen Entertainment Forum
Support => TNet 3 Support => Topic started by: Manmax75 on September 13, 2013, 05:27:29 AM
-
Networking reliable physics is quite a hefty and challenging task to undergo. Depending on how you design your game, you could be looking at designing systems like interpolation, extrapolation and client-side prediction. Each of which have their own pro's and con's associated. A great article to read, which I found helped me a lot, is Valve's Multiplayer Networking.
https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking (https://developer.valvesoftware.com/wiki/Source_Multiplayer_Networking)
However, some of the methods described in here may not be ideal for your game, it all depends on what you are trying to achieve.
I am trying to make a physics-heavy sandbox game (much like Garrysmod). Initially I thought a full authoritative approach would work best, however I quickly ran into problems with client-side prediction. As it so turns out, CLSP seems almost impossible in the current version of Unity, without sacrificing a lot of freedom, or by implementing some sort of 3rd party physics library. In terms of sacrificing freedoms, you could alternatively make your own Rigidbody class and use straight line motion formulas to predict movement and correct errors, and then use a CharacterController.Move command to get instant collison feedback. The downside of this being that all your predicted objects cannot be anything other than capsule colliders. And for the sakes and purposes of my game, I couldn't do this.
So I had to scrap the idea of CLSP and with it a fully authoritative approach. My next point of option was semi-authoritative. Where the client takes care of his own position and relays it to the server, which then relays it to the other connected clients. In order to prevent jittery movement I implemented an interpolation system which delays the players view back 100ms. Based heavily off the default networking script and this one here: http://wiki.unity3d.com/index.php?title=NetworkView_Position_Sync
However I found that players often moved around with slightly jitter movement. Not too bad, but noticeable enough to look unnatural. It seemed that the players would move at a much lower framerate. I synched packets at ~ 20 times per second, is this too much? Is this not enough?.
In my game, players also have the option to spawn in and manipulate objects, rotating, dragging them around, etc. My current implementation uses a semi-authoritative approach. The server is authoritative of all entities until a client picks one up, then the client tells the server where the object moves and interpolates between (once again slightly jittery). The downside I have yet to resolve is that once the player lets go of the object, and the server assumes control again, there is a noticeable jump as the server is not fully up to date with what the client was prior to exchanging ownership. Something I have also yet to fix.
Another thought occurred to me that perhaps I just let the individual players simulate their own physics, and let the server send out a heartbeat every once and awhile to correct errors, however I quickly found out that things went out of sync very fast and the correction of errors was far to noticeable to allow for any smooth sort of gameplay.
After weeks of networking and prototyping I'm becoming ever frustrated with approaches on how I can solve these issues.
What do you guys thinks?
EDIT: I noticed that with a lot of the examples here, that the colliders get updated to server position straight away, and the renders lag behind. Is this a better method than what I currently have? What would be the drawbacks, etc from using this over interpolating everything.
-
I say less is more in this case. Let clients predict movements, but sync immediately upon collision or applied force. Also sync periodically -- twice per second is more than enough. Depending on the game it can even be once every few seconds (set in space for example). In the Multi-Purpose Game Starter Kit I sync 4 times per second just to be on the safe side, and even that is too much. Jitter is mainly from synchronizing too frequently.
-
In my own implementation, I use a modified example ported over from uLink (I modified it to send messages periodically rather than every frame - ouch!)
It works like a dream. Basically, the client performs in the exact order:
- Gather input state
- Simulate physics with input state
- Send input state to server along with position (redundant data)
- Store input state (along with timestamp - I rolled my own network time system)
Server does upon receiving input state:
- Simulate physics with input state (I use FixedUpdate, so time delta is the same between client/server therefore output is nearly same)
- Check result against what client sent (redundant data). If client is too far off real value, send real value to client
Client does upon receiving a state correction:
- Find the stored input state corresponding to when that correction was sent (matching timestamps)
- Set state to server state
- Resimulate every input state
The last part is very important (resimulating input), because by the time the player receives the state correction, it will have been sent some time in the past and is therefore old data (so the client must extrapolate what the correct position will be)
Syncs rarely ever happen (as everything is done in FixedUpdate using the same timestep, the output of client/server almost exactly matches, aside from numeric drift), and normally everything is smooth as butter with the very occasional hitch (hey it happens in Source too so I'm not concerned).
-
After revising my interpolation code, I found that my calculation of network time was severely wrong. I ended up rolling my own method with the use of the StopWatch class and this article on syncing times: http://www.mine-control.com/zack/timesync/timesync.html (http://www.mine-control.com/zack/timesync/timesync.html)
Now I'm getting an accuracy of time within the 10-100 ms mark. The only thing I have to watch now is for time drift, so far I've ran both a client and server for two hours and have had a 2ms drift in time. Not that big of deal, but I will have to compensate eventually. Also my timestamps are fractional seconds expressed as a float, which gives me about ~27.75 hours of continuous runtime before the precision becomes too low to do any sort of meaningful interpolation without noticing some (albeit minimal) jitter. I went to change to a double but I noticed that Tnet doesn't support it? So I think I'll just represent it as an int when transmitting and divide by 1000 or whatever to get it into a decimal value. All though I'm curious if there is a better way about doing that.
Anyway, so I've managed to smooth out player movement, using a non-authoritative model for players. All my implementations of Client-Side prediction so far have given me unsatisfactory results.
Now I just have to work out how I'm going to handle the manipulation of objects.
-
All my implementations of Client-Side prediction so far have given me unsatisfactory results.
If you're still interested I would take a look at the uLink Snowball demo. They implement server-authoritative physics with clientside prediction. The main changes I made was moving most of the logic from Update to FixedUpdate (I thought it was kind of fugly to have number of messages sent depend on framerate, and additionally I reduce another vulnerability as client does not have to send delta time - it's always the same between client and server for every fixed update), and I created an InputData class which was marked as serializable, so I batch together an array of them and send them 5 at a time as a byte array serialized with BinaryFormatter (so 10 times per second, rather than every fixed update).
You may (and probably will) run into lots of hair-pulling frustrations like rubber banding. I certainly did (spent I think 3 days on getting it working to my satisfaction). The main thing is to step away, drink some coffee, come back a little while later and double-check that the client and server are both running the same code, in the same order, with the same inputs. In my case, it was usually things running out of order.
-
Masaaki, after a lot of debugging and prototyping, I found that the root of all my problems was that my movement code was not deterministic. Thank you for your advice, it worked perfectly ;D
Now I'm getting precision of about ~0.0001 between the real and predicted states, which is phenomenal!
I'm now implementing the rewind system. Currently my client sends packets of input and predicted results 33 times per second (based on the time from source multiplayer article). Because the input collects faster than the send rate, it buffers about 2 input states per packet. So the server receives two inputs, runs through them and compares the predicted and actual states. But I have a few questions, when the client sends a packet off with a few input states, should it wait for the server to verify it before sending more? Or should it just keep sending. Like the client would continue predicting movement, but it just wouldn't send anything to the server until its old state was verified?
Or if the client does keep sending packets when the server gets a mispredicted packet does it simply throw away any future packets it's buffered? And then wait for the client to readjust?
What would be the wisest thing to do here?
I have a verified input sort of system implemented right now, but it results in choppy movement on the server.
EDIT: I got rid of the "verified" system and now it's working pretty good. However I do have a question, collision with other objects usually causes prediction error and the client has to be corrected... causing noticeable view lag.
What's a good way to minimize this? I know a big flaw in CLSP is the fact that collision often causes prediction mishaps. Would the wisest solution be simply to sync a collision from the serverside with the client? Or perhaps the other way around? Or both?
-
EDIT: I got rid of the "verified" system and now it's working pretty good. However I do have a question, collision with other objects usually causes prediction error and the client has to be corrected... causing noticeable view lag.
What's a good way to minimize this? I know a big flaw in CLSP is the fact that collision often causes prediction mishaps. Would the wisest solution be simply to sync a collision from the serverside with the client? Or perhaps the other way around? Or both?
Collision does seem to be an issue. In general, probably because of floating point precision, if I walk against another player (both players are capsules) either the client slides away to one side, or the server does, and in this case the client eventually snaps to where they should be. I'm not sure anything can be done about this - other than limiting the amount of collisions that occur. Collision with the environment seems to be much less of a problem than other players, oddly enough.
With other players, you're bound to end up with prediction problems anyway, as you don't have the exact state of that player at the time the server is performing physics with the data you sent to it. So you'll see some jitter when bumping into other players, especially if they are moving around.
One idea is, instead of instantly snapping to the correct position, the client smoothly interpolates to it. For small corrections much less noticeable than snapping.
-
Collision does seem to be an issue. In general, probably because of floating point precision, if I walk against another player (both players are capsules) either the client slides away to one side, or the server does, and in this case the client eventually snaps to where they should be. I'm not sure anything can be done about this - other than limiting the amount of collisions that occur. Collision with the environment seems to be much less of a problem than other players, oddly enough.
With other players, you're bound to end up with prediction problems anyway, as you don't have the exact state of that player at the time the server is performing physics with the data you sent to it. So you'll see some jitter when bumping into other players, especially if they are moving around.
One idea is, instead of instantly snapping to the correct position, the client smoothly interpolates to it. For small corrections much less noticeable than snapping.
I think it purely comes down to that the server and client are running collisions at separate slightly separate times which causes these issues. Because the client can predict, it must also simulate physics. If it's simulating a collision that the server hasn't even caught up with yet (or vice versa perhaps) then issues would begin to happen. Surprisingly though, I found it only happens if the collision causes some sort of movement of the other object (which is probably why colliding with static things in the world doesn't really affect it that much). But like you suggested, the best way to minimize this effect is purely to lerp the correction so it's not as jittery.
Now the next issue(s) I have to face is object manipulation.
The user will have the ability to pick up and rotate objects much like in garrysmod (all down with AddForce commands, etc). I'm wondering if I should do client-side prediction for this as well. Run the AddForce commands on both client and server, and make corrections where necessary. But what about if you were to pick up a cube and smash it into another? From my brief testing, the cube already begins motion clientside before the server can start sending out interpolated results, which results in an initially jerky response.
Another method would be to give the player authority over the object but then you open up possibilities of cheating and other issues like what happens if you collide with someone else's cube that they currently have authority over.
-
In GMod, I don't believe they use any form of prediction when dragging physics objects. I think the default behavior in Source is that physical objects like ragdolls and props are always controlled by the server, and not predicted by clients.
-
Mmm, so the client just sends off inputs in that regard and waits for the server to update it.
-
The last part is very important (resimulating input), because by the time the player receives the state correction, it will have been sent some time in the past and is therefore old data (so the client must extrapolate what the correct position will be)
I am taking a look at doing that for my game (a simple networked physics car game), but I don't see how you can resimulate the whole physics on the client in Unity, assuming you're using Unity (since this is a TNet forum). Can you explain in more details what/how you're doing that thanks?
It seems you could hack something together with manipulating Time.fixedDeltaTime but I'm not sure the physics engine will appreciate that.
-
I am taking a look at doing that for my game (a simple networked physics car game), but I don't see how you can resimulate the whole physics on the client in Unity, assuming you're using Unity (since this is a TNet forum). Can you explain in more details what/how you're doing that thanks?
It seems you could hack something together with manipulating Time.fixedDeltaTime but I'm not sure the physics engine will appreciate that.
That is a problem, as you cannot really step the physics in Unity on a per-object basis. Unfortunately, I'm not really sure you CAN do this for vehicles and such just yet - unless you roll your own physics or grab a third party physics engine such as Jiggle or BulletX
In my case I was using CharacterController, which made it fairly easy to just call my own simulation code on-demand.
-
Yeah that's what I read elsewhere, maybe you had found another solution. Thanks
-
Yeah, unfortunately Unity doesn't support rewinding physics. However you could try and hack your own physics step system together and extrapolate where the car should go by playing with cubic splines... however this is far from accurate and will possibly be a bit unreliable. Then you have the whole other issue with detecting collisions, which you can't do on the fly except with a character controller.
-
I will make a quick proof of concept using Bullet, I made a quick wrapper so I can test stuff with networked physics. http://www.youtube.com/watch?v=rABptJuprmw (http://www.youtube.com/watch?v=rABptJuprmw)
I'll you know how that goes.
Another interesting article, not mentioned here (as far as I can tell :) ) is http://gafferongames.com/game-physics/networked-physics/ (http://gafferongames.com/game-physics/networked-physics/), he mentions the algorithms from this topic and also gives some very useful C++ code implementing the techniques discussed.
-
Good job on the Bullet Wrapper by the way. I was going to go down that route, but decided against it (for now).
I've seen that article too, great source of information. Especially in the comments section below the article itself, they begin to talk about alternative methods to Client-Side Prediction, since it's sort of an old system, and rewinding and replaying can be rather expensive to do for complex simulations.
-
If my understanding is correct, the Source engine doesn't do any sort of physics replay on the client. Instead, the clients are treated as though they exist slightly in the past (10 ms or so). This means that instead of having the extrapolate the position of where the object will be, the client can simply keep a buffer of positions where the object was (marked with timestamps) and interpolate where the object was 10 ms ago. Of course, having the clients "living in the past" no doubt creates its own set of issues, but it does seem to avoid the need to do physics extrapolation.
-
If my understanding is correct, the Source engine doesn't do any sort of physics replay on the client. Instead, the clients are treated as though they exist slightly in the past (10 ms or so). This means that instead of having the extrapolate the position of where the object will be, the client can simply keep a buffer of positions where the object was (marked with timestamps) and interpolate where the object was 10 ms ago. Of course, having the clients "living in the past" no doubt creates its own set of issues, but it does seem to avoid the need to do physics extrapolation.
It does no replay on objects that aren't completely under the player's control, but any object the player owns and controls (their player object, mainly, perhaps a vehicle they are riding, etc) does use physics replay. That "living in the past" bit is only for interpolating game state received from the server for objects the client doesn't directly control.
-
The server allows the client to handle collisions the player makes as part of its own prediction, if not then the client wouldn't predict properly in the first place, as the server would handle the collision but the client would keep moving. So in a way the client does *simulate* its own, yet very limited, version of physics. This simulated state of physics can be overwritten at any time of course by the server.
-
I wanted to get my vehicle physics working but the Bullet API for vehicles is a complete mess and quite hard to ".NETize". I don't want to spend too much time on making that work. I think I'll try using another more simple physics engine, I just want nice networked vehicles :)
I released what I got so far on the Bullet plugin. Here is the code + Unity3D project : https://github.com/speps/UnityBullet
The project is only cubes because that's the only shape I did so far, if anyone wants to contribute I'll gladly accept pull requests.
On another note, here is an interesting document I found recently while reading a DOOM3 code review : http://fabiensanglard.net/doom3_documentation/The-DOOM-III-Network-Architecture.pdf
It seems like they interpolate/predict everything :
The system presented here uses prediction on all entities in the PVS of the client to both improve
the responsiveness and to present a smoothly changing environment. This includes prediction of
the player movement because the avatar controlled by the player is also an entity in the PVS of
the client.
It's quite well written and easy to understand, and the source code is under GPL : https://github.com/TTimo/doom3.gpl (check the network tab for more up to date forks)
-
Due to the increasingly high amount of issues brought about by Unity's lack of efficient networking for physics heavy games, I've unfortunately been forced to swap engines from Unity to UDK. The built-in networking in UDK is phenomenal, and the physics system provides an efficient and effective way of doing smooth interpolation and prediction of objects and players as well allowing for complex simulations that are interactable for the client. Luckily I was still in the prototyping stage for my game so it's not a great big deal doing this transition. It is unfortunate that it came to this, but I knew I would be spending at least well over a year to bring the networking up to a stage I was happy with, and that is time I just do not have.
Unity is a great engine, there is no doubt about that, and TNet is a fantastic solution for the sub-par inbuilt networking for Unity, I would certainly recommend it to anyone. I'd also like to thank Aren for all his help, as well as every other community member on this board. Feel free to use this thread as a hub of knowledge for complex networking :P
Because I will be no longer needing my scripts anymore, you can all have them. Feel free to upgrade them where you see fit, they're all heavily commented and work generally well.
NetworkTime.cs is my own networking time solution. It accurately syncs time across all connected clients, and automatically checks for clock drift at a set interval and updates where necessary. The logic of it is based on this article: http://www.mine-control.com/zack/timesync/timesync.html (http://www.mine-control.com/zack/timesync/timesync.html)
SyncNetworkObject.cs is the script that you attach to any rigidbody you want to be networked. It will automatically update the position and rotation and physics data of your rigidbody or object over to all clients. It's primary function is to buffer changes over the ping time so it can smoothly interpolate effectively hiding lag.
Global.cs is just a global singleton class where I hold some variables that both scripts reference, not really important.
Good luck guys, if you have any questions feel free to ask!
-
I've always said that Unity was great for prototyping but there are huge areas of the engine which lack professional finish. Networking is one, physics is another. Try to combine the two in a game and good luck ! It's also why the Asset Store is thriving...
Thanks for the scripts, they might come in handy :)
-
Hello.
I am working on an arcade soccer game in Unity that Uses TNet for networking and so far so good. The only issue I have right now is how to get a ball that is moved by physics move the same way on both clients. All other elements move flawlessly but the ball that is ruled by physics does not work like I wanted.
Is there a simple solution for this issue without coding physics on the server side)?
-
The simple solution is to let physics do its thing. Send sync notifications from the object's owner when there is a collision, or periodically (1-4 times per second at most), and let each client do their own physics logic. That's what I did in the Multi-Purpose Game Starter Kit as well.
-
The simple solution is to let physics do its thing. Send sync notifications from the object's owner when there is a collision, or periodically (1-4 times per second at most), and let each client do their own physics logic. That's what I did in the Multi-Purpose Game Starter Kit as well.
So your recommendation for use of physics with TNet is to let each client run the full physics simulation and correct inaccuracies periodically (where the host/server is authoritative)? Given the state of Unity's physics, is there a smooth way to correct positions of objects influenced by physics? I have gotten the impression that it can become jittery pretty quickly? Thanks in advance!
-
Use lerps and slerps when there are discrepancies between host and client else. Plus make sure to update the position every so often. I use tno.send 1 time a second and tno.sendquickly 4 times a second to sync the position. My team has people all over the world and any lag seen is very negligible. one thing to note with the sendquickly is to time stamp the position so just in case a earlier position come in after a newer one you don't bounce backwards.
-
So your recommendation for use of physics with TNet is to let each client run the full physics simulation and correct inaccuracies periodically (where the host/server is authoritative)? Given the state of Unity's physics, is there a smooth way to correct positions of objects influenced by physics? I have gotten the impression that it can become jittery pretty quickly? Thanks in advance!
The key is to separate physics from rendering by putting the Renderer on a separate object set to follow its parent. I explained this concept quite a few times and used it on many occasions, including in the 2nd example that comes with TNet.
This way there will be jitter, but you won't notice it because it will be smoothly interpolated away.