Print Page - Nov 23, 2017 -- Integrating TNet with Steam Networking

Games => Dev Blog => Topic started by: ArenMook on November 23, 2017, 03:07:24 AM

Title: Nov 23, 2017 -- Integrating TNet with Steam Networking
Post by: ArenMook on November 23, 2017, 03:07:24 AM

Back in Windward days, I would sometimes get comments from players saying they can't join their friends' games, no matter what they tried. Let's face it, while us devs find it a trivial task to open up a port on the router, and TNet does indeed us UPnP to do this automatically, the players are less savvy and can sometimes be behind such firewalls that even UPnP can't breach. Fortunately, Steam has ways around it, and they've indeed taken care of all of this with their networking API. It uses UDP to simulate TCP-like functionality, but since it's UDP, NAT punchthrough is easy to do. Better still, if NAT punchthrough fails, Steam allows using its servers as relays to still make it possible for two players to play together.

Of course there are limitations: first, both players must be using Steam. But hey, let's face it -- Steam is the best platform out there for gamers. Is there a reason NOT to use it? Second, the packet size is limited to just over 1 MB -- but quite frankly if your game is sending out packets greater than 1 MB in size, you're probably doing something wrong. And last but not least, the API itself is a little... weird. To explain just what I mean by that, let's look at the steps required with the latest version of TNet (from the Pro repository as of this writing).

First, you will want to grab Steamworks.NET here: https://github.com/rlabrecque/Steamworks.NET

Next, let's start by making a new controller / wrapper class. I called mine "Steam" for simplicity.

using UnityEngine;
using Steamworks;
using TNet;
 
public partial class Steam
{
    CSteamID userID;
 
    void Awake ()
    {
        SteamAPI.Init();
        SteamUserStats.RequestCurrentStats();
        userID = SteamUser.GetSteamID();
        DontDestroyOnLoad(gameObject);
    }
 
    void OnDestroy () { SteamAPI.Shutdown(); }
 
    void Update () { SteamAPI.RunCallbacks(); }
}

With the script attached to a game object in your first scene, Steamworks API will be initialized, and it will be shut down when your application does. The Update() function simply lets Steam do its thing.

Next, we need to create a special connection wrapper class for Steam to use with TNet. By default TNet will use its sockets for communication, but since we'll be using Steam here, we should bypass that. Fortunately the latest Pro version of TNet has a way to specify an IConnection object for every single TcpProtocol which essentially inserts its operations in between of TNet's sockets, making all of this possible. I chose to make this class inside the Steam class, but it's up to you where you place it.

public partial class Steam
{
        [System.NonSerialized] static System.Collections.Generic.Dictionary<CSteamID, TcpProtocol> mOpen = new System.Collections.Generic.Dictionary<CSteamID, TcpProtocol>();
        [System.NonSerialized] static System.Collections.Generic.HashSet<CSteamID> mClosed = new System.Collections.Generic.HashSet<CSteamID>();
 
        class P2PConnection : IConnection
        {
                public CSteamID id;
                public bool connecting = false;
                public bool disconnected = false;
 
                public bool isConnected { get { return !disconnected; } }
 
                public bool SendPacket (Buffer buffer) { return SteamNetworking.SendP2PPacket(id, buffer.buffer, (uint)buffer.size, EP2PSend.k_EP2PSendReliable); }
 
                public void ReceivePacket (out Buffer buffer) { buffer = null; }
 
                public void OnDisconnect ()
                {
                        if (!disconnected)
                        {
                                disconnected = true;
 
                                var buffer = Buffer.Create();
                                buffer.BeginPacket(Packet.Disconnect);
                                buffer.EndPacket();
                                SteamNetworking.SendP2PPacket(id, buffer.buffer, (uint)buffer.size, EP2PSend.k_EP2PSendReliable);
                                buffer.Recycle();
 
                                lock (mOpen)
                                {
                                        mOpen.Remove(id);
                                        if (!mClosed.Contains(id)) mClosed.Add(id);
                                }
 
                                if (TNManager.custom == this) TNManager.custom = null;
                        }
                }
        }
}

So what does the P2PConnection class do? Not much. It keeps the Steam ID identifier since that's the "address" for each "connection". I use both terms in quotations because instead of addresses, Steam's packets are sent directly to players, and players are identified by their Steam ID. Likewise, there are no "connections" established with Steam's API. Remember how I said that the API itself is a bit weird? Well, this right here is what I meant. Instead of the expected workflow where a connection must first be established and acknowledged before packets start flowing, Steam's approach is different. You simply start sending packets to your friend like you're the best buddies in the world. Your friend's client gets the packets along with a special notification of whether to accept the incoming packets or not. If the client chooses to accept the packets, they can be received immediately. There is no "decline" option. In fact, no notification is sent back to the first player at all, and trying to do so will actually auto-accept the packets! So the options are: accept packets, or ignore them, leaving the other player wondering.

Anyway, so back to P2PConnection. ReceivePacket() can't be handled here, because packets don't arrive via sockets. Instead they arrive in one place, and must then be queued in the right place -- which we'll get to in a bit. For now, the only two useful functions in that class are SendPacket -- which simply calls the appropriate SteamNetworking API function, and the OnDisconnect notification. This one needs some explanation.

Since there is no concept of "connections" with Steam's API, we have to account for this ourselves. So to keep it short, we're simply sending a Disconnect packet to the other player when we're done. We're also keeping a list of known open and closed "connections" (and I'm going to tire of using quotation marks by the end of this post...). So to sum it up, when TNet says that the connection is closed, we still send out a Disconnect packet to the other player, ensuring that they know to stop sending us packets.

Moving on -- we have the custom connection class for TNet. We should now use it. Let's start by writing the Connect function:

        void ConnectP2P (CSteamID id)
        {
                if (TNManager.custom == null && !TNManager.isConnected && !TNManager.isTryingToConnect && mInst != null)
                {
                        CancelInvoke("CancelConnect");
 
                        var p2p = new P2PConnection();
                        p2p.id = id;
                        p2p.connecting = true;
                        TNManager.custom = p2p;
                        TNManager.client.stage = TcpProtocol.Stage.Verifying;
 
                        var buffer = Buffer.Create();
                        var writer = buffer.BeginPacket(Packet.RequestID);
                        writer.Write(Player.version);
                        writer.Write(TNManager.playerName);
                        writer.Write(TNManager.playerData);
                        var size = buffer.EndPacket();
                        SteamNetworking.SendP2PPacket(id, buffer.buffer, (uint)size, EP2PSend.k_EP2PSendReliable);
                        buffer.Recycle();
 
                        Invoke("CancelConnect", 8f);
                }
#if UNITY_EDITOR
                else Debug.Log("Already connecting, ignoring");
#endif
        }

Inside the ConnectP2P function we create our custom P2PConnection object and assign it as TNManager.custom -- meaning it will be used by TNManager's TcpProtocol for all communication instead of sockets. We also immediately send out a packet requesting the ID. TNet does this whenever a TCP connection is established, so we should follow the same path. This packet will be received by the other player (the one hosting the game server), and a response will be sent back, actually activating the connection.

One other thing the function does is it calls the "CancelConnect" function via a delayed invoke, which will simply act as a time-out:

        void CancelConnect ()
        {
                var p2p = TNManager.custom as P2PConnection;
 
                if (p2p != null && p2p.connecting)
                {
                        TNManager.client.stage = TcpProtocol.Stage.NotConnected;
                        TNManager.onConnect(false, "Unable to connect");
                        TNManager.custom = null;
                }
        }

It's also useful to have a String-accepting version of the Connect function, for convenience:

        static public bool Connect (string str)
        {
                ulong steamID;
 
                if (mInst != null && isActive && !str.Contains(".") && ulong.TryParse(str, out steamID))
                {
                        mInst.ConnectP2P(new Steamworks.CSteamID(steamID));
                        return true;
                }
                return false;
        }

So -- we now have a way to start the connection with a remote player. We now need to handle this operation on the other side. To do that, we need to subscribe to a few events. First is the P2PSessionRequest_t callback -- this is the notification that effectively asks you if you want to receive packets from the other player. Ignoring it is one option, but simply calling AcceptP2PSessionWithUser is more useful. Just in case though, we only do it if there is a game server running. We also need to handle the error notification:

        // Callbacks are added to a list so they don't get discarded by GC
        List<object> mCallbacks = new List<object>();
 
        void Start ()
        {
                // P2P connection request
                mCallbacks.Add(Callback<P2PSessionRequest_t>.Create(delegate (P2PSessionRequest_t val)
                {
                        if (TNServerInstance.isListening) SteamNetworking.AcceptP2PSessionWithUser(val.m_steamIDRemote);
                }));
 
                // P2P connection error
                mCallbacks.Add(Callback<P2PSessionConnectFail_t>.Create(delegate (P2PSessionConnectFail_t val)
                {
                        Debug.LogError("P2P Error: " + val.m_steamIDRemote + " (" + val.m_eP2PSessionError + ")");
                        CancelInvoke("CancelConnect");
                        CancelConnect();
                }));
        }

With this done, the server-hosting client is now able to start accepting the packets. We now need to actually receive them. To do that, let's expand the Update() function:

        // Buffer used to receive data
        static byte[] mTemp;
 
        void Update ()
        {
                SteamAPI.RunCallbacks();
 
                uint size;
                if (!SteamNetworking.IsP2PPacketAvailable(out size)) return;
 
                CSteamID id;
 
                lock (mOpen)
                {
                        for (;;)
                        {
                                if (mTemp == null || mTemp.Length < size) mTemp = new byte[size < 4096 ? 4096 : size];
 
                                if (SteamNetworking.ReadP2PPacket(mTemp, size, out size, out id))
                                        AddPacketP2P(id, mTemp, size);
 
                                if (!SteamNetworking.IsP2PPacketAvailable(out size))
                                {
                                        UnityEngine.Profiling.Profiler.EndSample();
                                        return;
                                }
                        }
                }
        }

The code above simply checks -- is there a packet to process? If so, it enters the receiving loop where data is read into a temporary buffer, and then placed into individual buffers that TNet expects. Basically the stuff that TNet does under the hood when receiving packets. Since we're doing the receiving, we also need to do the splitting. Each packet is added to the appropriate queue by calling the AddPacketP2P function which we will write now:

        static void AddPacketP2P (CSteamID id, byte[] data, uint size)
        {
#if UNITY_EDITOR
                if (!Application.isPlaying) return;
#endif
                TcpProtocol tcp;
 
                if (mOpen.TryGetValue(id, out tcp))
                {
                        // Existing connection
                        var p2p = tcp.custom as P2PConnection;
                        if (p2p != null && p2p.connecting) p2p.connecting = false;
                }
                else if (TNServerInstance.isListening)
                {
                        // New connection
                        var p2p = new P2PConnection();
                        p2p.id = id;
 
                        lock (mOpen)
                        {
                                tcp = TNServerInstance.AddPlayer(p2p);
                                mOpen[id] = tcp;
                                mClosed.Remove(id);
                        }
                }
                else if (TNManager.custom != null)
                {
                        // New connection
                        var p2p = TNManager.custom as P2PConnection;
                        if (p2p == null) return;
 
                        p2p.id = id;
                        tcp = TNManager.client.protocol;
 
                        lock (mOpen)
                        {
                                mOpen[id] = tcp;
                                mClosed.Remove(id);
                        }
                }
                else return;
 
                tcp.OnReceive(data, 0, (int)size);
        }

The AddPacketP2P function checks if it's an existing connection first. If it is, the connection is marked as no longer trying to connect, and the packet is added to the TcpProtocol's receiving queue. If the connection is not yet open, we check to see if a game server is running. If it is, a new P2PConnection is created and a new player gets created on the server. This player won't have an IP address or an open TCP socket. Instead, it has the reference to the P2PConnection which it will use for communication.

Last but not least, if the game server is not running, the function checks to see if the TNManager has its own P2P reference set. We assigned it in ConnectP2P(), so this means that this check effectively makes sure that we are trying to connect. If this check passes, the packet is added to the TNManager client's incoming queue.

If all else fails, the packet is simply ignored.

So that's that! This is all you need to be able to effectively overwrite TNet's networking functionality with Steam Networking. Before you go though, you may want to make it possible for people to be able to right-click their friends in the Steam friends list and use the "Join game" option. To do this, you need to set the rich presence's "+connect" key:

        public void AllowFriendsToJoin (bool allow)
        {
                if (allow) SteamFriends.SetRichPresence("connect", "+connect " + userID);
                else SteamFriends.SetRichPresence("connect", "");
        }

Simply call Steam.AllowFriendsToJoin(true) when you want to make it possible for them to join. Personally, I placed it inside a function called from TNManager.onConnect, but it's up to you where you need it to be.

You will also need to subscribe to the Join Request like so:

                // Join a friend
                mCallbacks.Add(Callback<GameRichPresenceJoinRequested_t>.Create(delegate (GameRichPresenceJoinRequested_t val)
                {
                        var addr = val.m_rgchConnect;
                        addr = addr.Replace("+connect ", "");
                        if (!Connect(addr)) TNManager.Connect(addr);
                }));

When a player chooses the "Join Friend" option from within a game, GameRichPresenceJoinRequested_t will be triggered. When a player uses the same option while the game isn't launched, GameRichPresenceJoinRequested_t won't be sent. Instead a "+connect <string>" command-line option will be sent to the game's executable -- so you will want to handle that yourself.

Anyway, that's it! This is all you need to make your TNet-powered game be able to use Steam Networking. I hope this helps someone!

You can grab the full file here (http://www.tasharen.com/tnet/Steam.cs).

Title: Re: Nov 23, 2017 -- Integrating TNet with Steam Networking
Post by: jingato on March 20, 2021, 11:22:56 PM

Hey Aren, I am looking to use TNet in my next project, which will be in VR and on the oculus Quest 2. Oculus provides a some platform features such as rooms, matchmaking, and p2p connection. I'm wondering if this same interface would be easy to use to run all the data through the oculus protocol? If so, do you know how that would be implemented best? I am going through your Steam example and there a few things I don't quite understand. The Oculus API uses a standard connection / disconnection so it's a bit different than the steam implementation, though I think the data still all goes through a single common function (not entirely sure yet). It also seems like your class that implements the Connection interface represents a connection to a specific user, yet it seems like you assign it to a single static instance, so I am unsure exactly what it is supposed to represent. Is the userId of that supposed to be the local user or the user you are connecting to for example.

Also, does this interface also allow you to route UDP calls or only TCP?

Here's (https://developer.oculus.com/documentation/unity/ps-p2p/) the page to the Oculus p2p. If you could just let me know if this is doable with your system that would be great. If you could help me understand how it all works and how this would be implemented with the Connection interface, that would be even more amazing.

Thanks!

Tasharen Entertainment Forum

Games => Dev Blog => Topic started by: ArenMook on November 23, 2017, 03:07:24 AM