When To Start Server?

[IOwaska]

I have been thinking a ways down the road trying to foresee any issues I may encounter. I am not sure if, or how much Age of Empires II HD you have played but I had planned to set up my navigation through menus and such as that game does. IE Main Menu-> Single Player, Multiplayer. Additionally I had planned to release the game through Steam, taking advantage of the friends list, achievements, etc. I planned to use an Amazon server for actual server storage and just using Steam for its gateway to players. This mostly being because Steam's matchmaking abilities and such are completely independent of TNet and I wanted to avoid that. Is this wise? Even possible to right-click a friend and Join-Game without Steam's servers?

To get to the root of my post. I notice when I start Age of Empires, Steam updates to show I am playing the game. I understand the game has to notify Steam of this, correct? If so, how can you notify Steam unless you actually connect to the external lobby server? From my interpretation of how Age of Empires does it, is that the lobby server isn't connected to/shown until the player would click Multiplayer, and if they clicked Single Player, they would use a virtual server - to use TNet lingo. Is my interpretation of how they are likely doing it wrong? Are they really connecting to a lobby server at game start and just not showing it until multiplayer is clicked? But if so, how would you handle single player? If the user is already connected to a lobby server, you can't start a virtual server, right?  Furthermore, how would one handle the case of no connection to a connection exists during a game, would they have to cycle lobby servers from virtual to tcp when they go back to the main menu?

I perfectly understood the examples in other posts referring to a WoW like flow (Login/Regions(game server)/Ironforge(channel), but without a log in, and using Steam Credentials as player ID/stats, wouldn't they always need to be in a lobby server? At start up, would you check if a lobby server existed externally and connect to it, and if not, default to a virtual server?

I am sure this doesn't sound 100% clear so let me know if I can clarify any more.

It's a small differentiation from your example so I am just a little unsure and wanted to make sure I had the right frame of mind before continuing. WoW is just a bit different in that you are guaranteed an internet connection, or the game would just be a login screen, so just not sure about the offline component.

Thank you for your time and I appreciate any responses =)

[cmifwdll]

I think you're mostly asking about Steam integration right? Once you register an account for Steamworks SDK you'll have access to its documentation. Unity projects typically use Steamworks.NET, which is a free .NET wrapper for Steamworks.

Steamworks works mostly through callbacks. When Steamworks is initialized using your AppID, your Steam client will show as in-game.
ArenMook posted a wrapper for most of the good stuff (a wrapper for the wrapper, making the wrapper a wrapped wrapper):
http://www.tasharen.com/forum/index.php?topic=13569.msg60971#msg60971
At the bottom is how right-click and join a friend works.

To get to the root of my post. I notice when I start Age of Empires, Steam updates to show I am playing the game. I understand the game has to notify Steam of this, correct?

The game notifies the Steam *client* (the Steam.exe application running on your PC), not Steam's servers.

Also of note: Steamworks is probably best integrated at the end of development, you don't need to worry about it right now.

[ArenMook]

Steam's API is pretty awesome and simple enough to use. With TNet you won't be using the networking part of Steam's API though -- that API is for smaller scale things, peer to peer networking. TNet can scale to hundreds of concurrent players without any trouble, as I've seen first hand with Windward. You'll use the Steam's API to advertise the status of your game, such as the snippet I posted at the bottom of the post cmifwdll linked -- which will make it possible for friends to right click the player's name in their Steam friends list and choose to join their game. In short, use Steam for everything -- IDs, saving player profile data, friends list, etc -- except networking.

[IOwaska]

Awesome. Thank you both for the help. The Steam-side is mostly as I had thought -- essentially adding interface functions to set those types of data and use Steam.Net to fill them out. @ArenMook, how do you go about using an external server without login credentials? Such that, your package sets up a server on the host's machine and others join to it. To use an external server, would the game auto connect to the server (how to have dynamic IP if server changes without changing build?) and when a player creates a game, it'd be added to the external server's lobby? Like this:

External Server (Amazon)
(Lobby Server)
     |_server started -- lobby server created
     |_(user starts game, game auto-connects to server; loading all player info via steam)
     |__ (user creates game -- becomes host) -- Lobby Server (1 game server active)
     |____Lobby server displays game to join?
     |_(user2 starts game, game auto-connects to server; loading all player info via steam)
     |__(user 2 can see user1's game)

OR

Is it 'better' (better obviously being subjective) to have the user be the lobby and game server (setup via TCP - user now a host) and other players join it? I am just not sure how other players would 'know' about the new game without direct IP.

(User 1 starts game -- loads info from Steam)
  |_____Single Player -> Virtual Server locally
  |_____Multiplayer ->Create Game: Starts TCP Server on user's machine -- user 1 becomes host
      (Now here, if the host created their own server, they are the lobby and game server.

(User 2 starts game -- loads info from Steam)
  |____Multiplayer -> Find Game
       (How would user 2 know about User 1's game?)

I have the UDP and Virtual Servers setup on my local machine but cannot connect any other machines without a direct IP. I am just not sure when to start servers and which servers to start (lobby v game). Again, I understand the setup with your example of World of Warcraft, but for other games like Age of Empires I am not sure how the setup is different, exactly. Also, I am not sure if dedicated server is needed for only 'real-time' style games, but if it matters in terms of these examples, the game is turn based-civ-esque so not a whole lot of RTS/FPS like calculations going on.

Perhaps I am missing a connection somewhere? Thank you for the direction and your time.

[cmifwdll]

In the second scenario, you're right about user 2 having no way of knowing about user 1's game.
The first scenario you presented is the proper way: having the lobby server on a public server.

When initializing a game server (TNServerInstance.Start(...)) you can specify the remote lobby server to connect to.
Your flow is perfectly accurate in scenario 1, no connections missed.
You can use TNet's gameserver to store player data. You could also store it locally and sync it with whatever server you connect to too if you want. Alternatives to using Steam for storage.

One issue you may run into is NAT. When user 2 tries to connect to user 1 (let's pretend they're behind NAT), user 1's router doesn't know which local device user 2 is trying to reach so it drops the connection attempt. User 1 would have to first try connecting to user 2 so the router creates an entry in its lookup table. I wonder if we could convince ArenMook to add punchthrough facilitating logic to the lobby server. It's a bit tricky with TCP. I wonder if NAT (or, more specifically, the connection issues brought by NAT) will exist when IPv6 is fully rolled out. I'm really behind on my networking knowledge and I'm slowly forgetting everything I learned.

Oh, I just re-read your last paragraph, calculations are done on each game client. TNet has no knowledge of your game or what's happening in it: the server simply passes data between clients. If you need an authoritative server, you'll have to rewrite large portions of TNet. There's plenty of ways to control authority among your clients though.
Also, you shouldn't need to type an IP to connect: make sure the gameserver is set to register with the lobby server, and that the client is connecting to that lobby server and parsing the ServerList.

[IOwaska]

Yeah I am a little foggy in the networking stuff as well. Did some in school on a base level and trying to wrap my head around the various constructs and models people have used. Authoritative is held in the highest regard as it heavily stifles cheating, no host migrations, etc. However, having one client host and run the logic is more 'down to earth' and well used in even games like Call of Duty and Age of Empires as 'host migrations' -- at least that is my interpretation -- compared to say, CounterStrike dedicated servers. Authoritative would be ideal, but I don't see too much wrong with going the client-host route.

All of my local connections work great. It's when I were to go on another pc via LAN, it won't pick up the server and I have to connect via IP on TCP. Perhaps my main problem is that I am using my dev machine for everything. Perhaps I should do what others have done and run my server on a remote raspberry pi or something, to help me isolate game and lobby server logics -- as I think I am conflicting them too much locally. However, if I type in the IP, it works like a charm. So the really connection I am missing is the client connecting to the correct lobby server. To sort this out, I think I'm better off splitting them up from the start.

As for NAT, I guess that is a battle I will have to fight when I get there. From what I had read in a previous forum, I don't recall exactly which ones, but Aren was explaining that it should get around NAT for PC, np. It's consoles and mobiles which can get a little hairy, and I am shooting for PC -- at least at first.

Your response was very insightful and glad to hear I seem to have the right approach and thought-framework for going forward. It's all a learning experience so thanks again, cmifwdll, et all.

[IOwaska]

When initializing a game server (TNServerInstance.Start(...)) you can specify the remote lobby server to connect to..

What is the best way to store the remote lobby ip to connect to? If a user has a build out, and I update the server for some reason, I would need to push a new build or have a primary database somewhere that would store the lobby server location....but that would (should) be the same server.

So is it good practice to essentially hard-code that IP? Just forcing all users to update/patch to get the updated ip?

[cmifwdll]

Ah, if you're connecting to a LAN server you need to use its LAN address. I believe the ServerList provides you with both LAN and WAN? Can't remember if that's a change I made or if it's in base TNet. Using its (your) WAN address results in router hairpinning which your router might not like.

I'm not sure TNet addresses NAT. UPnP is used to open the port, but I don't think that creates a NAT mapping. Maybe I'm wrong. Regardless, though, UPnP seems to be really, really hit or miss (mostly miss) depending on each user's configuration. It may prove a difficult battle to win. I'm interested in how others tackle the issue.

You could register a domain name and use that. You can freely modify the IP address your domain name points to. Though, I believe changing its record may take a day or two to propagate? Just plan ahead if you do need to change IPs: keep the old IP active for a few days. If a domain name isn't an option then hardcoding the IP:Port or storing it in a config file would be fine, it's done quite often.

[IOwaska]

I'm not sure TNet addresses NAT. UPnP is used to open the port, but I don't think that creates a NAT mapping. Maybe I'm wrong. Regardless, though, UPnP seems to be really, really hit or miss (mostly miss) depending on each user's configuration. It may prove a difficult battle to win. I'm interested in how others tackle the issue.

Ah, I see. IDK, it seems many have used TNet successfully without issues, except the Failport from Apple -- go figure. And that is what I was thinking about, UPnP. Perhaps I took the forums' word a bit too highly? I assume we will see.

You could register a domain name and use that.

Of course! Duh! Total brain fart, that'd work perfectly.

[cmifwdll]

Correction: TNet's UPnP does create a port mapping, thereby allowing NAT traversal.

[IOwaska]

Sweet, thanks for the clarification. I thought I had read something similar, I just couldn't remember where. The caveat is consoles and AirPort, correct? Or has that been solved also? Either way, thanks for the update.

[ArenMook]

UPnP is for IPv4, yes. From everything I've seen it's not necessary for IPv6 at all. TNet supports both. As long as you keep your lobby on a public server and have your players register their own servers with it, all will be able to see the full list of active servers. Connections via IPv6 shouldn't worry about NAT, but IPv4 you will run into issues in some cases, that much is true. In Windward's case I have an official public game server hosted on a public-accessible VPS, so that if players can't get their own hosted, they can still play together with their friends.