Level Up! your manual tests: Client vs. Server, part 1
Back in the day, games didn't connect to the internet and if you wanted to play multiplayer, you shared your sofa with a buddy and plugged in a second controller; ah - what simple times! Nowadays, nearly all games require a connection to an online service on startup, even if you want to play singleplayer. Add in online multiplayer, matchmaking and integration with third parties like Facebook, Twitch or Twitter and suddenly you've got connections to online services all over the place.
Unlike our rather hilarious video above where you'd test what is visually in front of you on the screen, modern games testing requires us to understand the back-end services that hold our games up, how they're built and how they're all linked together. Manual testers that are technically competent enough to understand how to test back-end services are in short supply, so if you can get it right then it's a massive plus your skillset. With that in mind, lets get cracking!
What is a game client? The game 'client' is the software that you think of as 'the game'. You download it to your PC, console or device and it runs locally on that device. It's usually developed in a language like C++ and every player runs an instance of it. This is what most test teams will get delivered as daily builds.
What is an online service? In the context of a game. It's a separate program (piece of software) from the game that runs on a server somewhere, not of your local device. It's always on and can always be accessed by the game through a secure web address. It's kinda like a website in this respect. It's written in a different programming language than the game itself, something like PHP and uses HTTP or HTTPS communication protocols. It runs a one-to-many relationship with the game clients running on players' devices; this means that many instances of the game running on many devices all connect to the same instance of the online service (like many computers all connect to the same website, right!?). Remember that online service code doesn't need to display anything to a screen, it outputs logging data and communicates with the game clients and that's all, you'll never 'see' the online service during testing. Online services are required to connect to things like user databases to process player logins to an online system; they also process requests from game clients for things like matchmaking and in-game store content (more on this later).
What is a game server? Yet another piece of software. Games that have online multiplayer modes will run an instance of the game server program for the duration of each online session and then it close down afterwards. These 'game servers', somewhat confusingly, also run on a physical server somewhere. The number of instances of this program running at the same time depends on how many online sessions are running at that time. This program is solely responsible for tracking things like the position of players and other objects within the world and broadcasting the state of the game to each of the players in the session which it does many times each second. Game servers use a different type of communication to persistent online services, instead of HTTP, they'll use something like UDP (learn more about this online) because of the type of data that is being sent and received.
So, how do these things affect our day-to-day work as testers? The first hurdle is understanding that bugs and fixes can be made to either the client, online services or game server. Testers have gotten used to bug fixes and feature changes arriving in new client builds overnight, but changes can be deployed to an online service at any time and sometimes without notice.
I've seen testers come across this scenario and become entirely confused: "I've been testing today's build and trying to reproduce a bug with matchmaking, but about half an hour ago the behaviour changed and I can't reproduce the bug I'd seen earlier"
The tester hasn't been told that new, updated services code has been deployed and caused the behaviour of the game to change without a new client build. It's super important that testers understand this fact, that changes can be made without the need for a new client build and also that they know what parts of the game are powered by connections to online services. Lets talk about some examples now so that you know what to look for. (Note: In nearly all of the examples below, you can usually expect to see some kind of 'loading' animation within the UI while the game waits to get data back from the services too, which is a bit of a clue).
Multiplayer setup - Any menus that appear before starting an online game session will be driven by online services. Matchmaking results, player lobbies and pre-game chat will all be populated by data that is supplied by online services.
In-game stores - In-game stores are nearly always populated by online services. The game will usually be setup in such a way that it's capable of displaying all sorts of different purchasable content on each store page and then when you navigate to the store at any time, the game will ask the online service what content to display. This is usually for security reasons but also to allow the shop to be updated at any time, without releasing an update to the game.
On Launch - Nearly all games make calls to online services on launch. The title will usually have it's own user database that you're logging in to, particularly for online play. Your profile data is probably stored on a server somewhere for security and it grabs that data on game launch - it's this kind of process that allows you to run the game on a new device and still maintain your profile progress. Games will also make other checks with online services during login; checking for updates, sending telemetry data or contacting third party services like Xbox LIVE or the Playstation Network. Speaking of which...
Third-Party Services - Games frequently have connections to the platform that they run on; to perform operations like logging in, triggering achievements/trophies, submitting leaderboard scores and grabbing all sorts of other data from the player profile. As an example, perhaps the game uses the player avatar in some way, it needs to 'ask' the platform holder service for that information before it can display it. This goes for Xbox Live, PlayStation network, Steam, iOS GameCenter, Android GooglePlay or any of the online stores that Nintendo operate.
Telemetry + Logging - Data is continuously sent and logged for pretty much every action that you can take in the game. This data is submitted silently and will be batched up locally if it can't send the data off straight away. You won't be able to 'see' this happening without some tooling, but it's definitely something that can and should be tested.
Notifications, Gifts and Message Centers - Some games have a message center or notification system which presents game news, update logs or provides gifts during anniversaries and other events. All of these UI areas will be driven and populated by online services - expect to see brief loading animations while the data is fetched from online.
User Generated Content - Weapon skins, car vinyls, battle replays, card decks; all this kind of stuff is stored in a database and accessed through online service calls. UI pages within the game that allow you to browse this type of content will be populated from online services in the same way.
There are other areas, but these should definitely get your mind working on the right path. Knowing what parts of the game are populated by online services helps us understand whether bugs are likely to be service bugs or client bugs, they also give us clues when we're performing tests.
For example, if I run a low bandwidth test and navigate to a UI page that has a list populated from an online service, then I'm going to see a longer loading time before the information pops into the page; running the same test on any other UI page wouldn't show the same result.
Now that we are aware of the online services behind the games that are under test, how do we directly test those services if we can't directly 'see' them? This is a very good question that requires a much bigger conversation, because it's not straight forward! We can run generic bandwidth tests using tooling or mobile networks, we can also test the end-result in game.
On top of this, there are specific tools that allow us to intercept and spy on the messages being sent between the client game and the various services that it connects to. Two tools that I've personally used are Charles debugging proxy and Fiddler web debugging proxy which provide a 'window' into communications with the back-end. If you can become component with these tools, then you'll be able to take that test expertise with you to every new project.
In part 2, I'll go into more detail about these tools and the specific types of testing that we might do against online services. We'll also talk about the nature of the communications between the client game and the online services, in that they operate through a series of 'requests' and 'responses'. The more detail we know about the nuts and bolts of how this works the more we can apply smart test techniques to exercise them.
I hope this helps demystify some things! Happy testing. Now go check out that old games testing Ad!