News
Reviews
   -Icewind Dale
   -TA: Kingdoms - The Iron Plague
   -Deus Ex
   -Terminus
   -Vampire
   -and TONS more...
Previews
   -Elite Force
   -Max Payne
   -Theme Park World
   -and TONS more...
Interviews
   -Space Empires IV
   -Ian Winter
   -Tomas Jachimczak
   -Kelvin Phillips Jolt
   -Sujoy Roy XSi
   -and TONS more...
Features
   -Beta Testing
   -The Best RPGs
   -Mousepadz
   -DotCom Security
   -Gamers Desktops
   -and TONS more...
Forums
Columns
  Submit News
Help Needed
The Admins
Disclaimer
Advertising

  Get Hosted!
Site Directory
Admin Sections
   -Online Manual

   -Account Manager
Hosting Terms


  GibWorld
GibSeek
AnimePlanet
Moddage


GibSeek Search:-


GibWorld Features - Beta Testing Tips
Beta Testing Tips
Author - Brad Burton

An article for the tester AND test coordinator
 

     That new game you want will be out next year. Too bad you’ll have to wait. You head over to their site to take a look at it and – what’s this?! beta signup? Well, it looks like that game you want next year is accepting applications for their closed beta test coming in a few weeks. Maybe you won’t have to wait after all. But most likely you will if you only want the beta to play the game.
     A beta test is usually the second test (after alpha) of a product and is usually the first test by its actual customers. Beta testing a product can become very tedious for the designer to do all by himself. It's also not recommended because it's likely that you'll overlook several things. That's where us beta testers come in! You, the designer, send your product to a select few of the general public and HOPE we get back to you. Though by doing this you'll always catch some of those "unreliable testers" that just USE your program for their own benefit, you'll hopefully, thanks to the "reliable testers," find out where the bugs in your program are and get valuable feedback. Beta testers should do certain things when they're testing a product. The designer (or lead tester) should also do some things to prepare for testing and communicate this to the tester; they need to keep the tester active.
     Filling out the beta form on the game’s site can be tedious, but it's extremely important. You need to know your computer settings and configuration. This alone can be difficult if you don’t know a lot about your computer. You may need to answer questions relating to the game’s genre and list previous games you’ve played. Most likely they’ll ask how much experience you have as a beta tester and how much time you’re willing to spend testing the game. These two are the most important. In order to be accepted for a closed beta, you need experience and a LOT of time. If what you fill out meets the requirements of the game and is impressive to the company, then they’ll choose your application over others; thus you’re accepted as a tester. You will be contacted and asked to sign an NDA (non-disclosure agreement) which asks you to keep everything about the game confidential until they say it's okay. Yep, this means you can't show off… You’ll have to wait to show your friends until the release of the game. Once everything is signed they’ll send more information on how to get the beta (whether it’s available via ftp, or by Fed Ex to your home). Many times they’ll send you a free copy of the game when it’s released, or other rewards such as a T-shirt.
     When you fill out the form put a LOT in the “comments” box. List previous games you've tested and what you plan to do as a tester. You must be willing to devote hours of your time to help with testing. When I was accepted to my first closed beta, it helped me get into other closed betas, which in turn, helped me into other closed betas, until I could recite a whole list of games I’ve tested for experience. 
     Beta testing is like proof-reading a paper. Once you finish typing a paper there're tons of spelling, grammar, sentence structure errors, etc (for me at least). For some reason when you proof-read it by yourself, you tend to pass up a lot. That's why I try to have a friend proof-read it also because they'll have a fresh look. The designer is probably so used to his (or her) creation by now that he's VERY likely to overlook MANY, MANY things that a person with a fresh look on the project won't. When a product has finally gone through its design and alpha stages it may function, but it's not going to be used by the public (much less purchased) unless it's absolutely flawless…well, for the most part at least. Sometimes it's impossible to get every little bug out of a program, but if one thing (even a little thing) doesn't seem right then the person that's using it will probably get the feeling that the entire product is flawed, especially if you haven't established yourself as a company. That's why it's important for you as a designer to get multiple testers on your project and for you as a tester to catch these bugs and report them.
     Testing isn't all about technical errors though. It's also about enhancing the game - making it look better, sound better, and be more fun. Many designers are so wrapped up in worrying about technical errors that they forget to listen to or don't take seriously the tester's suggestions. This feedback can be invaluable. Randy Trulson, President of Neuron Entertainment, Inc. agrees:

"I think a lot of game designers should also lend an ear to the tester's thoughts about the game itself such as gameplay, character movement, ease of control, and aspects like that as well as the functionality. Too many times I have seen testing done primarily for the purpose of finding functional or logical bugs in the game. When it comes right down to it, having a boring, uninteresting, or difficult to play game is a big bug in itself."

Some people think they can't get into a competitive beta unless they have the newest model computer, best graphics card, etc. This is not at all true. The company wants to make sure that their product works well and is compatible on EVERYONE'S computer. They want a DIVERSE group of testers with all kinds of different hardware, not just people that have the latest stuff. 
     Depending on the game and the number of applicants, a beta test can be HIGHLY selective, but your inclusion into these "HIGHLY selective" tests doesn't depend on your system specifications as much as it does on the kind of person you are. To get into some of these competitive betas you need to prove to the company that you're a dedicated tester; that you're willing devote lots of your time to TEST not just play games. The more enthusiastic you seem the more they're gonna want you. 
     The company is looking for a variety of testers, both amateur and hardcore. They are looking for someone that knows about games AND has no idea what they're doing. They wouldn't want you testing their RTS game if you don't know what an RTS game is, right? Not necessarily, you don't always need background on the genre to be an effective tester. Though I do believe the optimal tester knows what he's doing, Jay Adan, Marketing Director at Cyberlore Studios makes a good point:

"You can get some VERY good feedback from people that don't normally play the style of game that you have created…I believe that these days many developers hope that they'll get some of the mainstream interested in [the] game as well. If you ONLY get feedback from hardcore gamers how will you possibly know how intuitive it is to people that don't normally play your style of game? This kind of feedback is important." 

     But still…don't just blindly sign up for anything you see if you don't plan on actually testing and giving feedback. It's a waste of time for both you and the company and isn't a way to gain experience.
     Alpha testing is quite a bit different than beta. The general public doesn't usually participate in this testing phase. Though it's different with every company, generally this is done in-house with the programmer and a few of his friends (employees in some cases), however there will be some chances for the public to get in on an alpha test. Alpha testing usually comes in after the basic design of the program has been completed. You will look over the program and make suggestions or give ideas to the designer to help improve it. You'll also report and give ideas to help get rid of or work around any MAJOR problems. Don't report the little things. There's bound to be a number of bugs after a program is created and the designer most likely knows about many of them. This isn't the main concern in alpha testing.
     The developer usually has three main goals when going into a beta test phase: stress testing, hardware compatibility testing, and in-depth game feedback including bug reporting and aesthetic problems. The number of people that get in on a test depends on which goal the developer is focusing on. For example, stress testing, which usually comes last, requires a large number of people to be logged in at once so many people are needed for the test. When testing a multi-player game, stress tests are very common. A stress test is used to put a game “beyond its limits”. The company may have a set date and time of when to log on to the game so they can get as many people playing at once as possible. This is one of the reasons why so many companies do public beta testing. Since so many people are accepted for stress testing, many companies will choose the testers randomly rather than looking for "experienced" testers. Sometimes they just can't look over so many applications.
     When beginning a testing phase it works best for testers to have a set guideline, a summary, or checklist of some sort. If you're starting a testing phase for your product, try to implement this and let the tester know beforehand that they'll be expected to go through a specific series of tests. This way they're more likely to spend their time TESTING the game instead of just playing it. If you’re a tester and haven't been given any guidelines for testing, make up your own. It'll impress the designer if you write up a little report of the things you tested. It's also extremely important to COMMUNICATE with your testers. Even if there's not much to say just let them know that you haven't forgotten about them. Otherwise, most likely they'll forget about you. Sometimes those "unreliable testers" weren't really "unreliable" at all. It's just that they didn't think they were needed anymore. In many cases it's the designer's fault if a tester has become "inactive."
     When I got the StarSiege beta CD in the mail, I read the letter that came with it first. It looks like I will get a free copy of the game when it’s released! It told me where to report bugs, and where I could find a list of known bugs on the Internet. After looking at the list I will remember not to report these bugs when testing the game. For EverQuest, I won’t get a free release copy of the game for testing, but it’s still rewarding to know I’ve helped make a game better. For testing 7 Kingdoms II not only will I get a free final release copy for testing it, but if I do a good job I’ll also get my name in the credits under a “Special Thanks” section! I don’t always get a beta CD in the mail though. Sometimes it’s required that you download the game off their ftp server. A LOT of free memory is needed on your computer to do this, and it’s almost necessary to have a cable modem.
     The first part of beta testing can sometimes be very frustrating. It’s called the installation test. It’s quite simple; make sure the installation process works correctly. If it doesn’t, it’s extremely important to report it. 
     Your goal as a beta tester is to give feedback on changes you'd make to the game, and to report bugs. You must act like you're the developer of the game. What would you change to make it better? To “draw in” the player? How could you make this game fun not only for beginning gamers, but also for experienced gamers that have been playing the game for months? What can you do to add to the features of the game? Are the graphics good enough? Does the sound add to the game? Does it make the game better, or does it get annoying? Also try to find bugs in the game. Try to make it crash and make it do something wrong. Always have a notebook and pen handy because if the game freezes and you have to restart you'll need to write down any error messages received. You should also write down every step you took to encountering the bug so you don't forget. Then restart the computer, open the game, and try to retrace the bug. Look at your notebook and do the same things you did before to find the bug again. Hopefully the same thing happens again. Report the bug, restart the computer, and try to find another one. I always include in my bug report my system configuration, and any other programs I had running.
     In EverQuest there were quite a lot of options and buttons to press. So I had a little “strategy” to testing it. I would first press every button to make sure they worked properly (Guess what! Pressing one of the buttons caused the game to crash). Then I pressed each one while fighting a monster to make sure they worked in combat. Then I killed the monster using every technique possible. I let it kill me. Then I earn experience, gain levels, and then play using the more advanced options. I walk along walls to make sure my character doesn't get stuck, I test out my skills and spells trying them where I normally wouldn’t be able to use them (fire spell in water). These are known as function tests. I report all the bugs I find, and all my suggestions to make the game better. What do you know! In a few days they’ve come out with a new patch. They’ve fixed the bugs I reported. I make sure that the repaired bug or newly enhanced feature works correctly in the new patch. This can be called regression testing. Usually there’s not a problem (but you never know!).
     The whole point to beta testing a game is to find bugs and reproduce them. Trying to reproduce a bug is, by far, the most difficult part of testing, but it's also the most important. List on the bug reports the EXACT steps you took to encountering the bug. This is where testing can be tedious and often frustrating. Mark Kundinger, Associate Producer of Ultimation Inc. emphasizes the importance of reproducibility:

"When writing a bug report about a legitimate bug (crash bug or similar), it's best to give VERY explicit instructions to recreate the problem. Make it so simple that even a retarded monkey could find the bug. This way, it becomes a snap for a programmer to reproduce. Bugs that can be reproduced tend to get fixed much faster than bugs that cannot. If the game has something like a scenario editor, it may be useful to create a custom mission that makes it easy to reproduce the bug under "laboratory" conditions, and send that to the developer."

     When I was playing StarSiege I was in the middle of a battle with another mech when the game crashed. Well, I went back to try to reproduce it and found that this would be close to impossible. There were so many little things I did (and the littlest thing does matter) that no matter how hard I tried it probably wouldn't happen again. I keep playing until I encounter another bug. Then I make an attempt to reproduce it, just like the one before that. It’s very rewarding if I am able to reproduce it and may help out quite a lot if I report the bug properly.
     I've been through my share of beta tests, and it's a different experience each time. It ranges from very pleasant and rewarding to "They must have forgotten me!" It's a rewarding experience when the producers communicate with me, acknowledge any ideas and bugs I've reported or testing reports submitted, and notify me of when the product's been released. It's also really nice to see my name in the credits or get a free final copy of the product (hehe).
     I always like being one of the first few people to see a game before the release. If I’m testing a game I don’t have to worry about waiting for delays in the final product shipment, etc. As a beta news reporter I have insight to the game, will know the conditions of the beta and future betas, and with permission of the company may be able to write up a preview or post some screen shots. It’s a nice feeling if you receive the final release of the product in the mail when it’s all done. Then you can add the game to your beta-testing resume!


Article written by Brad Burton
Be sure to visit his website!


Feature Index

Henry VIII had six wives, but who had eight dogs?

So you've opted to be a beta tester and get games ahead of all your friends, but is it all that is cracked up to be? In this article we discuss the job itself, and the work it entails.

So you've opted to be a beta tester and get games ahead of all your friends, but is it all that is cracked up to be? In this article we discuss the job itself, and the work it entails.

GW Admins
BloAtEd -
[13/08/2000]
D4N73 - [10/08/2000]
Bobertchin - [03/08/2000]
Spooky - [28/07/2000]
Man1c-M0g - [21/07/2000]
eXecutioner - [16/07/2000]

GW Guests
diesel -
[13/08/2000]
Woolley - [13/08/2000]
Crespo - [07/08/2000]
flopz - [21/07/2000]
Horus - [18/07/2000]
MINS - [15/07/2000]
reaps - [12/07/2000]
wrECK - [05/07/2000]
fab_fiona - [02/07/2000]
Cage - [27/06/2000]
K3nZ - [17/06/2000]
Gaz - [17/06/2000]
Ven - [08/06/2000]
wabut - [04/06/2000]
KenwooD - [27/05/2000]
avatar - [18/05/2000]
smiler - [13/05/2000]
UkSiR - [13/05/2000]
templar - [03/05/2000]
Bl4sT - [28/04/2000]
Melachi - [25/04/2000]
Kain - [24/04/2000]
Kaos - [18/04/2000]
Slash - [06/04/2000]
Gunner - [02/04/2000]
kevman - [27/03/2000]
Scum - [20/03/2000]
 
List Columners
List ALL Postings
 Signup For A Column


Mod-Scene

Looking for the latest mod news, reviews, previews, and info for a multitude of games? Mod-Scene has it all, and more!

 

 

 

 

 































 

 

© Copyright 1999 - 21st Century, GibWorld Networks. All Rights Reserved.