Why Beta Sucks (Rant)

Bukola on July 16th, 2008

buggySome people are really struggling with beta; both the definition of the word and its execution. This is a problem that I’ve encountered with many applications online, as well as with certain Internet moon games.

Speaking as a former software development manager, I get it. You’ve just coded a really cool app, and you want to release it. You know that it’s not quite ready for prime time so you slap the ‘beta’ label on it, perhaps thinking that end users will be happy to help you iron out the kinks. Well, you’re probably wrong.

Today, “beta” seems to be an excuse for crappy coding and poor planning. Applications are released in extended, or perpetual beta, which is a public dumping of crappy code by developers who refuse to inform users of a solid release date. Twitter’s team didn’t even bother to label their micro-blogging client “beta,” they simply dumped their garbage code and listened to people complain about it on a daily basis.

Something similar happened to LunarWars. When the game opened it experienced a gold rush of new users who left after they had buggy experiences. With limited budgets to spend on advertising, Indie games live and die by word of mouth marketing, and if new players encounter a broken game they will not recommend it to their friends.

I don’t like to complain/report/post about three bugs while hundreds of other people complain about the same three bugs. Don’t you think its easier to manage the complaints of a smaller group of people who actually want to help you fix the problems they come across? The creators of SocialMedian think so. The developers knew their code wasn’t perfect so they released the app in private alpha to a devoted group of beta testers (including myself). This used to be the norm, but now it’s actually newsworthy.

Back in the day software developers released their first confident batch of code in alpha form to a tiny group of people, then a beta version of the product was made open to a wider (but still limited) group of folks before it was finally released to the public. To me, this seemed like a pretty good system. What happened?

Random Salad is full of things much less interesting than this. Whether you've come to this page by a link or by voodoo sex magic, pay a visit to the homescreen before you go.

16 Comments:

link Christoph Malcolm
July 16th, 2008 at 6:54 am

I think there are different assumed definitions of “beta” for different kinds of applications.

Obviously with software it means that the program is still incomplete and probably full of bugs. Beta releases are normally made available publicly using the “beta” label as a disclaimer, before a full commercial release is shipped.

With online applications things change quite a bit. The fact that users don’t have to download each new version makes all the difference, and because of that the changes are all incremental. For me, it’s assumed that any website is going to be continually improved and modified, but with online apps the term “perpetual beta” became something of a buzzword. I blame Flickr for popularizing it.

When it comes to browser-based games, the “beta” suffix really does give the impression that it’s incomplete, and for me it comes with the assumption that everything will be reset at some point. Lunar Wars is one case where the developer decided “perpetual beta” applied to games in the same way it does to cloud computing, which was probably a bad decision. The difference between a game and something like Google Docs being that when Google developers update the code for their Spreadsheets app, the data in my spreadsheets isn’t going to change.

I don’t really have a problem with Web 2.0 companies saying that everything is in “perpetual beta”, because version numbers for incremental changes would only be more confusing and less useful, but I’d really like to play a completed game.

link Locke
July 16th, 2008 at 7:57 am

Agreeing 100% with you Bukola. The beta label often seems to just be an excuse for some developers to fall back on these days.

link Grazor
July 16th, 2008 at 10:15 am

It doesn’t have to have the ‘beta’ label to be buggy. Commercial pressures often require products (whether hardware or software) to be released early to gain market share over competitors. This usually means shortcuts during development and testing prior to initial release. Couple that with a commercially viable ‘lifespan’ for some of the latest products to be a couple of years (or less) before they’re superceded/obsolete/out-of-fashion and you can see why.

Kids are great testers - in my experience they will press/try anything and everything, even if it doesn’t make sense to do so. If you’ve not nailed down every possible permutation then they will completely fuck up your creation somehow - little bastards!

link Joey Michaels
July 16th, 2008 at 12:53 pm

Then there’s the whole issue of bands stuck in perpetual beta…

link Locke
July 16th, 2008 at 1:57 pm

I clicked on that link expecting to be able to get some betamax tapes and was disappointed.

What we need is some more patches.

link Kent C. Tugood
July 16th, 2008 at 2:00 pm

As a computer scientist who’s looking to get into the video-game/browser game business, I know how dang important it is to get out of that ‘beta’ stage. To be honest, the concept of a public beta doesn’t usually work out the best in my opinion. A small group of dedicated people can usually spot a good portion (Not all) of the bugs in an application, so that if a reset is necessary to go into Alpha, there is not a considerable user lashback against the move.

Of course, you’re not going to be able to find all bugs in any program in beta, or even alpha for that matter. As Grazor said, some people will tear an application apart looking for exploits, bugs, glitches, anything that will cause unusual behaviour in a program. To be honest, those are the people I’d like on my team in a beta-test. So if I’ve released a finished product, which works well, spotlessly, and correctly 90% of the time, I wouldn’t lose sleep updating my final product every so often to fix that remaining 10%, and I don’t think the userbase would have much complaints about that.

As for that ‘perpetual beta’ buzzword, as long as the gradual changes improve an already working product, I don’t think that deserves to be labelled as a ‘beta’ application.

link Genitals McKool
July 16th, 2008 at 7:43 pm

so kent why are you not helping joey?

link Kent C. Tugood
July 16th, 2008 at 9:03 pm

I should be helping Joey, but I don’t know enough about Browser Based jazz yet, which this next year of school should give me a good enough lesson on.

Also: I have my own Browser-Based game in the works. It’s gonna be ugly to code at times, due to HEXAGONS.

Also Also: Anyone who knows how to make decent code and graphic display of a hexagon map online, LET ME KNOW.

link Joey Michaels
July 17th, 2008 at 2:24 am

Haha we just decided against maps.

link Bukola
July 17th, 2008 at 7:34 am

@Christoph Malcolm: Heh I blame Google for putting “Perpetual beta” in vogue. I disagree with your point about web companies in perpetual beta. Participating in an incomplete social network is just as bad as playing an incomplete game.
@Locke: Thanks :D Beta has definitely become a refuge for mediocre software/game developers.
@Grazor: You’re absolutely right! Video game developers are doing this now. They release games with KNOWN BUGS. They do this to meet production schedules. They also know that they can just release a patch to fix the problems later down the line. Its ridiculous. Why should I pay 50 dollars for an incomplete game?
@Joey Michaels: Lol beta bands XD. So you’re making a game? Do you need an experience marketing gal on staff? What’s a hexagon map?
@Kent C. Tugood: I’d love to talk with you about gaming career stuff. How exciting for you!! The process that you laid out for testing is sound. I wish more developers used it. Also, what is a hexagon map?

link Christoph Malcolm
July 17th, 2008 at 9:09 am

Normally a website in a state of “perpetual beta” is complete by the time it reaches the public, but “complete” changes with technology. It’s just an industry term now to describe sites which are continually improved.

Twitter is actually a “complete” site, for example. They have added a few new features, but the core functionality of Twitter was fully realized before it was released. The problem they’ve had is that their servers couldn’t handle all of the traffic, and Ruby on Rails has apparent hindrances to scalability.

So in the case of a lot of Web 2.0 companies, “beta” really does have an entirely different definition. The sites aren’t unfinished, they’re just saying that they will always be working to make them even better.

link Kent C. Tugood
July 17th, 2008 at 12:53 pm

@Joey: But that takes all the Geopolitical Junk out of it!

@Bukola: A Hex Map is like this, but I’m having trouble thinking of easy ways to store data, move across, and display on a browser a hexagon. My team has thought of a couple of ways, but none of them thus far have looked pretty from a design perspective.

link Joey Michaels
July 17th, 2008 at 3:07 pm

We’re going to add the map during Beta. Which is to say “whenever we feel like it.”

link Kent C. Tugood
July 17th, 2008 at 5:06 pm

What’s the concept of the game, in a quick synopsys?

link Joey Michaels
July 17th, 2008 at 6:30 pm

I don’t want to say too much, but Huns may figure prominently in the final version. Or Mongols.

link Kent C. Tugood
July 17th, 2008 at 10:45 pm

Oooooh, intrigue!

Post a Comment:

Register your nickname if you plan on sticking around.
Sign up for Gravatar if you want a nifty little icon.
RandomSalad.com is owned and operated (barely) by Christoph Malcolm and Kimber Loo. The content belongs to the people who create it, under a Creative Commons license. Everything else belongs to Christoph, probably. © 1998-2008, etc. Don't even fuck around or I'm calling your parents.