Does gw2 need a PBE?

[CETheLucid.3964]

@PyrateSilly.4710 said:NoThere are things/bugs that do not turn up until you have it on the live server, just the nature of the beast.Most will just see what is coming up and stock pile materials just to make more gold. They will only try out new things just to be the first do them when it goes live. Various things like these is what has happened on so many other beta servers and that's why most just don't do them anymore.

That's the sum of it. Way too much for way too little. It wouldn't accomplish anything and generally be a waste of resources.

[FrizzFreston.5290]

Considering the nature of such things I think a closed beta experience would be better.

[Inculpatus cedo.9234]

ArenaNet already has test servers; they just aren't as public as some may desire. You can sign up to be a volunteer test player if you should so desire.

Good luck.

[Illconceived Was Na.9781]

It turns out to be hugely expensive to maintain a public test environment, without generating much useful intel. Too many people who play beta do so to learn and prepare for the upcoming content, including theorycrafting, exploring new areas to figure to efficient farming techniques, speculating on where they might make a killing on the trading post (or its equivalent). And all too few people who report issues do so in a way that makes it easy to action them, e.g. "this didn't work" instead of "the skill say X, but when I used it, I got Y." There's also a lot of confusion between, "I don't like this" and "it's broken."

Worse, running a test environment means more work to keep surprises hidden, to manage economic changes, to avoid spoilers for those who don't "test," and to prevent people from burning out even before the update. And finally, it adds to the behind-the-scenes costs, because it requires keeping track of one more 'build' of the code & keeping track of differences. Besides what we see 'live,' there's also going to be the 'next to go live', the 'testing now', and 'developing now' versions of the code (among others). It's part & parcel of software development to have robust tools to keep track of each and 'promote' only the approved changes, but each additional 'official' environment adds to the complexity.

And to top it off, even in environments that include public betas, plenty of bugs go unmentioned.

Putting it all together: it's a lot more effort without it preventing a lot more bugs.

I share the frustration of the OP that this game sometimes gets hit by what seems like easily-preventable issues (and I'm sure, in some cases, ANet would agree: there's no good reason for those). All the same, while it sounds like a good idea in theory, in practice... not so much. Which is why ANet has actively considered it (more than once) and concluded (more than once): it's not worth it.

[The V.8759]

@"Revilrad.1962" said:No because people who participate would start wanting "special" , "prestigious" rewards for their effort and the whole thing will become just another recurring Celebration Event.

Why would they do that? theres plenty of games with PBE's, havent seen a single one giving rewards. If a players wants to test in pbe, its their choice. You know there is no reward

[Westenev.5289]

I feel it's a non-issue, since few people here has incentive to give unbias opinions on balance. Personally, I'd prefer more frequent hit-and-miss balance patches.

[ReaverKane.7598]

@Airdive.2613 said:

@Game of Bones.8975 said:

@"Etheri.5406" said:I have more faith in the couple hundred top end pvp players than in the thousands of silver and bronze players with strong opinions. Same for PvE. Same for WvW.

If 90% of your customers are "silver and bronze players" those are the people you need to get your information from. On a bell-curve, the high-end players are the outliers.

You don't necessarily need to get any information at all.It might be valuable, or it might undermine the developer's creative inspiration. Ultimately ANet has every reason to develop their game to the best of their abilities, not their playerbase's. They are the content creators, not consumers.

Have you been playing this game?Seriously? Have you? because it feels you haven't...They seriously need a PBE, there hasn't been an instance of balance where they have got it right and that didn't have unforseen consequences when it hits live.

Most people i've read voting no, don't seem to grasp the concept of how a PBE works, and what it's for.

First of all... They already have something like a PBE. Except it's restricted to content creator partners. That's why there's been issues with players leaking recipes and dealign with insider trading. They were content creators that had access to that early build servers and exploited the information. That's been dealt with though. Just serves as proof it exists.

So the groundwork is done already.

Second, like currently, access to such servers is usually highly vetted, not just anyone gets access, so usually the feedback will be more on point, it's not just a early version of every balance patch.

Third, having 1000 players play-testing a build is probably already tens of times more people than their internal testing teams can provide. And while they can simulate concurrent players for some errors, it's impossible to simulate player choice and it's impossible to emulate the "hive-mind" of millions of players pushing a new meta after a release. Even 1000 players in a PBE would be a spec of dust, but already a great improvement. And even without any written feedback from those players just the data stream from a PBE environment alone would be enough to show how effective and ineffective changes are.

Fourth, GW2 has a catastrophically slow balance pace, we can't really afford to spend 3 months with broken metas after each balance patch (which tends to happen frequently). If their pace was monthly or bi-weekly, then i'd say it would be frequent enough that balance would be restored after a failed attempt. But with such a slow pace, they really need that pre-balance-release feedback so that live servers don't have to be ruined by their own failures constantly.

Seriously, just look at the market out there. Most mobas and similar games have a PBE, have more frequent balance updates, and still they have instances of broken metas. If they want to keep the quarterly pace, Arena Net should really stack the deck in their favour and get a PBE environment to make sure their planned changes don't fizzle or blow back in our faces has is common.

[Airdive.2613]

@ReaverKane.7598 said:

Seriously, just look at the market out there. Most mobas and similar games have a PBE, have more frequent balance updates, and still they have instances of broken metas.

Say what? So they spend tons of resources on PBEs and there are still broken metas. Why keep spending them, then, except for PR?Also, balancing GW 2 professions is inherently much harder than, say, League champions because those have about 4 abilities each with the same list of passives to choose from, whereas the number of theoretically possible combinations in GW 2 is mind-boggling. It is best they don't bother too much and just roll out stuff for us players to explore and enjoy.

[ReaverKane.7598]

@Airdive.2613 said:

@ReaverKane.7598 said:

Seriously, just look at the market out there. Most mobas and similar games have a PBE, have more frequent balance updates, and still they have instances of broken metas.

Say what? So they spend tons of resources on PBEs and there are still broken metas. Why keep spending them, then, except for PR?Also, balancing GW 2 professions is inherently much harder than, say, League champions because those have about 4 abilities each with the same list of passives to choose from, whereas the number of theoretically possible combinations in GW 2 is mind-boggling. It is best they don't bother too much and just roll out stuff for us players to explore and enjoy.

Yeah, no... They mess up sometimes, but guess what? They mess up a lot less consistently in those games than in GW2. Also they fix them more readily than GW2.Is it mind boggling? Is it greater than League of Legends for example? Because GW2 balances classes mostly in a vaccum. LoL balances their interaction, which is a different league ahead of what we get.

Yeah, 4 abilities, and a magnitude more synergies and item combinations than GW2 has. Just look at the latest changes, they're having to balance the game because not of a champion being too strong but because people were palying it a bit differently than usual (the funnel strat) which made champions grow way faster than intended (because extra money) and it was breaking the game. So they had to fix that.

@"Illconceived Was Na.9781" said:It turns out to be hugely expensive to maintain a public test environment, without generating much useful intel.They already have a server that is used for similar purposes. Just not as widespread.Too many people who play beta do so to learn and prepare for the upcoming content, including theorycrafting, exploring new areas to figure to efficient farming techniques, speculating on where they might make a killing on the trading post (or its equivalent). And all too few people who report issues do so in a way that makes it easy to action them, e.g. "this didn't work" instead of "the skill say X, but when I used it, I got Y." There's also a lot of confusion between, "I don't like this" and "it's broken."They can do what they already do in live servers, collect data and dismiss most of player feedback. Or they can have only vetted players join (which is what they do already, just need to expand it a bit) and use only their specialized feedback.

Worse, running a test environment means more work to keep surprises hidden, to manage economic changes, to avoid spoilers for those who don't "test," and to prevent people from burning out even before the update. And finally, it adds to the behind-the-scenes costs, because it requires keeping track of one more 'build' of the code & keeping track of differences. Besides what we see 'live,' there's also going to be the 'next to go live', the 'testing now', and 'developing now' versions of the code (among others). It's part & parcel of software development to have robust tools to keep track of each and 'promote' only the approved changes, but each additional 'official' environment adds to the complexity.

They already allow Content creators early access to content. And it doesn't get spread out. There was a lot of issues a few years back about people using the insider knowledge obtained from such servers to profit, by stockpiling mats that would be used in the "to be released" recipes and then selling them after their price inflated. This was already handled.

And to top it off, even in environments that include public betas, plenty of bugs go unmentioned.

Yes, lets not get a vaccine for a disease because some people can't be vaccinated. Lets not disinfect water because some people are immune to chlorine.Even if a few bugs slip through, less will.

Putting it all together: it's a lot more effort without it preventing a lot more bugs.

The majority of the effort was already undertaken.

I share the frustration of the OP that this game sometimes gets hit by what seems like easily-preventable issues (and I'm sure, in some cases, ANet would agree: there's no good reason for those). All the same, while it sounds like a good idea in theory, in practice... not so much. Which is why ANet has actively considered it (more than once) and concluded (more than once): it's not worth it.I just love how you're so wrong with such confidence.

[Ephemiel.5694]

Even if they had it, they would pull a Blizzard and constantly say "we're listening to your feedback" while ignoring it completely.

[Illconceived Was Na.9781]

@ReaverKane.7598 said:

@Illconceived Was Na.9781 said:It turns out to be hugely expensive to maintain a public test environment, without generating much useful intel.They already have a server that is used for similar purposes. Just not as widespread.

No, those aren't interchangeable tools. Setting up a PBE is nothing like the work to set up a private test server for specific purposes, with people under Non-Disclosure, with people specifically chosen because of their relationship to the game, the type of feedback they can offer. Under those conditions, they can use the same test environment that already exists. A public server has to go through a different amount of vetting, because the average person expects more stuff to work more often (just look at the sorts of reports we got from the official BETA preview for GW2's launch, for HoT).

With a private test environment, they can boot people who don't provide valuable feedback; with a test server, they can't tell which feedback is useful or not until they read it. That's a lot of reading, without any clear indication of whether the content will provide any actionable intel. Much like the video linked here: we're expected to watch 20 minutes without any clue as to whether there's anything interesting.

[Zaklex.6308]

I said No, they've had an Alpha team in the past, and for all I know they still might have one...there's plenty of testing, it's just that you can't test for everything a player base might do. When you have a million players you have the potential for a million different things to be done...and that is impossible to test for.

[Danikat.8537]

@ReaverKane.7598 said:Second, like currently, access to such servers is usually highly vetted, not just anyone gets access, so usually the feedback will be more on point, it's not just a early version of every balance patch.

That's not a public test server then. That's the system they already have where periodically groups of play testers are invited to try unreleased content under a non-disclosure agreement. I can't find the link now but there is/was a thing on the Anet website (www.arena.net, not the GW2 one) where you could sign up to be considered for future tests.

But if it's not publicly available it's not a public test environment. It's basically inviting players (or sometimes non-players) to be volunteers in the QA department.

[ReaverKane.7598]

@Danikat.8537 said:

@ReaverKane.7598 said:Second, like currently, access to such servers is usually highly vetted, not just anyone gets access, so usually the feedback will be more on point, it's not just a early version of every balance patch.

That's not a public test server then. That's the system they already have where periodically groups of play testers are invited to try unreleased content under a non-disclosure agreement. I can't find the link now but there is/was a thing on the Anet website (www.arena.net, not the GW2 one) where you could sign up to be considered for future tests.

But if it's not publicly available it's not a public test environment. It's basically inviting players (or sometimes non-players) to be volunteers in the QA department.

It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

[Illconceived Was Na.9781]

@ReaverKane.7598 said:It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

That still comes with all the issues of a public server, and none of the benefits of a private one. People with Plat+ or 80+ rank don't automatically become good testers, good at giving feedback, or even able to offer criticism that applies generally rather than their specific point of view. We have many such examples already of skilled players with myopic views about the game.

[ReaverKane.7598]

@Illconceived Was Na.9781 said:

@ReaverKane.7598 said:It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

That still comes with all the issues of a public server, and none of the benefits of a private one. People with Plat+ or 80+ rank don't automatically become good testers, good at giving feedback, or even able to offer criticism that applies generally rather than their specific point of view. We have many such examples already of skilled players with myopic views about the game.

Again, player feedback is as useful as they find it. The true virtue of such a server is data (not player feedback, but server logs and statistics) that approximates closer to how it will behave on live, as opposed to how it behaves when being tested exclusively by the QA team.

[Illconceived Was Na.9781]

@ReaverKane.7598 said:

@Illconceived Was Na.9781 said:

@ReaverKane.7598 said:It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

That still comes with all the issues of a public server, and none of the benefits of a private one. People with Plat+ or 80+ rank don't automatically become good testers, good at giving feedback, or even able to offer criticism that applies generally rather than their specific point of view. We have many such examples already of skilled players with myopic views about the game.

Again, player feedback is as useful as they find it. The true virtue of such a server is data (not player feedback, but server logs and statistics) that approximates closer to how it will behave on live, as opposed to how it behaves when being tested exclusively by the QA team.

And again, you're only looking at it from a player perspective. Yes, absolutely, it's valuable to find out that a particular skill or chat code only bugs out on a full map or that the stress-testing scripts missed a key factor of 100 humans playing together (or 10000 logged on at the same time). But it's an enormous amount of work to make it possible for the occasional situation in which that is the aggravating factor.

The point is that the amount of useful data is limited, and it's not favorable to the costs. There are set up costs, maintenance costs, and change & code management headaches. The hardware is the easy part; it's all the human resources that go into it that make it a bad deal overall.

Put another way, public test servers do prevent some issues, they create new ones, and they change expectations in unrealistic ways. We know that ANet has considered public testing, and rejected it. And from the remarks, it's likely that someone internally brings this up about as often as we do on Reddit or the forums, and each time, after consideration, they make a business decision that the game wouldn't be enough better off to make it worthwhile.

(Incidentally, there are many other things that ANet probably could change internally that would also result in fewer issues. Some of them are as simple as having the right people in change/code management positions. But like most process-oriented changes, they are more easily said than done.)

So by all means keep suggesting it. I think ANet should hear how unhappy people are with bugs, and especially those that seem to be preventable. Just keep in mind that, on their side of things, there are other considerations — it's a business decision whether it's the best way to deal with bug prevention. Make the suggestion and then let them decide whether its' "cheap" or easy or not.

[The V.8759]

@Illconceived Was Na.9781 said:

@ReaverKane.7598 said:

@Illconceived Was Na.9781 said:

@ReaverKane.7598 said:It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

That still comes with all the issues of a public server, and none of the benefits of a private one. People with Plat+ or 80+ rank don't automatically become good testers, good at giving feedback, or even able to offer criticism that applies generally rather than their specific point of view. We have many such examples already of skilled players with myopic views about the game.

Again, player feedback is as useful as they find it. The true virtue of such a server is data (not player feedback, but server logs and statistics) that approximates closer to how it will behave on live, as opposed to how it behaves when being tested exclusively by the QA team.

And again, you're only looking at it from a player perspective. Yes, absolutely, it's valuable to find out that a particular skill or chat code only bugs out on a full map or that the stress-testing scripts missed a key factor of 100 humans playing together (or 10000 logged on at the same time). But it's an enormous amount of work to make it possible for the occasional situation in which

that

is the aggravating factor.

The point is that the amount of

useful

data is limited, and it's not favorable to the costs. There are set up costs, maintenance costs, and change & code management headaches. The hardware is the easy part; it's all the human resources that go into it that make it a bad deal overall.

Put another way, public test servers do prevent some issues, they create new ones, and they change expectations in unrealistic ways. We know that ANet has considered public testing, and rejected it. And from the remarks, it's likely that someone internally brings this up about as often as we do on Reddit or the forums, and each time, after consideration, they make a business decision that the game wouldn't be enough better off to make it worthwhile.

(Incidentally, there are many other things that ANet probably could change internally that would also result in fewer issues. Some of them are as simple as having the right people in change/code management positions. But like most process-oriented changes, they are more easily said than done.)

So by all means keep suggesting it. I think ANet should hear how unhappy people are with bugs, and especially those that seem to be preventable. Just keep in mind that, on their side of things, there are other considerations — it's a business decision whether it's the best way to deal with bug prevention. Make the suggestion and then let them decide whether its' "cheap" or easy or not.

I honestly this shouldn't even be part of the discussion. We simply don't exactly know how much resources a PBE would cost, and yes maybe it's not worth it. I'm asking it at the players, so I'm looking for a player perspective. Saying yes doesn't mean that you think it's possible from ArenaNet's side, it means you think player testing would be a good option right?

[Illconceived Was Na.9781]

@"The V.8759" said:I honestly this shouldn't even be part of the discussion. We simply don't exactly know how much resources a PBE would cost, and yes maybe it's not worth it. I'm asking it at the players, so I'm looking for a player perspective. Saying yes doesn't mean that you think it's possible from ArenaNet's side, it means you think player testing would be a good option right?

I'm paraphrasing the reasons the devs have given in the past. They don't think it's cost effective. My opinion is moot.

If you want my best guess anyhow, then no, I can't see it as being an overall good thing. There will be some great feedback, some good data, but not enough. The amount of useful intel compared to distracting rants, the amount of usable data... it's a lot of trouble for little return. But, I have no personal experience with testing games. I have seen "public" testing for software products used by software developers and ... even then, the feedback & data is really weak, unless there are powerful incentives.

[starlinvf.1358]

@ReaverKane.7598 said:

@Illconceived Was Na.9781 said:

@ReaverKane.7598 said:It can be publicly available, but they can still veto who accesses it. League of Legends places restrictions on who accesses it.GW2 can do the same. They can for example launch a PvP test build, and only open the server to players with 80+ rank on PvP and x amounts of matches, or plat+.

That still comes with all the issues of a public server, and none of the benefits of a private one. People with Plat+ or 80+ rank don't automatically become good testers, good at giving feedback, or even able to offer criticism that applies generally rather than their specific point of view. We have many such examples already of skilled players with myopic views about the game.

Again, player feedback is as useful as they find it. The true virtue of such a server is data (not player feedback, but server logs and statistics) that approximates closer to how it will behave on live, as opposed to how it behaves when being tested exclusively by the QA team.

Thats not a consistent truth either. A test server is a control environment, so players are going to behave differently due to a multitude of factors. A world boss event for instance is massively different with PUGs then it is with the equivalent of a Guild Group (ie at least minimally coordinated); and on top of which, different skill levels of players can get away different types of strategies. Modern glass cannons being a glaring example of something good players can run with little trouble, but even moderately causal players would struggle with in fights that last more then 3 seconds.

If you want useful data, you have to replicate the environment. The problem though is any environment with new elements will need weeks to reach proper equilibrium..... same as a live environment. You want a wide testing pool to capture as much berth of data as possible, but also needs to be focused enough to cut down noise inherent in the environment. This makes an ideal test pool an oxymoron. It has to contain players who universally agree on and seek a realistic average of performance, but also has a level of expertise of the system that moves them to one end of the bell curve. But the more expertise players have, the more they lean toward a status quo that lets them keep a competitive edge. Its an inherent conflict of interest, and very few players have the scruples to maintain a neutral position. Limiting access to certain demographics only biases that pool further; and is only desirable if your game (or the game mechanics) only targets an easily defined group.

But theres also extra layer of issues with a Test server being in exclusion to the main game...... a conflict of incentive. Even during the POF Beta Weekend, I had to make a choice between trying out the new Especs in WvW, or giving up potential rewards earned for that weekend when the kill count would be insanely high. I went with the former, because it gave me a chance to try some radically wide range of builds with ZERO investment into gear. Giving me enough info to know what builds I might like and pursue post launch. My priorities were squarely self interested first, game conscious second. And a PBE would have to expect a significant chunk of the population to not be like that, AND devote a substantial amount of hours per week to generate enough data for useful metrics.

Since none of that is possible without being disruptive or not properly reflected in behavior in a controlled environment, I'm in full support of introducing a test batch into a live environment, as it is an accurate representation of the day 1 gold rush. Out of all the data they can collect, this is by far the most crucial to curb long term damage in a persistent environment. The test batch is temporary, and they're impact ONLY disruptive on the play environment. But if they find exploits or cheese tactics that generate undeserved gains in the economy or competitive modes, they can stay contained to the test batch while the Devs clean up the updates to address them for the live release. The only down side is Salty players who cling to the status quo, and might throw a tantrum because a disruption exists..... but often times you can quell that dissent by simply offering bonus rewards to get them involved. Both sides of this are getting something out of the situation; Test players get an experience and/or knowledge to put toward the real release (which 99% is gonna be shared data by the end of the test period), and Live players get extra stuff for "tolerating" a change thats going to inevitably happen when it goes live anyway.

[Carighan.6758]

I voted yes, but the cynist in me wants to say that the production environment works perfectly well as a beta testing one for 6 years now. Players are used to things being glitchy and broken.

[Sylvari.2953]

Weird, my comment mysteriously disappeared from the 1st page but I got no notice/reason from moderators. :o Spooky!

[b k.1648]

Public beta tests are usually ruined by unreasonable expectations and unreasonable feedback. The only time it makes sense is when you need to test scaling with large numbers (i.e., stress testing).

[Cynn.1659]

Most devs can't accept feedback anyway.

How dare this random nobody tell me how to do my job i did for "Insert arbitrary number of years"

[Alga.6498]

If this is like a PTR, then yeah, it would be so useful.Just like Blizzard does before their patches goes live.