The New Balance Philosophy needs a major improvement - LIVE TEST SERVERS for ALL

[RedBaron.6058]

Hello all,

I am confortable with the announced new balance philosophy but it is paramount that all players have access to LIVE TEST SERVERS so player's feedback is provided in real time, alongside the feedback provided by ANet testers, before any game patch is rolled out.

Thanks.

[Infusion.7149]

Arenanet had live public test servers for the Desert Borderlands and it didn't run the same on live servers.

I'm pretty sure Arenanet partners get early access to content already.

[Omega.6801]

While the idea of having feedback from ALL players maybe sounds good. It sounds fair and democratic, maybe even scientific.

People rarely consider the amount of work it would take to filter through everything, just to find out that players want their mains buffed, that losing is unfun and that balance is important. Don't get me wrong, sure they could do all the statistics and maybe come up with a good representative 10 pages long multiple choice test to evaluate thousands of participants. This process of gathering enough data, evaluating and iterating on it might take a few years though. For balance patches to come out in every reasonable amount of time, testing must be done but it must be done to scale. So instead of asking 100'000 players, they ask maybe 100. Instead of asking everyone, they ask people who they know, have some sort of expertise with the game, that ANet can account for.

Also: Free for all Test servers bring complaints about "that awesme drop you got but can't transfer to your main account. ANet fixploxwhen!?!?!?!?!?!?".

Also also: The moment you put something in the Test servers that can be transfered to the "real game", test servers will have to deal with players not there to actually test but to farm.

Also also also: I'm not a big buildcrafter and I never bothered to become one, get behind the math of stats, damage formulas and modifiers. I can read what skills and traits do and can get something together that works but I never went into detail with all that. That's for different players to do and I think their opinion, on wether or not a 0.5s change to a CD will break the game or not, is more valuable to ANet.

[RedBaron.6058]

4 minutes ago, lokh.2695 said:

While the idea of having feedback from ALL players maybe sounds good. It sounds fair and democratic, maybe even scientific.

People rarely consider the amount of work it would take to filter through everything, just to find out that players want their mains buffed, that losing is unfun and that balance is important. Don't get me wrong, sure they could do all the statistics and maybe come up with a good representative 10 pages long multiple choice test to evaluate thousands of participants. This process of gathering enough data, evaluating and iterating on it might take a few years though. For balance patches to come out in every reasonable amount of time, testing must be done but it must be done to scale. So instead of asking 100'000 players, they ask maybe 100. Instead of asking everyone, they ask people who they know, have some sort of expertise with the game, that ANet can account for.

Also: Free for all Test servers bring complaints about "that awesme drop you got but can't transfer to your main account. ANet fixploxwhen!?!?!?!?!?!?".

Also also: The moment you put something in the Test servers that can be transfered to the "real game", test servers will have to deal with players not there to actually test but to farm.

Also also also: I'm not a big buildcrafter and I never bothered to become one, get behind the math of stats, damage formulas and modifiers. I can read what skills and traits do and can get something together that works but I never went into detail with all that. That's for different players to do and I think their opinion, on wether or not a 0.5s change to a CD will break the game or not, is more valuable to ANet.

ANet just needs to do what other good games do, LIVE TEST SERVERS for all players interested in providing feedback, which in GW2 case are not for sure as you said "...100'000 players...".

I really do not understand your remark about "awesome drops" in test servers, everyone knows that test servers content is temporary and do not transfer to REAL LIVE SERVERS...

Are LIVE TEST SERVERS too much to ask from an AAA game in 2022?

[Labjax.2465]

I have mixed feelings about public test servers. On paper, it sounds helpful. In practice, I've seen games where stuff shows up on test servers early, the feedback given is almost entirely ignored, and it goes live as it was on the test servers anyway. Best I can figure is it's that thing where by the time it's reached public test servers, they're just looking for game-breaking bugs, they're too far into the dev process to do much of anything.

[Ultramex.1506]

They could do a "beta" event like EoD

Or they can't separate the traits and skills? As in unable to change beta without affecting  current traits/skills?

[kharmin.7683]

Be careful what you wish for.  SWToR uses test servers and then ignores players' feedback anyway.

[Ashen.2907]

7 hours ago, Labjax.2465 said:

I have mixed feelings about public test servers. On paper, it sounds helpful. In practice, I've seen games where stuff shows up on test servers early, the feedback given is almost entirely ignored, and it goes live as it was on the test servers anyway. Best I can figure is it's that thing where by the time it's reached public test servers, they're just looking for game-breaking bugs, they're too far into the dev process to do much of anything.

This has been my experience as well (with CO being a notable exception years ago).

[Zok.4956]

9 hours ago, Infusion.7149 said:

Arenanet had live public test servers for the Desert Borderlands and it didn't run the same on live servers.

As far as I remember, those test servers where you had to use a different client were not public (everyone can join) but closed (invite only).

We also had closed test servers for EotM, where a few invited/selected WvW guilds provided feedback (did not work well in the beginning).

We also had closed test servers for the HoT raids where a few invited/selected players/guilds tested the raids (it did work very well with the exception of the drama, that was caused because one guild used their prior knowledge for doing a "worlds first clear" after HoT was released).

We had public tests for the elite specs, but in my opinion this was just some marketing campaign, because Anet did not really change elites as response to the (overall good and detailled) feedback, players had given.

Of course, public test servers are not the one-for-all solution for everything. But they can give more and sometimes better feedback because players, that are invested in a game mode or content-type and that are trying to make it better / more fun, can self-select and can give feedback.

[Gehenna.3625]

3 hours ago, kharmin.7683 said:

Be careful what you wish for.  SWToR uses test servers and then ignores players' feedback anyway.

I can confirm this, though I wouldn't call it ignore all the time. I think there are three reasons for that in SWTOR's case

1. They don't know how to fix it
2. It doesn't fit their philosophy
3. They don't have the resources to fix it before it releases

Reason number 2 would be the ignore part but doesn't seem to be the main one. And in case of reason 3, they generally fix things in the least sensible way.

Still, I don't think that test servers have all the answers and I think that releasing what they're planning to do early is the best way to do it. Give people time to give feedback and yourself time to change things accordingly if you agree with the players.

[branwen.1049]

9 hours ago, Labjax.2465 said:

I have mixed feelings about public test servers. On paper, it sounds helpful. In practice, I've seen games where stuff shows up on test servers early, the feedback given is almost entirely ignored, and it goes live as it was on the test servers anyway.

You can namedrop World of Warcraft, it's okay.

 

On a more serious note, test servers/realms are hit-or-miss, depending entirely on how they're handled, and how the feedback given is considered. As much as I would love them to have things pre-tested so bugs/balancing can get ironed out before it goes into the live game, that actually requires them to take that feedback into consideration and deem it worthy to actually fix by whatever margin of people tested it vs. the overall game population. 😕

Edited October 29, 2022 by Giovanelli.6071

[mythical.6315]

People overall tend to be horrible when it comes to giving feedback and can often be selfish when it comes to balancing.  It's better to have a smaller group who are more knowledgeable and have proven that they're good at providing less biased feedback.

[Randulf.7614]

20 hours ago, RedBaron.6058 said:

ANet just needs to do what other good games do, LIVE TEST SERVERS for all players interested in providing feedback, which in GW2 case are not for sure as you said "...100'000 players...".

I really do not understand your remark about "awesome drops" in test servers, everyone knows that test servers content is temporary and do not transfer to REAL LIVE SERVERS...

Are LIVE TEST SERVERS too much to ask from an AAA game in 2022?

No, but I’m not sure GW2 is quite a AAA game either

Im not against live servers, but I’d prefer Anet to be much better at reactive fixing of any issues instead of waiting months for data collection. Metrics can be a hindrance at times.

[RedBaron.6058]

The upcoming changes hinted by the New Balance Philosophy will transform completely the current classes and even the heritage philosophy from the original ANet Manifesto.

Such huge changes need to be showed, discussed and changed within the whole GW2 community before hitting real live servers.

I love GW2 and I dont want it to be gutted before getting better after months and months of tweaks, especially because 3 or 4 great new MMO will launch during the next 12 months and will easily drain a lot of unhappy GW2 players.

Long live GW2!

[Labjax.2465]

On 10/29/2022 at 10:38 AM, Giovanelli.6071 said:

You can namedrop World of Warcraft, it's okay.

 

On a more serious note, test servers/realms are hit-or-miss, depending entirely on how they're handled, and how the feedback given is considered. As much as I would love them to have things pre-tested so bugs/balancing can get ironed out before it goes into the live game, that actually requires them to take that feedback into consideration and deem it worthy to actually fix by whatever margin of people tested it vs. the overall game population. 😕

Was thinking of SWTOR mostly, tbh. But this shows it's more than one game that has had the problem lol.

[Obtena.7952]

The game doesn't need such a thing. There isn't actually much risk with releasing the content to the live server for 'player testing' as they do now. There might be a benefit but the question is if it outweighs the cost/time to have it. Plus, I don't really see what the real impact would be to have it because while player feedback is important, so is actual usage numbers and statistically relevant in-game numbers, things you don't get from a test live environment. There are simply insights to player behaviour that don't reveal themselves until things are in the game. 

Edited November 1, 2022 by Obtena.7952

[Zok.4956]

On 10/30/2022 at 2:47 AM, mythical.6315 said:

People overall tend to be horrible when it comes to giving feedback and can often be selfish when it comes to balancing.

Only a very few players are really selfish and try to manipulate their feedback in this way.

Overall, people tend NOT TO BE horrible when it comes to giving feedback if they have some guidelines for giving feedback and it is explained to them what is expected, what is good feedback, what is bad feedback etc... Sure, without those guidelines, feedback can loose  a lot of it's value.

 

On 10/30/2022 at 2:47 AM, mythical.6315 said:

It's better to have a smaller group who are more knowledgeable and have proven that they're good at providing less biased feedback.

Selecting this small group is the problem, it creates selection bias that can make all of the feedback useless if the opinions of the selected group do not represent the opinions of the population they should represent.

Just one example:  The Desert Borderland in WvW was tested before release with a small group of WvW players/guilds. In the end it was the wrong group. Because after the new map was released it was so bad, at lot of WvW players left and it nearly killed the game mode. 

Another example: If the selected group consists mostly of "content creators" ("content" only means mostly twitch/youtube streams and videos) it is clear that the feedback is biased, because those will usually try to earn money with their twitch/youtube activities and wil try to get more audience/viewers/subscribers/followers. So their feedback is usually biased from their own point of view and they really do not represent "all players".

The bigger the testing group / the testers sample, the less systematic and less impactful possible selection bias errors could be. With players self-selecting for public test servers you could get the biggest test samples/groups and the company/studio/Anet could still set priorities, how they balance the feedback of different groups  or player types.

[Gwynnion.7364]

9 hours ago, Zok.4956 said:

Overall, people tend NOT TO BE horrible when it comes to giving feedback

From recent experience, it's the developers who tend to be horrible about listening to feedback.  During the EoD class betas, there was a ton of player feedback about each of the new elites, most of it straightforward, and many of those issues have still never been addressed.

9 hours ago, Zok.4956 said:

Selecting this small group is the problem, it creates selection bias that can make all of the feedback useless if the opinions of the selected group do not represent the opinions of the population they should represent.

We've all seen what happens when elite players and content creators are the ones privileged to give feedback -- especially when they're the only ones the devs listen to.  Yeah, some of them may have a very thorough understanding of the class, but they do not share the average player's concerns or experience.  And content creators are notorious for glossing over problems in order to stay in a company's good graces for continued access.

[mythical.6315]

14 hours ago, Zok.4956 said:

Only a very few players are really selfish and try to manipulate their feedback in this way.

And it takes time to sort those ones out.

14 hours ago, Zok.4956 said:

Overall, people tend NOT TO BE horrible when it comes to giving feedback if they have some guidelines for giving feedback and it is explained to them what is expected, what is good feedback, what is bad feedback etc... Sure, without those guidelines, feedback can loose  a lot of it's value.

You must not have seen the threads in the bugs subforum.

14 hours ago, Zok.4956 said:

Selecting this small group is the problem, it creates selection bias that can make all of the feedback useless if the opinions of the selected group do not represent the opinions of the population they should represent.

Just one example:  The Desert Borderland in WvW was tested before release with a small group of WvW players/guilds. In the end it was the wrong group. Because after the new map was released it was so bad, at lot of WvW players left and it nearly killed the game mode. 

Another example: If the selected group consists mostly of "content creators" ("content" only means mostly twitch/youtube streams and videos) it is clear that the feedback is biased, because those will usually try to earn money with their twitch/youtube activities and wil try to get more audience/viewers/subscribers/followers. So their feedback is usually biased from their own point of view and they really do not represent "all players".

The bigger the testing group / the testers sample, the less systematic and less impactful possible selection bias errors could be. With players self-selecting for public test servers you could get the biggest test samples/groups and the company/studio/Anet could still set priorities, how they balance the feedback of different groups  or player types.

Let's say we have 90 people with no medical background and 10 people who specialize in it.  Since those 90 outnumber the 10, this must mean that they know better?  Balance is just shifting everything to how the majority want it?

[Zok.4956]

8 hours ago, mythical.6315 said:

Let's say we have 90 people with no medical background and 10 people who specialize in it.  Since those 90 outnumber the 10, this must mean that they know better?  Balance is just shifting everything to how the majority want it?

I think your comparison is flawed. The 10 people (the devs) are doctors who have developed a new treatment and the 90 people (the players) are patients who provide feedback on whether the treatment is working as hoped and whether it has side effects.

Let's look at the elite-specs public beta tests. The devs did not really gave any guidelines how to give good feedback or what kind of feedbeck they wanted or about what exactely they want feedback. So it was actually to be expected that the feedback from the players would be chaotic and unstructured. And some of the players feedback was. 

But I was impressed by how many players gave very good, detailed and well-founded feedback, which they put a lot of time and energy into, despite the unclear guidelines.

Just imagine how the game could be improved if the developers really tried to use this kind of power.

P.S. Looking back, it wasn't really a public beta test, just a marketing campaign and the devs didn't really care about player feedback. That was a shame and players are rightly upset about it.

 

[Zok.4956]

9 hours ago, mythical.6315 said:

And it takes time to sort those ones out.

It always takes time to sort feedback out. But the better the preparations and guidelines, the less time it takes.

[JusticeRetroHunter.7684]

5 hours ago, Zok.4956 said:

I think your comparison is flawed. The 10 people (the devs) are doctors who have developed a new treatment and the 90 people (the players) are patients who provide feedback on whether the treatment is working as hoped and whether it has side effects.

Let's look at the elite-specs public beta tests. The devs did not really gave any guidelines how to give good feedback or what kind of feedbeck they wanted or about what exactely they want feedback. So it was actually to be expected that the feedback from the players would be chaotic and unstructured. And some of the players feedback was. 

But I was impressed by how many players gave very good, detailed and well-founded feedback, which they put a lot of time and energy into, despite the unclear guidelines.

Just imagine how the game could be improved if the developers really tried to use this kind of power.

P.S. Looking back, it wasn't really a public beta test, just a marketing campaign and the devs didn't really care about player feedback. That was a shame and players are rightly upset about it.

 

Not disagreeing fully here with you but I think player feedback is most of the time 99% not good. The kind of question asked will reflect on the quality of feedback. 
 

Example with the doctors: I’m a doctor and I ask you: Are you in pain? Show me where and what kind of pain? The patient tells you “yes,” and shows you their knee and says it is throbbing. This is an example of good feedback because the question to the patient made sense.

 

Conversely: I’m the doctor and I asked you: so what medication should I give to you?

 

You can imagine the patient could probably say anything at this point and 99% of whatever they say could actually kill them by mistake. The patient could even be thinking rationally, but because they have no idea what medicines actually do to the human body their feedback is not only gonna be probably wrong but could lead to worse things.

 

In the case of Guild Wars 2 the doctor doesn’t know what the medicines really do either.

 

So, it is like Mythical points out…the majority being able to influence what is deemed “true” and “correct” is deeply flawed. Anet needs help from professionals that actually know what to do and how to design games. and I would extend this bracket of knowledge to scientific fields in particular biologists (people that study the most naturally balanced system known to man kind: Planet Earth) and system designers that know how to put functioning systems together. 
 

one thing I want to add and to say just so my stance is clear: that a professional patient is not the same as a professional doctor…

where…being a good patient doesn’t mean that patient would know what medicines to take.

The analogy here is that top players of the game are not in this group of people that give good feedback either…especially to the games overall balance (and this is what I agree with ZoK on about using a diverse pool to avoid any kind of selection bias towards any particular viewpoint…that’s how it’s done in science anyway)

Edited November 4, 2022 by JusticeRetroHunter.7684

[Zok.4956]

1 hour ago, JusticeRetroHunter.7684 said:

The kind of question asked will reflect on the quality of feedback. 

I can agree 100% to this.

 

1 hour ago, JusticeRetroHunter.7684 said:

I’m a doctor and I ask you: Are you in pain? Show me where and what kind of pain? The patient tells you “yes,” and shows you their knee and says it is throbbing. This is an example of good feedback because the question to the patient made sense.

 

Conversely: I’m the doctor and I asked you: so what medication should I give to you?

Agreed. The doctor should ask the right questions. The patient cannot do the doctor's job (they can't, even if they graduated at Youtube-Academy 😉) of deciding which drug is best. But the patient can answer whether the drug prescribed by the doctor helped to relieve pain or had unwanted side effects.

Likewise, players should not do Anet's job (they can't, even if they graduated at Twitch-Academy 😉) . But a lot of players are good judges of whether the balance/game changes "prescribed" by Anet are working well or have unwanted side effects.

 

2 hours ago, JusticeRetroHunter.7684 said:

In the case of Guild Wars 2 the doctor doesn’t know what the medicines really do either

Well, in the case of doctors in the real world, especially GPs,/family doctors many of them don't know this either unless the problem is obvious and unfortunately medical treatments are sometimes trial and error and that's why patient feedback is so important.

Because of this, I believe that a public test server could be used as a very valuable tool by the developers to collect player feedback to help the developers doing their job.