WvW Performance and Map Cap Testing

[primatos.5413]

If the lag was gone, we had soething to prevent overstacking (alliances maybe?) and the balance would be proper in a way.. could be a nice game to play..

[Diku.2546]

@primatos.5413 said:If the lag was gone, we had soething to prevent overstacking (alliances maybe?) and the balance would be proper in a way.. could be a nice game to play..

Hmm...I agree...that would be nice.

[Diku.2546]

Since we're looking at Adjusting Capacity...Can we make it part of a larger plan to fix the WvW game mode?

A ) REBOOT - Can we Reboot WvW before we start?

Let's go back to the basics that made WvW uniquely server community based - For full context - Click - WvW-Reboot

B ) REDESIGN - Then Redesign & Replace the game mode by swapping the old one for a new one to minimize risk.

Match-Ups redesigned to be Player Driven, but Anet Controlled - For full context - Click - WvW-Redesign

C ) REOPEN - Then let the New Redesigned game mode adjust capacity to impact both performance & competition in a healthy way.

Examples to adjust capacity:

7. Adjusting the number of maps within a Globe helps to increase or decrease "seat" capacity of a Globe to host more or less players that are battling inside
8. Maximum Globe "Seat" Capacity is set by ANet
9. Seats are Reserved based on the following - [ Home Team - 45% ] / [ Enemy Raider = 40 %] / [ Referee = 15% ] - (Percents Adjustable)

Redesign should be built-in & naturally allow capacity to scale based on player demand - For full context - Click - WvW-Match-Up

We ought to have A Serious Step-by-Step Road Map & Vision to save the WvW game mode by transforming it...imho

[Anchoku.8142]

What has changed in the code to create such problems with lag?

[Chaba.5410]

@Anchoku.8142 said:What has changed in the code to create such problems with lag?

Considering that there hasn't been any announced performance and optimization passes to the new skills and traits introduced and old skills modified since the last time they've done it (after HoT), one could say everything's changed.

[Anchoku.8142]

@Chaba.5410 said:

@Anchoku.8142 said:What has changed in the code to create such problems with lag?

Considering that there hasn't been any announced performance and optimization passes to the new skills and traits introduced and old skills modified since the last time they've done it (after HoT), one could say everything's changed.

Perhaps; there could have been server hardware and/or network connection changes, too. Cost reductions, including traffic prioritization changes could have happened. It would be a shame if a cost-saving choice resulted in reduced performance but it has been known to happen.

[Chaba.5410]

@Anchoku.8142 said:

@Chaba.5410 said:

@Anchoku.8142 said:What has changed in the code to create such problems with lag?

Considering that there hasn't been any announced performance and optimization passes to the new skills and traits introduced and old skills modified since the last time they've done it (after HoT), one could say everything's changed.

Perhaps; there could have been server hardware and/or network connection changes, too. Cost reductions, including traffic prioritization changes could have happened. It would be a shame if a cost-saving choice resulted in reduced performance but it has been known to happen.

You're right. There were those changes too when they moved to Amazon servers. I'm pretty sure they moved some time after the last major look at optimization. It's good they're looking into this again.

[DemonSeed.3528]

I'm just deathly scared if there will be no follow through since this is my biggest issue with the game. I just hope they are going through the results of the test and working on a next stage, and letting us know about it. I still wish they'd give people an option to tunnel through a closer geolocated aws to connect to the main server (seperate issue from skill lag). In some cases it would help with latency/ping stability/routing issue.

[Riba.3271]

@Mil.3562 said:

@briggah.7910 said:reset NA spent first 40 mins in EoTM. then 1 hour in EBG. no skill lag but 1 spike for a few seconds. 19 in queue took 40 mins to get into EBG

Yes. Before the map capping test, during the first hour on reset days, we have map queues only in one and maybe two maps. Two map queues was very rare. And even then the queue is short, like less than 20. And, there are days where there is no queue at all on all maps just 30 mins after reset.

Yesterday's reset? WOW!All maps were on queue and some even as high as 40+ in queue. I didn't know that 7 is such a magical number...

edit: I told youu ♫

Queue isnt necessarily a bad thing. It means more people are enjoying WvW due to reduced lags. If it keeps up, some of The guilds and pugs will just go to a server with less qs.

Also reset as overhyped event shouldnt be measurement for issues. How were qs on saturday Prime?

[Ya Ya Yeah.7381]

Sounds to me like Amazon pushed back and told them it wasn't they fault, so then Anet has decided to look in to it or at least do something, which is good. Lag has always been here, even since pre launch. It reached the height of its problem during the WvW seasons when guilds would run up to take an objective and have absolutely no control over there abilities for roughly a min straight and then when everything caught up would either be standing over bodies or were actually the bodies on the ground. It happened all the time and caused loads and loads for frustration. In the past Anet has tried culling with various efforts to cull players in the field which lead to invisible zergs, they finally got that to a reasonably balanced state. Hope they figure it out and fix it.

[DemonSeed.3528]

It's laggy this week compared to the last, feels like they undid something. No word from them about the tests.

[stzy.7892]

I've been having so much stop and go lag since this has came into place. Most times, it's not a ping issue. It gets worse every week, I've literally seen very little change but that could also be my internet as well, but most times when I do have issues, it's not from my side.

[DoggyPaddler.3586]

Get rid of commander tags

[CrimsonRise.7650]

The lag does seem better. I only had a one moment on Reset, where I couldn't pop any skills for a few seconds.

[GROMIT.7829]

@"Diku.2546" said:Since we're looking at Adjusting Capacity...Can we make it part of a larger plan to fix the WvW game mode?

A ) REBOOT - Can we Reboot WvW before we start?

Let's go back to the basics that made WvW uniquely server community based - For full context - Click - WvW-Reboot

B ) REDESIGN - Then Redesign & Replace the game mode by swapping the old one for a new one to minimize risk.

Match-Ups redesigned to be Player Driven, but Anet Controlled - For full context - Click - WvW-Redesign

C ) REOPEN - Then let the New Redesigned game mode adjust capacity to impact both performance & competition in a healthy way.

Examples to adjust capacity:

7. Adjusting the number of maps within a Globe helps to increase or decrease "seat" capacity of a Globe to host more or less players that are battling inside
8. Maximum Globe "Seat" Capacity is set by ANet
9. Seats are Reserved based on the following - [ Home Team - 45% ] / [ Enemy Raider = 40 %] / [ Referee = 15% ] - (Percents Adjustable)

Redesign should be built-in & naturally allow capacity to scale based on player demand - For full context - Click - WvW-Match-Up

We ought to have A Serious Step-by-Step Road Map & Vision to save the WvW game mode by transforming it...imho

If they shut it down to do work do you honestly think it'll get turned back on again ?.

[GROMIT.7829]

@Anchoku.8142 said:What has changed in the code to create such problems with lag?

They bought the budget Amazon server package with no roll back feature.

[Diku.2546]

@"GROMIT.7829" said:If they shut it down to do work do you honestly think it'll get turned back on again ?.

---

Emphasis on the underlined word:

3. Create a Parallel WvG game mode

See link - for full context:https://en-forum.guildwars2.com/discussion/comment/1064948/#Comment_1064948

---

WvW will continue to be used with no changes until it's replaced by a New game mode.

Weekend events are held to test a New Parallel game mode.

THR - 11:59 pm - WvW is removed from service.FRI - New game mode is put into service & temporarily replaces WvW.

SAT - New game mode is tested for proof of concept

SUN - 11:59 pm - New game mode is removed from service.MON - WvW is put back into service

Repeat above weekend event until Parallel New game mode is fully tested & ready to replace WvW.

---

I'd like to emphasize the need for design mechanics that help to create Match-Ups that encourage Healthy Competition that's "Wholesome" for the players...imho

Redesign should be built-in & naturally allow capacity to scale based on player demand - For full context - Click - WvW-Match-Up

Examples to adjust capacity:

7. Adjusting the number of maps within a Globe helps to increase or decrease "seat" capacity of a Globe to host more or less players that are battling inside
8. Maximum Globe "Seat" Capacity is set by ANet
9. Seats are Reserved based on the following - [ Home Team - 45% ] / [ Enemy Raider = 40 %] / [ Referee = 15% ] - (Percents Adjustable)

[Tiawal.2351]

Lag is back again during weekend at least: while ping is under 50ms, the skill lag is about 2000ms. This was today on EU Gunnar/FoW during US morning, midday EU.Even in 10 vs. 30 fight, which is an almost "empty" EBG map.This looks like bad service from Amazon side, because if it was from the Guild Wars 2 code then would be almost constant.Good for a turn based game maybe, not WvW.

[corwin.8356]

I've also been Wiresharking my connection while playing wvw during reset and I never see my upload rate for gw2 go over 70kbps, and my download rate never got over 1.8Mbps, even in a zerg on zerg in EBG. What I did see was a spike in round trip time between when a packet is sent and acknowledged, almost 1.5 seconds for each packet. Only during big zerg fights.

It is also important to remember that FPS drops, lag , and server latency are not the same thing. The have similar effects on your experience because of the way the client tries to compensate. Just know that using your FPS as a troubleshooting tool for this particular problem is misleading.

Since GW2 is using AWS EC2 instances I'm going to have to assume they are using the cheaper T2 variety to save costs. The main problem with the EC2 platform is general is that it's software load balancing instead of hardware and this is how bottlenecks happen. The load balancing application has to compensate for burst traffic and that performance is predicated on how good the hardware resources are to begin with, then how much has been earmarked for that Virtual Machine. In all honesty, even the T3 servers would have problems. You almost have to move to dedicated hardware with very powerful hardware load balancing to address this. WvW would have to be a cash cow to facilitate this kind of change because that solution is expensive. It's not.

Also there is no difference in the packet on the transport side between a targeted skill and an aoe skill. The FPS drops people have due to those are client side. What I imagine does happen on the server side is making sets of multiple dmg calculations from an aoe bomb. It's most likely why the scourge aoes had targets reduced; it wasn't a balance problem so much as a computational problem.

Note: this is all conjecture using what I can see easily from my end and my knowledge of how AWS EC2 servers work. I have no in depth knowledge of how the code is structured, but I make my living as a network design engineer so I have a better than average grasp of how data flows.

tldr: it's most likely server-side influx traffic and the servers can't compensate fast enough to keep from having delayed packets back to the clients. What we see as users are skills "blinking" and what the client is waiting for is the packet from the server to say it got the skill "send", once this cascades (as in big fights with scores of players sending bundles of skill activations) it gets worse until one side is dead. Once the fight is over, the packet queues clear and the latency returns to normal.

[ShadowCatz.8437]

@"Blaeys.3102" said:It's already hard enough to get my guild group into WvW.

This isnt a move in the right direction - making it that much harder to play with friends.

//bad joke incomingYou aren't expected to play with your "friends", only to kill the enemy....when they show up

//end of joke

[Woop S.7851]

@Cal Cohen.2358 said:The key feedback that we’ll need from all of you is to get a sense of how the maps feel with fewer players. We still want WvW to feel like WvW, and we realize that any reduction to player counts will have a negative impact on the overall gameplay experience.

@"corwin.8356" said:tldr: it's most likely server-side influx traffic and the servers can't compensate fast enough to keep from having delayed packets back to the clients. What we see as users are skills "blinking" and what the client is waiting for is the packet from the server to say it got the skill "send", once this cascades (as in big fights with scores of players sending bundles of skill activations) it gets worse until one side is dead. Once the fight is over, the packet queues clear and the latency returns to normal.

Thanks for the tech analysis Corwin! :) Since packet delay or loss has become exacerbated by software or hardware issues, what do you think of the below scenario where instancing & population control are borrowed from Edge of the Mist (legacy map design & code)? Would that alleviate issues in terms of packet bloating & server relay latency spikes?

Current EBG Map:

• Server resource fixed due to map on 24/7
• No instancing
• No timer for matches
• Players wait for Queue during peak times
• Skill lags due to Blob clashes

Current EOTM Map: (from wiki)

• Each match is 4 hours in duration broken up into 5 minute ticks.
• The scoring portion has a duration of 3 hours and 52 minutes when a winner is decided.
• This is followed by a 5-minute grace period to allow players to finish up.
• Players are removed from the map while the map is resetting. The next match starts 3 minutes afterwards.

Questions from a packets' perspective:

• Based on the current EOTM's map design where focus is on divide & conquer + capping points instead of blobbing, would instancing with a timer minimize packets loss & relays from the client side?
• Would server resources then be easier to balance with micro-'instances'? (some minimal server-side coding probably required)
• In terms of Amazon's servers, would a 50 vs 50 (something Amazon is using for their own upcoming game for example) segmented in instances with 4 hour durations run better while decreasing packet loss?

Any thoughts welcomed! :) thanks!

[DemonSeed.3528]

Anet, can we get some updates on this please?

[CrimsonRise.7650]

@CrimsonNeonite.1048 said:The lag does seem better. I only had a one moment on Reset, where I couldn't pop any skills for a few seconds.

Now tonight's reset seemed quite bad even though it wasn't bad as usual, was anything changed?

[DemonSeed.3528]

@CrimsonNeonite.1048 said:

@CrimsonNeonite.1048 said:The lag does seem better. I only had a one moment on Reset, where I couldn't pop any skills for a few seconds.

Now tonight seemed quite bad even though it wasn't bad as usual, was anything changed?

For me the following week after the initial test was when I felt like they undid something, and now it's fluctuating especially after the new episode.

[XenesisII.1540]

Skill lag was back tonight, fat 3 way in garrison.Also in fat 3 way in open field around swt.Also stuttering stepping in one instance.But not as bad as before... I guess.