Turret Changes: A Technical Talk about why.

[isu]

People suggested lot of good ideas, how to improve this situation:

1. more expensive turrets with bigger damage

2. turrets using more bullet per hit for more damage

3. more types of ammo (AP, flame,...)

 

But if WC are not planning to add this, lot of things needs to be nerfed:

1. Tanking dinos are to strong 

-  Nerf their HP / armor on saddles

- Remove BP's platforms, leave only primitive one (now saddles on paracer/brontos are useless)

- remove ability to place c4 from stegos

 

2. Speed on characters is OP and connected with rocket launchers/c4 are deadly combinations

- add 2-3 seconds as targeting for Rocket Launchers

- cap speed of characters

[natman]

3 minutes ago, Arksouls said:

How do you balance the amount of time and effort ive put in on my 1000 turret base to some small tribe with 100 turrets how is it fair that we have the same defense when offline... You have basicly added a cap on how good your base can really be. now effectively everyone will have the same 100 turrets regardless of playtime

are you that little creative? it is not like you are limited to 100 turrets, just 100 per 10000 units. and those 10000 units are measured from and for each single turret, so in theory you could place a chain of far more than 100 turrets from one end of the map all the way to the other one, as long as there is a bit of space between them. Base design will become more important and demanding with this change rather than stale. And big tribes will still have more turrets than small ones.

[Arksouls]

1 minute ago, natman said:

are you that little creative? it is not like you are limited to 100 turrets, just 100 per 10000 units. and those 10000 units are measured from and for each single turret, so in theory you could place a chain of far more than 100 turrets from one end of the map all the way to the other one, as long as there is a bit of space between them. Base design will become more important and demanding with this change rather than stale. And big tribes will still have more turrets than small ones.

 

Well no they wont they will be the same design for all bases because your bases are limited to 100 turrets meaning smaller bases with more compact designs eliminating big bases in general...

[IsrealFox]

Obviously you guys dont check out the official servers very often, so let me clue you in. These so called insane builds you speak of on new server are nothing compared to the ones on legacy servers. Therefore that point is moot. Rag starts lagging extremly bad when there are around 30 survivors connected. Everytime someone joins you get a lag spike, mixed with the amount of dc zones on rag you see how this becomes an issue. It also begins happening on the center as well around 30 - 40 survivors but center lag is much different. I honestly just think your servers cant support the traffick. I would delete those dead ass legacy servers full of duped gear and see if that helps first before nerfing our only means of defense. If you follow through with this nerf there better be a soaker/veggie cake nerf too because it will be pointless to even build. There have to be other areas of the game where you can tone down textures and graphics to help with fps.

[Arksouls]

3 minutes ago, natman said:

are you that little creative? it is not like you are limited to 100 turrets, just 100 per 10000 units. and those 10000 units are measured from and for each single turret, so in theory you could place a chain of far more than 100 turrets from one end of the map all the way to the other one, as long as there is a bit of space between them. Base design will become more important and demanding with this change rather than stale. And big tribes will still have more turrets than small ones.

 

And another point here its isnt that i care that i cant build the same bases its more the fact that if you are offline i know for a 100% fact that me and my alliance will be able to breach your base no matter the design no mater the bullets because you are locked to 100 turrets

[Demerus]

7 hours ago, TheRightHand said:

Hey guys, sure has been a crazy day hasn't it?

So, I know we have that megathread over there, and guess what my weekend gets to be (It's reading that and every other thread.) but I wanted to make a new post here just so that information was in a good, easy place to see.

Let's just get right into it:

This is some profiling output from a fun little tool we made, which analyzes the server and finds each thread that is running and times how long it takes for the server to tick. It does a lot more than that, but that's what you need to know to understand this:

Total Duration 765.60 MS 1.31 FPS

StructureTurretBaseBP_C :     7977 234.98 MS
StructureTurretPlant_C :     9722 119.4 MS

This is turrets. BaseBP is normal metal turrets, Plant is SpeciesX.

Now, with this information, you need to have some other information:

Since about October or so, we saw a MASSIVE spike in the size of stalls and poor performance on our dedicated servers, but only some of them, most of them Ragnarok Servers.

So we investigated, and we found that an interesting change in the meta for base building had occured: People were making new, super-dense bases with insane turret counts, far beyond anything we'd ever seen before. The bases of these types were on all of our servers, but primarily belonged to Alpha Tribes who had set up bases on the Ragnarok servers. Similar setups caused equally bad performance on Center and Island servers in the rare cases we found them.

I want you to understand what all of this means to us, so I'll try to be brief.

Turrets, were very suddenly being used in a configuration that was atypical, and up to 7x the number of turrets we would see on any other server, even if it was a highly populated, well trafficked server. There were definitely other cases of densely packed bases, but this was higher by many magnitudes.

Let me emphasize now, that for the last two years, turrets have been a pain in our butt, but we have not encountered people making monstrosities like this. Every alpha tribe in the game has existed just fine, and gotten along well enough without these insanely dense crazy bases.

So, we looked at what we could do about it. Let me wrap this all up in context, these things I'm going to put here are facts:

Our turrets are highly optimized for what they do with respect to gameplay. They use a very fast bitmasked octree overlap to return all of specific types of actors (excluding irrelivant actors) in the aoe that they defend, and then act according to the settings you configure, early-outing from any extra calculations or wasted instructions if their shot is invalid. We could make that area they search smaller, which would be faster, but then they would need correspondingly smaller ranges.

We could make them search for less things in their overlap check, but then you'd have turrets that didn't fire on some things.

We could make them acquire targets less frequently, but then they'd be much less accurate and in many cases would not pick up things like rocket projectiles that entered their range between scans.

We could make them scan at different ranges for different things, but then their functionality would be weirdly ambiguous and what things would we reduce scan range for anyways? Anything that a turret should be shooting at is important.

Additionally, even if we did one or all of these things, the per-frame cost of each turret would not be greatly reduced (fractions of milliseconds) and 15k+ turrets would still cripple servers.

Another truth is that in the worst case, servers were ticking at as low as 1 frame per second. That means that no matter what you're seeing in game, the server can only update the whole game state one frame per one second. That is the worst our servers have ever run, and is completely unplayable, in every way.

No matter who you are, I think you can agree that it's not enjoyable to play the game like that.

As such, we are left with a dillema:

We can reduce the number of turrets that people can place, and attempt to recover some of the performance loss that happens because of them, or we can not, or we can try and find some medium where we save as much performance as we can, while still allowing players to defend themselves.

On a server with 70 people playing on it, after reducing turret density, this is the new framerate on the server:

Total Duration 222.86 MS 4.5 FPS

Now, this is a worst case scenario, but the difference is absolutely clear to us, and one thing that we cannot under any circumstances forget, is the impact that servers running at 1 fps has on all of the players on the server. 4-5 fps is actually playable-- it can be a little bit choppy, but it's not infuriatingly awful.

Okay, so, if you're still with me, you understand what the raw numbers are that went into this, and I'll provide a to-now TLDR:

A certain method of building bases with turrets caused server fps to plummet to historic lows. The culprit is undeniably turrets: in some cases costing between 400 and 500ms per frame on the server tick. Our turrets are already extremely well optimized, but players are placing tens of thousands of them. That's too many. Our changes showed a change of 1.1 -> 4.5 fps. Reducing their numbers was now not a question of "if" but of "how much".

So what do we actually do?

Well, we try to solve it. And we try to talk about how best to solve it. Are there going to be some edge cases we have to deal with? Sure. Will we maybe have to adjust the balance of some things? Probably. We're letting you know weeks in advance as part of a method of opening up that communication channel. So that you can make decisions about what to do well ahead of time. Crunch numbers. Adjust strategies. Yell at us, and let us know what some of the things that are important to you guys are that we might not be thinking of-- hundreds of thousands of people play this game in all sorts of different ways every single day, we simply can't know what all of those ways are.

Now you know the technical issues we face. There is no other place to "save" 500ms of server tick time and let you just keep turrets how they are. There's no way to make turrets faster in any meaningful way, without fundamentally changing turrets and how they work.

As an important note, this change will also make all of the turrets you DO have much more accurate and reliable, because the low server fps made them inaccurate and miss often. Kind of a compounding problem.

https://gyazo.com/e307c4f0324ee5fc239862ace0365c38

This is a graph of the tick times of our worst servers, right off our backend. A delta time of 500 simply means that the server is locked at 2.5fps (it can actually run slower than that, but that's as low as our recording goes.)

PC servers are far, far worse off than console servers, due to their increased caps on everything, and each and every one of these servers suffers enormously from turret-induced slowdown. PvE doesn't even come close. Out of the top 150 slowest servers we have, only 15 of them in fact are PvE servers. This is a pvp issue, and it's really bad. There is no truth to the claim that PvE runs worse. It is a myth.

So let's talk about it. I am going to post this, and then I'm going to sit down and try to have dinner, and when I come back from dinner, I'm going to read more of what people have had to say. We want you to still be able to defend your bases and all of the stuff you've spent enormous amounts of time to make. We also want you to be able to play the game at higher than 1 fps.

- The Right Hand

 

@Jen @Jatheish @Jeremy Stieglitz @lilpanda 

I don't PVP but I appreciate you taking the time to share with the community the technical reasoning for these changes. Transparency is VERY important.

I just wish you guys would share these things more often (especially on the main website) across ALL social media for ALL major technical changes and adjustments so people are more informed and understand what goes on behind the scenes. It goes beyond just simple trimmed down patch notes, it goes beyond a community crunch.

If certain things cause lag and it is confirmed by developers then the community has a concrete reason to make changes that will benefit the servers they play on. I hate the current state of the servers just as much as the next guy, but the only reason I tolerate it is because I enjoy the game, I enjoy playing it with my friends and most of all, I have invested too much of my time into it to just up and leave. Many people I know and traded with on these forums have either quit the game completely or moved to private servers. Its a real shame and I hate to lose contact with good and friendly players I have rubbed shoulders with since legacy.

So to continue my point, if a dev says for example: "Behemoth gate walls around your base significantly contributes to server lag and decreases performance" This official confirmation can be spread from player to player to cut down on using these sorts of things, especially on PVE where server community is most important. Players are ignorant and like to debate among themselves thinking they know it all, but if I were to tell someone something about the game that decreases server stability/performance and link them to an official dev post backing up what I tell them to be fact; I guarantee that person would be more willing to cooperate. This simple communication adjustment will encourage change, build trust, and most important rebuild the community's confidence in WC that seems to be decreasing lately.

Thank you for the work you do and I hope the servers improve from these efforts.

 

[invincibleqc]

15 minutes ago, MrNomad said:

( would be actually very nice if you could give a more proper defintion of that "area" because i got no idea what it means)

A foundation is approximately 150 units larger so 10,000 / 150 = 67 foundations.

[ListofDeath]

If the turrets are only going to be set as 100 per 67 foundation radius, then why don't WC make an "Upgrade" system to the turrets. You would put the same mats that you would get for the turrets into an "Item" which you bring to the turrets and it upgrades them. Or if you bring the same mats to make a turret, to the turret you can choose the "Upgrade" option. The upgrade would increase the turret damage by 10% or 100% and make it stack so many times and put a cap to it?

I know it might take some time to produce, but it's just an idea.

[Dirg]

I have seen elsewhere it is 35? Where did you find this?

[natman]

3 minutes ago, Arksouls said:

And another point here its isnt that i care that i cant build the same bases its more the fact that if you are offline i know for a 100% fact that me and my alliance will be able to breach your base no matter the design no mater the bullets because you are locked to 100 turrets

Again, you are not locked to 100 turrets. Other than that, i dont doubt that raiders will have a too easy time, i said that myself in my first post in this topic and also made suggestions in order to balance the attackers possibilities vs the upcoming defensive possibilities.

[FIGHTY]

the Thing with this 10000 Units is the area is to big. you cant defend a big base with big outside area with this ammount of turrets guys. more dmg on the turrets is not helping if you cant place on every Corner turrets to defend all the parts of your base and your dinos.

The second Thing is this Change is to big for a released game !!! 

they should done it before in the ea time.

Wildcard you give us every week farming and breeding Events... For what ???? we spend a lot of time in the  base and the breeding and after some months you Change something big and we cant protect this anymore ?

makes no sense sorry.

People take sometimes hollydays from work or what ever so they just can Play a week more ark and you want destroy it with one click ? lol

 

[ausYenLoWang]

5 minutes ago, Dirg said:

I have seen elsewhere it is 35? Where did you find this?

35 radius, 67 diameter

, someones simplifying the radius should be 33.5

[ooStuStu]

8 minutes ago, Dirg said:

I have seen elsewhere it is 35? Where did you find this?

33.5 in each direction including up and down so on a x, y and z axis it is a 33.5 radius .... 67 foundation long if you go in a straight line

 

[natman]

3 minutes ago, FIGHTY said:

the Thing with this 10000 Units is the area is to big. you cant defend a big base with big outside area with this ammount of turrets guys. more dmg on the turrets is not helping if you cant place on every Corner turrets to defend all the parts of your base and your dinos.

The second Thing is this Change is to big for a released game !!!

Everybody in this thread has said at least once that bases will be very hard to defend with the nerf how it is currently planned...

And while you are right, it would have been better to do things like this while ark was still in EA, using that as a reason to not make significant changes when they are direly needed and instead letting the game go down can not be an alternative.

[Arksouls]

7 minutes ago, natman said:

Again, you are not locked to 100 turrets. Other than that, i dont doubt that raiders will have a too easy time, i said that myself in my first post in this topic and also made suggestions in order to balance the attackers possibilities vs the upcoming defensive possibilities.

 

IT IS CAPPED AT 100 TURRETS PER 32 FOUNDATIONS I DONT KNOW HOW YOU EXPECT TO BUILD A BIG BASE WITHIN THAT MATE

[MrNomad]

Is it possible to get an official confirmation that this "10000 Units" thingy is meant to be structures within a certain range. Because i still dont get it.

How many turrets in which area / radius are meant to be allowed ?

[Arkimedes]

With all the talk about "upgraded turrets" seeming to be the most suggested solution, I'd like to stress that will ONLY work if turret health is greatly increased.

How many of us have fired a lucky rocket while the soakers doing it's thing & squeaked a rocket through & hit the turret wall. Soon as it hits RIP your turret wall.

Taking the time/resources to "upgrade" a turret is only feasible if they can't be destroyed with 1 lucky rocket.

Even if you have 20 "upgraded" turrets on each side of your base, you better pray that guy with mad movement speed doesn't get that rocket off, or you will inevitably be dealing with one hell of a blind spot before your base defenses really get in the fight... & soon as a vulnerability opens... RIP your base.

& of course this applies more to offline raid situations, but let's be honest, we build these intricately defended bases in hopes it can hold off attackers till someone can log... more power to those with 24/7 coverage... but I have a job/life...

[invincibleqc]

9 minutes ago, Dirg said:

I have seen elsewhere it is 35? Where did you find this?

Foundations are approximately 150 units:

Turrets will be limited to 100 within 10k units:

Do the math based on that info and you get 67 rounded.

[natman]

1 minute ago, Arksouls said:

IT IS CAPPED AT 100 TURRETS PER 32 FOUNDATIONS I DONT KNOW HOW YOU EXPECT TO BUILD A BIG BASE WITHIN THAT MATE

It is capped at 100 turrets per 67 foundations if you build in a straight flat line. But you can still put another turret next to the last one in that line because it would be out of the67 foundation range of the first one in that line. You are not limited to 100 turrets and also not to 32 foundations.

[DarthaNyan]

1 hour ago, invincibleqc said:

A foundation is approximately 150 units larger so 10,000 / 150 = 67 foundations.

 

1 hour ago, Dirg said:

I have seen elsewhere it is 35? Where did you find this?

To clarify: 67 foundations is a diameter of a sphere with a turret in a center. So its 33-34 foundations radius around the turret.

 

@TheRightHand

If the "density" is the problem - wouldn't it be a better solution if you implemented a small no build zone (1-1.5 foundations wide) for other turrets around a turret?

Also give PsX an armor melting property (like the arthroplueras' spit have), and drastically increase rate of fire of arthroplueras and make them sturdier in turret mode because currently they are derpy and useless.

Reduce mate-boost and (max)rider bonuses to 25% damage bonus and 20% damage reduction. Yuty's defensive buff is way too powerful.

Make the beer's defensive buff to slow down the player: heavily intoxicated individuals are almost never the most agile ones even compared to sober themselves.

Edit: With the turret change i would also propose limiting tribe size to 10-15 players and alliance size to 3 tribes (yours and 2 others).

[Arksouls]

2 minutes ago, natman said:

It is capped at 100 turrets per 67 foundations if you build in a straight flat line. But you can still put another turret next to the last one in that line because it would be out of the67 foundation range of the first one in that line. You are not limited to 100 turrets and also not to 32 foundations.

 

well its 32 foundation in a circler raudius so i still stand by what ive said...

[Dirg]

I posted a couple pages back that if it isnt the turret itself that is the issue but the target scan it makes, group the turrets so they dont make redundant scans unless an adjacent scanning turret activates.

[ToyGod]

This video pretty much sums up on how easy it is to drain the turrets and you wont need to drain 100 turrets you will only need to drain one side of building. once inside you can have your way with the dinos and base. 

a few hours of raiding should not = wiping somebody's 6 months work 

The huge issue is how poorly designed building is in the game. Most areas of the map require you to use pillers and ceilings instead of a foundation resulting in more turrets needed to fill in blind spots. 

Why don't we have more options for building why have you not incorporated some of the mods out there into the game S+?

Offensive is way to easy. 

Character speed needs be nerfed and capped so that you cant sprint under turrets.

Turrets need to be able to fire in all directions. 

Nerf Stegos and Brontos. 

5 hours ago, s3aHaWKS said:

@TheRightHand since you guys clearly only care about what important people have to say... please watch both of these videos. You guys say that you listen to the community, but you clearly don't, so how about you actually listen to us for ONE DAMN TIME. Even more ironic, you guys don't listen to the people that are actually playing official PvP, because clearly you guys don't play it. 

 

 

 

 

[Arkimedes]

The suggestion made in another thread was a great idea, why don't all the wildcard devs get together & build a competative base compliant with this change on an official server. & that's just basic base/shell, not contemplating trying to secure an area to hold your 700ish dinos...

Then open that puppy up to let the community try to raid it. Those are some statistics id like to see... how long does that baby last?

I think if we all SAW a setup that was feasible, it would make us feel much better...

[J0hnnySw1f7]

The day that ark pvp died...