Turret Changes: A Technical Talk about why.

[Rofelo]

Please remove the dragons ability to carry larger dinos, mainly the stego.

One of the current metas is flying stegos witg riders and dropping them on the roof and having the stego rider throw C4 from the stego.

[Duckalot]

As someone who havnt played for a long while and just came across this dilemma in the forum. I just want to pop in a few (maybe) possible solutions.

1.) If reducing the number of turrets allowed, then the damage needs to be adjusted so players still can defend themselves. This, without making a Solo turret too op.
      What if you increase the individual damage with the amount of turrets placed. I dont know the exact damage values but bear with me.
      If you add 1 turret, it has 100dps. Now add a second turret and the damage of both those turrets increase by 1DMG.
      Turret + Turret = 2Turrets + 2dps.
      T + T + T = 3 T + 3dps

     And then just scale this so when you get to a hundred turrets, you still have the same defensive capability of lets say 1000+ turrets.

2.) As some people already suggested.  Make another type of turret with more DMG.

3.) Dont know how viable this is but just gonna put it here anyway. Increase Turret Damage VS unmanned tamed dinos. Sending useless animals to their death to tank turrets shouldnt be so viable, if you cant have as many turrets. Atleast in my opinion.

4.) More drastically, rebalance the dinos health overall. Now im sure i havnt thought of everything this will stirr up, but in the in the end, Dinos would become much more valuable. Which is personally something i would like to see.

 

[FIGHTY]

One big Problem with your turret Change is the ammount you Need for a big base. 

We have a 19 x 19 Base and its i think 30 walls high. 100 turrets is allready not enought for this Building. and it says 1300 turrets to much.

And arround this big mainbuilding we have a really big Area for alot of brontos rexes and all the other dinos.

If you look now at the turret message it says on every turret Tower arround the base (big area  i think 100 x 100) hundreds to much turrets. 

But you Need for a big base with big outside area the ammount of turrets and you still dont have every Corner safe for the outside dinos.

More damage is not helping on 100 turrets because the size of the bases.

if you want, come over and look self.

[natman]

First of all, thanks for reaching out to us with this important background information!

I think your decisions and additions of the past have led to the game becoming self destructive in respect to perfomance and gameplay (increased rates, key resources being way easier to get in the environment and especially through tames (beaver, snail, frog, penguin, dung beetle, quetz) and unlimited server transfers have led to the formation of huge server locking tribes/alliances with insane bases covered in turets from top to bottom). We have reached a point, where changes need to be made.

But the complaints are justified, the announced changes will make bases too easy to raid for what it takes to build them up and what they are meant to protect. The slot and damage increase wont make up for this. And i repeat what others here have said, if you make these changes, you have to make a rebalance of the related mechanis at the same time. Just increasing the turret damage is the wrong approach for these reasons:

4 hours ago, Drenith said:

The issue here is cost to damage ratio. If you increase the damage of a single turret by 5x to 7x you're making individual turrets way more powerful. This makes configurations that were previously under the 100 turret cap far more powerful than they were before.

I think it would benefit the game much more if the reasons why people built so many turrets were adjusted:

• lower dino hp
• more turret hp/different ammo types
• C4 cant be stuck onto dinos anymore and rockets cant be fired while riding a creature anymore
• the big mess of unbalanced bonuses, buffs and debuffs gets straightened (imprinting, mate boost, yuty, consumables, resistances...)
• tribe and alliance size gets limited
• maybe even limit transfers

[ausYenLoWang]

Natman:  lets not stuff around with base dino stats again. the retuning that woudl then be required of boss fights etc etc would be a total nightmare

different ammo types would be cool,

a setting on the turret to increase Rate of Fire between 1-5x increases dps and can make a turret count as more than 1

[aeNaty]

5 hours ago, TheRightHand said:

Hey guys, sure has been a crazy day hasn't it?

So, I know we have that megathread over there, and guess what my weekend gets to be (It's reading that and every other thread.) but I wanted to make a new post here just so that information was in a good, easy place to see.

Let's just get right into it:

This is some profiling output from a fun little tool we made, which analyzes the server and finds each thread that is running and times how long it takes for the server to tick. It does a lot more than that, but that's what you need to know to understand this:

Total Duration 765.60 MS 1.31 FPS

StructureTurretBaseBP_C :     7977 234.98 MS
StructureTurretPlant_C :     9722 119.4 MS

This is turrets. BaseBP is normal metal turrets, Plant is SpeciesX.

Now, with this information, you need to have some other information:

Since about October or so, we saw a MASSIVE spike in the size of stalls and poor performance on our dedicated servers, but only some of them, most of them Ragnarok Servers.

So we investigated, and we found that an interesting change in the meta for base building had occured: People were making new, super-dense bases with insane turret counts, far beyond anything we'd ever seen before. The bases of these types were on all of our servers, but primarily belonged to Alpha Tribes who had set up bases on the Ragnarok servers. Similar setups caused equally bad performance on Center and Island servers in the rare cases we found them.

I want you to understand what all of this means to us, so I'll try to be brief.

Turrets, were very suddenly being used in a configuration that was atypical, and up to 7x the number of turrets we would see on any other server, even if it was a highly populated, well trafficked server. There were definitely other cases of densely packed bases, but this was higher by many magnitudes.

Let me emphasize now, that for the last two years, turrets have been a pain in our butt, but we have not encountered people making monstrosities like this. Every alpha tribe in the game has existed just fine, and gotten along well enough without these insanely dense crazy bases.

So, we looked at what we could do about it. Let me wrap this all up in context, these things I'm going to put here are facts:

Our turrets are highly optimized for what they do with respect to gameplay. They use a very fast bitmasked octree overlap to return all of specific types of actors (excluding irrelivant actors) in the aoe that they defend, and then act according to the settings you configure, early-outing from any extra calculations or wasted instructions if their shot is invalid. We could make that area they search smaller, which would be faster, but then they would need correspondingly smaller ranges.

We could make them search for less things in their overlap check, but then you'd have turrets that didn't fire on some things.

We could make them acquire targets less frequently, but then they'd be much less accurate and in many cases would not pick up things like rocket projectiles that entered their range between scans.

We could make them scan at different ranges for different things, but then their functionality would be weirdly ambiguous and what things would we reduce scan range for anyways? Anything that a turret should be shooting at is important.

Additionally, even if we did one or all of these things, the per-frame cost of each turret would not be greatly reduced (fractions of milliseconds) and 15k+ turrets would still cripple servers.

Another truth is that in the worst case, servers were ticking at as low as 1 frame per second. That means that no matter what you're seeing in game, the server can only update the whole game state one frame per one second. That is the worst our servers have ever run, and is completely unplayable, in every way.

No matter who you are, I think you can agree that it's not enjoyable to play the game like that.

As such, we are left with a dillema:

We can reduce the number of turrets that people can place, and attempt to recover some of the performance loss that happens because of them, or we can not, or we can try and find some medium where we save as much performance as we can, while still allowing players to defend themselves.

On a server with 70 people playing on it, after reducing turret density, this is the new framerate on the server:

Total Duration 222.86 MS 4.5 FPS

Now, this is a worst case scenario, but the difference is absolutely clear to us, and one thing that we cannot under any circumstances forget, is the impact that servers running at 1 fps has on all of the players on the server. 4-5 fps is actually playable-- it can be a little bit choppy, but it's not infuriatingly awful.

Okay, so, if you're still with me, you understand what the raw numbers are that went into this, and I'll provide a to-now TLDR:

A certain method of building bases with turrets caused server fps to plummet to historic lows. The culprit is undeniably turrets: in some cases costing between 400 and 500ms per frame on the server tick. Our turrets are already extremely well optimized, but players are placing tens of thousands of them. That's too many. Our changes showed a change of 1.1 -> 4.5 fps. Reducing their numbers was now not a question of "if" but of "how much".

So what do we actually do?

Well, we try to solve it. And we try to talk about how best to solve it. Are there going to be some edge cases we have to deal with? Sure. Will we maybe have to adjust the balance of some things? Probably. We're letting you know weeks in advance as part of a method of opening up that communication channel. So that you can make decisions about what to do well ahead of time. Crunch numbers. Adjust strategies. Yell at us, and let us know what some of the things that are important to you guys are that we might not be thinking of-- hundreds of thousands of people play this game in all sorts of different ways every single day, we simply can't know what all of those ways are.

Now you know the technical issues we face. There is no other place to "save" 500ms of server tick time and let you just keep turrets how they are. There's no way to make turrets faster in any meaningful way, without fundamentally changing turrets and how they work.

As an important note, this change will also make all of the turrets you DO have much more accurate and reliable, because the low server fps made them inaccurate and miss often. Kind of a compounding problem.

https://gyazo.com/e307c4f0324ee5fc239862ace0365c38

This is a graph of the tick times of our worst servers, right off our backend. A delta time of 500 simply means that the server is locked at 2.5fps (it can actually run slower than that, but that's as low as our recording goes.)

PC servers are far, far worse off than console servers, due to their increased caps on everything, and each and every one of these servers suffers enormously from turret-induced slowdown. PvE doesn't even come close. Out of the top 150 slowest servers we have, only 15 of them in fact are PvE servers. This is a pvp issue, and it's really bad. There is no truth to the claim that PvE runs worse. It is a myth.

So let's talk about it. I am going to post this, and then I'm going to sit down and try to have dinner, and when I come back from dinner, I'm going to read more of what people have had to say. We want you to still be able to defend your bases and all of the stuff you've spent enormous amounts of time to make. We also want you to be able to play the game at higher than 1 fps.

- The Right Hand

If you are to increase turret damage you will have to make changes to armor so you dont get completely wrecked by 1 turret on foot and then changes to armor for saddles so on and so on, either way to fix this problem it wont be a quick easy fix there will be many things that will have impact as there has to be some way to raid a base after to all its PVP not PVE, and please if you are to further change things for the sake of pvp make sure its a change you would be happy to play with.

Also another suggestion could you try to make pvp more fun and not so much grinding as in the end of the day its a game not life, no normal person with a fulltime job can take 11 days to raise a giga

[natman]

23 minutes ago, ausYenLoWang said:

Natman:  lets not stuff around with base dino stats again. the retuning that woudl then be required of boss fights etc etc would be a total nightmare

different ammo types would be cool,

a setting on the turret to increase Rate of Fire between 1-5x increases dps and can make a turret count as more than 1

Yes, reworking dinos hp and also the buffs and debuffs is a big thing, but i feel like it is needed. They should grab the problem by its roots instead of releasing another band aid. Also i think reworking dino stats will be actually part of the upcoming dino tlc pass.

Increasing rate of fire would increase lag as well i guess. I would rather go with @invincibleqc´s suggestion of being able to configure the power of a turret.

[Beatzzzz]

HEY I HAVE A GREAT IDEA   LETS WASTE MONTHS OF PPLS LIFE AND LET EM HVE 100 TURRETS #RIPMEGASANDALPHAS

[invincibleqc]

12 minutes ago, natman said:

Yes, reworking dinos hp and also the buffs and debuffs is a big thing, but i feel like it is needed. They should grab the problem by its roots instead of releasing another band aid.

Increasing rate of fire would increase lag as well i guess. I would rather go with @invincibleqc´s suggestion of being able to configure the power of a turret.

Yes, it would most likely ends up being the same. If a turret need to do the calculations more often because it shoots faster, this result in the same code running more often as well which negates the need to limit the turrets or at least makes it useless. Based on the same idea, the turret could "upgrade" using the materials you have in your inventory (much like when you repair something) and it increases in HP, consumes more bullets per shot and deals more damage. That way you still have to "pay" for the upgrade while also benefiting the HP increase.

4 minutes ago, Beatzzzz said:

Ha lol there turning the game to SH\IT

  there gonna notice when there player count drops #STOPWILDCARD id rather have lag and more turrets then 100turrets and LAG

So you would prefer watching a movie on a 60 inches tv that blinks over a 32 inches that works perfectly? Makes sense.

3 minutes ago, Beatzzzz said:

HEY I HAVE A GREAT IDEA   LETS WASTE MONTHS OF PPLS LIFE AND LET EM HVE 100 TURRETS #RIPMEGASANDALPHAS

HEY I HAVE A GREAT IDEA TOO... LETS SHARE OUR SOLUTIONS/IDEAS SO WE CAN HAVE A BALANCED GAME-PLAY WHILE ALSO PROFITING FROM BETTER PERFORMANCE.

[HiImZech]

bruh I think its a good implementation, only a few years too late. if it was like that in the beginning people won't be complaining about it so much.

 

if there's anything else that might cause the server to lag, they eventually WILL cause the server to lag unless a build limit is implemented.

since there is no control, people will spam it as much as they can. unless there is control, there will always be people doing what is not intended for the game to do.

 

for example. someone make a card game on an app. that company starts a tourney on that app. since there is no control, people are going to play their cards at the very last second, causing each round to last as long as 3 mins just so they can get those impatient opponents to ragequit or disconnect from unstable connection.

it may be "rare", but a chance is a chance, no matter how slim the occurrence may be. people have a higher chance to die in a traffic accident than getting bitten by a shark. though true, that does not mean no one gets bitten by a shark.

 

it will be wise imo to look for more of these "rare occurrences" and solve them quickly before someone abuses them

[Beatzzzz]

4 minutes ago, invincibleqc said:

Yes, it would most likely ends up being the same. If a turret need to do the calculations more often because it shoots faster, this result in the same code running more often as well which negates the need to limit the turrets or at least makes it useless. Based on the same idea, the turret could "upgrade" using the materials you have in your inventory (much like when you repair something) and it increases in HP, consumes more bullets per shot and deals more damage. That way you still have to "pay" for the upgrade while also benefiting the HP increase.

So you would prefer watching a movie on a 60 inches tv that blinks over a 32 inches that works perfectly? Makes sense.

HEY I HAVE A GREAT IDEA TOO... LETS SHARE OUR SOLUTIONS/IDEAS SO WE CAN HAVE A BALANCED GAME-PLAY WHILE ALSO PROFITING FROM BETTER PERFORMANCE.

No I would rather live in a base with deffence where 1 stego cant raid it and my tribe just finished a death wall with 1k turrets

[Dirg]

The issue is each turret scans for itself. This increases accuracy of target acquisition  and performance. So instead of running a scan for every turret let the servers optimize by bundling groups of turrets into a single scan. How this works? The server(or players) set turrets in a group when placed and the turrets on the outside edges of this group are all that scan in most instances. We all know that 99% of the time turrets are idle. If the scanning turrets detect something then ALL turrets scan until a period of time passes with no detection and they go back to passive.

Turrets only work en mass. 100 turrets wont defend anyone, I personally figure this is a shock value number you have trotted out to panic people and then you will relent and say "ok 400" or something to this affect hoping people will accept this in relief. I do not deny a problem but I challenge Wildcard to place 100 species X on a 10x10 wall then attack it with a 20k health 50%imprinted giga in a 50 armor saddle, repeat it with a mated pair of the same. Feel free to post the resulting video and your thoughts on your "fix".

[thedattrigga]

I really hope that you guys rethink this, it will genuinely ruin the game, this would mean that i could raid nearly any base with a couple of stegos, before you guys do this, stop the swaying of dino’s and if that doesn’t fix the lag issue then i think this would be a decent patch, but not this low, in my opinion, i don’t think something as small as a turret lags a game too much, as i used to be server 10 alpha and currently play on a ragnarok i believe it is the servers, but if you could please test out by stopping the swaying and then if that doesn’t work, change the turret limit in 30 foundations to 200, if that doesn’t work 150, if that doesn’t work, you ultimately switch it to 100, but if after all of this nothing works, then you can really see that it’s the servers and not a turret issue, i really think that the sway will fix most of the lag, thank you for your time

[Zederia]

3 minutes ago, thedattrigga said:

before you guys do this, stop the swaying of dino’s and if that doesn’t fix the lag issue then i think this would be a decent patch,

1

did you read? Animations are done client side, not server side. animations have ZERO impact on server performance

[Dirg]

8 minutes ago, Zederia said:

did you read? Animations are done client side, not server side. animations have ZERO impact on server performance

This begs the question why i dont have a box to turn that crap off! =)

 

[eViLxGoYaBeans]

Why don’t ya get rid of Dino tail swinging and those types of animation to help reduce lag and base lag also 

[Pendrodor]

THE BRONTO TAIL SWAY HAS NOTHING TO DO WITH SERVER PERFORMANCE ITS CLIENTSIDE 

[Pendrodor]

but honestly if they wanna implement something like this they should do a voting on it after all we play the game not them 

[wimthetim]

Read the comments. Do the decent thing and don't go ahead with the change as proposed. 100 turrets of all types is not enough. Increase the number to 200 - 300

That would still be a considerable reduction when 1000+ can fit into the designated space.

[invincibleqc]

Just now, Pendrodor said:

but honestly if they wanna implement something like this they should do a voting on it after all we play the game not them 

Votes are useless because this is a vicious circle. People complains about lags, rubber-banding, crashes, timeout, etc. Wildcard investigate and finds out that the turrets are mainly to blame. People then cries because they will lose their turrets. In conclusion, they are never happy. And I'm baffled about players attitude. I mean, they are constantly complaining "the devs doesn't communicate, our opinions doesn't matters, they are not transparent, etc.". But when they tell us their plan, and ask for our opinion and give us the chance to share our concerns and what also need to be addressed for this to be viable players are just raging and signing ridiculous petition... oh well.

[ooStuStu]

I am seeing the claim that bear traps, and plant Y traps are counted in the 100 turret limit ... can someone of authority please clear this up for the window lickers? @TheRightHand

[Arksouls]

How do you balance the amount of time and effort ive put in on my 1000 turret base to some small tribe with 100 turrets how is it fair that we have the same defense when offline... You have basicly added a cap on how good your base can really be... before it was only structure cap stopping you but structure cap was pretty high so if you were smart you could get around it, but now effectively everyone will have the same 100 turrets regardless of playtime which will make some bases impossible to be built on pvp some bases require more then 100 turrets id even say most bases require more then 100 turrets. Please atleast add some test servers to see how this turret update plays before you ruin all of our HARD WORK. And maybe look at turret damage in most bases having hundereds of turrets was more of a strategy to kill tankers very fast without turrets depleting too many bullets because you dont want to be running around filling up turrets during a defense, but even with more bullets it will still be far eaiser for tribes to just throw 20 brontos at your base and have people run up and c4 in... 

Like no offense to wildcard, I love your game but if you actually think this is going to help your pvp player base then your very very disconeted from your pvp community... 

I 100% believe that with the current changes mentioned, PVP servers will be over... 

I would litterally rather play in a 15 frame per second server then play on a server with capped turrets...

[MrNomad]

As posted in the steam forum before :

Hey Wildcard, even if i doubt that anyone of the devs is going to read this :

I want to give you a slight guess what the turret change will cause :  First of all , you can imagine that quite some tribes and players do the math in before, how many turrets they will need in order to keep away certain griefers or raiders. 

For example, i got (on official) 300 turrets in an area ( would be actually very nice if you could give a more proper defintion of that "area" because i got no idea what it means). In my case they are all in a range of like 25 FOundations radius. Like 180 Autoturrets, the rest is plant X.

I calculated what the outcoming dps has to be, to lets say keep away griefers with 5 brontos, and small stuff, when i am online to defend. Of course in an offline raid that wont help. So now your new patch is telling me, that as of 5th of december, 200 of those turrets will stop working. because they are in that imaginary area of effect. For me its like i either gotta give up that base, and build somewhere smaller, which means i cannot house like half of my tames anymore.

In addition now i am an even much easier target for smaller group of griefers, because each side of the base will be left with like 25 turrets, which is nothing these days in ark. They will have an easy task raiding me, killing my tames, and so on. For tribes that have people online 24/7 to defend its also more difficult, but for singleplayers its a mess. Like this you are more and more scaring casual players away from the game, those, who cannot be online 24/7, because of that crazy poop called life, work, and family.

And quite frankly, i have to say, that there are way more options to improve the game running more smoothly , rather than to cut structures out. There are several games which consist also a lot of high resolution structures to calculate, and they run far better at higher FPS. My impression is that the game is still calculating the structures 1 by 1... ( you will notice what i mean when you get closer to a large base from the distance), rather than putting them together in groups...

And last but not least : Increasing the turret damage by 20% wont help... not at all. 
As of the example given from my base, to keep the same level of defense, you would have to raise the damage by 200% , Auto- Tek and plant x turrets.

[y0himba]

Players could try building smaller bases.  Reduce the size of objects contained in a base like the Tek stuff, the fabricator and so forth to allow for more compact bases and try not taking up so much land that isn't even used. Huge bases are just for show, degrade the game, and lead to these insane amounts of turrets, which again, are used for showing off.

This reduction in size would not only require less turrets to defend, but would use less resources and would help make bases easier to defend.

 

[Tanek]

15 minutes ago, invincibleqc said:

Votes are useless because this is a vicious circle. People complains about lags, rubber-banding, crashes, timeout, etc. Wildcard investigate and finds out that the turrets are mainly to blame. People then cries because they will lose their turrets. In conclusion, they are never happy. And I'm baffled about players attitude. I mean, they are constantly complaining "the devs doesn't communicate, our opinions doesn't matters, they are not transparent, etc.". But when they tell us their plan, and ask for our opinion and give us the chance to share our concerns and what also need to be addressed for this to be viable players are just raging and signing ridiculous petition... oh well.

I think part of it might be that, over time, the fear is that initial fixes are done with a chainsaw rather than a scalpel, and the mess does not get cleaned up later. 

For example, the dino cap.  Was something like that needed to prevent servers being crushed under the weight of tames?  Probably, yes. That seemed to be the biggest server performance issue of the time based on what Wildcard was saying, and I had little reason to doubt them (on our server, big structures caused (and still do cause) more issues than dinos, but I can also see where thousands of tames on a server can be a problem).

But was it the only solution? The best solution? Or just the fastest solution? There are ideas out there to tackle one of the biggest reasons to tame a few boatloads of dinos, but we have not seen movement on it yet. Kibble farms. How many tribes have a good % of tames for no other reason than eggs? Might a kibble redesign take some of the pressure off servers?

This time Wildcard has a buffer of a couple weeks where people can talk this out. I am interested in seeing the result. Will the "nerf" just go in as planned? will there be other ideas? Tweaks? Maybe. There has to be something, right? If this turret issue really took off just last month, how were people defending bases before? Why was the game not 'broken and impossible' back then for tribes that kept turret counts reasonable? There is some factor I seem to be missing here.