Cluster Servers Randomly Adding PW & Changing to PvP - Please Help!

[Foximus]

Good day illustrious Ark Community,

We have been operating an Ark Cluster for just over a year now and have had two different experiences. The first wipe period, I created all the servers manually using .bat files and did not use ASM. For the entirety of this time period, there was no issue with the following problem I'm going to describe.

After wiping (a couple months ago), our servers will randomly change from PVE to PVP and Password Protect themselves with no Password actually required. This time around, I used ASM to initially create the servers, but after having these problems tried to re-install Windows and do it with .bat files. It has been many hours of dissecting my API Plugins, Configs, Firewall Settings, and ultimately continue to suffer this issue.

When this happens, the server will no longer show on the Cluster, but will still show on the main server list. Our players have been transferring to any server still showing on the cluster, logging out, then re-joining the intended destination server to download their character in. After a couple days, eventually all servers will end up in this state and players cannot transfer anywhere until I restart them.

This has been an exceptional headache for my team and would appreciate any high level insight of why this could be happening. I'm planning on another reformat hoping I can restore everything as it is while fixing the problem. The only "funny" thing I have seen is that bAutoPvETimer forces itself on the config. I did not use this setting the first time around, but have since been fighting to keep it as bAutoPvETimer=False. The configs will force bAutoPvETimer=True above my =False entry?

Any constructive criticism on why this would be happening would be greatly appreciated. 
Thank you for the support.

~FoX

[Larkfields]

-nowrite   on the command line will prevent the config files from being overwritten.  

bAutoPvETimer allows automatically switching between PVP and PVE.  See https://ark.fandom.com/wiki/Server_configuration
ServerPVE=true  should keep you in PVE.

Is it possible that someone else is editing your setup?

[Foximus]

Thank you @Larkfields for the insight! I'm going to restart our servers this evening with the changes you have mentioned.

Our prior wipe I just had ServerPVE=True, omitted bAutoPvETimer, and did not have any problem.

There should not be anyone else editing the configs.

I will follow up with my results of the -nowrite command, thank you again!

[Foximus]

- Added -nowrite to the command lines
- Verified ServerPVE=True
- Verified bAutoPvETimer is not on the configs

Already, two of the Seven servers turned PVP, one of which now has a password .

[Larkfields]

Wierd !
Are you stopping the server before you edit the files.  The server ignores any changes you make while it is running.

[Foximus]

Indeed! Worldsave, doexit, change configs, and reboot .

I have been unable to find a solution and it's really odd this didn't happen the first time to me! Only difference was a brief use of ASM.

[invincibleqc]

On 2/20/2022 at 12:55 PM, Foximus said:

our servers will randomly change from PVE to PVP and Password Protect themselves with no Password actually required

This sounds like a query issue to me. Though, this would not disables PvP but they would be displayed as such into the server list because by default, all servers show as PvP unless "SESSIONISPVE" is set, have an empty "SERVERPASSWORD", as well as an empty "CLUSTERID" (which would make them not listed into a transmitter because the queried ID does not match, etc). If the response clients receive times out, or is invalid/incomplete, this would results in most of the issues you are describing. I would suggest you ensure your ports are properly configured, that your proxy/firewall is not blocking queries, and that you monitor your network activity for any relevant logs.

[Larkfields]

I'm not familiar with ASM.  Could it have left a background process running that is still trying to maintain your server?

Just seen Invincible's post...  He's more experienced than I am, but I'm puzzled by the fact that Foximus's server was running for a year before ASM was introduced.

[Foximus]

Thank you for the further replies @Larkfields. I too am surprised the issues started with an ASM attempt and with the config's changing I thought there was something behind the scenes in default config I could have been missing this whole time.

 

Many thanks for joining the conversation and input @invincibleqc . Reviewing your post, my GameServerSettings has ServerPVE=True, ServerPassword= , and the clusterid is set in the launch .bat file. I'm confused because our first wipe the password was blank as well. I'm going to go back into the Firewall settings and look there. I didn't do anything differently between the two wipes but with the Windows Reformat I certainly could have missed something. I'm worried it's more networking related as you pointed out regarding the query timeout.

[invincibleqc]

22 minutes ago, Foximus said:

Reviewing your post, my GameServerSettings has ServerPVE=True, ServerPassword= , and the clusterid is set in the launch .bat file. I'm confused because our first wipe the password was blank as well.

Your configuration has nothing to do with it, and in fact, your server is likely no set to PvP (e.g. you couldn't actually damage other players in-game, etc.). That is, if my assumptions are right, which I believe them to be based on the symptoms you are reporting. Basically, it's the response (or lack-of) received that is interpreted as if the server is PvP, has a password (although any password would be considered valid because the server doesn't actually requires a password to join), and that its cluster ID is empty and/or not present whatsoever, don't have their BE icon for secured servers, etc. In other words; this only affects what clients are listing in their server browser.

[Foximus]

45 minutes ago, invincibleqc said:

Your configuration has nothing to do with it, and in fact, your server is likely no set to PvP (e.g. you couldn't actually damage other players in-game, etc.). That is, if my assumptions are right, which I believe them to be based on the symptoms you are reporting. Basically, it's the response (or lack-of) received that is interpreted as if the server is PvP, has a password (although any password would be considered valid because the server doesn't actually requires a password to join), and that its cluster ID is empty and/or not present whatsoever, don't have their BE icon for secured servers, etc. In other words; this only affects what clients are listing in their server browser.

Thank you again for the quick reply. What exactly will happen is that all servers will be just fine after restarting them all. When transferring between servers a few times, they then begin to no longer appear on the cluster list.  Which servers vanish does not always correspond with joining a specific one as, over time, they will all eventually be gone whether transferring there or not. If you look at the public server list and not through the cluster, you can see the servers are marked PVP and Password Protected. PVP and Password protection are not paired and right now my server list has two servers marked PVP with no password but 100% are not PVP. Those two are the ones which will not show on the cluster transfers.

Is there a possibility of a networking hiccup or other situation where the server will erroneously show a different setting to clients? As you pointed out regarding a password, you can still join those servers without a password by entering through the prompt. You're correct as well that when you join through that "password" that is not existent on the incorrectly marked PVP server, it is actually not PVP. If players can connect after restart and see all the servers for a period of time, could that really be my networking settings (I truly don't know)? Could it be something with SQL... though I again followed the same installation path as the first time?

To be honest, when we wiped ... my biggest fear was something like this going wrong! I used almost the same configuration settings only had ASM drive the original config. After having an issue where EPIC players could not connect (first wipe they could - suddenly this time they cannot), I thought it was ASM and reinstalled windows again to start over. Manually loaded all the files through .bat's and the issue we're discussing remains. We have moved on from Epic players connecting (which is disheartening), but am at wit's end with the server issue at hand . Looking through the Firewall information, I used screenshots from the first time to config the settings. Sadly, I don't see anything obvious that would cause a problem.

**Edit - I don't know if this is a lead but my command line is as follows -ClusterDirOverride=Z:\CLUSTER\CLUSTERS -clusterid=XXXXXXXXXX. When looking on the server, the path is Z:\CLUSTER\clusters\clusters\XXXXXXXXXX. Could it have something to do with that double folder of \clusters\clusters? The directory I provided is where daily activity occurs but not in the specific one on the command line.

 

[Foximus]

@invincibleqc - Do you believe this sounds network related? I would love to get in touch somehow to review if you have an idea of what could be the root cause. After several day's of continued analysis I still find myself struggling to find a solution.

[invincibleqc]

13 hours ago, Foximus said:

@invincibleqc - Do you believe this sounds network related? I would love to get in touch somehow to review if you have an idea of what could be the root cause. After several day's of continued analysis I still find myself struggling to find a solution.

It's definitely a query issue. It being network related is hard to tell but that would be my first guess. Either that, or the parallel task stop running for some reasons. First thing you should do is monitor your traffic and ensure that;

• Queries are being received (from memory, they should have a "Source Engine" header).
• Responses are being sent (the datagram should contains k,v pairs with extra data -- specifically what is not sent to clients through VMS -- such as "SERVERPASSWORD', "SESSIONISPVE", etc.).

If queries are not received; check your inbound rules to make sure they are not blocked or rate limited. If queries are received but not responded to; check your outbound rules. If everything seems fine at a network level, and since you seem to have trouble with EGS based on your previous comment, you could try to completely disable the EOS subsystem by setting the following to your servers:

# ../Engine.ini

[OnlineSubsystem]
DefaultPlatformService=Steam

 

[Foximus]

Thank you for the Reply, I will advise in a few day's after further review!