Linux server woes

[Theheadslacker]

I'm at the end of my rope, and I've given up on finding the answer with search engines, so I've signed up on the forum to maybe get answers directly from somebody who knows what they're doing.

I've started trying to run a Linux dedicated server on a spare box in my basement. It worked fine vanilla, but as soon as I tried adding mods I started segfaulting (signal 11) immediately at launch.

So I dug around, found tutorials, fixes, workarounds, etc. I've followed just about any advice I could find. I unpacked the steamcmd content into the Engine file tree. I pared down mod choices until I was down to structures+ which is really the only must-have mod imo.

When I couldn't even get that to work, I tried uninstalling all mods, removing references to them from config files, and removing automanagedmods from my run script (which I had already tried on its own after manually scp-ing mod files over from my desktop), and now the server still won't even run vanilla.

It looks like it starts normally, but ram usage slowly creeps up to around 1.5G before I get another signal 11. I tried running a different map, and the server ate up to 4G of RAM before faulting again, but subsequent tries also ran into the same pattern of hanging around 1.5G. I'm assuming the extra RAM was just from generating a new ark or something.

I've tried removing server files so a new world would be generated, in case the .ark file was corrupted. I've renamed my SavedArks folder and made a new one so the game could repopulate it, characters and all. I've deleted the server binary and run a validation so the game would reinstall missing files and folders.

Is it time to just wipe the installation and start over? Is there a recently-updated list of instructions for running a Linux server with mods? I can play vanilla if needed, but I don't see why the exact same settings and mods that work single player can't work on a server.

When I get home in the morning I can provide any logs, settings, scripts, etc. I'm not at home now though, and I can't ssh in to the box from outside my LAN.

[Theheadslacker]

Also, what's the logging situation in ark? I'm not getting much useful info even with logging turned on in my startup script.

I might be able to figure out for myself what the problem is, if I had more to go on than "oopsies, server borked!"

[Larkfields]

Try starting a new vanilla map to see if that works.
Maps can use around 8GB RAM when running, so 4GB isn't unusual.  How much RAM do you have on your server - and do you have Virtual RAM enabled (it should be on by default)?

Also: what distro are you using?

Some people say a Sig 11  can be caused by faulty RAM - so it's probably worth running a memory test.
This program is free... https://www.memtest86.com/download.htm

[Theheadslacker]

On 9/23/2020 at 3:07 AM, Larkfields said:

Try starting a new vanilla map to see if that works.
Maps can use around 8GB RAM when running, so 4GB isn't unusual.  How much RAM do you have on your server - and do you have Virtual RAM enabled (it should be on by default)?

Also: what distro are you using?

Some people say a Sig 11  can be caused by faulty RAM - so it's probably worth running a memory test.
This program is free... https://www.memtest86.com/download.htm

Vanilla map works after a reinstall. I'm using almost 7GiB and another 8GiB as buffer (crystal isles) once it's visible to join over LAN.

I'm not sure what you mean about virtual RAM. It's not a VM but a dedicated machine running in my basement, but it has 16GB of memory and I think I have it set for 32GB of swap space.

Distro is Ubuntu server, since that's what the wiki seemed to recommend. Which is fine with me because it's a familiar distro.

It turns out I did indeed have some faulty RAM, but I've replaced with two fresh sticks and am still faulting out immediately as soon as try to run mods.

Thanks for taking the time to help, at least. I'm sure replacing the bad RAM is pushing me closer, and I'd have never thought to check on my own since this is the first time (I'm aware of) that I've had a stick of memory fail.

[Larkfields]

The faulty RAM could have caused the save file to become corrupt - in which case you'll need to find a "good" save from a backup.
It could also have caused the Ark server software to be installed badly, so it'll be worth validating or reinstalling the Ark Server.

"Virtual RAM" = swap space.

[Theheadslacker]

Yeah I've already deleted the entire folder and reinstalled from scratch.

Looks like I'll just be playing without those sweet, sweet auto-opening doors

[Theheadslacker]

After more poking and prodding, my gut tells me this is either a problem inherent to Ark unpacking mods on linux, or it's some kind of issue with Ark and SteamCMD interfacing each other. Swapping which mod i'm trying to use doesn't seem to have any impact; as soon as i include (most) available avenues for including mods, the game faults immediately on start with Signal=11. Sometimes it will attempt to start for a minute or two and then fault out with the same Signal=11. I'll get various other errors involving F:\ drive, which i assume is some kind of workaround the devs have put in under the hood to rig up cross compatibility.

If i run my server via command line with the GameModIds option, it seems to have no impact on its own. The server will start successfully, but i don't see any effects of any mods. More on this below.

If i run my server via command line with the -automanagedmods flag (with or without GameModIds or any other Game.ini or GameUserSettings.ini permutations) the server instantly crashes.

If i run my server with any configuration including [ModInstaller] in the Game.ini file the server instantly crashes.

If i run my server with some configurations that include an ActiveMods list in GameUserSettings.ini file the server starts to run but crashes after a couple of minutes. Watching RAM usage in another terminal, i can see it get up to about 3Gb before the server faults out.

When using solely GameModIds in the shell command, if the listed mods are not in the game folders, then there's no change to the game. If i manually copy over the mod (folder and .mod file) from my desktop to the server's ShooterGame/Content/Mods/ directory then the mod appears to load into the game in a non-functional manner. The server list shows the server as a modded server, and i get the "loading mods" step when joining, but none of the mods actually function. For example, i've gotten structures plus to "load" and its engrams show up in my engram list, but none of them can be learned. Moby-sausage-less still has leeds smashing my rafts, etc.

[arkark]

47 minutes ago, Theheadslacker said:

After more poking and prodding, my gut tells me this is either a problem inherent to Ark unpacking mods on linux, or it's some kind of issue with Ark and SteamCMD interfacing each other. Swapping which mod i'm trying to use doesn't seem to have any impact; as soon as i include (most) available avenues for including mods, the game faults immediately on start with Signal=11. Sometimes it will attempt to start for a minute or two and then fault out with the same Signal=11. I'll get various other errors involving F:\ drive, which i assume is some kind of workaround the devs have put in under the hood to rig up cross compatibility.

If i run my server via command line with the GameModIds option, it seems to have no impact on its own. The server will start successfully, but i don't see any effects of any mods. More on this below.

If i run my server via command line with the -automanagedmods flag (with or without GameModIds or any other Game.ini or GameUserSettings.ini permutations) the server instantly crashes.

If i run my server with any configuration including [ModInstaller] in the Game.ini file the server instantly crashes.

If i run my server with some configurations that include an ActiveMods list in GameUserSettings.ini file the server starts to run but crashes after a couple of minutes. Watching RAM usage in another terminal, i can see it get up to about 3Gb before the server faults out.

When using solely GameModIds in the shell command, if the listed mods are not in the game folders, then there's no change to the game. If i manually copy over the mod (folder and .mod file) from my desktop to the server's ShooterGame/Content/Mods/ directory then the mod appears to load into the game in a non-functional manner. The server list shows the server as a modded server, and i get the "loading mods" step when joining, but none of the mods actually function. For example, i've gotten structures plus to "load" and its engrams show up in my engram list, but none of them can be learned. Moby-sausage-less still has leeds smashing my rafts, etc.

Signal 11 is SIGSEGV, and in this case its probably being issued by the kernel due to a memory violation of the ark process. Try this, execute the server as you normally do to trigger the segfault and immediately check the system log via /var/log/syslog or dmesg. See if there is any relevant message and paste it.

[Barbarello]

@Theheadslacker Have you made any progress?

I had a modded Linux server (S+) working from January to September. But, since then, I haven't been able to get any mod to work. The vanilla server works and I can play my saved games, but with the modded features missing (like S+ doors and wall missing in buildings). I've tried reinstalling the server, the mods, and so on, but with mods the server keeps crashing.

I tested with two mods, 1404697612 and 731604991 (S+). Neither work alone and they don't work together.

systemctl status ark:

$ sudo systemctl status ark

● ark.service - ARK Survival Evolved
   Loaded: loaded (/lib/systemd/system/ark.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: core-dump) since Fri 2020-10-23 20:47:21 EDT; 3s ago
  Process: 6875 ExecStop=/bin/killall -TERM srcds_linux (code=exited, status=203/EXEC)
  Process: 6957 ExecStart=/home/ark/server/ShooterGame/Binaries/Linux/ShooterGameServer Ragnarok?listen?SessionName=XXX?ShowFloatingDamageText=true?Port=7777?QueryPort=27015 -NoBattlEye -automanagedmods -server -log (code=dumped, signal=SEGV)
  Process: 6932 ExecStartPre=/home/ark/steamcmd +login anonymous +force_install_dir /home/ark/server +app_update 376030 +quit (code=exited, status=0/SUCCESS)
 Main PID: 6957 (code=dumped, signal=SEGV)

 

I got the impression there was a change in the permissions of the mods in September but I'm not sure, and I've tried changing the permission without any luck.