ARK Tools v0.6.4 - tools for reading and manipulating ark savegame files

[MirageUK]

Firstly I'd like to thank @Qowyn for providing the Java code which allowed me to make my own Windows program wrapper to list the wild and tames in grid format.

From this I then found a C# library based off of the code provided by Qowyn and made a slightly more feature rich version fo my own servers (https://github.com/ark-mod/ArkSavegameToolkitNet?

It currently:

• Lists locations of various structures: Terminals/Obelisks, Beaver Dams, Deino Nests, Wyvern Nests, Gas Veins, Water Veins and Artifact crates.
• Summarised count, min and max levels of all Wild and Tamed creatures.
• Lists details of each creature when selected in the Summary dropdown showing wild level points (and tamed for tamed creatures) with location.  
• Visualises positions on the following game maps:  The Island, Scorched Earth, Aberration, Extinction and Valguero.
• Custom map markers for showing caves/your bases etc.

Works with any offline ARK save file (so long as one of the maps I implemented), direct FTP server downloads and Single Player games.

Was hoping to release something to you all in the new year but then found this thread and looks like I need to work in including mutation stats, tribe and player data.

 

[MirageUK]

Now also works with Ragnarok and added more marker icons into the selection for people relates stuff (Man/Woman/Ally/Target).  

To get tribe and player data it will also need access to .arkprofile and .arktribe files so looking into that for the next feature change.

[DaedheldirPR]

On 12/16/2019 at 9:19 AM, MirageUK said:

Firstly I'd like to thank @Qowyn for providing the Java code which allowed me to make my own Windows program wrapper to list the wild and tames in grid format.

From this I then found a C# library based off of the code provided by Qowyn and made a slightly more feature rich version fo my own servers (https://github.com/ark-mod/ArkSavegameToolkitNet?

It currently:

• Lists locations of various structures: Terminals/Obelisks, Beaver Dams, Deino Nests, Wyvern Nests, Gas Veins, Water Veins and Artifact crates.
• Summarised count, min and max levels of all Wild and Tamed creatures.
• Lists details of each creature when selected in the Summary dropdown showing wild level points (and tamed for tamed creatures) with location.  
• Visualises positions on the following game maps:  The Island, Scorched Earth, Aberration, Extinction and Valguero.
• Custom map markers for showing caves/your bases etc.

Works with any offline ARK save file (so long as one of the maps I implemented), direct FTP server downloads and Single Player games.

Was hoping to release something to you all in the new year but then found this thread and looks like I need to work in including mutation stats, tribe and player data.

 

Is there a download for this? Would love to have this tool. Would be perfect for me as is.

[MirageUK]

Not just yet - I'm working on a few other bits and adding encryption to the server config saves containing ftp login information (plain text atm so not good for server security).

It should be ready in the next couple of weeks and I'll create a new post of my own about it.

[MirageUK]

It was just going to be a server admin tool but my good friend and tester (breaker more like lol) has given me a challenge to make it work for trusted users of private servers who's admin want to offer the functionality.

[DaedheldirPR]

1 minute ago, MirageUK said:

Not just yet - I'm working on a few other bits and adding encryption to the server config saves containing ftp login information (plain text atm so not good for server security).

It should be ready in the next couple of weeks and I'll create a new post of my own about it.

Thanks my dude was hoping to use this for my single player save. Not to much into server stuff yet.

[DaedheldirPR]

Cool if you ever need a tester with some single player stuff hit me up. Thanks again.

[MirageUK]

It's mainly useful for breeding on single player due to the limited spawns of wild around the player but you're more than welcome to have a copy once it's done.

1 minute ago, DaedheldirPR said:

Thanks my dude was hoping to use this for my single player save. Not to much into server stuff yet.

 

[ooviixoo]

On 12/12/2019 at 11:08 AM, ooviixoo said:

Anyone know if mutation data is able to be exported?  Combed through the tamed data from an export but aside from maleAncestors/femaleAncestors (showing just the dino parentals and family line) I am unable to find values for the # of mutations from mother and father lines.  Am I missing it, or is it not available in exports?

Working on a Breeding Assistant / Calculator and it would be valuable data.

Would also be valuable to know what a female dino's countdown for mating is.  I would assume these values have to be recorded in the save file, are they just not exported via json?

[MirageUK]

7 hours ago, ooviixoo said:

Would also be valuable to know what a female dino's countdown for mating is.  I would assume these values have to be recorded in the save file, are they just not exported via json?

Well there are properties on the TamedCreature class which would hint to mating so I will have to try em out and see what needs to be done to include them:-

• LastUpdatedGestationTime
• LastUpdatedMatingAtTiume
• BabyGestationPRogress
• BabyNextCuddleTime

I think I would have to also include multiplier settings in my app to calculate the NEXT times from the last times as these are stored in the game config and not the save game.

[funtimes]

On 12/18/2019 at 1:39 AM, MirageUK said:

Well there are properties on the TamedCreature class which would hint to mating so I will have to try em out and see what needs to be done to include them:-

• LastUpdatedGestationTime
• LastUpdatedMatingAtTiume
• BabyGestationPRogress
• BabyNextCuddleTime

I think I would have to also include multiplier settings in my app to calculate the NEXT times from the last times as these are stored in the game config and not the save game.

Are you able to see dinos in cryo?

[MirageUK]

I'm not sure on this one - I will have to see if the library has data available for cryo storage and how it handles the creatures details inside.  At the moment I would say no but I will add it to my list of things to investigate.

[funtimes]

I know somebody has made one that reads it, but it's not public, and they aren't willing to share it. But these where some of the comments they made about reading the cryo data in the world save files.

Cryopod data is essentially a save in a save.
Every cryopod is its own save.
Much like escaped json in a json.
 

[MirageUK]

I can't see any obvious properties in the toolkit that surface such information.

I have done a quick forcetame and cryo'd it with the name of "Cheaty Cryo" and was ab to find this in the save file so hopefully I can parse it out directly if not available in toolkit itself.

Won't be a high priority on this one though as I want to get the other stuff out before adding more features.

[ooviixoo]

Been considering a solution that is more dynamic than exporting to json.  Anyone think an active read of server memory could be a thing?  I dumped the heap on my dedicated linux box, but... yeah at 9GB, not a thing.  With the structs pretty well defined it should be possible to just peek around in active memory space?

[MirageUK]

We have our server setup for 30 minute saves so it could be up to 30 minutes out of date - but displays in the viewer the date of the save.  You can adjust the save time down and still read from the save game data more frequently if required.

I have my server on shared hosting so don't have such access to the server memory - and if I did I'd probably crash the ARK.. or complete server.  

If its just wild creatures youre interested in you can always install the S+ mod and use the option on the S+ Transmitter to list real-time stuff in-game.

(edit)

Thinking about it, if that's a route you're interested in you might be able to write your own server mod - which provides an external api to query the in-game data at real-time?

[McBane]

Looks like this tool isn't able to read tribe data (CMD => ./ark-tools.sh tribes ...) from Genesis savefiles. Only file I get as result is a "0.json" with one tribe (maybe the first?) inside. Also I don't get any error message at all.

On all other maps this is still working as always.

EDIT:
OK. Found the issue. They renamed "TriebID" to "TribeId" in Genesis, which makes this tool not be able to find the right proberty.
So a possible fix is this:

 

--- src/main/java/qowyn/ark/tools/data/Player.java	2020-02-28 11:33:12.726086823 +0100
+++ src/main/java/qowyn/ark/tools/data/Player.java	2020-02-28 12:39:46.265206495 +0100
@@ -106,7 +106,7 @@
     uniqueId = myData.getPropertyValue("UniqueID", StructUniqueNetIdRepl.class);
     savedNetworkAddress = myData.getPropertyValue("SavedNetworkAddress", String.class);
     playerName = myData.getPropertyValue("PlayerName", String.class);
-    tribeId = myData.findPropertyValue("TribeID", Integer.class).orElse(0);
+    tribeId = myData.findPropertyValue("TribeID", Integer.class).orElse(myData.findPropertyValue("TribeId", Integer.class).orElse(0));
     playerDataVersion = myData.findPropertyValue("PlayerDataVersion", Integer.class).orElse(0);
     spawnDayNumber = myData.findPropertyValue("SpawnDayNumber", Integer.class).orElse(0);
     spawnDayTime = myData.findPropertyValue("SpawnDayTime", Float.class).orElse(0.0f);
--- src/main/java/qowyn/ark/tools/data/Tribe.java	2020-02-28 11:33:12.726086823 +0100
+++ src/main/java/qowyn/ark/tools/data/Tribe.java	2020-02-28 12:40:27.756913747 +0100
@@ -64,7 +64,7 @@
 
     tribeName = tribeData.findPropertyValue("TribeName", String.class).orElse("");
     ownerPlayerDataId = tribeData.findPropertyValue("OwnerPlayerDataID", Integer.class).orElse(0);
-    tribeId = tribeData.findPropertyValue("TribeID", Integer.class).orElse(0);
+    tribeId = tribeData.findPropertyValue("TribeID", Integer.class).orElse(tribeData.findPropertyValue("TribeId", Integer.class).orElse(0));
 
     ArkArrayString membersNames = tribeData.getPropertyValue("MembersPlayerName", ArkArrayString.class);
     if (membersNames != null) {

You can download the fixed version here:
https://github.com/McBane87/ark-tools/releases/download/v0.6.4-1/ark-tools.zip

 

[andi79h]

On 2/28/2020 at 9:02 AM, McBane said:

Looks like this tool isn't able to read tribe data (CMD => ./ark-tools.sh tribes ...) from Genesis savefiles. Only file I get as result is a "0.json" with one tribe (maybe the first?) inside. Also I don't get any error message at all.

On all other maps this is still working as always.

EDIT:
OK. Found the issue. They renamed "TriebID" to "TribeId" in Genesis, which makes this tool not be able to find the right proberty.
So a possible fix is this:

 

You can download the fixed version here:
https://github.com/McBane87/ark-tools/releases/download/v0.6.4-1/ark-tools.zip

 

Seems as though the fix does not help in our case, as it is crashing with this message:

 

Need to load map, this may take some time...
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:27)
        at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87)
        at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:287)
        at qowyn.ark.tools.App.main(App.java:125)
steam@Ubuntu-1604-xenial-64-minimal:~$
 

Any ideas?

Cheers

[andi79h]

On 12/21/2019 at 11:04 AM, MirageUK said:

I can't see any obvious properties in the toolkit that surface such information.

I have done a quick forcetame and cryo'd it with the name of "Cheaty Cryo" and was ab to find this in the save file so hopefully I can parse it out directly if not available in toolkit itself.

Won't be a high priority on this one though as I want to get the other stuff out before adding more features.

I had the same problem, needed to find all Gachas in cryptos, because they violate the law on some of our servers.

That's my solution using the plain export function of ark-tools:

./ark-tools.sh export --class Gacha_Character_BP_C $filename gacha_plain.json --pretty-printing
./ark-tools.sh export --class PrimalItem_WeaponEmptyCryopod_C $filename gacha_cryo.json --pretty-printing

To get the names of the cryopod, one has to loop over the entries in this json. For some reason, there are some more classes exported but PrimalItem_WeaponEmptyCryopod_C , so ignore them...

 

$content = file("$path/$class");
$content = implode('', $content);
$rows = json_decode($content, true);

        $dinos = [];
        $duras = [];

        foreach ($rows as $row) {
            if (isset($row['class']) && $row['class'] == 'PrimalItem_WeaponEmptyCryopod_C') {

                foreach ($row['properties'] as $property) {
                    $props[$property['name']] = $property['value'];
                }
                if(!isset($props['CustomItemDatas'])) {
                    continue;
                }

                foreach ($props['CustomItemDatas'][0] as $itemData) {

                    $foundGacha = false;

                    if(isset($itemData['name']) && $itemData['name'] == 'CustomDataStrings') {
                        if(isset($itemData['value']) && is_array($itemData['value'])) {
                            foreach($itemData['value'] as $dino) {

                                if(!$foundGacha && substr($dino, 0, 20) <> 'Gacha_Character_BP_C') {
                                    continue;
                                }
                                $foundGacha = true;

                                $dinos[$row['names'][0]][] = $dino;
                                $duras[$row['names'][0]] = $props['SavedDurability'];
                            }
                        }
                    }
                }
            }
        }
 

 

 

[MirageUK]

Ohh sorry, I managed to find the data.  The byte array is another ark archive in and of itself.  I udpated the ARKSaveGameToolKit and ARKSaveGameToolkitNet repositories on my Github and added pull requests to update the main ones.

If you want to check out my latest updates I add them to my own forum post at 

 

Up to Preview 29 at the moment with a LOT of added features since I posted on here, including genesis map support and cryopod creatures.

Btw they also updated the use of TargetingTeam property on tamed creatures for Genesis - new property added "TamingTeamID".

[andi79h]

On 2/28/2020 at 9:02 AM, McBane said:

Looks like this tool isn't able to read tribe data (CMD => ./ark-tools.sh tribes ...) from Genesis savefiles. Only file I get as result is a "0.json" with one tribe (maybe the first?) inside. Also I don't get any error message at all.

On all other maps this is still working as always.

EDIT:
OK. Found the issue. They renamed "TriebID" to "TribeId" in Genesis, which makes this tool not be able to find the right proberty.
So a possible fix is this:

 

You can download the fixed version here:
https://github.com/McBane87/ark-tools/releases/download/v0.6.4-1/ark-tools.zip

 

Seems as though the fix does not help in our case, as it is crashing with this message:

Need to load map, this may take some time...
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:27)
        at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87)
        at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:287)
        at qowyn.ark.tools.App.main(App.java:125)
steam@Ubuntu-1604-xenial-64-minimal:~$
 

Any ideas?

Cheers

[McBane]

17 hours ago, andi79h said:

Seems as though the fix does not help in our case, as it is crashing with this message:

Need to load map, this may take some time...
Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:27)
        at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87)
        at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:287)
        at qowyn.ark.tools.App.main(App.java:125)
steam@Ubuntu-1604-xenial-64-minimal:~$
 

Any ideas?

Cheers

I've compiled this with OpenJDK-11. Maybe you need Version 11 as well to run this.

Edit:
I've tried to build with Version 8 (1.8) compability. Can you redownload the zip file and try again?

[andi79h]

5 hours ago, McBane said:

I've compiled this with OpenJDK-11. Maybe you need Version 11 as well to run this.

Edit:
I've tried to build with Version 8 (1.8) compability. Can you redownload the zip file and try again?

I tried the today's jar but got the error still:

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:27)
        at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87)
        at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:287)
        at qowyn.ark.tools.App.main(App.java:125)

steam@Ubuntu-1604-xenial-64-minimal:~$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

I'll upgrade to 11 to be sure its not the java version and head back to you

[McBane]

Alternatively, you can try to compile it yourself

17 hours ago, andi79h said:

I tried the today's jar but got the error still:

Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at qowyn.ark.FileFormatBase.readBinary(FileFormatBase.java:27)
        at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:87)
        at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:287)
        at qowyn.ark.tools.App.main(App.java:125)

steam@Ubuntu-1604-xenial-64-minimal:~$ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-8u242-b08-0ubuntu3~16.04-b08)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)

I'll upgrade to 11 to be sure its not the java version and head back to you

Hmm...OK. I nearly have no knowledge about Java. Everything I've done so far with this was trial and error + knowledge of script languages in general.
If you still have issues, you can try to build it yourself:

 

# Debian Buster
#----------------
apt-get install maven openjdk-11-jre openjdk-11-jdk git
git clone https://github.com/Qowyn/ark-savegame-toolkit.git
git clone https://github.com/McBane87/ark-tools.git
#
export MAVEN_OPTS='-Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dmaven.wagon.http.ssl.ignore.validity.dates=true'
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
#
cd ark-savegame-toolkit
mvn install
#
cd ../ark-tools
mvn install

 

[andi79h]

13 hours ago, McBane said:

Alternatively, you can try to compile it yourself

Hmm...OK. I nearly have no knowledge about Java. Everything I've done so far with this was trial and error + knowledge of script languages in general.
If you still have issues, you can try to build it yourself:
 

Good news: with openjdk version "11.0.5" 2019-10-15 it works pretty smooth
No clue why jdk8 hates it tho^^