Jump to content

ARK Tools v0.6.4 - tools for reading and manipulating ark savegame files


Qowyn

Recommended Posts

On 17.11.2016 at 11:05 PM, FatDude said:

Great progress.

Would it be possible to also extract player's lat/lon as you do with the dinos from the worldsave ?

Among all the other stuff, I'm also building a player heatmap and need the player locations for that

Thanks for your great effort on this tool

How are you doing this heatmap?  Which solution do you use to create that?

Wanna do the same for the wild dinos, but haven't found any simple solutions to do such thing.

Link to comment
Share on other sites

  • Replies 369
  • Created
  • Last Reply
On 21/11/2016 at 10:43 AM, Toni said:

How are you doing this heatmap?  Which solution do you use to create that?

Wanna do the same for the wild dinos, but haven't found any simple solutions to do such thing.

I'm currently using Microsoft Power BI

See the example image for tamed dinos on one of my servers (removed tribe name here)

I can then create reports that can be shown on a website based on the Ark data tools.

Can then different reports with players/tribes, tribelogs, kills, tamed dinos everything that can be pulled out with this handy tool.

With scheduled refresh the reports are showing live data (or semi live, since I currently extact data each hour)

 

Map_ex.jpg

Link to comment
Share on other sites

16 hours ago, FatDude said:

I'm currently using Microsoft Power BI

See the example image for tamed dinos on one of my servers (removed tribe name here)

I can then create reports that can be shown on a website based on the Ark data tools.

Can then different reports with players/tribes, tribelogs, kills, tamed dinos everything that can be pulled out with this handy tool.

With scheduled refresh the reports are showing live data (or semi live, since I currently extact data each hour)

 

Map_ex.jpg

Thx, but too late ^^

I received the link of your page and had a look there. It looks very nice. But for me personally the performance is too bad, therefore i've programmed it myself.

Looks like that now and is very fast:

https://born2frag-notifications.ch/wild_dinos_map.php

 

Link to comment
Share on other sites

On 23/11/2016 at 1:05 PM, Toni said:

Thx, but too late ^^

I received the link of your page and had a look there. It looks very nice. But for me personally the performance is too bad, therefore i've programmed it myself.

Looks like that now and is very fast:

https://born2frag-notifications.ch/wild_dinos_map.php

 

 

Toni, you did everything I have planned, just faster, better and more elegant ;-)

 

My route with powerbi kinda sucks due to limitations in powerbi and what it should be used for. (Could based, data refresh restrictions etc)

I have everything sourced in my DB and got the data model in place, just not found the best way to visualize.

 

Your project, is that something you would want to share with me or the community ?

Would really like implement similar on my servers as well.

 

Also I'm working on some additions like an ingame chat bot with commands like !TopTribe which will use rcon command for announcing ingame

 

I know time is money, so if that's the issue I'm sure we can find out something if needed.

 

Let me know

 

Link to comment
Share on other sites

15 hours ago, FatDude said:

 

Toni, you did everything I have planned, just faster, better and more elegant ;-)

 

My route with powerbi kinda sucks due to limitations in powerbi and what it should be used for. (Could based, data refresh restrictions etc)

I have everything sourced in my DB and got the data model in place, just not found the best way to visualize.

Your project, is that something you would want to share with me or the community ?

Would really like implement similar on my servers as well.

Also I'm working on some additions like an ingame chat bot with commands like !TopTribe which will use rcon command for announcing ingame

I know time is money, so if that's the issue I'm sure we can find out something if needed.

Let me know

I do not want to share my code. Reasons: Its many hours of work / My code is designed for exactly my servers and would take hours to adjust it to make it public and configurable / I wanna be the only server which has all this features in one place (You know... not much ark players around anymore ^^). Maybe i will redesign it to make it public some time, but this won't happen before there are enough ark players back and all servers has enough players again.

But if you need help to build something similar, i'm willing to help you. I just do not want to provide all my code. For example the Map-Function isn't such complex as it looks. If your using php too, i can help you with that. Its simply 1 div which has the size of the map... and many divs for the points (css absolut position with the values you have for lat / lon). If for example the lat is 47.2 / 56  simply place the point div at absolut position 47.2%  56%. Its a 100 x 100 map, so lat 47.2 is the same as 47.2% of the map picture from top.

The best is when you send me Pn's with your specific questions.

 

Regarding your ingame chat bot. I had this idea too, but at the moment, there isn't enough informations available to do something cool with that.

The best tribe at the moment is simply the tribe which has the most structures, items, dinos.

The serverlog is still bugged (causing server crashes), so you can't use any PvP actions (kills/destroys) to include into this calculation. The same for the server chat log.

As long as this 2 files aren't available, it doesn't make any sense for me to start programming something.

As soon as this 2 files are available, you can  1. Implement the ingame chat of your servers into the website and combine the chat of many servers into one chat.  2. Display PvP Actions in the chat.

If you simply wan't to announce something all 30 minutes for example, create a batch file which uses mcrcon.exe and sends the text to the server. Use the jobscheduler (or cron on linux) to run this script all 30 minutes. At the moment the easiest way.

 

 

Link to comment
Share on other sites

7 hours ago, Toni said:

I do not want to share my code. Reasons: Its many hours of work / My code is designed for exactly my servers and would take hours to adjust it to make it public and configurable / I wanna be the only server which has all this features in one place (You know... not much ark players around anymore ^^). Maybe i will redesign it to make it public some time, but this won't happen before there are enough ark players back and all servers has enough players again.

But if you need help to build something similar, i'm willing to help you. I just do not want to provide all my code. For example the Map-Function isn't such complex as it looks. If your using php too, i can help you with that. Its simply 1 div which has the size of the map... and many divs for the points (css absolut position with the values you have for lat / lon). If for example the lat is 47.2 / 56  simply place the point div at absolut position 47.2%  56%. Its a 100 x 100 map, so lat 47.2 is the same as 47.2% of the map picture from top.

The best is when you send me Pn's with your specific questions.

 

Regarding your ingame chat bot. I had this idea too, but at the moment, there isn't enough informations available to do something cool with that.

The best tribe at the moment is simply the tribe which has the most structures, items, dinos.

The serverlog is still bugged (causing server crashes), so you can't use any PvP actions (kills/destroys) to include into this calculation. The same for the server chat log.

As long as this 2 files aren't available, it doesn't make any sense for me to start programming something.

As soon as this 2 files are available, you can  1. Implement the ingame chat of your servers into the website and combine the chat of many servers into one chat.  2. Display PvP Actions in the chat.

If you simply wan't to announce something all 30 minutes for example, create a batch file which uses mcrcon.exe and sends the text to the server. Use the jobscheduler (or cron on linux) to run this script all 30 minutes. At the moment the easiest way.

 

 

 

Sure, I understand your reason.

I don't really need help atm, for me its more about the lack of time due to irl work ;-/

The going from scratch php route just takes more time, but also gives the best result. Will have to consider.

My pvp calculations are based on the tribe logs where I extract the killed by player/dino/which tribe and do calc on top of that(semi working atm)

While serverlogs/chatlogs would be the best option, I know its bugged and have it disabled, so using the tribelog for kill info

 

On retrieving ingame chat I can give you hint. rcon getchat command ;-)

( Patch 185.4: Chat can now be retrieved in RCON -- you just have to use the RCON command "getchat" to return the latest chat buffer.  )

Only problem is it will clear the chat buffer, so cant also use an external rcon system for retrieving the chat at the same time (batllemetrics rcon system in my case)

 

My pvp and point system is among other used for commands like !putbounty 'Playername 'points', which will deduct the points from the player and give to the killer once completed

For this to work propper I need to check for unique player names and automatically rename last created player with same name via rcon tool.

Open/collected bouties can be shown on both website and ingame either automated or via the chatbot

 

Via mcrcon.exe as you mentioned you can do lots of fancy stuff. Convert point to rewards (rcon giveitemtoplayer) and remove the points in the db.

So you can basically automate the reward system any way you like. Like 1000 points needed for xx skin etc.

 

I have many more crazy ideas floating in my head atm, just gotta think out of the box ;-)

 

So lots of stuff planned, some is working already and some is wip and running as poc(proof of concept) on my test server, just neeed more time :-/

Link to comment
Share on other sites

6 hours ago, Qowyn said:

0.4.0 is almost ready, just trying to add one last thing: extended options for tribes command, to extact extended data about PCs, structures and creatures. New version will come today (CET). If that last feature takes too long it will be pushed back to 0.4.1

Wow great.

Would it be possible to also extract player's lat/lon as you do with the dinos from the worldsave ?

Also would if be possible to include tribeid and playerid in the dino extact, since tribe name and player name might not be unique and causes join problems

Link to comment
Share on other sites

Decided to be too tired to work on this today. So 0.4.0 is out now.

Changelog:

  • Added reading and writing support for sav files (except for text properties)
  • Added --write-all-fields switch to creature commands, should write (almost) all fields
  • Added team, playerId, and ownerName to creature commands
  • Added --tribeless switch to tribe data extraction, this will list everything not belonging to a tribe in tribeless.json
  • Added GPS data for PGAs
  • Added --max-age option to ignore unchanged player profiles
  • Added --positions switch to player command, this will load the map to get the current position of the player
  • Improved player data extraction time by lazy loading tribe data
  • Improved crash protection against broken files by catching all unexpected errors for player profiles, tribe data and cluster data
  • Improved detection of tamed creatures
  • Improved detection of array of struct properties
  • Fixed tribe command ignoring equipped items
  • Fixed crash while processing saves from Volcano map

team field: this is a tribe id if > 999999999, otherwise a player id

playerId field: always points to a player, only set if the creature got tamed before joining a tribe or (probably) for personally owned creatures

ownerName field: same as playerId, but containing the name of the player

Link to comment
Share on other sites

21 hours ago, Qowyn said:

Decided to be too tired to work on this today. So 0.4.0 is out now.

Changelog:

  • Added reading and writing support for sav files (except for text properties)
  • Added --write-all-fields switch to creature commands, should write (almost) all fields
  • Added team, playerId, and ownerName to creature commands
  • Added --tribeless switch to tribe data extraction, this will list everything not belonging to a tribe in tribeless.json
  • Added GPS data for PGAs
  • Added --max-age option to ignore unchanged player profiles
  • Added --positions switch to player command, this will load the map to get the current position of the player
  • Improved player data extraction time by lazy loading tribe data
  • Improved crash protection against broken files by catching all unexpected errors for player profiles, tribe data and cluster data
  • Improved detection of tamed creatures
  • Improved detection of array of struct properties
  • Fixed tribe command ignoring equipped items
  • Fixed crash while processing saves from Volcano map

team field: this is a tribe id if > 999999999, otherwise a player id

playerId field: always points to a player, only set if the creature got tamed before joining a tribe or (probably) for personally owned creatures

ownerName field: same as playerId, but containing the name of the player

--positions is trowing some error about missing file.

I copy world.ark and *.arktribe, *.arkprofile from the ark server to process on another server. What else I need to copy (If other files are needed)

You wrote it read the map, I also need to copy the map from ?

Link to comment
Share on other sites

It looks like the new Dino Groups for whistling are throwing errors when processing player files.  Fortunately, this does not seem to stop the files from being processed.

"Warning: Unknown Struct Type DinoOrderGroup at 5680 trying to read as StructPropertyList"

For reference, the command I'm using is:

ark-tools players "%~dp0\sourcefiles\TheIsland.ark" "%~dp0\players" --inventory long --pretty-printing

 

Link to comment
Share on other sites

  • 2 weeks later...
On 12.12.2016 at 7:27 AM, Failoe said:

Additionally, if possible I'd love a feature that gives an option for x,y,z coordinates on the structures display for a tribe.

That will come with the next version. And I'm developing a mod to enhance the features of ark-tools, this mod will allow you to get the ingame names of item classes and other information too. And it will be possible to extract most information right from the running game and post it to some HTTP(S?) endpoint.

Link to comment
Share on other sites

On 12/25/2016 at 6:17 AM, Qowyn said:

That will come with the next version. And I'm developing a mod to enhance the features of ark-tools, this mod will allow you to get the ingame names of item classes and other information too. And it will be possible to extract most information right from the running game and post it to some HTTP(S?) endpoint.

That will be awesome

EDIT:  I just gave you your 100th reputation point.   :-)     (stupid forum software keeps deleting the smiles)

 

 

Link to comment
Share on other sites

On 2017-01-05 at 1:45 AM, Failoe said:

You need to call his executable via command line.

As an example:


cd C:\path_to_ark-tools\ark-tools
ark-tools tribes "C:\Ark\Island\ShooterGame\Saved\SavedArks\TheIsland.ark" "C:\Ark\output_directory" --pretty-printing --items long --structures

 

I feel like such an idiot, but i cannot simply get this to work, created a batch file, edited the lines to match my system but it won't output jack poop.

Would really appriciate if OP made a quickstart guide to help us noobs, i could probably manage if i'm able to get the help section to be printed.

Link to comment
Share on other sites

I responded to your PM but I'll put it here to help any future viewers.  The reason that this isn't working is because you're running it off a batch file. The batch file is executing its code, then going "My work is done here! I must go, my people need me!" and closing the window.  If you just run it in an open command prompt instead of a batch file you'll see the output because the command prompt won't close.

Link to comment
Share on other sites

37 minutes ago, Failoe said:

I responded to your PM but I'll put it here to help any future viewers.  The reason that this isn't working is because you're running it off a batch file. The batch file is executing its code, then going "My work is done here! I must go, my people need me!" and closing the window.  If you just run it in an open command prompt instead of a batch file you'll see the output because the command prompt won't close.

"'ark-tools' is not  recognized as an internal or external command, operable program or batch file."

Does not matter if i run it as an administrator or not, neither of the folders i've downloaded and extracted will give anything but this message when i've tried doing it the way you explain above.

Resolved ^^^^

Link to comment
Share on other sites

@Qowyn i am getting out of memory errors when running the tool, i have put in a issue request at https://github.com/Qowyn/ark-tools/issues/5

Quote

Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Unknown Source)
at java.nio.DirectByteBuffer.(Unknown Source)
at java.nio.ByteBuffer.allocateDirect(Unknown Source)
at qowyn.ark.ArkSavegame.(ArkSavegame.java:68)
at qowyn.ark.tools.CreatureListCommands.listImpl(CreatureListCommands.java:109)
at qowyn.ark.tools.CreatureListCommands.tamed(CreatureListCommands.java:62)
at qowyn.ark.tools.App.main(App.java:114)

anyone else get this? tried reinstalling java and replaced the tools files, rant he command in admin prompt ... nothing works.

 

Fixed the issue by using the java command rather than the ark-tools.exe command.

java -Xmx2G -jar ark-tools.jar tamed TheIsland.ark dinos

 

Link to comment
Share on other sites

On 8.1.2017 at 6:56 AM, DrunkMunki said:

@Qowyn i am getting out of memory errors when running the tool, i have put in a issue request at https://github.com/Qowyn/ark-tools/issues/5

anyone else get this? tried reinstalling java and replaced the tools files, rant he command in admin prompt ... nothing works.

Try to run the tool with additionally -Xmx8G.

Like this:

java -Xmx8G -jar ark-tools.jar cluster parameter1 parameter2.....

 

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...