Jump to content

ARK Tools v0.6.4 - tools for reading and manipulating ark savegame files


Qowyn

Recommended Posts

  • Replies 369
  • Created
  • Last Reply
  • 2 weeks later...
On 03/09/2016 at 10:12 PM, Cilya said:

Hi !

Very nice work on this tool ! Surprisingly it's hard to find with google, but I was very happy to find it as it seems very powerful. I have a problem using it though. I can't seem to import a tamed creature data from one saved ark to another. I use your tool this way :


ark-tools.exe export TheIsland_src.ark --creature "Foo" creature.json
ark-tools.exe import TheIsland_dest.ark creature.json TheIsland_edited.ark

 

I was able to export a dino and then import it into a secondary file, im assuming you cant do this with a live file?
would be nice to restore dinos from abuse in a live file instead of having to shutdown the server, export dino from current file... import it into a separate one, rename and then start up.

Link to comment
Share on other sites

On 23/03/2017 at 6:38 PM, FatDude said:

Mysql is fine for the purpose, since we are talking low data volume anyway.

Have you tried any of the opensource php etl frameworks that support json files ?

A quick search gave me this one, though I never used it: https://github.com/leomarquine/php-etl

Also the dino info is coming from the world save which usually saves every 20-30min depending on settings. You can make a trigger php script that checks the world save timestamp and only starts your etl scripts once it's been updated. That will save you etl time.

If you source in the players and tribes to own tables, you can start joining the data together to show ex all dino for a tribe.

 

are you able to opensource this great system, or offer it for sale, i am looking at developing such a system but only have basic coding skills :( and as someones already developed such a system, would be counter productive to hire someone to develop it.

Link to comment
Share on other sites

On 2/5/2017 at 0:08 PM, DrunkMunki said:

 

are you able to opensource this great system, or offer it for sale, i am looking at developing such a system but only have basic coding skills :( and as someones already developed such a system, would be counter productive to hire someone to develop it.

My system is not really shareable in its current form and a bit overkill / complicated, I kinda built it as a proof of concept for fun.

Would need to drop the ssis packages and do the data loading / slowly changing dimension (type 2) via stored procedures instead.

Also since I got MSDN access, I'm using the enterprise version of MS sql server. Would need to convert it to work on the free edition.

Anyway I will think about it.

Link to comment
Share on other sites

ark-tools.jar patched for v257 => https://ufile.io/behuv

works with wild, tamed, tribes and players commands, I don't use the others so I don't know if they works

If you want to do it by yourself, rebuild ark savegame toolkit 0.6.0 with this modification => https://github.com/Qowyn/ark-savegame-toolkit/issues/1

then rebuild ark-tools with the new ark-savegame-toolkit-0.6.0.jar

Have fun guys

Link to comment
Share on other sites

15 hours ago, MaeL said:

ark-tools.jar patched for v257 => https://ufile.io/behuv

works with wild, tamed, tribes and players commands, I don't use the others so I don't know if they works

If you want to do it by yourself, rebuild ark savegame toolkit 0.6.0 with this modification => https://github.com/Qowyn/ark-savegame-toolkit/issues/1

then rebuild ark-tools with the new ark-savegame-toolkit-0.6.0.jar

Have fun guys

Great work dude, will have a look

Link to comment
Share on other sites

Just a heads up for people who like to be part of a big project and are interested in creating dino heatmaps and stuff with the ark-tools. I'm server admin for 10+ servers hosted in europe and we are looking forward on improving our project with this tool.

If you are interested, send a PM ;)

---

@MaeL Thanks dude, I appreciate the update to the .jar :) Helps us a lot.

Link to comment
Share on other sites

On 18/03/2017 at 8:02 PM, StriderUk said:

A plea to the php/db gods.

 

Hi chaps, I have managed to sort out the retrieval of wild dino data, via this fantastic tool.

 

I have at the moment, got the json files loading into a mysql db and then this is published on my website.

I feel this is a pretty dirty way around of doing it, but I'll be honest i dont know a better way to do this.

I have scanned the ww and just get lost in all the information out there.

 

I know a few of you have managed to turn this data into a thing of beauty and I know i will never get to that with my skills, but would love it, if you could throw me some pointers.

Stack overflow is a great resource, but i feel i am not properly asking what I require to know over there.

 

Any pointers guys?

 

Thanks in advance.

I know this post is a month old, but I feel like PHP/Mysql (or mariaDB or percona) deserve some more attention, so I'm happy to give you some pointers.

Its correct that storing the json output in your database is not the ideal situation. It makes it hard (not impossible) to create relations between certain records and you're also gonna find yourself storing a lot of data that you'll never use. The most important thing is that you have no way (that i know of) to make indexes. In pretty much any database, you want to create indexes, to fetch certain sets of data very quickly. 

My advice to you would be: try to iterate (loop) through the different files, per type. Make a table similar to that type and store only the information you want in that table. In other words: grab all the dino's, loop through all the dino's and store them in your database one by one.
Keep in mind that you might want to extend in the future, so setting up something abstract is something you would want to do.

Try to break things down in small chunks that you can work with :)

Good luck!

Link to comment
Share on other sites

17 hours ago, MaxPremiumARK said:

I know this post is a month old, but I feel like PHP/Mysql (or mariaDB or percona) deserve some more attention, so I'm happy to give you some pointers.

Its correct that storing the json output in your database is not the ideal situation. It makes it hard (not impossible) to create relations between certain records and you're also gonna find yourself storing a lot of data that you'll never use. The most important thing is that you have no way (that i know of) to make indexes. In pretty much any database, you want to create indexes, to fetch certain sets of data very quickly. 

My advice to you would be: try to iterate (loop) through the different files, per type. Make a table similar to that type and store only the information you want in that table. In other words: grab all the dino's, loop through all the dino's and store them in your database one by one.
Keep in mind that you might want to extend in the future, so setting up something abstract is something you would want to do.

Try to break things down in small chunks that you can work with :)

Good luck!

Thanks for the advice mate, I did manage to get it all working in the end.

We tried out a few methods, firstly parsing through SQL 2016 using the new native json tools in ssis, this worked a treat, but the whole security aspect of allowing anon log ins to view the ssrs side of things was becoming too much more work, so we canned that.

In the end we have managed to load the json files directly using an ajax query, its nothing like some of you guys have done, but it fits the purpose for what we require.

We think of it as our top trump of wild dinos -

 

http://warpspeed.zapto.org:7771

We are next hoping to bring in the player data next, so our community can find their lost dinos, again its not going to be on the scale of what some of you guys have produced, but it fits the needs for what we want.

 

 

 

Link to comment
Share on other sites

Sure thing!
I noticed some other people using the direct json output, that this tool provides, on production sites as well. Please do keep in mind that this might cause security risks. If attackers find a way to inject / execute unwanted commands, you're in big trouble. If you're gonna allow direct access, I would recommend using your webserver as a proxy, to obscure / block all direct access to this tool using a firewall, with your webserver's IP in the whitelist. This will also give you the option to cache the output ;)

 
Another unpleasant side-effect might be noticeable when you have a lot of visitors on the website. Ideally you would want to separate the web traffic from your gameserver traffic, so your gameserver is not affected by any traffic spikes. Its a good thing to be in control when and how often your gameserver is called for data :)

Link to comment
Share on other sites

On 15/5/2017 at 3:42 PM, StriderUk said:

Thanks for the advice mate, I did manage to get it all working in the end.

We tried out a few methods, firstly parsing through SQL 2016 using the new native json tools in ssis, this worked a treat, but the whole security aspect of allowing anon log ins to view the ssrs side of things was becoming too much more work, so we canned that.

In the end we have managed to load the json files directly using an ajax query, its nothing like some of you guys have done, but it fits the purpose for what we require.

We think of it as our top trump of wild dinos -

 

http://warpspeed.zapto.org:7771

We are next hoping to bring in the player data next, so our community can find their lost dinos, again its not going to be on the scale of what some of you guys have produced, but it fits the needs for what we want.

 

 

 

Dude, if you got everything imported in ms sql db, then why drop it ?

Drop the SSRS part, those reports sucks for this purpose anyway

Just connect directly to the db from your php/whatever webpages. Use some of the free java table builder like https://datatables.net/ or similar to build the tables..

The use the php steam login api (or similar) to match the steamid to the player logged in

Link to comment
Share on other sites

@StriderUk I'd like to have a look at that savegame. Next version of ark-tools is almost ready and should solve all problems with the volcano map and other maps using those fields.

In addition to the new command for updating ark_data.json there will be a command to get the version of ark-tools. There have been a lot of internal changes, resulting in fewer error messages about unknown properties and other stuff.

Link to comment
Share on other sites

I am using the tamed, wild, tribe and player export quite frequently. Instead of pushing it directly into some SQL, I rather run a PHP script over it and link some stuff like dino classes. For example we already have ~6k dinos within 2 larger maps into one table now, which is pretty cool to let player search for their dinos instead of using mods for this.

Now I am facing a minor problem. It seems as though the export is limited to bytes on dino's stats, is this true? We got some admin dinos which are pretty high in level and their stats seem to overflow a lot:

https://dl.dropboxusercontent.com/u/9831256/ark/arktools1.gif

Is there a chance you change those bytes to say words (16 bit) or something?

Cheers

 

PS: For some reason the forum software wont let me post the image from my dropbox nor does it insert it right into the post. You may copy the link the old fashion way^^

arktools1.gif

Link to comment
Share on other sites

And 0.5.0 is finally out. Tried to cram two additional commands into this release but ran out of time.

Changelog:

  • Many changes to ark-savegame-toolkit to improve stability and useability
  • Breaks compatibility with old json formats (thingToJson and jsonToThing commands)
  • Added support for previously unknown field in map saves (crash fix)
  • Added support for the new PlayerLocalData.arkprofile Version 3
  • Added updateData command: checks for and downloads newer ark_data.json files
  • Added version command: outputs the installed version of ark-tools
  • Fixed output of unsigned 8 bit values: attributes and colors of creatures
  • Removed ark_data.json from release

That last point is important for new installations: You have to run

ark-tools updateData

at least once for new installations if you want human readable names for creatures, items or anything else.

The new commands that didn't make it are planned for the next version:

  • Command db: reads all available data from savegame, profiles, tribes and cluster (if available) and writes it to the specified URI (or file path) using the specified driver
  • Command db-drivers: prints a list of all available drivers, their settings and all supported protocols
  • Drivers can support writing to local files or URIs and they are either intergrated into ark-tools.jar or loaded from external *-ark-tools-driver.jar files
  • The first driver will be called "json" and will support writing to local files and POSTing to HTTP and HTTPS URLs or uploading to FTP URLs
Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.


×
×
  • Create New...