StevenB Posted January 12, 2017 Share Posted January 12, 2017 Hi, Thanks for this great tool! I am having an issue exporting and importing dinos to a cluster file (havent tried other things). This a feature that is currently working or maybe I am using it wrong? Many more errors like the one shown here while exporting, but they are the same just different location in the file. ./ark-tools.sh export restore/2017-01-08_17.30.10/TheIsland.ark snav/argent.json --dinoid 1147157684391425702 --pretty-printing Warning: Unknown Struct Type SPLightSettingsStruct at 39dd5f9 trying to read as StructPropertyList ./ark-tools.sh import snav/76561198156293811 snav/argent.json snav/76561198156293811_new Warning: Unknown Struct Type ArkInventoryData at a9 trying to read as StructPropertyList java.lang.NullPointerException at qowyn.ark.tools.EditingCommands.importIntoClusterData(EditingCommands.java:421) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:273) at qowyn.ark.tools.App.main(App.java:114) ./ark-tools.sh import snav/76561198156293811 snav/argent.json snav/76561198156293811_new --file-format cluster java.lang.IllegalArgumentException: No enum constant qowyn.ark.tools.FileFormat.cluster at java.lang.Enum.valueOf(Enum.java:238) at qowyn.ark.tools.FileFormat.valueOf(FileFormat.java:9) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:237) at qowyn.ark.tools.App.main(App.java:114) Link to comment Share on other sites More sharing options...
Failoe Posted January 13, 2017 Share Posted January 13, 2017 On that note, coordinates for all player corpses and dropped player inventories would be great. Finding player bodies is something admins end up spending too much time on. Link to comment Share on other sites More sharing options...
CyanicEmber Posted January 15, 2017 Share Posted January 15, 2017 Every time I try to run the .exe it momentarily opens and then immediately exits...? Link to comment Share on other sites More sharing options...
StevenB Posted January 16, 2017 Share Posted January 16, 2017 @CyanicEmber This is a console / command line only program atm. You should run cmd and cd to your arktools directory to run it. When you do run it inside a command prompt you will then see the available commands and options printed out. Best of luck! Steven. Link to comment Share on other sites More sharing options...
Whata Posted January 16, 2017 Share Posted January 16, 2017 Is there a way to use this tool and extract dataset from cluster information when a player uploads a dino in the terminal to move from map to another? As part of the clustering and allowing up/downloads of dinos, you can specify the share cluster directory. I have some ideas to use this process and dino stats. I tried using this an example: C:\ARK Survival Evovle Stuff\ark-tools>ark-tools.exe c2j clusterPlayerFilenameHere PlayerFilename.jsonWarning: Unknown Struct Type ArkInventoryData at a8 trying to read as StructPropertyList I did get majority of the dataset, but I think its having issue reading some parts and not getting dino colorset, tribe name ... etc? Link to comment Share on other sites More sharing options...
Toni Posted January 18, 2017 Share Posted January 18, 2017 On 24.12.2016 at 6:17 PM, Qowyn said: That will come with the next version. And I'm developing a mod to enhance the features of ark-tools, this mod will allow you to get the ingame names of item classes and other information too. And it will be possible to extract most information right from the running game and post it to some HTTP(S?) endpoint. Sounds nice Keep up your nice work Link to comment Share on other sites More sharing options...
cokll Posted January 19, 2017 Share Posted January 19, 2017 21 hours ago, Toni said: Sounds nice Keep up your nice work I use players error can not be used,But there's no problem with Tribes [root@localhost ~]# java -jar ark-tools.jar players /root/Saved/TheIsland.ark /root/bak/ java.lang.NullPointerException at org.glassfish.json.UnicodeDetectingInputStream.fillBuf(UnicodeDetectingInputStream.java:90) at org.glassfish.json.UnicodeDetectingInputStream.detectEncoding(UnicodeDetectingInputStream.java:129) at org.glassfish.json.UnicodeDetectingInputStream.<init>(UnicodeDetectingInputStream.java:76) at org.glassfish.json.JsonParserImpl.<init>(JsonParserImpl.java:76) at org.glassfish.json.JsonReaderImpl.<init>(JsonReaderImpl.java:68) at org.glassfish.json.JsonProviderImpl.createReader(JsonProviderImpl.java:131) at javax.json.Json.createReader(Json.java:220) at qowyn.ark.tools.CommonFunctions.readJson(CommonFunctions.java:110) at qowyn.ark.tools.CommonFunctions.readJsonRelative(CommonFunctions.java:123) at qowyn.ark.tools.DataManager.loadData(DataManager.java:33) at qowyn.ark.tools.PlayerListCommands.players(PlayerListCommands.java:81) at qowyn.ark.tools.App.main(App.java:114) [root@localhost ~]# [root@localhost ~]# java -jar ark-tools.jar cluster /root/Saved/TheIsland.ark /root/bak java.lang.NullPointerException at org.glassfish.json.UnicodeDetectingInputStream.fillBuf(UnicodeDetectingInputStream.java:90) at org.glassfish.json.UnicodeDetectingInputStream.detectEncoding(UnicodeDetectingInputStream.java:129) at org.glassfish.json.UnicodeDetectingInputStream.<init>(UnicodeDetectingInputStream.java:76) at org.glassfish.json.JsonParserImpl.<init>(JsonParserImpl.java:76) at org.glassfish.json.JsonReaderImpl.<init>(JsonReaderImpl.java:68) at org.glassfish.json.JsonProviderImpl.createReader(JsonProviderImpl.java:131) at javax.json.Json.createReader(Json.java:220) at qowyn.ark.tools.CommonFunctions.readJson(CommonFunctions.java:110) at qowyn.ark.tools.CommonFunctions.readJsonRelative(CommonFunctions.java:123) at qowyn.ark.tools.DataManager.loadData(DataManager.java:33) at qowyn.ark.tools.PlayerListCommands.cluster(PlayerListCommands.java:725) at qowyn.ark.tools.App.main(App.java:114) [root@localhost ~]# java -jar ark-tools.jar tribes /root/Saved/TheIsland.ark /root/bak [root@localhost ~]# Link to comment Share on other sites More sharing options...
DrunkMunki Posted January 28, 2017 Share Posted January 28, 2017 im trying to import a player JSON file but cant seem to get it to work, i get the below error: >ark-tools.exe j2p 76561198xxx900789.json 76561198xxx900789.arkprofile java.lang.NullPointerException at org.glassfish.json.JsonObjectBuilderImpl$JsonObjectImpl.getInt(JsonOb jectBuilderImpl.java:213) at qowyn.ark.ArkProfile.readJson(ArkProfile.java:131) at qowyn.ark.ArkProfile.<init>(ArkProfile.java:58) at qowyn.ark.tools.ConvertingCommands.jsonToProfile(ConvertingCommands.j ava:114) at qowyn.ark.tools.App.main(App.java:114) i have tried using ark-tools.exe and using relative and absolute locations of the files, generates the same error. Link to comment Share on other sites More sharing options...
McBane Posted January 29, 2017 Share Posted January 29, 2017 Unfortunately my Export for Tribes and Dinos isn't working anymore for my The Center Server. Do you maybe know whats causing this? java.lang.UnsupportedOperationException: Found unexpected Value 2 at 2132 at qowyn.ark.ArkSavegame.readBinary(ArkSavegame.java:145) at qowyn.ark.ArkSavegame.<init>(ArkSavegame.java:78) at qowyn.ark.tools.PlayerListCommands.tribes(PlayerListCommands.java:343) at qowyn.ark.tools.App.main(App.java:114) EDIT: I'm sure you put the Error-Checking in "ArkSavegame.java" on purpose, but for me this was the showstopper. And that's why I commented the part out and now everything seems to work again: public void readBinary(ArkArchive archive, ReadingOptions options) { readBinaryHeader(archive); if (saveVersion == 6) { // Name table is located after the objects block, but will be needed to read the objects block readBinaryNameTable(archive); } readBinaryDataFiles(archive, options); readBinaryEmbeddedData(archive, options); int unknownValue = archive.getInt(); if (unknownValue != 0) { // McBane: This was my showstopper //if (unknownValue > 1) { //throw new UnsupportedOperationException("Found unexpected Value " + unknownValue + " at " + (archive.position() - 4)); //} for (int n = 0; n < unknownValue; n++) { int unknownFlags = archive.getInt(); int objectCount = archive.getInt(); String name = archive.getString(); } } readBinaryObjects(archive, options); readBinaryObjectProperties(archive, options); } Link to comment Share on other sites More sharing options...
Stalefish Posted March 15, 2017 Share Posted March 15, 2017 @Qowyn bump, any news about ark-tools or that mod you were working on? Link to comment Share on other sites More sharing options...
StriderUk Posted March 18, 2017 Share Posted March 18, 2017 A plea to the php/db gods. Hi chaps, I have managed to sort out the retrieval of wild dino data, via this fantastic tool. I have at the moment, got the json files loading into a mysql db and then this is published on my website. I feel this is a pretty dirty way around of doing it, but I'll be honest i dont know a better way to do this. I have scanned the ww and just get lost in all the information out there. I know a few of you have managed to turn this data into a thing of beauty and I know i will never get to that with my skills, but would love it, if you could throw me some pointers. Stack overflow is a great resource, but i feel i am not properly asking what I require to know over there. Any pointers guys? Thanks in advance. Link to comment Share on other sites More sharing options...
StevenB Posted March 18, 2017 Share Posted March 18, 2017 @StriderUk I am no God with regards to php/db or html for that matter, but I managed to piece together something both visually pleasing and usable using mr Qowyns own ark-tools-web as a starting point. This uses the raw jsons that the tool provides as a datasource. Best of luck! Regards Steven. edit: seems I am not able to post URL's, but just go to Qowyns github and you'll find it. Link to comment Share on other sites More sharing options...
StriderUk Posted March 18, 2017 Share Posted March 18, 2017 thanks for that, I will check that out Link to comment Share on other sites More sharing options...
StriderUk Posted March 19, 2017 Share Posted March 19, 2017 Hey Steven, I dont suppose I could borrow you, or anyone for that matter? I have all the polymer server etc installed, webtools are there too, but unable to import in the dino data and not sure what I am doing wrong, any points or pms would be massively appreciated Cheers. Link to comment Share on other sites More sharing options...
FatDude Posted March 21, 2017 Share Posted March 21, 2017 On 18/3/2017 at 8:02 PM, StriderUk said: A plea to the php/db gods. Hi chaps, I have managed to sort out the retrieval of wild dino data, via this fantastic tool. I have at the moment, got the json files loading into a mysql db and then this is published on my website. I feel this is a pretty dirty way around of doing it, but I'll be honest i dont know a better way to do this. I have scanned the ww and just get lost in all the information out there. I know a few of you have managed to turn this data into a thing of beauty and I know i will never get to that with my skills, but would love it, if you could throw me some pointers. Stack overflow is a great resource, but i feel i am not properly asking what I require to know over there. Any pointers guys? Thanks in advance. Sure what do you need help with ? I've created below for my players: http://ark.sharpplay.com:81 I'm using MS SQL and etl packages, but the same can be done in mysql Any special task you need help with ? Link to comment Share on other sites More sharing options...
StriderUk Posted March 21, 2017 Share Posted March 21, 2017 Hi mate, Im using mysql, i know not ideal and with the free dev edition of ms sql, maybe I should be looking to change it. I am using a dirty tool called json to my sql to get the files into my db, via batch scripts, that load them in every 5 mins. I feel there must be a better way, every so often, my files retrieve the wrong column information in the etl process, but quickly correct theirselves. I have scoured the web, but come across so much conflicting information. I'd love to build something as you have done, but I really dont have the time, with work commitments. So am quite happy with a simple table based site, as i have done here, (its not pretty trust me). http;/warpspeed.zapto.org:1080/joomla/ Thanks in advance for any help at all you can give me, if only pointing me in the right direction Link to comment Share on other sites More sharing options...
FatDude Posted March 23, 2017 Share Posted March 23, 2017 On 21/3/2017 at 8:27 PM, StriderUk said: Hi mate, Im using mysql, i know not ideal and with the free dev edition of ms sql, maybe I should be looking to change it. I am using a dirty tool called json to my sql to get the files into my db, via batch scripts, that load them in every 5 mins. I feel there must be a better way, every so often, my files retrieve the wrong column information in the etl process, but quickly correct theirselves. I have scoured the web, but come across so much conflicting information. I'd love to build something as you have done, but I really dont have the time, with work commitments. So am quite happy with a simple table based site, as i have done here, (its not pretty trust me). http;/warpspeed.zapto.org:1080/joomla/ Thanks in advance for any help at all you can give me, if only pointing me in the right direction Mysql is fine for the purpose, since we are talking low data volume anyway. Have you tried any of the opensource php etl frameworks that support json files ? A quick search gave me this one, though I never used it: https://github.com/leomarquine/php-etl Also the dino info is coming from the world save which usually saves every 20-30min depending on settings. You can make a trigger php script that checks the world save timestamp and only starts your etl scripts once it's been updated. That will save you etl time. If you source in the players and tribes to own tables, you can start joining the data together to show ex all dino for a tribe. I parse the tribe data into a separate table with the tribeid as key, so I can join it on the tribes For tribes and players, I also source in the filename (removing the extensions) and use that as key See image: Since I use slowly changing dimensions (have history on players, tribes and dinos) I sourced into staging tables first (src.*) and then load the dim.* and fact.* tables You dont really need that, you can just load directly into you "normal" tables and do you reports on those of those. Link to comment Share on other sites More sharing options...
StriderUk Posted March 26, 2017 Share Posted March 26, 2017 Thanks for your help FatDude, this is exactly what i was looking for. Cheers Stryder Link to comment Share on other sites More sharing options...
MrJonesDK Posted April 6, 2017 Share Posted April 6, 2017 Is it possible to find the Unicorn? If I use this cmd line: ark-tools.exe wild c:\at\theisland.ark c:\at\output\ I cannot find it.. Edit: I can only find the once I tamed by using this: ark-tools.exe tamed c:\at\theisland.ark c:\at\output\ My ACM says, there should be a wild one spawned.. Gonna test if I can see one, if I kill my tamed Unicorns.. Edit: When I killed my Unicorn and after a couple of wild dino wipes, the Unicorn spawned and I could find it via this program :-) Link to comment Share on other sites More sharing options...
Stalefish Posted April 9, 2017 Share Posted April 9, 2017 There can only be one unicorn at a time. Link to comment Share on other sites More sharing options...
FatDude Posted April 10, 2017 Share Posted April 10, 2017 Hey mate. I dont know if you still work on this project but as I'm stream lining my import and moving it to azure could I have a few feature requests. Add playerid_fileid (filename) to player output file Add tribeid_fileid (filename) to tribeout output file Both are needed in order to join correctly (as tribe names can change, but the file id will not. Way better and also easy to track history on tribes and players) command -SingleFile (outputs all players/tribes to single file instead of multiple files. 1 Single file for players and 1 single file for tribes. Above id's would need to be added) This would give an huge performance gain when importing to a DB (also make importing a lot easier) Link to comment Share on other sites More sharing options...
andi79h Posted April 12, 2017 Share Posted April 12, 2017 On 12.1.2017 at 5:59 PM, StevenB said: Hi, Thanks for this great tool! I am having an issue exporting and importing dinos to a cluster file (havent tried other things). This a feature that is currently working or maybe I am using it wrong? Many more errors like the one shown here while exporting, but they are the same just different location in the file. ./ark-tools.sh export restore/2017-01-08_17.30.10/TheIsland.ark snav/argent.json --dinoid 1147157684391425702 --pretty-printing Warning: Unknown Struct Type SPLightSettingsStruct at 39dd5f9 trying to read as StructPropertyList ./ark-tools.sh import snav/76561198156293811 snav/argent.json snav/76561198156293811_new Warning: Unknown Struct Type ArkInventoryData at a9 trying to read as StructPropertyList java.lang.NullPointerException at qowyn.ark.tools.EditingCommands.importIntoClusterData(EditingCommands.java:421) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:273) at qowyn.ark.tools.App.main(App.java:114) ./ark-tools.sh import snav/76561198156293811 snav/argent.json snav/76561198156293811_new --file-format cluster java.lang.IllegalArgumentException: No enum constant qowyn.ark.tools.FileFormat.cluster at java.lang.Enum.valueOf(Enum.java:238) at qowyn.ark.tools.FileFormat.valueOf(FileFormat.java:9) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:237) at qowyn.ark.tools.App.main(App.java:114) Same here. I tried to alter a mammoth but I also get a NPE $ ./ark-tools.sh import centerPVE/export/76561198026916269 centerPVE/export/random200.json centerPVE/export/76561198026916269_new java.lang.NullPointerException at qowyn.ark.PropertyContainer.getPropertyValue(PropertyContainer.java:63) at qowyn.ark.PropertyContainer.getPropertyValue(PropertyContainer.java:58) at qowyn.ark.tools.EditingCommands.importIntoClusterData(EditingCommands.java:403) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:273) at qowyn.ark.tools.App.main(App.java:114) And also the cluster format does not work well. $ ./ark-tools.sh import Missing argument SAVE. Imports all objects from JSON into SAVE. Usage: ark-tools import SAVE JSON NEW_SAVE [OPTIONS] Option Description ------ ----------- --file-format Select format of input and output <cluster|localprofile|map|profile|tribe> files. $ ./ark-tools.sh import centerPVE/export/76561198026916269 centerPVE/export/random200.json centerPVE/export/76561198026916269_new --file-format=cluster java.lang.IllegalArgumentException: No enum constant qowyn.ark.tools.FileFormat.cluster at java.lang.Enum.valueOf(Enum.java:238) at qowyn.ark.tools.FileFormat.valueOf(FileFormat.java:9) at qowyn.ark.tools.EditingCommands.importThing(EditingCommands.java:237) at qowyn.ark.tools.App.main(App.java:114) On the other hand, I was able to c2j a file, modify it (its a bit tough to understand the structure) and then j2c it back into the cluster. Link to comment Share on other sites More sharing options...
Evilcious Posted April 15, 2017 Share Posted April 15, 2017 I was really excited about all of this but couldn't make a lot of the commands work. Maybe because of updates or because of the maps I'm using (no official maps : Ragnarok, Skies of Nazca and Roraima). Do you know if @Qowyn is still working on it ? Link to comment Share on other sites More sharing options...
StriderUk Posted April 17, 2017 Share Posted April 17, 2017 @FatDude I'm really interest actually how you got this to load in ssis. Have you used one of the free tools, or have you managed to cook something yourself that loads the json into a staging table? I have finally got around to getting sql 2016 onto my server and will probably end up going down this route for loading my data. I have tried to load the data, using the new sql commands built into 2016, would be interested to know how you got this working if you dont mind? cheers Phill Link to comment Share on other sites More sharing options...
FatDude Posted April 17, 2017 Share Posted April 17, 2017 5 hours ago, StriderUk said: @FatDude I'm really interest actually how you got this to load in ssis. Have you used one of the free tools, or have you managed to cook something yourself that loads the json into a staging table? I have finally got around to getting sql 2016 onto my server and will probably end up going down this route for loading my data. I have tried to load the data, using the new sql commands built into 2016, would be interested to know how you got this working if you dont mind? cheers Phill Im currently using one of the free opensouce json ssis compoments, If works fine but is a bit weird to setup and need to loop though all the files in a loop https://jsonsource.codeplex.com/ But Im playing with the json support in the new sql server which would probably be a better and easier way.. Some sample code you can look at below ( using a blob storage, but can easily be changed to read from a local file(s) instead) Could write some stored procs to load the data to source tables SELECT BulkColumn FROM OPENROWSET (BULK '1022680475.json', DATA_SOURCE = 'MyAzureBlobStorageTribes', SINGLE_CLOB) as j CROSS APPLY OPENJSON(BulkColumn) SELECT players.* FROM OPENROWSET (BULK '76561197960815112.json', DATA_SOURCE = 'MyAzureBlobStoragePlayers', SINGLE_CLOB) as j CROSS APPLY OPENJSON(BulkColumn) WITH( id int, playerName nvarchar(100),name nvarchar(100), level int, tribeId int,tribeName nvarchar(100)) AS players Some screen dumps of the the ssis setup: Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.