Jump to content


  • Content Count

  • Joined

  • Last visited

  • Feedback


Community Reputation

0 Gathering Thatch

About FatDude

  • Rank

Recent Profile Visitors

838 profile views
  1. Dude, if you got everything imported in ms sql db, then why drop it ? Drop the SSRS part, those reports sucks for this purpose anyway Just connect directly to the db from your php/whatever webpages. Use some of the free java table builder like https://datatables.net/ or similar to build the tables.. The use the php steam login api (or similar) to match the steamid to the player logged in
  2. My system is not really shareable in its current form and a bit overkill / complicated, I kinda built it as a proof of concept for fun. Would need to drop the ssis packages and do the data loading / slowly changing dimension (type 2) via stored procedures instead. Also since I got MSDN access, I'm using the enterprise version of MS sql server. Would need to convert it to work on the free edition. Anyway I will think about it.
  3. Im currently using one of the free opensouce json ssis compoments, If works fine but is a bit weird to setup and need to loop though all the files in a loop https://jsonsource.codeplex.com/ But Im playing with the json support in the new sql server which would probably be a better and easier way.. Some sample code you can look at below ( using a blob storage, but can easily be changed to read from a local file(s) instead) Could write some stored procs to load the data to source tables SELECT BulkColumn FROM OPENROWSET (BULK '1022680475.json', DATA_SOURCE = 'MyAzureBlobStorageTribes', SINGLE_CLOB) as j CROSS APPLY OPENJSON(BulkColumn) SELECT players.* FROM OPENROWSET (BULK '76561197960815112.json', DATA_SOURCE = 'MyAzureBlobStoragePlayers', SINGLE_CLOB) as j CROSS APPLY OPENJSON(BulkColumn) WITH( id int, playerName nvarchar(100),name nvarchar(100), level int, tribeId int,tribeName nvarchar(100)) AS players Some screen dumps of the the ssis setup:
  4. Hey mate. I dont know if you still work on this project but as I'm stream lining my import and moving it to azure could I have a few feature requests. Add playerid_fileid (filename) to player output file Add tribeid_fileid (filename) to tribeout output file Both are needed in order to join correctly (as tribe names can change, but the file id will not. Way better and also easy to track history on tribes and players) command -SingleFile (outputs all players/tribes to single file instead of multiple files. 1 Single file for players and 1 single file for tribes. Above id's would need to be added) This would give an huge performance gain when importing to a DB (also make importing a lot easier)
  5. Mysql is fine for the purpose, since we are talking low data volume anyway. Have you tried any of the opensource php etl frameworks that support json files ? A quick search gave me this one, though I never used it: https://github.com/leomarquine/php-etl Also the dino info is coming from the world save which usually saves every 20-30min depending on settings. You can make a trigger php script that checks the world save timestamp and only starts your etl scripts once it's been updated. That will save you etl time. If you source in the players and tribes to own tables, you can start joining the data together to show ex all dino for a tribe. I parse the tribe data into a separate table with the tribeid as key, so I can join it on the tribes For tribes and players, I also source in the filename (removing the extensions) and use that as key See image: Since I use slowly changing dimensions (have history on players, tribes and dinos) I sourced into staging tables first (src.*) and then load the dim.* and fact.* tables You dont really need that, you can just load directly into you "normal" tables and do you reports on those of those.
  6. Sure what do you need help with ? I've created below for my players: http://ark.sharpplay.com:81 I'm using MS SQL and etl packages, but the same can be done in mysql Any special task you need help with ?
  7. --positions is trowing some error about missing file. I copy world.ark and *.arktribe, *.arkprofile from the ark server to process on another server. What else I need to copy (If other files are needed) You wrote it read the map, I also need to copy the map from ?
  8. Wow great. Would it be possible to also extract player's lat/lon as you do with the dinos from the worldsave ? Also would if be possible to include tribeid and playerid in the dino extact, since tribe name and player name might not be unique and causes join problems
  9. Sure, I understand your reason. I don't really need help atm, for me its more about the lack of time due to irl work ;-/ The going from scratch php route just takes more time, but also gives the best result. Will have to consider. My pvp calculations are based on the tribe logs where I extract the killed by player/dino/which tribe and do calc on top of that(semi working atm) While serverlogs/chatlogs would be the best option, I know its bugged and have it disabled, so using the tribelog for kill info On retrieving ingame chat I can give you hint. rcon getchat command ;-) ( Patch 185.4: Chat can now be retrieved in RCON -- you just have to use the RCON command "getchat" to return the latest chat buffer. ) Only problem is it will clear the chat buffer, so cant also use an external rcon system for retrieving the chat at the same time (batllemetrics rcon system in my case) My pvp and point system is among other used for commands like !putbounty 'Playername 'points', which will deduct the points from the player and give to the killer once completed For this to work propper I need to check for unique player names and automatically rename last created player with same name via rcon tool. Open/collected bouties can be shown on both website and ingame either automated or via the chatbot Via mcrcon.exe as you mentioned you can do lots of fancy stuff. Convert point to rewards (rcon giveitemtoplayer) and remove the points in the db. So you can basically automate the reward system any way you like. Like 1000 points needed for xx skin etc. I have many more crazy ideas floating in my head atm, just gotta think out of the box ;-) So lots of stuff planned, some is working already and some is wip and running as poc(proof of concept) on my test server, just neeed more time :-/
  10. Toni, you did everything I have planned, just faster, better and more elegant ;-) My route with powerbi kinda sucks due to limitations in powerbi and what it should be used for. (Could based, data refresh restrictions etc) I have everything sourced in my DB and got the data model in place, just not found the best way to visualize. Your project, is that something you would want to share with me or the community ? Would really like implement similar on my servers as well. Also I'm working on some additions like an ingame chat bot with commands like !TopTribe which will use rcon command for announcing ingame I know time is money, so if that's the issue I'm sure we can find out something if needed. Let me know
  11. I'm currently using Microsoft Power BI See the example image for tamed dinos on one of my servers (removed tribe name here) I can then create reports that can be shown on a website based on the Ark data tools. Can then different reports with players/tribes, tribelogs, kills, tamed dinos everything that can be pulled out with this handy tool. With scheduled refresh the reports are showing live data (or semi live, since I currently extact data each hour)
  12. Great progress. Would it be possible to also extract player's lat/lon as you do with the dinos from the worldsave ? Among all the other stuff, I'm also building a player heatmap and need the player locations for that Thanks for your great effort on this tool
  13. Thanks for the last update, really appreciated. Really nice with the dino ids. Already used it to track some lost dino between different world saves. Man its really helpfull Some stuff to look at when you have time. 1/3 of my ark profiles says possible corrupted and is not exported it seems. (On a few days old server running Scorched Earth)
  14. While analyzing the tamed dinos data it seems to include dead dinos or perhaps there is some data issues in the extract ? TP'ed to the location on the centermap and the tamed dinos was not there nor other dinos of that type (dodo) Im using the creatures to export all dinos. Another funny thing the somewhat bugged spawns on the Center map assuming the extract is just somewhat correct. I knew the mosa spawns was bugged (overspawning), but coel and trilo ? type nr C:\ark-tools\dinos\Trilobite_Character_C.json 19235 C:\ark-tools\dinos\Coel_Character_BP_C.json 8093 C:\ark-tools\dinos\Mosa_Character_BP_C.json 3627 C:\ark-tools\dinos\Dragonfly_Character_BP_C.json 3626
  • Create New...