CSV to structured Dictionary

Question

Is it possible to recreate a structured dictionary from a csv file?

Suppose we want a dictionary like:

onready var Library = {

    "Books":{
        #EXAMPLE OF A BOOK STUFF
        "Cooking 101":{
            "RECIPES":10,
            "SOUND":"COOK_BOOK_SND",},     

        "Psychology": "BOOK_STUFF",
        "Fishing tips": "BOOK_STUFF",
        "Dresses of 20 century": "BOOK_STUFF"
    }
}

In the CSV is like:

How we would import the csv in order to recreate that dictionary?

The goal is to manipulate BIG amount of data using CSV, which will avoid duplicating dictionary structures inside godot scripts.

I tried various methods but always ended up scratching my head as to how implement this.

-Godot 3.2.1 stable official

So far my research has found the following topics:

How to use CSV formated Files with Godot
https://www.youtube.com/watch?v=FiavSnFC8HE

How can I import a .csv or .txt file?
https://godotengine.org/qa/40566/how-can-i-import-a-csv-or-txt-file

GMS function load_csv() in Godot
https://godotforums.org/discussion/20075/gms-function-load-csv-in-godot

Answer 1

Woops, forgot to post the answer, my bad. I Got it to work properly, it's really quite simple, you just need to wrap your head around a system to use the lines.

for LINECSV in DATA_CSV: # Executed "for" each line of CSV
    if !LINECSV == 0 and LINECSV != DATA_CSV.size()-1: 
        # == 0 because they are my categories, not the data I want.
        #size()-1 is because my DATAFromCSV adds a extra ghost line.    

        var ID = DATAFromCSV[LINECSV][0]
        var Title = DATAFromCSV[LINECSV][1]
        var SubTitle = DATAFromCSV[LINECSV][2]

        #ID, TITLE_CATEGORY and TITLE
        LIBRARY[ID] = {Title:SubTitle}

Comments

Mh ? How is it working ? I paste your code in a project and DataFromCSV is not recognized, can you explain more please ?

I'm working on it since 1 entire day and didn't manage to get my CSV in a double dictionary...

Answer 2

Anything is possible :) I haven't dealt with CSV in Godot but it would certainly be possible to write a parser that gives you dictionary objects.

You might want to reconsider CSV for your database, there are some cool tools that can output JSON in a way that is much easier to parse in Godot eg. CastleDB - take a look at this video https://www.youtube.com/watch?v=TdWWiVd1IxQ

Comments

Looks like a great tool, I just looked it up, but I still wanted to use CSV because I already had a data structure to use and I didn't wanted to change it :>, I eventually got it to work, I post it as an answer.

---

I am looking into that particular situation myself and I tried CastleDB. It does make the exporting into JSON easy but it lacks very important feature, like sorting columns, filtering, searching, editing multiple values at the same time, etc. A modern spreadsheet editor has much better tools for that. The nice thing about CastleDB is that is links data between tables, so I still have no good solution for creating, tweaking, balancing my big database.

---

While yea JSON has some advantages like faster parsing, I liked the way I could design a compiling structure from my raw string data imported from CSV files, so far I have not had any problems with perfomance, so I sticking with it.

I needed a lot of data organized and the way I have done was through .CSV files, in the end everything worked as expected, however, later I runned into some minor issues.

-> When exporting the game, no .csv files were exported and the console would loop with errors like "Needs to be open before use", the problem was that I couldn't include .csv files format in the exported game in any way shape or form. The workaround was simply to rename them to .tres, which makes godot export them automatically, and I still could open them with spreadsheet programs.

-> When editing the files, the program that I used was locking them, and Godot couldn't run the game, which made a pain for testing the game, as I had to change the values that I needed, close the program, them test on Godot, open the spreadsheet later, edit, repeat. The solution was to duplicate the data folder outside of the project and run with a thirdparty program a simple folder sync to the godot project folder, so I could edit multiple data files, press a sync button, and test the game without worrying about closing the program.

Eventually you only need 1 single file for all data, you can discriminate multiple structures in 1 file. For example, you could just put a special character in the csv files that when compiling in Godot make it ignore the spreadsheat field if it contained that character. Something similar like my code but better imo:

if !LINECSV.has("#") and LINECSV != DATA_CSV.size()-1:  #Compile data

This will make you able to split multiple data structures in one file, you could do say:

 if DATA_CSV[LINECSV][0].has("WEAPON_"): 
    var WEAPON_ID = DATA_CSV[LINECSV][0] #Because it has WEAPON_ as prefix, the compile structure here can consider it as a weapon, you could do that with NPC_Data, MAP_Data, STAT_Data, SKILL_Data, or anything you want.

The key with compiling methods from the csv files was the string built-in functions in Godot, for example, I could split a string everytime it encoutered a '/' , and creates a array from that, or I could split between two numbers by ':' and consider them a vector2, then in my spreadsheet I could write values like: [32:64] and godot would read as Vector2(32,64).

It's really powerful what you can do with simple values in a spreadsheet, I seriously hope they implement in Godot a way to edit csv files directly, it's too dam powerful, we can make use of databanks for any purpose we want, and some games cannot be developed without them.