Suggestion: Expose function for creating valid field names

[iainmwallace]

Hi,

It would be really useful to expose the function that the import function uses to convert the column names in a file to the column name that appears in the solvebio.
This would avoid any issues when supplying custom field definitions.

Based on trial and error, I believe the function removes spaces and converts all text to lowercase, but I am not sure if there are other edge cases

Cheers,

Iain

[davecap]

Hi Iain,

Can you provide an example of what you're seeing? Is it from a CSV/TSV file?

It's possible that the field transform is only done on CSV/TSV headers, but not for other formats like JSON. If that's the case, it will do the following:

x.replace(' ', '_').lower()

It actually may not be necessary for us to do this, we'll look into it and see if we can leave the fields as-is.

[iainmwallace]

Yeah, it was a CSV file. If it is possible to not change the field names, I would strongly recommend doing that.

I would also suggest using readr to guess the column types. The import auto-guess feature doesn't seem to work as well. I am planning on using readr to guess the column types, and then create the appropriate template. I can let you know how I get on.

Also, I tried to use the import function to import directly rather than via file, but that gave an error. It would be very nice if one could import a tibble directly from this.
Specifically, the error was:
DatasetImport.create(dataset_id=dataset$id, data_records=x)
Error in DatasetImport.create(dataset_id = dataset$id, data_records = x) :
Either an upload ID or manifest is required.