Sql – Is using multiple tables an advisable solution to dealing with user defined fields


I am looking at a problem which would involve users uploading lists of records with various field structures into an application. The 2nd part of this would be to also allow the users to specify fields to capture information.

This is a step beyond anything ive done up to this point where i would have designed a static RDMS structure myself. In some respects all records will be treated the same so there will be some common fields required for each. Almost all queries will be run on these common fields.

My first thought would be to dynamically generate a new table for each import and another for each data capture field spec.Then have a master table with a guid for every record in the application along with the common fields and then fields that specify the name of the table the data was imported to and name of table with the data capture fields.

Further information (metadata?) about the fields in the dynamically generated tables could be stored in xml or in a 'property' table.

This would mean as users log into the application i would be dynamically choosing which table of data to presented to the user, and there would be a large number of tables in the database if it was say not only multiuser but then multitennant.

My question is are there other methods to solving this kind of varaible field issue, im i going down an unadvised path here?

I believe that EAV would require me to have a table defining the fields for each import / data capture spec and then another table with the import – field – values data and that seems impracticle.

Best Solution

I hate storing XML in the database, but this is a perfect example of when it makes sense. Store the user imports in XML initially. As your data schema matures, you can later decide which tables to persist for your larger clients. When the users pick which fields they want to query, that's when you come back and build a solid schema.