Bulk Import & Relationship

[Bahalzamon]

I have a couple of large CSV files that I am looking to import into the Dataverse.

There are hundreds of thousands of records, I am wondering how I go about filling in the Lookup portions of the tables. I think I could do it through Power Automate over a considerable amount of time due to the volume. But From past experience I have received throttling when attempting to have power automate run a large number of things through it.

As a side note, is there a way to import that data into multiple tables? What I am referring too is importing one table and assigning the data into 2 different tables as the data structure has changed.

[bipinshan]

Hi @Bahalzamon ,

Have you looked at the dataflows for powerapps?

https://docs.microsoft.com/en-us/powerapps/maker/data-platform/create-and-use-dataflows

I think you will need to to split your csv into 2 two and import the file which does not have dependency then import second file.

Another approach is to create dummy text field in your first entity and write plugin to create related table based on those text fields values.

Regards,

Bipin Kumar
Follow my Blog: https://xrmdynamicscrm.wordpress.com/

Kudos and Accept my answer if helpful!

[dpoggemann]

Hi @Bahalzamon ,

I utilize Excel import primarily but agree this would take a while with hundreds of thousands of records!  🙂 

https://docs.microsoft.com/en-us/powerapps/user/import-data 

If we have complex relationships / large numbers of records, we usually utilize eOne Solutions SmartConnect to accomplish as it has great capabilities for mapping to many different sources and destinations and can do more complex mapping relationships as needed.  This is not a free tool though...  As @bipinshan mentioned, Dataflows is an option as well, I have just not spent a ton of time working with this tool.

[Bahalzamon]

@bipinshan ,

I am attempting to use this and am coming across a couple of issues.

• There doesn't seem to be ANY way to relate these records as I am working on importing a small data set and the mapping portion for the Lookup column doesn't even display to tell it to find something. Do you know of a way to map either after the fact or during?
  • I did find THIS, but not 100% on if it can apply in my situation or even how to accomplish it without some additional research. And if it would mess up any of the existing records that are related.
• Lastly Choice columns. I have a choice column with all of the USA States as 2 letters (CA, AZ, TX, NY, etc...) and the field I am mapping to it is exactly that, but I am assuming it is having an issue as choice fields in the Dataverse are stored as ID values and displaying the text value. Does anyone know of a way to still get this info imported into the table properly?