cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Gary_Eden
Helper II
Helper II

Connecting to Multiple Dataverse Environments (including Dataverse for Teams)

Hi, 
There are a few topics on moving data between Dataverse environments but they don't really cover what I'm looking for.

 

I have two scenarios that I want to address:

Scenario 1

We use the typical Dev/Test/Live set up on a client Tennent and need a simple way to back-fill data from the live environment back to test and dev. I know you can do this using csv files to export and import but this is clunky and as the client wants to do this on regular (probably monthly) basis and there is a lot of data to migrate.

I've built and successfully used scripts (formulas) in PowerApps to compare and migrate data from one source to another and back again e.g. Salesforce to Azure SQL to Salesforce or between SharePoint lists and Dataverse etc. These scripts are quick and reliable and just need a click from the user to execute them. 

What I need is the ability to directly connect to two or more Dataverse environments so I can reuse the scripts I've used before - one click could then populate multiple Dataverse instances at the same time.

 

Scenario 2

If scenario 1 is possible this should be a doddle but just in case a different method is needed.

We have a client that has both standard and Teams versions of Dataverse. The majority of end users on the use the apps a few times a year so for licencing it makes sense to use PowerApps for Teams/Dataverse for Teams. However, we have a requirement for the data stored within the Dataverse for Teams environments to be visible in apps built in the standard way. This data needs to be seen live and doesn't need to be exported/imported.  

 

 

I suspect that it may not be possible to connect one PowerApp directly to multiple Dataverse environments - I'd be very happy if proven wrong as it would make life easier.

 

Assuming it's not possible to connect to multiple Dataverse environments my thoughts are to use the following:

1. For data transfer use a series of SharePoint lists as a carrier for the data transfer. One click export from the app in one environment and one click import/update from the app in the other environment.

2. For the live data connection use a Flow to act as the bridge between the various Dataverse environments.

 

Am I missing anything obvious?

 

Thanks in advance.

 

1 ACCEPTED SOLUTION

Accepted Solutions
SBax
Impactful Individual
Impactful Individual

Hi Gary,

 

I know a much simpler solution.

 

Use the Dynamics 365 Data Migration utility to transfer the data between your environments, it's a lot less effort and will do exactly what you need 

 

Details here

View solution in original post

4 REPLIES 4
Gary_Eden
Helper II
Helper II

Hi 
Further to my earlier post I've been experimenting with Dataflow to migrate the data. This is successful up to a point but the blocker is the migration of the autogenerated values. This is confirmed by some test code I've ran in PowerApps which returns the error "The specified column is generated by the server and can't be specified".

 

The problem I have is that the column throwing the error is the one which has the same name as the table and holds the record GUID. There are 40+ tables and the GUIDs are used throughout as foreign keys.

 

Is there any way to bypass the auto generation and save the GUIDs from one table to another?

Yes I know what GUID stands for 🤣 - as my original post describes the goal is to create a mirror of the production data in Dev & Test as the volume of live data is now causing performance issues and we need to improve the code to compensate for this.

 

BTW I didn't build it, I've had enough bad experiences with Dataverse (CDS back then) in the past to be cautious about which 'helpful' bits to use in Dataverse. 

  

SBax
Impactful Individual
Impactful Individual

Hi Gary,

 

I know a much simpler solution.

 

Use the Dynamics 365 Data Migration utility to transfer the data between your environments, it's a lot less effort and will do exactly what you need 

 

Details here

Gary_Eden
Helper II
Helper II

Thanks @SBax 

I've just had a quick skim read and it looks promising. 

I'll give it a go and report back... fingers crossed.

 

Hi Sbax,

Thanks for point me in the right direction.

For others finding this thread the step-by-step notes are here

Note: The Data Migration Utility works off the back of Solutions - this caught me out at first, you need to create a Solution with your tables first and them export/import the Solution to it's new home.

 

Tip: use an obvious name for your solution to make it easier to find in the utility.

 

System generated values (GUIDs etc.) and lookups, choices, etc. are all replicated in tact ensuring lookups and references work exactly the same where compared to the original location. Looking at the exported XML file, images are referenced twice (by imageid and image) - image holds the Base64 value. 

 

(Having an XML file also makes it easier in the future to take exports/backups and copy over to other databases types if needed.)

 

Important: The utility will not migrate files/attachments. 

 

The XML doesn't have a field that appears to relate to an attached file.

 

I recall stumbling across a thread sometime ago discussing where Dataverse stores uploaded files. From what I remember this came across as a 'black art' as no one was able to find out where files were stored or more importantly how to retrieve them directly from Dataverse (rather than through the app).

 

Does anyone know where the Dataverse files are kept?

And/or, has anyone used a Flow (Power Automate) to automatically retrieve and save files locally? 

 

Cheers

Gary

Helpful resources

Announcements
Ignite 2022

WHAT’S NEXT AT MICROSOFT IGNITE 2022

Explore the latest innovations, learn from product experts and partners, level up your skillset, and create connections from around the world.

Power Apps Africa Challenge 2022

Power Apps Africa Challenge

Your chance to join an engaging competition of Power Platform enthusiasts.

Top Kudoed Authors
Users online (1,558)