cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Doxelnaut
Regular Visitor

Best Practice For Automated Data Synchronization

Hi,

I am working on a Power Platform product that will require an automated integration with a foreign system and am looking for some input regarding best practices to accomplish my goals. The integration needs to run every few hours and connect to the other system via a REST API endpoint to collect data that will need to undergo some transformation and then be processed into my Model Driven App.

 

So far I have been working with Data Factory to accomplish this task, however I am finding that due to the frequency and nature of the transformations this solution is not cost effective. I also have concerns about the productization of such a solution.

 

Is Data Factory the wrong answer for my stated goals? I am looking into the possibility of switching to either Azure App Service or Azure Logic App Service as alternatives. Any suggestions or personal experiences are welcome.

 

Thank you!

1 ACCEPTED SOLUTION

Accepted Solutions
cchannon
Continued Contributor
Continued Contributor

Depending on the volume of data being moved, there may be other elegant solutions available to you.

 

With PowerAutomate, you can set up custom connectors and webhooks so that a PA Flow could connect to this external data source and migrate the data on a transactional basis. This would not be a good solution for bulk movement, but if you can set it up transactionally it would be a very easy way to keep the systems in constant sync.

 

Probably MSFT would recommend you use DataFlows for this though. They are another great tool for managing unidirectional data movement and they also come with a ton of baked-in connectors, and MSFT is starting to push them as a catchall unidirectional data movement tool for powerplatform and beyond. You can use them to schedule and process the transformation and migration of data on an ongoing basis with practically zero code through a Power Query. There are some limitations, but on the whole you can do just about any transformation and processing you need and they will handle reasonable data migrations in reasonable timeframes.

View solution in original post

2 REPLIES 2
alrez
Microsoft
Microsoft

Hi,

 

Data Factory should work for your solution that you have proposed. Unfortunately, I'm not certain about the other services. I only have limited knowledge into them. If anyone else from the communities has any opinions on this solution feel free to chime in.

 

Regards,

 

Alex

 

-------

 

Community Support Team _ Alex Rezac
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

cchannon
Continued Contributor
Continued Contributor

Depending on the volume of data being moved, there may be other elegant solutions available to you.

 

With PowerAutomate, you can set up custom connectors and webhooks so that a PA Flow could connect to this external data source and migrate the data on a transactional basis. This would not be a good solution for bulk movement, but if you can set it up transactionally it would be a very easy way to keep the systems in constant sync.

 

Probably MSFT would recommend you use DataFlows for this though. They are another great tool for managing unidirectional data movement and they also come with a ton of baked-in connectors, and MSFT is starting to push them as a catchall unidirectional data movement tool for powerplatform and beyond. You can use them to schedule and process the transformation and migration of data on an ongoing basis with practically zero code through a Power Query. There are some limitations, but on the whole you can do just about any transformation and processing you need and they will handle reasonable data migrations in reasonable timeframes.

View solution in original post

Helpful resources

Announcements
PA User Group

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

secondImage

Are Your Ready?

Test your skills now with the Cloud Skill Challenge.

secondImage

Demo Extravaganza is Back!

We are excited to announce that Demo Extravaganza for 2021 has started!

Users online (67,205)