cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
Frequent Visitor

DataFlow to DataLake - Automatic transfer without programming

Hello, 

 

I have created dataflow which gets data from Dynamics (through ODATA). I am able to see all entities created as custom entities in Entity option. I want to export the data to Data Lake. 

 

I am even able to export these custom entity to data lake using Entity --> Export to Data lake option. I also see option from main menu as "Export to data lake"

 

Could you please confirm is there any option available which will automatically push the data to Data Lake without manually pushing data. I am checking option without Powerapp code/programming. 

 

Is there any better option to setup automatically transfer data to Data Lake

 

pbagdiya_0-1595616294204.png

 

pbagdiya_1-1595616426401.png

 

 

 

4 REPLIES 4
Super User
Super User

it is automated continuous incremental replication

See
https://docs.microsoft.com/en-us/powerapps/maker/common-data-service/export-to-data-lake

Export to Data Lake provides these features:

  • Linking or unlinking the Common Data Service environment to Data Lake Storage Gen2 in your Azure subscription.
  • Continuous replication of entities to Data Lake Storage Gen2.
  • Initial write, followed by incremental writes for data and metadata.
  • Replication of both standard and custom entities.
  • Replication of create, update, and delete (CrUD) transactions.
  • Continuous snapshot updates for large analytics scenarios.
  • Facilitated metadata discovery and interoperability between data producers and consumers such as Power BI, Azure Data Factory, Azure Databricks, and Azure Machine Learning.

 

Thanks for the response. I understand following resolution, however I am not able to understand how should I make "Export to data Lake" option automatically from the given notes/docs. 

 

As I said previously, I can export dataflow to data lake manually (through "Export to data lake") option but can schedule/automate it. 

Automate means, As soon as I create dataflow, it should load and transfer data to Data Lake. I guess this might not be possible as well since custom entity always needs "Change Tracking" flag enable into to export. As per my knowledge this flag is false during creation.  

see the link I posted. Once you export to data lake it becomes automatic. Like dataflows you cannot configure the schedule.

Thank you for answering my question. 

 

Does it mean that I have to refresh data only once while creating Dataflow? 

Helpful resources

Announcements
New Badges

New Solution Badges!

Check out our new profile badges recognizing authored solutions!

New Power Super Users

Congratulations!

We are excited to announce the Power Apps Super Users!

Power Apps Community Call

Power Apps Community Call: February

Did you miss the call? Check out the Power Apps Community Call here.

Microsoft Ignite

Microsoft Ignite

Join digitally, March 2–4, 2021 to explore new tech that's ready to implement. Experience the keynote in mixed reality through AltspaceVR!

Top Solution Authors
Top Kudoed Authors
Users online (89,900)