cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
CharlesS
Kudo Kingpin
Kudo Kingpin

Dataflow Engine V2 - What exactly changed?

I noticed an important difference today while creating a new dataflow, a new dataflow type called Standard V2. In the past, my new dataflows were always created as V1, but today (without doing anything differently, I think) my new dataflow was a V2.

 

Does anybody know what the changes are between V1 and V2?

 

I suspect there are changes in relation to the way the dataflow engine interprets dates and choices, because my "old way" of doing things now results in failed loads.

 

CharlesS_0-1612567234020.png

 

5 REPLIES 5
CharlesS
Kudo Kingpin
Kudo Kingpin

Here a few tips for those having issues with the Standard V2 dataflow engine:

  1. When working with dates, ensure that the date format in your dataflow is exactly the same as the date format in Dataverse. In particular, look out for slashes and dashes. If your date data is stored with dashes in dataverse (2020-02-06), make sure your dates are also structured with dashes in your dataflow. The Standard V1 dataflow engine was more "permissive" to this effect, accepting slashes instead of dashes. This no longer seems to be the case with Standard V2.
  2. The Standard V2 dataflow engine doesn't like importing null data in choices. This wasn't a problem with the Standard V1 engine, but it is now. My recommended solution is the following: create a new choice called "None" (or anything else you like). Take note of the numerical value associated with your newly created choice. In your dataflow, replace all null fields with that numerical value. 

If you disagree with my findings, please comment below, I have been known to be mistaken from time to time... 😉Nevertheless, the solutions above worked fine for me. If you have any other tips relating to the Standard V2 engine, please comment below!

Hi @CharlesS ,

 

in addition to your findings I've recognized with Standard V2 Dataflows there are no "integration keys" available anymore after first run of dataflow. With changing of alternate key incl. lookup columns ypou can't set up the right integration columns anymore cause there's no entry to your table anymore...

Till now theres nothing better with Standard V2 engine for me...:/

 

BR,

 

Sven

Hi @CharlesS ,

 

just ran into the same issue with sensitve dateformat for standardv2 dataflows.

I've an alternate key for a table with 2 columns:

- crefb_year (text type)

- crefb_validfrom (date type store as yyyy-mm-dd)

 

Have tried to set up the dataflow but I'm not able to "format" the date column matching for the following odata request (behind the dataflow action):

 

It MUST be like this

KSven_0-1615808828026.png

When I set up the "crefb_validfrom" column as string with Power query I will get the following error protocol:

"/crefb_marketforecasts(crefb_validfrom='2021-01-01',crefb_year='2021')""

--> So there are some apostrophe before and after my date value.

 

So I've tried to set up the date column as date column with Power query. Here I will get the following error protocol:

"/crefb_marketforecasts(crefb_validfrom=1/1/2021,crefb_year='2021')""

--> So there are no apostrophe anymore, but not the "yyyy-mm-dd" format.

 

My question:

How to set up date column in Power Query to ensure "yyyy-mm-dd" value without apostrophe?

Already have tried the english (sweden) locale format, but of course it will stay as "d/m/yyyy" format.

 

BR,

Sven

Hello @KSven ,

 

Sorry for my late reply. I think the Microsoft team are making backend changes. I just posted a new thread this morning. I have a dataflow used to import data into Date Only fields. It has been working fine for several weeks. This morning, surprise, the dataflow is failing:

New Dataflow v2 problem with Date Only imports - Power Platform Community (microsoft.com)

 

I trust Microsoft will look into these date/time issues and ensure any fixes are fully functional, whatever locale they are using.

 

To be continued...

 

Keep 'em flowin'

 

Charles

Hi @CharlesS ,

 

no problem, already have created MS Support Ticket some days before. Haven't received some more information about this bug, but I have 2 more Tickets about reference links via Datflows. So I guess its a Main Problem with Standard V2 Dataflow (especially regarding to the integration key setting with standard v1 I guess they tried to set it automatically with v2, but failing till now 😄 )

 

BR,

Sven

Helpful resources

Announcements
Power Apps Africa Challenge 2022

Power Apps Africa Challenge

Your chance to join an engaging competition of Power Platform enthusiasts.

Super User 2 - 2022 Congratulations

Welcome Super Users

The Super User program for 2022 - Season 2 has kicked off!

September Events 2022

Check out all of these events

Attend in person or online, there are incredible conferences and events happening all throughout the month of September.

Government Carousel

New forum: GCC, GCCH, DoD - Federal App Makers (FAM)

In response to the unique and evolving requirements of the United States public sector, Microsoft has created Power Apps US Government.

Users online (2,306)