cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
calerof
Advocate II
Advocate II

DataFlow does not bring all records to the final table in Dataverse

Hi,

I'm creating a DataFlow appending 3 tables from different sources:

  1. Excel file in OneDrive for Business
  2. SharePoint List
  3. Imported table from SQL Server

After several attempts I got it working but when checking the actual data, I see some records missing, but in the DataFlow they seem to be correct.

error1.png

Here's an screen-shot of one item with more than 15 rows in the DataFlow.

 

error2.png

Here is a filter of the table in Dataverse for the same item, it only has 8 rows.

 

Any help would be extremely appreciated.

Thanks,

Fernando

1 ACCEPTED SOLUTION

Accepted Solutions

I'm thinking the table you extracted from Dataverse shows that the missing records are actually not in Dataverse. You can always check with a view as well to be sure they are not there.

 

If they are indeed missing, here is what I would do:

1. Filter your dataflow to include only the 15 rows you are trying to import.

2. Once the refresh is complete, check the refresh history (see@EricRegnier 's reply) and open the log to validate how many upserts you got. If someone else in your organization is running that dataflow, ask him or her to do the same. If the upserts number is not the same, then @dpoggemann idea about user rights to data would need to be investigated further.

3. What are you using as a primary key? Is there any way the data could be overwriting itself?

 

 

 

View solution in original post

10 REPLIES 10
mahoneypat
Solution Sage
Solution Sage

Just to clarify.  You are using a Dataflow to combine 3 sources which is working correctly, but when you view the Dataverse table in Excel rows are missing?

 

Regards,

Pat

Hi @mahoneypat 

The DataFlow creates an appended table Cardex. When I look at the data in the DataFlow everything looks fine, but when I look at the actual Cardex table it's incomplete. No, I don't use Flow.

EricRegnier
Super User II
Super User II

Hi @calerof,

Just to clarify, the 3 sources import into 3 different tables, correct? If so, which one is causing issues? Have you checked in advanced find without any filters to confirm that records are missing? Also, can you check that there are no errors/warnings after a dataflow run? To check go to make.powerapps.com --> Data --> Dataflows, click the little warning icon next to the Last Refresh column:
2020-06-13_8-19-14.png

Cheers

 

Hi @EricRegnier ,

The Dataflow image above shows the original tables and the new appended one is called "cardex". In this new table within the Dataflow all records appear to be complete.

I didn't find the Advanced Find, this is a Canvas app, but I checked for example one record of date 13-feb-2021, which appears in the Dataflow image above with DocNum 1100 and Qty of 10 two times, and in the final appended table it does not exist.

error1.png

Also, there are no errors nor warnings in the Dataflow refresh:

error2.png

Still I don't know what's going on.

Thanks for the help.

Regards,

Fernando

dpoggemann
Resident Rockstar
Resident Rockstar

Hi @calerof,

 

Quick question, have you verified that the records showing in your dataflow were ever created in the Dataverse?  The 15 records you show have DocNum values where in your screenshot from the 8 rows in the dataverse only 5 of the 8 DocNum values match and if you look at these 5, none of the Fecha or CodigoTransaccion values match between the two.  I am guessing none of these records have been written to the cardex table.

 

Am I looking at something incorrectly?

 

Thanks,


Drew

Hi @dpoggemann ,

Thank you for your reply. Here are two screen shots of the complete Dataflow, all records from the appended table "Cardex" are in there:

Dataflow1.pngDataflow2.png

Fernando

dpoggemann
Resident Rockstar
Resident Rockstar

Hi @calerof,

 

Sorry, I am a bit confused here.  You mentioned the records are not making it to the dataverse but in your last communication you are identifying they are "all there"?  Are you comfortable the data is in the dataverse and it is just a challenge with viewing the records?  If so, could this be simply a difference between running the dataflow as a user with admin rights (or elevated rights at BU or Oganization level) vs you querying dataverse as your own user that may have a role where you can view only your one records in the Cardex table?  Can you verify this from a security perspective with user roles assigned? 

 

Sorry if still off base, trying to understand exactly what the challenge is here.

 

Thanks,

 

Drew

I'm thinking the table you extracted from Dataverse shows that the missing records are actually not in Dataverse. You can always check with a view as well to be sure they are not there.

 

If they are indeed missing, here is what I would do:

1. Filter your dataflow to include only the 15 rows you are trying to import.

2. Once the refresh is complete, check the refresh history (see@EricRegnier 's reply) and open the log to validate how many upserts you got. If someone else in your organization is running that dataflow, ask him or her to do the same. If the upserts number is not the same, then @dpoggemann idea about user rights to data would need to be investigated further.

3. What are you using as a primary key? Is there any way the data could be overwriting itself?

 

 

 

View solution in original post

Hi @dpoggemann ,

The Dataflow records are complete, all the records in the appended table in Dataverse are included in the Dataflow, but not all the records in the Dataflow are included in the appended table.

I am System Administrator in the environment. 

Thanks,

Fernando

Helpful resources

Announcements
PA_User Group Leader_768x460.jpg

Manage your user group events

Check out the News & Announcements to learn more.

Power Query PA Forum 768x460.png

Check it out!

Did you know that you can visit the Power Query Forum in Power BI and now Power Apps

Carousel 2021 Release Wave 2 Plan 768x460.jpg

2021 Release Wave 2 Plan

Power Platform release plan for the 2021 release wave 2 describes all new features releasing from October 2021 through March 2022.

R2 (Green) 768 x 460px.png

Microsoft Dynamics 365 & Power Platform User Professionals

DynamicsCon is a FREE, 4 half-day virtual learning experience for 11,000+ Microsoft Business Application users and professionals.

Users online (2,358)