Performance with Patching to Azure

[vsslasd]

The following simple patch statement takes almost one entire second per row:

ForAll(Filter(Collection,ActiveStatus="Changed" ),

Patch('[dbo].[Table]', First(Filter('[dbo].[Table]', RID=RowId)),

{

Complete:Complete

}

)));

The "Complete" is a bit type and RowId is an indexed identity column. So this should be very fast, but it is not.

The database is Azure.

How can we boost performance? The patch operation is updating row by row, as opposed to a Batch Update operation.

Anyway a batch update operation can be invoked?

We are not looking to utilize Flow in this case.

[v-monli-msft]

Hi @vsslasd ,

According to the documentation here:

https://powerapps.microsoft.com/en-us/blog/performance-refresh-forall-multiple-field-lookups-531/

"

You may ask why we named our new function ForAll instead of For or ForEach as many other languages do.   That’s because our system is designed for asynchronous and parallel operations.  The formula for one record may be paused waiting on a data operation, and while we wait, we may start evaluating the formula for another record.   The best way to think about it is that we could be evaluating the formula for all the records at the same time, with the implication that we can’t guarantee the order in which the formula for each record will be evaluated.  For most functions and services that are stateless and do not have side effects, including those used above, this is perfectly fine and allows us to optimize performance.  We are hoping that the All in the function name will differentiate it from other For constructs that imply an order.

...

But as we think it may be common, we also added the ability to use the Patch function on a set of records instead of just modifying one record at a time.

"

So if you want to bulk update at one time, use Patch function directly. For example, if your Source and Destination have the same column names, you can use a simple Patch statement. ChecklistItemsSource and the CheckedItems collections, have the same column names. Hence you can use the below formula to update the source at once with all the changes.

Patch(ChecklistItemsSource , CheckedItems)

Regards,

Mona

[vsslasd]

Good Morning, Mona, and thank you for your input.

I was reading your note and I don't understand.

Let's say we have one collection, called CollectionA and we need to insert the collection into an Azure Table in scenario 1, and need to update the Azure Table in scenario 2.

Scenario 1 (Insert Patch):

ForAll(Filter(CollectionA,ActiveStatus="New" ),

Patch('[dbo].[AzureTable]', Defaults('[dbo].[AzureTable]'),
{
    Service_Call_ID: Service_Call_ID,
    Equipment_ID: Equipment_ID,
    Task_List_ID: Task_List_ID,
    Task_Code: Task_Code,
    Task_Description: Task_Description,
    Skill_Level: Skill_Level,
    PRM_Asset_Id: PRM_Asset_Id,
    Complete: Complete,
    NA: NA,
    Reading1: Reading1,
    Reading2: Reading2,
    Reading3: Reading3,
    Comments: Comments,
    RecordLongitude: RecordLongitude,
    RecordLatitude: RecordLatitude,
    ModifyUser: ModifyUser,
    Timestamp: Timestamp
    }
    ))));

Scenario 2 (Update Patch):

ForAll(Filter(CollectionA,ActiveStatus="Changed" ),

Patch('[dbo].[AzureTable]', First(Filter('[dbo].[AzureTable]', RID=RowId),
{
    Service_Call_ID: Service_Call_ID,
    Equipment_ID: Equipment_ID,
    Task_List_ID: Task_List_ID,
    Task_Code: Task_Code,
    Task_Description: Task_Description,
    Skill_Level: Skill_Level,
    PRM_Asset_Id: PRM_Asset_Id,
    Complete: Complete,
    NA: NA,
    Reading1: Reading1,
    Reading2: Reading2,
    Reading3: Reading3,
    Comments: Comments,
    RecordLongitude: RecordLongitude,
    RecordLatitude: RecordLatitude,
    ModifyUser: ModifyUser,
    Timestamp: Timestamp
    }
    ))));

In reviewing your suggestion, I am understanding you to say I could do the following:

Patch('[dbo].[AzureTable]',CollectionA)

since I am considering the source to be: CollectionA and the Destination Azure Table to be "AzureTable"

But that syntax does not work.

Sounds like we want to forego the ForAll option altogether.

Is it possible you could show me the syntax in these two contexts, I'm certain that will shed the necessary light.

Thank you

[vsslasd]

Thank you, Mona.

I have read the materials and do not understand, the syntax.

We are happy to do a simple patch, and not utilize the ForAll if this helps with our key objectives: performance and best practice.

Here is a scenario:

"CollectionA" is a PowerApps collection containing data which needs to be patched to AzureSQL.

"AzureTable" is the AzureSQL Table we want to patch to.

We need to Patch Insert Rows into Azure SQL if those rows don't yet exist, and we need to Patch Update Rows into Azure SQL if those rows already exist.

Our current way of doing this is:

Insert New Rows:

If(Connection.Connected,
Set(vRID,
ForAll(Filter(CollectionA,ActiveStatus="New" ),
Patch('[dbo].[AzureTable]', Defaults('[dbo].[AzureTable]'),
{
    Service_Call_ID: Service_Call_ID,
    Equipment_ID: Equipment_ID,
    Task_List_ID: Task_List_ID,
    Task_Code: Task_Code,
    Task_Description: Task_Description,
    Skill_Level: Skill_Level,
    PRM_Asset_Id: PRM_Asset_Id,
    Complete: Complete,
    NA: NA,
    Reading1: Reading1,
    Reading2: Reading2,
    Reading3: Reading3,
    Comments: Comments,
    RecordLongitude: RecordLongitude,
    RecordLatitude: RecordLatitude,
    ModifyUser: ModifyUser,
    Timestamp: Timestamp
    }
    ))));

Update Existing Rows:

If(Connection.Connected,
ForAll(Filter(CollectionA,ActiveStatus="Changed" ),
    Patch('[dbo].[AzureTable]', First(Filter('[dbo].[AzureTable]', RID=RowId)),
    {
        Complete: Complete,
        NA: NA,
        Reading1: Reading1,
        Reading2: Reading2,
        Reading3: Reading3,
        Comments: Comments,
        RecordLongitude: RecordLongitude,
        RecordLatitude: RecordLatitude,
        ModifyUser: ModifyUser,
        Timestamp: Timestamp
        }
        )));

And then we Update the Collection, so we can keep track that it was Patched:

If(Connection.Connected, UpdateIf(CollectionA, ActiveStatus = "New" || ActiveStatus="Changed",{ActiveStatus:"Saved"}));

In reviewing your suggestion, it would appear that in order to Patch Update, all we would need to do is the following:

Patch('[dbo].[AzureTable]', CollectionA)

But this syntax does not work.

We still need to Patch Insert. Again, we are fine without the ForAll. And we would like to batch update and batch insert in both scenarios, or whatever other prevailing method which accomplishes best performance and best practice.

Can you perhaps elaborate further on this subject, so that I can get more clarity?

None of the documentation on this is very clear.

Thank you for your assistance.

[vsslasd]

Hello, Mona, @v-monli-msft 

Did you have any response to this question?

Thank you,

[vsslasd]

Good Morning, v-monli-msft  and @Mr-Dang-MSFT

Do you have any feedback on this ?

This is an important question/ideology/concept for us because it impacts all of our patch statements.

[vsslasd]

Or perhaps, do you have thoughts: @TopShelf-MSFT  ?

[vsslasd]

I have opened a Microsoft Support case on this topic.

[PaulD1]

Haven't tried the Patch syntax suggested, but regularly use Collect to do this:

Collect([SQL Table],CollectionA)

As noted, the field names must match and data types be compatible. If necessary, use RenameColumn, DropColumn, AddColumn, etc to ensure your collection matches your SQL table field for field.

[vsslasd]

Thank you, Paul.

I still need to know how to update just one or two fields at a time, so my question hasn't been addressed yet, particularly with the ForAll concept asked above.

Do you have thoughts on that ?


Mona ? @v-monli-msft