cancel
Showing results for 
Search instead for 
Did you mean: 
Reply
jesenavaranjan
Helper V
Helper V

Slow filtering and sorting

I am having some extreme slow issues on my power apps. I have everything stored in local collections but the sorting and filtering is very slow.

Can anybody help me optimize this formula? There are various search and filter components.

If(
    MyALL.Value = false && DGD.Selected.Value = "ALL",
        SortByColumns(
            Filter(
                MilestoneTempS,
                SearchInput.Text in Title,
                CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                DateDiff(
                    from.SelectedDate,
                    ForecastDate
                ) >= 0,
                DateDiff(
                    ForecastDate,
                    to.SelectedDate
                ) >= 0,
                Manager.Text in MilestoneManager
            ),
      If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )),
    If(
        MyALL.Value = false && DirectorateD.Selected.Value = "ALL",
            SortByColumns(
                Filter(
                    MilestoneTempS,
                    SearchInput.Text in Title,
                    CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                    DateDiff(
                        from.SelectedDate,
                        ForecastDate
                    ) >= 0,
                    DateDiff(
                        ForecastDate,
                        to.SelectedDate
                    ) >= 0,
                    Manager.Text in MilestoneManager,
                    OwningOrgDG = DGD.Selected.Value
                ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
        If(
            MyALL.Value = false && DivisionD.Selected.Value = "ALL",
                SortByColumns(
                    Filter(
                        MilestoneTempS,
                        SearchInput.Text in Title,
                        CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                        DateDiff(
                            from.SelectedDate,
                            ForecastDate
                        ) >= 0,
                        DateDiff(
                            ForecastDate,
                            to.SelectedDate
                        ) >= 0,
                        Manager.Text in MilestoneManager,
                        OwningOrgDG = DGD.Selected.Value,
                        OwningOrgDir = DirectorateD.Selected.Value
                    ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
            If(
                MyALL.Value = false,
                    SortByColumns(
                        Filter(
                            MilestoneTempS,
                            SearchInput.Text in Title,
                            CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                            DateDiff(
                                from.SelectedDate,
                                ForecastDate
                            ) >= 0,
                            DateDiff(
                                ForecastDate,
                                to.SelectedDate
                            ) >= 0,
                            Manager.Text in MilestoneManager,
                            OwningOrgDG = DGD.Selected.Value,
                            OwningOrgDir = DirectorateD.Selected.Value,
                            OwningOrgDiv = DivisionD.Selected.Value
                        ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
                If(
                    MyALL.Value = true && DGD.Selected.Value = "ALL",
                        SortByColumns(
                            Filter(
                                MilestoneTempS,
                                SearchInput.Text in Title,
                                CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                                DateDiff(
                                    from.SelectedDate,
                                    ForecastDate
                                ) >= 0,
                                DateDiff(
                                    ForecastDate,
                                    to.SelectedDate
                                ) >= 0,
                                User().FullName = MilestoneManager || User().FullName = MilestoneOwner,
                                Manager.Text in MilestoneManager
                            ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
                    If(
                        MyALL.Value = true && DirectorateD.Selected.Value = "ALL",
                            SortByColumns(
                                Filter(
                                    MilestoneTempS,
                                    SearchInput.Text in Title,
                                    CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                                    DateDiff(
                                        from.SelectedDate,
                                        ForecastDate
                                    ) >= 0,
                                    User().FullName = MilestoneManager || User().FullName = MilestoneOwner,
                                    DateDiff(
                                        ForecastDate,
                                        to.SelectedDate
                                    ) >= 0,
                                    Manager.Text in MilestoneManager,
                                    OwningOrgDG = DGD.Selected.Value
                                ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
                        If(
                            MyALL.Value = true && DivisionD.Selected.Value = "ALL",
                                SortByColumns(
                                    Filter(
                                        MilestoneTempS,
                                        SearchInput.Text in Title,
                                        CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                                        DateDiff(
                                            from.SelectedDate,
                                            ForecastDate
                                        ) >= 0,
                                        User().FullName = MilestoneManager || User().FullName = MilestoneOwner,
                                        DateDiff(
                                            ForecastDate,
                                            to.SelectedDate
                                        ) >= 0,
                                        Manager.Text in MilestoneManager,
                                        OwningOrgDG = DGD.Selected.Value,
                                        OwningOrgDir = DirectorateD.Selected.Value
                                    ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   ),
                            If(
                                MyALL.Value = true,
                                    SortByColumns(
                                        Filter(
                                            MilestoneTempS,
                                            SearchInput.Text in Title,
                                            CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                                            DateDiff(
                                                from.SelectedDate,
                                                ForecastDate
                                            ) >= 0,
                                            User().FullName = MilestoneManager || User().FullName = MilestoneOwner,
                                            DateDiff(
                                                ForecastDate,
                                                to.SelectedDate
                                            ) >= 0,
                                            Manager.Text in MilestoneManager,
                                            OwningOrgDG = DGD.Selected.Value,
                                            OwningOrgDir = DirectorateD.Selected.Value,
                                            OwningOrgDiv = DivisionD.Selected.Value
                                        ),If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(
                SortOldM1,
                Descending,
                Ascending
            )   )
                            )
                        )
                    )
                )
            )
        )
    )
)

4 ACCEPTED SOLUTIONS

Accepted Solutions
Anonymous
Not applicable

Hi @jesenavaranjan ,

 

First thing, it's a good thing to work with a collection but like you experienced it, sometimes it's not enough.

 

From what i can see you filter everytime you change something on your screen, but the filter is the same.

 

Maybe you could try to do the filter in another collection in a OnVisible control of your page (happen at the loading of your screen).

Then call this another pre-filtered collection in the code you wrote on your post.

 

It will at least save the time of the filter at every different sorting.

 

I don't know if it will be enough though, but i hope this will give you some lead.

View solution in original post

RandyHayes
Super User III
Super User III

@jesenavaranjan 

What specific slowness are you experiencing?

 

I would first start by simplifying the formula you have.

Although I have only a cursory idea what your logic is by looking at the If statements and the conditions and I don't have your data or cases to test with, I looked at what you had and simplified to the following formula:

SortByColumns(
   Filter(MilestoneTempS, 
      SearchInput.Text in Title 
      && (CodeSearch.Text in Code || (Value(CodeSearch.Text) = MilestoneID)) 
      && (DateDiff(from.SelectedDate, ForecastDate) >= 0) 
      && (DateDiff(ForecastDate, to.SelectedDate) >= 0) 
      && Manager.Text in MilestoneManager
      && If(MyALL.Value, (User().FullName = MilestoneManager || User().FullName = MilestoneOwner), true)
      && If(DirectorateD.Selected.Value = "ALL", OwningOrgDG = DGD.Selected.Value, true)
      && If(DivisionD.Selected.Value = "ALL", OwningOrgDir = DirectorateD.Selected.Value, true)
      && If(!(DirectorateD.Selected.Value = "ALL") && !(DivisionD.Selected.Value = "ALL"), OwningOrgDG = DGD.Selected.Value && DirectorateD.Selected.Value && OwningOrgDiv = DivisionD.Selected.Value, true)
      ),
   If(SortOldS,"ConfidenceRating","ForecastDate"),
   If(SortOldM1, Descending, Ascending)
   )

You are much better off by putting your if statements where they need to be within your formula rather than repeating the same formula over and over within separate if statements.  This will make your Formula easier to read and troubleshoot.

 

Beyond that, I would be curious to know what slowness you are seeing and information about your situation - amount of records, data source, etc.

 

 

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
Check out my PowerApps Videos too!

View solution in original post

Hi @jesenavaranjan ,

Please take a try with the solution I provided above, then check if it could help in your scenario.

 

If you have solved your problem, please go ahead to click "Accept as Solution" to identify this thread has been solved.

 

Best regards,

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

@jesenavaranjan 

Definitely try the formula out.  The goal should always be to avoid a collection if possible and to make your logic as compact as possible as to never repeat any part of the formula.  This will provide a much easier to troubleshoot formula and will read easier when reviewing issues.  Doesn't say that it will perform better...just that you have a point to start.

To nail down performance issues in a formula, the best thing to do is to break apart the formula from the inner part out and use it in a seperate gallery to determine what part of your formula causes the issue.

 

Since you are over the record limit though, this does take you to needing a collection.  So, you will want to only populate that as infrequently as possible.  If you patch a change back to the data source, you are better off patching your local collection as well, rather than refreshing the entire collection.  This can get tricky, so choose your plan as needed - the exception rather than the rule.

 

As always...post back if you need more help.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
Check out my PowerApps Videos too!

View solution in original post

7 REPLIES 7
Anonymous
Not applicable

Hi @jesenavaranjan ,

 

First thing, it's a good thing to work with a collection but like you experienced it, sometimes it's not enough.

 

From what i can see you filter everytime you change something on your screen, but the filter is the same.

 

Maybe you could try to do the filter in another collection in a OnVisible control of your page (happen at the loading of your screen).

Then call this another pre-filtered collection in the code you wrote on your post.

 

It will at least save the time of the filter at every different sorting.

 

I don't know if it will be enough though, but i hope this will give you some lead.

View solution in original post

RandyHayes
Super User III
Super User III

@jesenavaranjan 

What specific slowness are you experiencing?

 

I would first start by simplifying the formula you have.

Although I have only a cursory idea what your logic is by looking at the If statements and the conditions and I don't have your data or cases to test with, I looked at what you had and simplified to the following formula:

SortByColumns(
   Filter(MilestoneTempS, 
      SearchInput.Text in Title 
      && (CodeSearch.Text in Code || (Value(CodeSearch.Text) = MilestoneID)) 
      && (DateDiff(from.SelectedDate, ForecastDate) >= 0) 
      && (DateDiff(ForecastDate, to.SelectedDate) >= 0) 
      && Manager.Text in MilestoneManager
      && If(MyALL.Value, (User().FullName = MilestoneManager || User().FullName = MilestoneOwner), true)
      && If(DirectorateD.Selected.Value = "ALL", OwningOrgDG = DGD.Selected.Value, true)
      && If(DivisionD.Selected.Value = "ALL", OwningOrgDir = DirectorateD.Selected.Value, true)
      && If(!(DirectorateD.Selected.Value = "ALL") && !(DivisionD.Selected.Value = "ALL"), OwningOrgDG = DGD.Selected.Value && DirectorateD.Selected.Value && OwningOrgDiv = DivisionD.Selected.Value, true)
      ),
   If(SortOldS,"ConfidenceRating","ForecastDate"),
   If(SortOldM1, Descending, Ascending)
   )

You are much better off by putting your if statements where they need to be within your formula rather than repeating the same formula over and over within separate if statements.  This will make your Formula easier to read and troubleshoot.

 

Beyond that, I would be curious to know what slowness you are seeing and information about your situation - amount of records, data source, etc.

 

 

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
Check out my PowerApps Videos too!

View solution in original post

v-xida-msft
Community Support
Community Support

HI @jesenavaranjan ,

Based on the formula that you provided, I found that you create a nested If formula within your app. Please consider modify your formula as below:

If(
   MyALL.Value = false && DGD.Selected.Value = "ALL",     /* <-- Condition1 */
SortByColumns(
            Filter(
                MilestoneTempS,
                SearchInput.Text in Title,
                CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                DateDiff(
                    from.SelectedDate,
                    ForecastDate
                ) >= 0,
                DateDiff(
                    ForecastDate,
                    to.SelectedDate
                ) >= 0,
                Manager.Text in MilestoneManager
            ),
      If(SortOldS,"ConfidenceRating","ForecastDate"),
            If(SortOldM1, Descending, Ascending)
),
 MyALL.Value = false && DirectorateD.Selected.Value = "ALL", /* <-- Condition2 */
SortByColumns(
                Filter(
                    MilestoneTempS,
                    SearchInput.Text in Title,
                    CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                    DateDiff(
                        from.SelectedDate,
                        ForecastDate
                    ) >= 0,
                    DateDiff(
                        ForecastDate,
                        to.SelectedDate
                    ) >= 0,
                    Manager.Text in MilestoneManager,
                    OwningOrgDG = DGD.Selected.Value
                ),
If(SortOldS,"ConfidenceRating","ForecastDate"),

            If(SortOldM1, Descending, Ascending)
),
MyALL.Value = false && DivisionD.Selected.Value = "ALL", /* <-- Condition3 */
SortByColumns(...
...
...
MyALL.Value = true, /* <-- Condition8 */
SortByColumns(
                   Filter(
                           MilestoneTempS,
                           SearchInput.Text in Title,
                           CodeSearch.Text in Code || Value(CodeSearch.Text) = MilestoneID,
                           DateDiff(
                                    from.SelectedDate,
                                    ForecastDate
                                   ) >= 0,
                           User().FullName = MilestoneManager || User().FullName = MilestoneOwner,
                           DateDiff(
                                     ForecastDate,
                                     to.SelectedDate
                                   ) >= 0,
                           Manager.Text in MilestoneManager,
                           OwningOrgDG = DGD.Selected.Value,
                           OwningOrgDir = DirectorateD.Selected.Value,
                           OwningOrgDiv = DivisionD.Selected.Value
                  ),
If(SortOldS,"ConfidenceRating","ForecastDate"),

            If(SortOldM1, Descending, Ascending)
)
)

please take a try with above formula, check if the issue is improved.

 

In addition, also please consider take a try to turn on/enable the "Explicit column selection" option within Advanced settings of App settings of your app, then re-publish your app, check if the issue is solved.

 

Best regards,

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi thank you for this!
I will try the formula but the user can apply different filters based on their selection so changing the collection to match the filtering won't work right?

Hi!
Thank you for this, looks amazing - I just need to sit down and understand the logic. And I will try this later and let you know how it goes 🙂
I am using SQL - and up to 4000 records right now. I am using many collections and the app has become quite complex

Jese

Hi @jesenavaranjan ,

Please take a try with the solution I provided above, then check if it could help in your scenario.

 

If you have solved your problem, please go ahead to click "Accept as Solution" to identify this thread has been solved.

 

Best regards,

Community Support Team _ Kris Dai
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

View solution in original post

@jesenavaranjan 

Definitely try the formula out.  The goal should always be to avoid a collection if possible and to make your logic as compact as possible as to never repeat any part of the formula.  This will provide a much easier to troubleshoot formula and will read easier when reviewing issues.  Doesn't say that it will perform better...just that you have a point to start.

To nail down performance issues in a formula, the best thing to do is to break apart the formula from the inner part out and use it in a seperate gallery to determine what part of your formula causes the issue.

 

Since you are over the record limit though, this does take you to needing a collection.  So, you will want to only populate that as infrequently as possible.  If you patch a change back to the data source, you are better off patching your local collection as well, rather than refreshing the entire collection.  This can get tricky, so choose your plan as needed - the exception rather than the rule.

 

As always...post back if you need more help.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up. Solved your problem? - Click on Accept as Solution. Others seeking the same answers will be happy you did.
Check out my PowerApps Videos too!

View solution in original post

Helpful resources

Announcements
PA User Group

Welcome to the User Group Public Preview

Check out new user group experience and if you are a leader please create your group

MBAS Attendee Badge

Claim Your Badge & Digital Swag!

Check out how to claim yours today!

secondImage

Are Your Ready?

Test your skills now with the Cloud Skill Challenge.

secondImage

Demo Extravaganza is Back!

We are excited to announce that Demo Extravaganza for 2021 has started!

MBAS on Demand

Microsoft Business Applications Summit sessions

On-demand access to all the great content presented by the product teams and community members! #MSBizAppsSummit #CommunityRocks

Top Solution Authors
Top Kudoed Authors
Users online (34,535)