Solved: Re: Substitute multiple "words" in one string

adamtj · ‎12-19-2021

Hi!
Im trying to find a way to replace mutliple words in one string.

For example I have a list with words that I want to replace with "***" in a string, if i had only one replace argument then I could simply use Substitute( "MyString etc and so on", "etc", "***" ). However to replace all words in the string from the list I have to loop throu the list and do "the replace action".

So for I havent found a solution for this, any one that have any solution?

My main goal is to change mutliple words in one string dynamically. For example:
Input: "This is my source string"
Output: "This **** my **** string"

RandyHayes · ‎12-29-2021

@adamtj

No worries on the language - you are actually doing just fine translating from what I see.

It was me who didn't really read through as completely as I should have.

Grab some popcorn and relax...this is a long response!

I spent a little more time now and read through all the posts. The problem you were having is that ForAll was being treating as a ForLoop development function...it is not! ForAll in PowerApps is designed to return a table of records based on your record definition. In Message #3, you saw the result of this. Your Record definition for the ForAll had no column defined, so you were going to get a Table of records based on the Split formula...a Result column. But they are all going to be different and there is not a real good way to put them together.

So, you can't use ForAll like a ForLoop. PowerApps is not a development platform, so it really doesn't have those type of functions - it's all about tables and records.

Okay, that said...so what you are trying to do is not a problem, but it does take some steps to get there. The key is to be able to split the Notes text by the keywords. And, since the Split function requires a specific split text, and yours is variable, then Split is really not helpful. The next key is to abide by the No-Code platform that PowerApps is and adapt to the way that it needs to process data.

What needs to be done is to identify all the locations in the text where any of the keywords are found, then use the locations and lengths to rebuild the text and remove the "found" information.

So...here we go!

Using your example in Message #3 - The formula would be this:

With({_redactText: "****"; _keyWords: ["test@mail.com"; "999-123 45 67"]; _textToAlter: "Test string with an e-mail: test@mail.com and and phone: 999-123 45 67."};
    With({_items:
        With({_wds:
            // STEP 8: sort by location for purpose of rebuild of string
            Sort(
                // STEP 7: drop the table column - not used hereafter
                DropColumns(
                    // STEP 6: remove records with no word
                    Filter(
                        // STEP 5: Add the word to the record
                        AddColumns(
                            // STEP 4: Ungroup the table by the Value column to provide a table 
                               with inner records
                            Ungroup(
                                // STEP 1: iterate through keywords. 
                                   Note: no record defined so results will be a table in the Value column
                                ForAll(_keyWords;
                                    // Filter out empty locations
                                    Filter(
                                        // STEP 3: Group by the distinct locations
                                        GroupBy(
                                            // STEP 2: Cycle through entire string and gather all locations 
                                               where the keywords exist
                                            ForAll(Sequence(Len(_textToAlter)) As _seq;
                                                // return a record with the word, and location
                                                {wd:Value; loc:Find(Value; _textToAlter; _seq.Value)}
                                            ); 
                                        "loc"; "_locs"
                                        ); 
                                        !IsBlank(loc)
                                    )
                                );
                                "Value"
                            ),
                            "wd"; First(_locs).wd
                        );
                        !IsBlank(wd)
                    );
                    "_locs"
                );
                loc
            )};

            // STEP 9: iterate over a sequence based on the number of records in the _wds table
            ForAll(Sequence(CountRows(_wds));
                With({_prev:Last(FirstN(_wds; Value-1))};
                    // define record for Table as the prior record with the prev and len column added and values 
                       based on previous word
                    Patch(Last(FirstN(_wds; Value)); {prev: Coalesce(_prev.loc; 1); len:Len(_prev.wd)})
                )
            )

        )};

        // STEP 10: RETURN RESULTS
        If(CountRows(_items)=0;
            // Nothing to alter - return original
            _textToAlter;
            // Rebuild new string
            Concat(_items; Mid(_textToAlter; prev+len; loc-len-prev) & _redactText) & 
            With(LookUp(_items; loc=Max(_items; loc)); Mid(_textToAlter; loc + Len(wd)))
        )

    )
)

There are a lot of lines to the above because there are a lot of basic actions going on to get the data table shaped properly to be able to feed it to the final formula in the RETURN RESULTS section.

So here is what is going on - you need to read this from the inside out.

What we are doing is first building a table of all the locations of all the keywords. This is all being put into the scoped With variable called _wds. That variable will have a table of records with a wd and loc column. The values will be the word and the location found (note as well, the formula will account for ALL instances of a keyword, not just one).

To get that table, we first iterate through all the keywords table (STEP 1). This will give us our iterator. That is then used to look through the entire string (this is the inner-most ForAll function) - STEP 2. That is where we cycle through all the string based on the length of the string. That ForAll returns the word in wd and the location found in loc as a record. Since this will produce a lot of duplicated words and locations (because we just go through the string letter by letter), the formula is Grouping them by the loc (location) - STEP 3. This will return a table with a distinct list of locations. It will have a table column called _locs in it...we really don't need it except for the word in it. (NOTE, we also filter at this step to remove any blank locations).

So now we have a table from the Primary STEP 1 ForAll function. Records will have ONLY a Value column that contains a record. That record has a column called loc with a number and a column called _locs with a table.

In STEP 4, we remove the grouping of the Value column. This will result in a table that has records with a loc column and a _locs column. With that we then grab the word from the first record of the _locs column table and add it to the table we are building - STEP 5.

In STEP 6 we do a filter on the table we are building to remove any blank words.

STEP 7 we get rid of the _locs table column as we no longer need it (not really necessary, but just to be "clean").

STEP 8 we do a sort on all the loc columns as we want the table to be sorted by location in the string for rebuilding.

All of the above steps produce a formatted, sorted and clean table for the next process. This is all stored in the _wds With variable.

In STEP 9, we work on the _wds table to add some more vital information.

So, we iterate over that table based on the number of rows in the table. We get the previous record from the table first and assign it to a scoped With variable called _prev. This is the record before the record we are currently working on - we need it for some values.

Since we're in a ForAll, we need to define our record for the resulting table. In this case, our record is made from the Patch of the _wds record (wd and loc) and we add two more columns, the prev column (which will hold the value of the previous word record location - note: coalesced to avoid blanks and set to 1 as the start), and the len column (which will hold the length of the previous word.)

All of this (STEP 9) produces a table. Each record in the table will have a wd, loc, prev and len column with values. It is all assigned to a With scoped variable called _items.

And that brings us to the final step - STEP 10. We now have a formatted, sorted and data-full table with all the information we need to rebuild a string.

So, first off...if there happen to be no items in our table (no matches from prior steps), then we just return the original text.

Otherwise, we just iterate through the _items table in a Concat statement. In that we use the Mid function to return bits and pieces of the original text based on all the locations and information we gathered in our table. And then we append the redact text. This will have returned not found text up to that location and then have the redact text in place of the keyword found.

This then continues for the rest of the records in our table of locations.

Finally, we append the result with the Mid text to the end of the original text starting with the last location and length of word found.

SO...based on your example text, your resulting text from:

Test string with an e-mail: test@mail.com and and phone: 999-123 45 67.

will be:

Test string with an e-mail: **** and and phone: ****.

I hope this is all clear and helpful for you.

_____________________________________________________________________________________
Digging it? - Click on the Thumbs Up below. Solved your problem? - Click on Accept as Solution below. Others seeking the same answers will be happy you did.
NOTE: My normal response times will be Mon to Fri from 1 PM to 10 PM UTC (and lots of other times too!)
Check out my PowerApps Videos too! And, follow me on Twitter @RandyHayes
Really want to show your appreciation? Buy Me A Cup Of Coffee!

View solution in original post

WarrenBelz · ‎12-19-2021

Hi @adamtj ,

Do you mean something like

ForAll(
   YourListName,
   Substitute(
      YourString,
	  YourListFieldName,
	  "***"
   )
)

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

Visit my blog Practical Power Apps

adamtj · ‎12-19-2021

Hi @WarrenBelz
Thank you for replying!
I tried something like that but it did not change the source string, it returned another list with strings substituted with that current record in the list (the list in the ForAll loop)

In this example "TempStr" = 'Test string with an e-mail: test@mail.com and and phone: 999-123 45 67.'
Listitems in "GDPRContent" = 'test@mail.com' and '999-123 45 67'

UpdateContext({TempStr:{Text: varGdprLeadsNote.NotesHtmlText}});;

UpdateContext({TempCol:
    ForAll(
        Split(varGdprLeadsNote.GDPRContent;";");
        Patch(
            TempStr;
            {
                Text: Substitute(TempStr.Text; ThisRecord.Result; "****")
            }
        )
    )
})

Above code did not change the source string (TempStr). TempCol however returns a list:

{ Result: "Test string with a e-mail adress: **** and and phone: 999-123 45 67" },
{ Result: "Test string with a e-mail adress: test@mail.com and and phone: ****" }

WarrenBelz · ‎12-19-2021

Hi @adamtj ,

I am not a fan of either tables stored in Variables or Patching to them (that is what Collections and Collect) are for, however firstly try this

UpdateContext(
   {
      TempStr:{Text: varGdprLeadsNote.NotesHtmlText}
   }
);;
UpdateContext(
   {
      TempCol:
      ForAll(
         Split(
            varGdprLeadsNote,
            GDPRContent & 
            ";"
         ) As aPatch;
         Patch(
            TempStr;
            {
                Text: 
                Substitute(
                   TempStr.Text; 
                   aPatch.Result; 
                   "****"
                )
            }
         )
      )
   }
)

Please click Accept as solution if my post helped you solve your issue. This will help others find it more readily. It also closes the item. If the content was useful in other ways, please consider giving it Thumbs Up.

Visit my blog Practical Power Apps

adamtj · ‎12-20-2021

Hi!

I tried above solution but the results are the same unfortunately.

To clarify the "TempVar" and "TempCol" was only for my troubleshooting, the basic function looks like this:

ForAll(
    Split(varGdprLeadsNote.GDPRContent;";");
    Substitute(varGdprLeadsNote.NotesHtmlText; ThisRecord.Result; "****")
)

The problem is that I dont understand how to change the source string "varGdprLeadsNote.NotesHtmlText" to include changes made by the Substitute ForAll loop? The ForAll loop returns multiple strings, one string per item in "Split(varGdprLeadsNote.GDPRContent;";");" "List". How do I get that to be one string in the end?

I suppose to actually change the source string is not the solution, however is it possible to manipulate the result from ForAll to be one string incorporation the diffrent changes made by substitute?

Confused 🙂

WarrenBelz · ‎12-21-2021

Hi @adamtj ,

Please consider using a collection here - it is so much easier to manipulate the contents.

adamtj · ‎12-21-2021

Hi @WarrenBelz ,
If its easier then im more then willing to try that. However i would appreciate any pointers you could give me on the subject.

My main goal is to change mutliple words in one string dynamically. For example:
Input: "This is my source string"
Output: "This **** my **** string"

WarrenBelz · ‎12-21-2021

Hi @adamtj ,

On reading this again, you seem to be replacing a string with itself - what is varGdprLeadsNote.NotesHtmlText and varGdprLeadsNote.GDPRContent - is this a table with two fields and where are you getting the words to replace

adamtj · ‎12-21-2021

Hi @WarrenBelz ,

I will try and explain the whole picture 🙂

The function for the app is to store and track notes and progress in a new sales leads and its on-boarding process. According to laws in EU (GDPR) information such as e-mail, phone number and similar information that could identify a person is consider sensitive. The law states that this kind of information isn't allowed to store longer than necessary. Hence the information must be traceable and erasable or able to be anonymized.

So when the user inputs information and notes into the app a series of RegEx searches are made on the text, text that matches "sensitive data" gets marked and the actual words copies and stores in same record as the actual note.

When the user saves the information a record with the notes and marked keywords (matched as sensitive information) will be saved in a table. Both fields are text fields, NotesHtmlText and GDPRContent

The record looks like this:

Now to the tricky part. When the on-boarding are finished or the lead is terminated and appropriate time has past the data should be either anonymized or erased. I'm trying to anonymize the data.

So back to the question:

varGdprLeadsNote is the selected record in the "LeadNotesTable" that i want to anonymize.

varGdprLeadsNote.NotesHtmlText is the field that contains the actual note/information submitted by the user.
VarGdprLeadsNote.GDPRContent is the field that contains identified sensitive "words", mail phone number etc inside the submitted note/information. The "keywords" are stored as a string separated with ";".

So the thing I'm trying to do is to replace NotesHtmlText with a new string where all of the keywords in GDPRContent are replaced with "****".

However this formula only takes me halfway. The NotesHtmlText remains unchanged which I now understand is expected.

ForAll(
    //Creates a collection to loop throu with the keywords to anonymize, by parsing the 
    //comma separeted string stored in the 'GDPRContent' field.
    Split(varGdprLeadsNote.GDPRContent;";");

    //Replaces the keywords in the submited text.
    Substitute(varGdprLeadsNote.NotesHtmlText; ThisRecord.Result; "****")
)

If i put the result from the ForAll into a collection I end up with a new collection containing each result from the loop:

The result i want to write back should look like this:
<p>Bl bla **** med mer</p><p>Bla **** etc och så vidare</p>

As I said before any input are greatly appreciated!