Substitute multiple "words" in one string

[adamtj]

Hi!
Im trying to find a way to replace mutliple words in one string.

For example I have a list with words that I want to replace with "***" in a string, if i had only one replace argument then I could simply use Substitute( "MyString etc and so on", "etc", "***" ). However to replace all words in the string from the list I have to loop throu the list and do "the replace action". 

So for I havent found a solution for this, any one that have any solution? 

My main goal is to change mutliple words in one string dynamically. For example:
Input: "This is my source string"
Output: "This **** my **** string"

[adamtj]

@WarrenBelz, no worries I appreciate your time and effort!

[RandyHayes]

@adamtj 

So in that formula, you have your ForAll backward and are not capturing any of the results of it.  ForAll is a function that creates a table of records based on your second parameter to your ForAll.  It is not a ForLoop function like in a development platform.

So, your ForAll will be returning a table with a single column called Value that would have all the results of your Substitute in it.  You can assign that to a variable to use.  I would avoid collections as they are not needed in this case, and they are more difficult to maintain in an app (just try searching for where you define a collection and places where you update it!)  You're not going to be doing any other add/remove/edit functions on those results, so not needed.

So, I do this *almost* exact thing in a generic templating component.  But that one assumes pre-entered "keys" to replace.  So, getting your string back with out all the nasty substitute nesting I can get you, but where I am not clear...how are you determining sensitive information?

As I see it, if you wanted to do full substitution on the GDPRContent and that is just a list of semicolon separated values, then the following would do what you need.

With({_redact: "********"};

  Concat(
    Split(varGdprLeadNote.GDPRContent; ";");

    If(
        //Emails
        IsMatch(Result; Email); _redact;

        // Phone number
        IsMatch(Result; <phoneRegEx>); _redact;

        // etc..
        "BloodType" in Result; _redact;

        // otherwise, no need to redact, just return
        Result
    )
  )
)

This would return a text value with all the sensitive information replaced.

Putting in context with your original post...

Input : "This is my source string"

With({_redact: "***"};

  Concat(
    Split("This is my source string"; " ");

    If(
        //Is
        Result = "is"; _redact;

        // Source
        Result = "source"; _redact;

        // otherwise, no need to redact, just return
        Result
    ) & " "
  )
)

This would produce a result of:  "This **** my **** string"

I hope this is helpful for you.

EDIT: found syntax issue in above formula...corrected.

[adamtj]

@RandyHayes 
Thank you for that explanation it was very helpful and it gave me some ides! However splitting the string with " " and then look for matches did not work. For example , if the string contains a phone number " 999 123 45 67", if you split by " " the string wouldn't be a phone number any more.

I think I might have to re-think and redact the whole record instead of redacting sub-strings!

[RandyHayes]

@adamtj 
Well, I was taking it from the perspective of the example you gave in message #13 where you mentioned that the keywords in GDPRContent were separated with the semicolon.

Then you mentioned that you wanted to replace NotesHtmlText with the new string where all the keywords in GDPRContent are replaced with "****".

SO...is that the key point of your issue?  That you want to take all of the keywords in the GDPRContent column and then replace any of those keywords found in the NotesHtmlText with "****", or is it the GDPRContent column that you want to "redact"?  

Sorry if I misunderstood - I did come into this a little late in the message chain.  

So please clarify and then I can provide a solution for that one too.

[adamtj]

Hi @RandyHayes 
Okay I understand, i guess my explanation gets lost in translation. Its always a challenge to explain something in an other language 🙂

To clarify:

The, GDPRContent semicolon separated string, contains the words that i want to find and replace in NotesHtmlText, so yes its the NotesHtmlText column i want to redact.

For example:
GDPRContent = "test; string"
NotesHtmlText = "This is my test html string"

For all "words"  in GDPRContent i want to identify and replace in NotesHtmlText. This example should then output in NotesHtmlText = "This is my **** html ****" 

[RandyHayes · Accepted solution]

@adamtj 

No worries on the language - you are actually doing just fine translating from what I see.

It was me who didn't really read through as completely as I should have.

Grab some popcorn and relax...this is a long response!

I spent a little more time now and read through all the posts.  The problem you were having is that ForAll was being treating as a ForLoop development function...it is not!  ForAll in PowerApps is designed to return a table of records based on your record definition.  In Message #3, you saw the result of this.  Your Record definition for the ForAll had no column defined, so you were going to get a Table of records based on the Split formula...a Result column.  But they are all going to be different and there is not a real good way to put them together.

So, you can't use ForAll like a ForLoop.  PowerApps is not a development platform, so it really doesn't have those type of functions - it's all about tables and records.

Okay, that said...so what you are trying to do is not a problem, but it does take some steps to get there.  The key is to be able to split the Notes text by the keywords.  And, since the Split function requires a specific split text, and yours is variable, then Split is really not helpful.  The next key is to abide by the No-Code platform that PowerApps is and adapt to the way that it needs to process data.

What needs to be done is to identify all the locations in the text where any of the keywords are found, then use the locations and lengths to rebuild the text and remove the "found" information.

So...here we go!

Using your example in Message #3 - The formula would be this:

With({_redactText: "****"; _keyWords: ["test@mail.com"; "999-123 45 67"]; _textToAlter: "Test string with an e-mail: test@mail.com and and phone: 999-123 45 67."};
    With({_items:
        With({_wds:
            // STEP 8: sort by location for purpose of rebuild of string
            Sort(
                // STEP 7: drop the table column - not used hereafter
                DropColumns(
                    // STEP 6: remove records with no word
                    Filter(
                        // STEP 5: Add the word to the record
                        AddColumns(
                            // STEP 4: Ungroup the table by the Value column to provide a table 
                               with inner records
                            Ungroup(
                                // STEP 1: iterate through keywords. 
                                   Note: no record defined so results will be a table in the Value column
                                ForAll(_keyWords;
                                    // Filter out empty locations
                                    Filter(
                                        // STEP 3: Group by the distinct locations
                                        GroupBy(
                                            // STEP 2: Cycle through entire string and gather all locations 
                                               where the keywords exist
                                            ForAll(Sequence(Len(_textToAlter)) As _seq;
                                                // return a record with the word, and location
                                                {wd:Value; loc:Find(Value; _textToAlter; _seq.Value)}
                                            ); 
                                        "loc"; "_locs"
                                        ); 
                                        !IsBlank(loc)
                                    )
                                );
                                "Value"
                            ),
                            "wd"; First(_locs).wd
                        );
                        !IsBlank(wd)
                    );
                    "_locs"
                );
                loc
            )};

            // STEP 9: iterate over a sequence based on the number of records in the _wds table
            ForAll(Sequence(CountRows(_wds));
                With({_prev:Last(FirstN(_wds; Value-1))};
                    // define record for Table as the prior record with the prev and len column added and values 
                       based on previous word
                    Patch(Last(FirstN(_wds; Value)); {prev: Coalesce(_prev.loc; 1); len:Len(_prev.wd)})
                )
            )

        )};

        // STEP 10: RETURN RESULTS
        If(CountRows(_items)=0;
            // Nothing to alter - return original
            _textToAlter;
            // Rebuild new string
            Concat(_items; Mid(_textToAlter; prev+len; loc-len-prev) & _redactText) & 
            With(LookUp(_items; loc=Max(_items; loc)); Mid(_textToAlter; loc + Len(wd)))
        )

    )
)

There are a lot of lines to the above because there are a lot of basic actions going on to get the data table shaped properly to be able to feed it to the final formula in the RETURN RESULTS section.

So here is what is going on - you need to read this from the inside out.

What we are doing is first building a table of all the locations of all the keywords.  This is all being put into the scoped With variable called _wds.  That variable will have a table of records with a wd and loc column.  The values will be the word and the location found (note as well, the formula will account for ALL instances of a keyword, not just one).

To get that table, we first iterate through all the keywords table (STEP 1).  This will give us our iterator.  That is then used to look through the entire string (this is the inner-most ForAll function) - STEP 2.  That is where we cycle through all the string based on the length of the string.  That ForAll returns the word in wd and the location found in loc as a record.  Since this will produce a lot of duplicated words and locations (because we just go through the string letter by letter), the formula is Grouping them by the loc (location) - STEP 3.  This will return a table with a distinct list of locations.  It will have a table column called _locs in it...we really don't need it except for the word in it. (NOTE, we also filter at this step to remove any blank locations).

So now we have a table from the Primary STEP 1 ForAll function.   Records will have ONLY a Value column that contains a record.  That record has a column called loc with a number and a column called _locs with a table.

In STEP 4, we remove the grouping of the Value column.  This will result in a table that has records with a loc column and a _locs column.  With that we then grab the word from the first record of the _locs column table and add it to the table we are building - STEP 5.

In STEP 6 we do a filter on the table we are building to remove any blank words.

STEP 7 we get rid of the _locs table column as we no longer need it (not really necessary, but just to be "clean").

STEP 8 we do a sort on all the loc columns as we want the table to be sorted by location in the string for rebuilding.

All of the above steps produce a formatted, sorted and clean table for the next process.  This is all stored in the _wds With variable.

In STEP 9, we work on the _wds table to add some more vital information.

So, we iterate over that table based on the number of rows in the table.  We get the previous record from the table first and assign it to a scoped With variable called _prev.  This is the record before the record we are currently working on - we need it for some values.  

Since we're in a ForAll, we need to define our record for the resulting table.  In this case, our record is made from the Patch of the _wds record (wd and loc) and we add two more columns, the prev column (which will hold the value of the previous word record location - note: coalesced to avoid blanks and set to 1 as the start), and the len column (which will hold the length of the previous word.)

All of this (STEP 9) produces a table.  Each record in the table will have a wd, loc, prev and len column with values.  It is all assigned to a With scoped variable called _items.

And that brings us to the final step - STEP 10.  We now have a formatted, sorted and data-full table with all the information we need to rebuild a string.

So, first off...if there happen to be no items in our table (no matches from prior steps), then we just return the original text.

Otherwise, we just iterate through the _items table in a Concat statement.  In that we use the Mid function to return bits and pieces of the original text based on all the locations and information we gathered in our table.  And then we append the redact text.  This will have returned not found text up to that location and then have the redact text in place of the keyword found.  

This then continues for the rest of the records in our table of locations.

Finally, we append the result with the Mid text to the end of the original text starting with the last location and length of word found.

SO...based on your example text, your resulting text from:

     Test string with an e-mail: test@mail.com and and phone: 999-123 45 67.

will be:

     Test string with an e-mail: **** and and phone: ****.

I hope this is all clear and helpful for you.  

[adamtj]

@RandyHayes 
Okey, almost choked on my popcorn's there for a bit 🙂

Well actually this make sense, as you mentioned powerapps is not a developer platform. Coming from a more "code first" background there is a learning curve to how to make it "do what you want" i guess. I must be honest thou, I haven't fully understand your formula yet. However I do understand the concept now and the formula works just fine when testing it!

Thank you!

[RandyHayes]

@adamtj 
Yes, the popcorn and the formula is a bit to digest!

It is difficult coming from a development world to PowerApps.  I know it was a struggle for me way back when I made the switch.  But, keep at it because once you grasp the concepts, formulas like that just fly out of your mind and it all makes sense.

If you have any questions on any part of the formula let me know.  I tried to step-by-step it as much as possible, but I am sure I glossed over a few things.

Happy to help and have a happy new year!

[AndrewLansford]

Randy,

I found this incredibly useful. I was looking for a way to specify a table of find and replace pairs so that I could build a dynamic email templating system. Some small modifications and this worked perfectly.

1. In the deepest ForAll I included the replacement word as a field in the record (field named rw for replacementWord)

2. Add rw back as a column as well as wd

3. Instead of concatenating with _redactText, I use the replacement word I placed in that record

I implemented this in a component that has two input parameters:

InputText (replaces _textToAlter)

FindAndReplaceTable ( a table with the schema { find: "textToFind", replaceWith: "textToOutput" } )

I included my entire code in case someone else is looking to do the same

With(
    {
        _items: With(
            {
                _wds: Sort(//step 8: sort by the found locations so we can rebuild the string
                    DropColumns(//step 7: drop the _locs column, its not used any longer
                        Filter(//step 6: remove records with no word
                            AddColumns(//step 5: add the word and replaceWith as columns
                                Ungroup(//step 4: ungropu the table by the value column to produce a tablle with inner records
                                    ForAll(//iterate through all the keywords
                                        //Note: no record defined so results will be a table in the Value column
                                        Self.FindAndReplaceTable As _frt,
                                        Filter(//Filter out the empty locations
                                            GroupBy(//Step 3: Group by the distinct locations
                                                ForAll(//Step 2: Cycle through the entire string and gather all locations where the keywords exist
                                                    Sequence(Len(Self.InputText)) As _seq,//Return a record with the word found and its location
                                                    {
                                                        wd: _frt.find,
                                                        loc: Find(
                                                            _frt.find,
                                                            Self.InputText,
                                                            _seq.Value
                                                        ),
                                                        rw: _frt.replaceWith
                                                    }
                                                ),
                                                "loc",
                                                "_locs"
                                            ),
                                            !IsBlank(loc)
                                        )
                                    ),
                                    "Value"
                                ),
                                "wd",
                                First(_locs).wd,
                                "rw",
                                First(_locs).rw
                            ),
                            !IsBlank(wd)
                        ),
                        "_locs"
                    ),
                    loc,
                    Ascending
                )
            },
            ForAll(//step 9 iterate over a sequence based on the number of records in the _wds table
                Sequence(CountRows(_wds)),
                With(
                    {
                        _prev: Last(
                            FirstN(
                                _wds,
                                Value - 1
                            )
                        )
                    },
                    Patch(//define record for table as the prior record with the prev and len column added and values based on the previous word
                        Last (
                            FirstN(
                                _wds,
                                Value
                            )
                        ),
                        {
                            prev: Coalesce(
                                _prev.loc,
                                1
                            ),
                            len: Len(_prev.wd)
                        }
                    )
                )
            )
        )
    },
    If(//Step 10: return results
        CountRows(_items) = 0,//Nothing to alter, return the original text
        Self.InputText,
        Concat(//Rebuild the new string
            _items,
            Mid(
                Self.InputText,
                prev + len,
                loc - len - prev
            ) & rw
        ) & With(
            LookUp(
                _items,
                loc = Max(
                    _items,
                    loc
                )
            ),
            Mid(
                Self.InputText,
                loc + Len(wd)
            )
        )
    )
)