AI Builder flow syntax - Predict (form processing)

[RobBoyers]

Hi,

I am attempting to take an email attachment and apply the Predict action (form processing) for each attachment. I've tried to follow the instructions here https://docs.microsoft.com/en-us/ai-builder/form-processing-model-in-flow but when I use the expression base64(items('Apply_to_each')) it doesn't work.

I get this error message in the output: "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"InternalServerError\",\"message\":\"Status code action Unknown is not handled. error message : File content could not be parsed, was the mime type correct?\"}}"

The input message seems to look ok: 

item/request

"{\n \"base64Encoded\":… \"application/pdf\"\n}"

Has anyone out there managed to sucessfully apply the predict action (form processing) to an email attachment?

Thanks

Rob

[v-yamao-msft]

Hi @RobBoyers ,

Could you try if the following function will work:

base64(outputs('Get_attachment')?['body'])

I am trying to make more tests on this issue, will back if I have got any updates.

Best regards,

Mabel

[RobBoyers]

Hi, thanks for reaching out. I'm still struggling I'm afraid.

If it helps, My trigger action is Outlook: When a new email arrives, from a specific address with the subject of invoice. Has attachment = yes, Include attachments = yes.

Loop for each: select an output from previous step: 'Attachments'

Should be pretty simple, I must be missing something obvious here.

[RobBoyers]

Hi,

I have made some progress with the input syntax but it appears the Predict step is struggling to handle pdf files.

I used the expression 

base64(triggerOutputs()?['body/Attachments']) and when the flow fails (bad request), the input appears to be ok.

I added an if statement to ensure the attachment was of type application/pdf just to make sure.

Here's the first few lines of the input;

{"host":{"apiId":"subscriptions/24fa6447-0d7f-439d-9ee6-3b1984fec8b6/providers/Microsoft.Web/locations/westeurope/runtimes/europe-002/apis/commondataserviceforapps","connectionReferenceName":"shared_commondataserviceforapps","operationId":"Predict"},"parameters":{"modelId":"a3933888-a893-e911-a81c-000d3a479be2","item/request":"{\n   \"base64Encoded\": \"W3siQG9kYXRhLnR5cGUiOiIjTWljcm9zb2Z0Lk91dGxvb2tTZXJ2aWNlcy5GaWxlQXR0YWNobWVudCIsIklkIjoiQUFNa0FHTXpNVFZtT0RBeExUVTVPR010TkRRM01DMDVaRFV6TFRabVl6UmlabVEzTW1Wa1lnQkdBQUFBQUFDOTNEZWVmY2dFUXI4Y1EtaHB6YUlDQndBUE95UmpMQmp3VEwweG5OTEM1MU1CQUFBQUFBRU1BQUFQT3lSakxCandUTDB4bk5MQzUxTUJBQURyUjA3ZUFBQUJFZ0FRQURSTjBfVm1rYnBMbDl5WmNRdWZIVG89IiwiTmFtZSI6Ikludm9pY2UucGRmIiwiQ29udGVudFR5cGUiOiJhcHBsaWNhdGlvbi9wZGYiLCJTaXplIjoxMDU4MDQsIkNvbnRlbnRCeXRlcyI6IkpWQkVSaTB4TGpjTkNpWGk0OC9URFFveElEQWdiMkpxRFFvOFBDOUJkWFJvYjNJZ0tHSmlkV3hzWlhJcEwxQnliMlIxWTJWeUlDaEhjbUZ3WlVOcGRIa2dRekZHYkdWNFVtVndiM0owS1M5RGNtVmhkR2x2YmtSaGRHVWdLRVE2TWpBeE9UQTJNVGN4TVRJNU5UQXJNREVuTURBbktTOU5iMlJFWVhSbElDaEVPakl3TVRrd05qRTNNVEV5T1RVd0t6QXhKekF3SnlrK1BnMEtaVzVrYjJKcURRb3lJREFnYjJKcURRbzhQQzlNWlc1bmRHZ2dNalUyTnk5R2FXeDBaWElnTDBac1lYUmxSR1ZqYjJSbFBqNXpkSEpsWVcwTkNuaWNWSTdOQ3NJd0VJVHZDL3NPK3dScG1teCtlcTNGNGxFYjhBSGl6NlVxcGVEenUwMlZJcnVIK1dDWW1RbEJxK0JJTDdjS1UydkZ3VkIrSUV3SWphZkFUc1dHaEpuamo4YVZmTkJmRXVNZmJMNHp3ck5rV1dkS2svUnM3cVduT21qcVhnakg4dFcrWnFvTnBadm9mdEIwbjBWMDEvZXBiM2NEWmFHeWwrWXN1VzFDY05FcGJ4d0Z5OHBicG5SQitBQUFBUC8vaXRaZ1VHRUlZL0JtY0dVSVpmQmhjR2Z3WXZEUTFEWFNZR0JtTUdTSUFJcUZNWGdBeVpnWXpWaUZFQzllTHRjUWNtdzJNekxTc3pReFZqQTNBbnJHMkJSc013QUFBUC8vQXRxc0Q3UXhFR2luUDhoT1E1Q2QrZ3d1UUJFUHlpd3pzZFF6TXdGNjA4QmN6OERZREd3WkFBQUEvLzhDV3FZTjlFNDRFQVl4QkZKb3ZESFllRE5MWXowalkzT3c4UUFBQUFELy93SWFid3dNSlE5Z2FGSEJBaU16UFNOTG9BWG1sbm9teHBaZ0N3QUFBQUQvL3dJSEZpaG8zSUFrS0xwOFFOWlJaZzB3RlJnYkdpcVltUUZUQlNReEFBQUFBUC8vQW52RGxFRWNHaG5pd0tSaFFwRWRvUFJyYktwbmFtd01OaDhBQUFELy93S2Fid0JPVXN5YXBpYWdoT2ZERUFBTU5Gamtxd0tETGlZR25ON0NVSzAxSlRHaG01cm9HUmtaSzVnWldzTHRCZ0FBQVAvL0F0cHRybWx1ckFFMDNoOXFueUZERUdVaGFHQ0lZZ1VBQUFELy93SmFJY3dnd2lBRmhLSlFLeVNCZkJrZ0ZHWVFwamdnZ1RiQmNpMEFBQUQvL3dMYUpBazAzUXFjRU1KQkFXZ01zczBBR0lCaHdMQ0QySzBHekZjZ1BtWEpCR1N6Z1NrODF3SUFBQUQvL3dMYURMY1ZuTnBkR1B3b3Q4SFUwZ2llVlFFQUFBRC8vOEtTQ0xVb1NnMUc1aVo2bHVibUNxWVdKdkNvQWdBQUFQLy9BbHFpQTh4TmtjQUE4Z0htS1dBQW1zQ0t2QUJnVVFoTWdoUjV5ZGpjQk1VNkFBQUFBUC8vQXFjTUdCUmhrR0NRQmtJSnl0SzRpYUdlcFlVRmlqVUFBQUFBLy84Q1dxTUs5STgvS0lXREU0UTZNSHJDS1MwbmdLa2MyUklBQUFBQS8vK0NwbkloSUNrRkpFWEJQQm5LWXNnU1dCUVpBWVBNREZGV0FBQUFBUC8vQWljREQzQ2xGTVRnRG8yZkdFMmdRQkJEQU9XUmcyUVRBQUFBLy8raWdZZE16STMxTEkwdFVLd0JBQUFBLy8vQ2lCeGpZT1JBaWp0b2hXc0taSHVDL2V3Qkx1azlLSEtEc1FHd1pESTFWVEExTVlPN0FRQUFBUC8vd2hxb2tHS1JzcUlRSEtaSUZnRUFBQUQvL3dKYUpBb3NvcWdacHFhRzVucW13RFlFc2pVQUFBQUEvLy9Da1kyaFBnS1hUdjZVSjM0a0N3RUFBQUQvL3dKYTJBb3Nra0MrRVVUUHdqZ01CTFlYZ2VuYlVBR1RBYllGVlBvWkc0T2FlR1lHUmdvbUNrV3B2RnhwV3JCR0pWUXFGOVI0TTREeGNxQThRMk5FT3hLVkExY0hhVlJDL0cxSllwbHNhS1puWUdZQjlqWUFBQUQvL3dLM0V2MkJpZE1IM2pyRVVWa0h3cUxXbURUYlRDek45Y3dnalNrQUFBQUEvLzhDV2hhandSQU5qTDFRQmhkb0xXQU

Here's the output:

{"statusCode":400,"headers":{"Cache-Control":"no-cache","Set-Cookie":"ReqClientId=40abb475-1cba-4e49-8642-b4421b1cc6bc; expires=Fri, 21-Jun-2069 21:33:01 GMT; path=/; secure; HttpOnly,orgId=2e569944-1e69-404d-8df5-6e9b8ec42aaf; expires=Fri, 21-Jun-2069 21:33:01 GMT; path=/; secure; HttpOnly,ApplicationGatewayAffinity=7374892e0d5a6889c14a9b9e9065db1068800941fd22098d3410ac68d1f59365;Path=/;Domain=rboyerssandbox.crm4.dynamics.com","Server":"","x-ms-service-request-id":"d512e04a-bfb5-446b-8f48-ed1d9b4f368d","REQ_ID":"d512e04a-bfb5-446b-8f48-ed1d9b4f368d,d512e04a-bfb5-446b-8f48-ed1d9b4f368d","AuthActivityId":"7ff5b68e-ad66-48e0-b987-de8139f8afd3","x-ms-ratelimit-burst-remaining-xrm-requests":"59999","x-ms-ratelimit-time-remaining-xrm-requests":"11,997.82","OData-Version":"4.0","Public":"OPTIONS,GET,HEAD,POST","Date":"Fri, 21 Jun 2019 21:33:03 GMT","Content-Length":"3480","Allow":"OPTIONS,GET,HEAD,POST","Content-Type":"application/json; odata.metadata=minimal","Expires":"-1"},"body":{"error":{"code":"0x80040265","message":"{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"InternalServerError\",\"message\":\"Status code action Unknown is not handled. error message : File content could not be parsed, was the mime type correct?\"}}","innererror":{"message":"{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"InternalServerError\",\"message\":\"Status code action Unknown is not handled. error message : File content could not be parsed, was the mime type correct?\"}}","type":"System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=9.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]","stacktrace":"   at Microsoft.Crm.Extensibility.OrganizationSdkServiceInternal.Execute(OrganizationRequest request, InvocationContext invocationContext, CallerOriginToken callerOriginToken, WebServiceType serviceType, Boolean checkAdminMode, ExecutionContext executionContext, Dictionary`2 optionalParameters)\r\n   at Microsoft.Crm.Extensibility.OData.CrmODataExecutionContext.Execute(OrganizationRequest request, ExecutionContext executionContext)\r\n   at Microsoft.Crm.Extensibility.OData.CrmODataServiceDataProvider.ExecuteOperation(CrmODataExecutionContext context, EdmOperation edmOperation, Dictionary`2 parameters, Dictionary`2 boundParameters)\r\n   at Microsoft.Crm.Extensibility.OData.ActionController.ProcessOperationRequest(String operationName, Dictionary`2 operationParameters, EntityReference entityReference, String boundEntityName, String boundEntityType)\r\n   at Microsoft.Crm.Extensibility.OData.ActionController.<>c__DisplayClass11_0.<PostBoundAction>b__0()\r\n   at Microsoft.PowerApps.CoreFramework.ActivityLoggerExtensions.Execute[TResult](ILogger logger, EventId eventId, ActivityType activityType, Func`1 func, IEnumerable`1 additionalCustomProperties)\r\n   at Microsoft.Xrm.Telemetry.XrmTelemetryExtensions.Execute[TResult](ILogger logger, XrmTelemetryActivityType activityType, Func`1 func)\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()"}}}}

[nani5swamy]

Hi, I am not getting Predict option while adding next step.

[robboyers1]

You have to create your flow within a solution else you don't get the predict action.

[robboyers1]

An update on the syntax you need to use;

string(items('Apply_to_each')?['ContentBytes'])

is what you need to parse a pdf NOT base64.

[nani5swamy]

Hi,

As suggested, i have created a model driven app and try to create a flow within app but no use.

Please suggest where i can define flow in solution.

[nani5swamy]

Hi @robboyers1 ,

Thanks for the suggestion. Please guide how to create a flow in solution.

[robboyers1]

Hi,

Go to make.powerapps.com > solutions > create your solution. Then click 'New > Flow'.

You have to use make.powerapps.com, this doesn't work in the classic solution builder.

Rob

[jyotipch · Accepted solution]

This is actually straight forward:

[nani5swamy]

Thanks @jyotipch 

Its working!

[kcg633755]

I have a very similar flow built in solutions which when I trigger manually works, however when it is triggered by a new email arriving it fails.

Here is the error I recieve...

"code": "0x80040265",

"message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"InvalidPredictionInput\",\"message\":\"Bad JSON escape sequence: \\\\Z. Path 'base64Encoded', line 110, position 118.\"}}",

Any idea why it is failing with the email trigger?

[jyotipch]

---

@kcg633755 wrote:

I have a very similar flow built in solutions which when I trigger manually works, however when it is triggered by a new email arriving it fails.

Here is the error I recieve...

"code": "0x80040265",

"message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"InvalidPredictionInput\",\"message\":\"Bad JSON escape sequence: \\\\Z. Path 'base64Encoded', line 110, position 118.\"}}",

Any idea why it is failing with the email trigger?

---

Sorry not much idea. Normally escape sequence error means somewhere in your data there are special characters which is making the code treat it like a separator.

[jyotipch]

Recently, I found my flow to be failing. It was throwing an error "Unexpected character at ..." I looked into the email attachment output and found it's JSON schema. I replaced my request payload in Predict with the below:

{

  "base64Encoded":"EXPRESSION"

  "mimeType":"application/pdf"

}

where the EXPRESSION is  string(triggerBody()?['Attachments']?[0]?['ContentBytes'])

(Assuming you are extracting the contents of the first attachment)

That solved my error. If you are encountering such kind of errors, I suggest you analyse the input of the predict step and check it's schema.