Form Processing Errors - Large Files

[rcpratt88]

I've been working to set up some Form Processing functions using AI Builder -- just looking to capture various fields on a PDF file that we receive daily and store the data in an Excel table.

I'm having an issue with getting the Form Processing step to work properly.  I was able to train and build my function, although I did have to "trim" down some of the PDF files to the relevant pages in order to get the training and field selection to work properly.  The files are generally 20-25 pages and 1-2 MB, although they do have a LOT of data and fields crammed into several pages. 

When the flow runs, I get the following error after about 50-60 seconds of processing: "BadRequest".  The full error text is below.

Does this just mean that my file are too large/complex for AI Builder to process?  Is there any way to resolve or work around this error?

{

  "error": {

    "code": "0x80040265",

    "message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"DependencyTimeout\",\"message\":\"Timeout on Cognitive backend\",\"properties\":{\"DependencyHttpStatusCode\":\"408\"}}}",

    "innererror": {

      "message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"DependencyTimeout\",\"message\":\"Timeout on Cognitive backend\",\"properties\":{\"DependencyHttpStatusCode\":\"408\"}}}",

      "type": "System.ServiceModel.FaultException`1[[Microsoft.Xrm.Sdk.OrganizationServiceFault, Microsoft.Xrm.Sdk, Version=9.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35]]",

      "stacktrace": "   at Microsoft.Crm.Extensibility.OrganizationSdkServiceInternal.Execute(OrganizationRequest request, InvocationContext invocationContext, CallerOriginToken callerOriginToken, WebServiceType serviceType, Boolean checkAdminMode, ExecutionContext executionContext, Dictionary`2 optionalParameters)\r\n   at Microsoft.Crm.Extensibility.OData.CrmODataExecutionContext.Execute(OrganizationRequest request, ExecutionContext executionContext)\r\n   at Microsoft.Crm.Extensibility.OData.CrmODataServiceDataProvider.ExecuteOperation(CrmODataExecutionContext context, EdmOperation edmOperation, Dictionary`2 parameters, Dictionary`2 boundParameters)\r\n   at Microsoft.Crm.Extensibility.OData.ActionController.ProcessOperationRequest(String operationName, Dictionary`2 operationParameters, EntityReference entityReference, String boundEntityName, String boundEntityType)\r\n   at Microsoft.Crm.Extensibility.OData.ActionController.<>c__DisplayClass11_0.<PostBoundAction>b__0()\r\n   at Microsoft.Crm.Extensibility.OData.CrmODataUtilities.<>c__DisplayClass10_0`2.<InvokeActionAndLogMetric>b__0()\r\n   at Microsoft.PowerApps.CoreFramework.ActivityLoggerExtensions.Execute[TResult](ILogger logger, EventId eventId, ActivityType activityType, Func`1 func, IEnumerable`1 additionalCustomProperties)\r\n   at Microsoft.Xrm.Telemetry.XrmTelemetryExtensions.Execute[TResult](ILogger logger, XrmTelemetryActivityType activityType, Func`1 func)\r\n   at lambda_method(Closure , Object , Object[] )\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ActionExecutor.<>c__DisplayClass10.<GetExecutor>b__9(Object instance, Object[] methodParameters)\r\n   at System.Web.Http.Controllers.ReflectedHttpActionDescriptor.ExecuteAsync(HttpControllerContext controllerContext, IDictionary`2 arguments, CancellationToken cancellationToken)\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActionAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Controllers.ActionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from previous location where exception was thrown ---\r\n   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()\r\n   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Web.Http.Dispatcher.HttpControllerDispatcher.<SendAsync>d__1.MoveNext()"

    }

  }

}

[JoeF-MSFT]

Hi @rcpratt88,

That error usually indicates that the document you are trying to process is too large in number of pages, so the recommendation here to process PDFs with only the pages you want to extract data from. 

Having said that, we are actively working on increasing support for larger files. 

[andyinv]

Now getting the same when it was working earlier. No error part suggests it's size related tho, and at 80kb and two pages PDF I'd be surprised....

{

  "error": {

    "code": "0x80040265",

    "message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"DependencyFailure\",\"message\":\"Timeout on Cognitive backend\",\"properties\":{\"DependencyHttpStatusCode\":\"408\"}}}"

  }

}

[JoeF-MSFT]

Hi @andyinv,

Thanks for letting us know about this. We are investigating an issue impacting certain customers regarding timeouts. I'll update this thread with any update.

[JoeF-MSFT]

Hello again @andyinv,

The timeout issue for small documents is now fixed. Apologies for the impact of this incident.

[andyinv]

Thanks @JoeF-MSFT - couple hours I spent staring at that wondering if I'd changed anything (I was still developing the flow when it happened - ofc, it only started the minute I went to video the process to send to the user to illustrate the completed task...)

Glad it wasn't me !

[Martinp2c]

Hi @JoeF-MSFT ,

Just had a multiple flows failed due to the same error.

{
"error": {
"code": "0x80040265",
"message": "{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"DependencyFailure\",\"message\":\"Timeout on Cognitive backend\",\"properties\":{\"DependencyHttpStatusCode\":\"408\"}}}"
}
}

Any idea when this will be fixed, or is this an server crash issue?

[JhonatanRS]

Hi Joe I am running into the same issue, it was working 1 hour ago and now it is not working with the same files.

is there an issue now?

[emahon]

I too am having the same issues.  First happened last night approx 9pm then again this morning.

[JoeF-MSFT]

Hi all,

I want to apologize for these issues yesterday and today. We're currently investigating what has caused this and working on ensuring this doesn't cause more inconveniences in the future. The reliability of AI Builder is a top priority for us, and we know this is what you expect for a Microsoft service.

Thank you for your understanding and apologies once again for the impact of this incident.

[Martinp2c]

Thank you @JoeF-MSFT 
Appreciate the feedback.

[BO55VXR]

Just jumping on the "Me too" bandwagon... 

I've had three failures of the past week, my PDFs are actually single page and about 72KB in size.... There are simple training certificates.

[andyinv]

It's kinda hard to justify investing in this when availability is so...... flakey. 😕

[BO55VXR]

Thing is ... if a re-execute the flow it works fine second time round 

[Garethterb]

I'm also getting this error on small 2 page documents now and re-running the flow does not work either.

{\"operationStatus\":\"Error\",\"error\":{\"type\":\"Error\",\"code\":\"DependencyFailure\",\"message\":\"Timeout on Cognitive backend\",\"properties\":{\"DependencyHttpStatusCode\":\"408\"}}}

[antoinec]

Hi Garethterb, We unfortunately hit a scalability issue this morning for customers using the Europe geography. I'll echo what Joe wrote earlier by saying that solving these types of problems is top of mind of us. We identified a couple of issues which were fixed after the previously reported incidents but it seems we still have some to work through. Apologies for the inconvenience, please do let us know how things are going for you now.