A dataflow usually contains multiple queries, and the results of a few of these queries get loaded into entities. This works fine most of the time. The problem is when one of these queries fails. My expectation is that when this happens, none of the queries will load their result into CDS. However, this is not the case. Currently all the queries but the failing one will load data into CDS. This makes little sense, see some examples below. Please make the dataflow atomic (transaction), so that either every query designated to load data in CDS does so, either none does. Maybe even add a setting to toggle this atomic/transaction behavior on/off for each dataflow. Example 1: In a dataflow with 4 queries where 1 query is for a parent entity and the other 3 are for child entities. Why load the children into CDS if the parent could not be loaded? Example 2: I am loading data from JSON files. To avoid my dataflow getting slower and slower as files pile up, I keep track of the files I processed (loaded data from) and will not process them again. I have a ProcessedFiles entity in CDS where I store the list of files. One of the queries at the beginning of the dataflow will filter out from the input files the ones I already processed, then subsequent queries will only load data from the new files. Finally the dataflow loads new entities to CDS for both the data loaded and the files I loaded them from. However, if the query that loads the data from the files fails, the query that writes the new files to the ProcessedFiles entity still executes.
... View more