Missing UTF-8 BOM in .csv after flow copy file to another OneDrive for Business

[AMHo]

Hi Community,

I do copy .csv files from one OneDrive for Business to another OneDrive for Business with a flow (template sync, "Copy files from one OneDrive for Business account or folder to another").

It's working fine, but, after the files have been moved, the UTF-8 BOM is missing in all files. (noticed that all files in the destination folder are 3 Byte smaller than the files in source folder and tested files)

I did build a similar flow from scratch without template - still the missing UTF8-BOM. 

Anyone ever had problem like that and could suggest anything that helps preserving UTF-8 BOM in the copy process of the flow? 

Thank you.

[v-yuazh-msft]

Hi @AMHo,

Could you please explain more details about the UTF8-BOM that you mentioned?

Best regards,

Alice

[AMHo]

Hi Alice, thanks for your answer.

A bit of background info: we provide product information as .csv, xml and txt, our customers import that data with many different tools in their various ERP systems, shops etc. We deliver data in 14 european languages every day in a 10 sec push rate. One of the requirements is to provide data with the corresponding BOM. 

I manage 100+ flows for different customers and scenarious, we never had problems as we saved email attachments to OneDrive for Business, UTF8-BOM never disapeared. We do now handle 30+MB files, email is not the option to deliver and flow it to the OneDrives, one reason is that it is limited to 20 MB attachments, other is the failure rate.

As discribed, the flow from OnDrive for Business to OneDrive for Business does not keep the UTF8-BOM when the file has file extension .csv. It does when the file is .txt or .xml, so why is the behaviour different? I expected to get the exact file from one place to another disregarding the file extension. I could not find any setting to change this behaviour for .csv. It is not possible to rename the file before flow and then rename it back in OneDrive for Business to OneDrive for Business Flow (its not a Sharepoint Flow) so I got stuck. We can't change anything else in the this data delivery process, we need to deliver .csv files with UTF-8 BOM.

There is the definition of BOM: https://en.wikipedia.org/wiki/Byte_order_mark

I hope I explained right. Any help and suggestions are very much welcome, thank you in advance.

Andrea

[FumiakiM]

Hi, @AMHo  !

I had a similar issue dealing with CSV in Japanese, and found a solution to add BOM using concat.

https://blog.karamem0.dev/entry/2020/09/30/120000

# the second step is "Create a file (SharePoint)"

the function is "concat(decodeUriComponent('%EF%BB%BF'), body('Create_CSV_Table'))"

Hope this helps.