You can use Parse CSV action from Plumsail Documents connector. It allows you to convert CSV into an array and variables for each column. Please read this article demonstrating how it works.
In your case you will need to parse your CSV into an array using this action, then you can use Create XLSX document from template action from the same connector to generate an XLSX file. Please read this article describing how to work with XLSX templates.