To get them to export in the desired format you would need to convert them to text by preceding them with a ' e.g.
If there is a long list this would be tedious. Say the numbers are in Column A starting at row 1 then in B1 enter
and copy the formula down. Then select column B, Copy then Edit > Paste Special and choose Values. Then delete the original column A. Save the file as a .CSV
Open the CSV in Notepad, Edit > Replace, enter ! for what to replace and leave the replace with box blank.