Can fprintf UTF-8 text, but cannot use xmlwrite

I have a structure as:

q.txt(1:100) % contains 100 UTF-8 arabic words

I can dump a range to a file as:

fprintf(fid,'%s', char(strjoin(q.txt(30:40))));

The resultant file is UTF-8 encoded " with BOM " and can be viewed by any text editor such as NotePad++. But when I try to store my text in xml, I get UTF-8 encoding " without BOM ". Notepade++ also detects its encoding as: ANSI as UTF-8. No UTF-8 arabic characters are written.

docNode = com.mathworks.xml.XMLUtils.createDocument('qdoc');
toc = docNode.getDocumentElement;
toc.setAttribute('version','0.01');
simitem = docNode.createElement('simitem');
simitem.appendChild(docNode.createTextNode(char(strjoin(q.txt(30:40)))));
toc.appendChild(simitem);
xmlwrite('info.xml',docNode);

The produced xml file does not properly show arabic strings, the encoding is not correct. But other tags and english attributes show normally.

Answer 1

[Noam Greenboim]

Many Matlab functions don't add BOM to the UTF-8 files, although the encoding was set to UTF-8 (I think it's the default). This includes writetable, writecell and more.

Therefore, you need to add it manually.

I had the same issue, and I made a work-around -

https://www.mathworks.com/matlabcentral/fileexchange/75708-update-bom-for-unicode-text-files

Usage:

BOM('info.xml', 'UTF-8')

This should add the BOM to your file. Note that it returns the currect BOM (before the change), if it exists.

The idea behind the hack is to edit the file as a binary, and then add the 3 characters BOM. They are invisible if you just use a text editor.

[Walter Roberson]

The UTF-8 recommendation is not to add BOM.

[Noam Greenboim]

Maybe. But when you open it with Excel, for example, it would mix-up the non-English text.