How can you display a unicode string, say:
x <- "•"
using its escaped equivalent?
y <- "\u2022"
identical(x, y)
# [1] TRUE
(I'd like to be able to do this because CRAN packages must contain only ASCII, but sometimes you want to use unicode in an error message or similar)
After digging into some documentation about iconv
, I think you can accomplish this using only the base
package. But you need to pay extra attention to the encoding of the string.
On a system with UTF-8 encoding:
> stri_escape_unicode("你好世界")
[1] "\\u4f60\\u597d\\u4e16\\u754c"
# use big endian
> iconv(x, "UTF-8", "UTF-16BE", toRaw=T)
[[1]]
[1] 4f 60 59 7d 4e 16 75 4c
> x <- "•"
> iconv(x, "UTF-8", "UTF-16BE", toRaw=T)
[[1]]
[1] 20 22
But, if you are on a system with latin1
encoding, things may go wrong.
> x <- "•"
> y <- "\u2022"
> identical(x, y)
[1] FALSE
> stri_escape_unicode(x)
[1] "\\u0095" # <- oops!
# culprit
> Encoding(x)
[1] "latin1"
# and it causes problem for iconv
> iconv(x, Encoding(x), "Unicode")
Error in iconv(x, Encoding(x), "Unicode") :
unsupported conversion from 'latin1' to 'Unicode' in codepage 1252
> iconv(x, Encoding(x), "UTF-16BE")
Error in iconv(x, Encoding(x), "UTF-16BE") :
embedded nul in string: '\0•'
It is safer to cast the string into UTF-8 before converting to Unicode:
> iconv(enc2utf8(enc2native(x)), "UTF-8", "UTF-16BE", toRaw=T)
[[1]]
[1] 20 22
EDIT: This may cause some problems for strings already in UTF-8 encoding on some particular systems. Maybe it's safer to check the encoding before conversion.
> Encoding("•")
[1] "latin1"
> enc2native("•")
[1] "•"
> enc2native("\u2022")
[1] "•"
# on a Windows with default latin1 encoding
> Encoding("测试")
[1] "UTF-8"
> enc2native("测试")
[1] "<U+6D4B><U+8BD5>" # <- BAD!
For some characters or lanuages, UTF-16
may not be enough. So probably you should be using UTF-32
since
The UTF-32 form of a character is a direct representation of its codepoint.
Based on above trial and error, below is probably one safer escape function we can write:
unicode_escape <- function(x, endian="big") {
if (Encoding(x) != 'UTF-8') {
x <- enc2utf8(enc2native(x))
}
to.enc <- ifelse(endian == 'big', 'UTF-32BE', 'UTF-32LE')
bytes <- strtoi(unlist(iconv(x, "UTF-8", "UTF-32BE", toRaw=T)), base=16)
# there may be some better way to do thibs.
runes <- matrix(bytes, nrow=4)
escaped <- apply(runes, 2, function(rb) {
nonzero.bytes <- rb[rb > 0]
ifelse(length(nonzero.bytes) > 1,
# convert back to hex
paste("\\u", paste(as.hexmode(nonzero.bytes), collapse=""), sep=""),
rawToChar(as.raw(nonzero.bytes))
)
})
paste(escaped, collapse="")
}
> unicode_escape("•••ERROR!!!•••")
[1] "\\u2022\\u2022\\u2022ERROR!!!\\u2022\\u2022\\u2022"
> unicode_escape("Hello word! 你好世界!")
[1] "Hello word! \\u4f60\\u597d\\u4e16\\u754c!"
> "\u4f60\u597d\u4e16\u754c"
[1] "你好世界"
Collected from the Internet
Please contact [email protected] to delete if infringement.
Comments