Special Unicode Characters are not removed in Python 3
Himanshu Ahuja
I have a keys list including words. When I make this command:
for key in keys:
print(key)
I get normal output in terminal.
but when I print the entire list using print(keys), I get this output:
I have tried using key.replace("\u202c", ''), key.replace("\\u202c", ''), re.sub(u'\u202c', '', key) but none solved the problem. I also tried the solutions here, but none of them worked either:
Replacing a unicode character in a string in Python 3
Removing unicode \u2026 like characters in a string in python2.7
Python removing extra special unicode characters
How can I remove non-ASCII characters but leave periods and spaces using Python?
I scraped this from Google Trends using Beautiful Soup and retrieved text from get_text() Also in the page source of Google Trends Page, the words are listed as follows:
When I pasted the text here directly from the page source, the text pasted without these unusual symbols.
riteshtch
You can just strip out the characters using strip.
>>> keys=['\u202cABCD', '\u202cXYZ\u202c']
>>> for key in keys:
... print(key)
...
ABCD
XYZ
>>> newkeys=[key.strip('\u202c') for key in keys]
>>> print(keys)
['\u202cABCD', '\u202cXYZ\u202c']
>>> print(newkeys)
['ABCD', 'XYZ']
>>>
Tried 1 of your methods, it does work for me:
>>> keys
['\u202cABCD', '\u202cXYZ\u202c']
>>> newkeys=[]
>>> for key in keys:
... newkeys += [key.replace('\u202c', '')]
...
>>> newkeys
['ABCD', 'XYZ']
>>>
Collected from the Internet
Please contact [email protected] to delete if infringement.
edited at
- Prev: Separate columns with regular expressions
- Next: ValidationError HTTP Code for 'user already exist'
Related
Related Related
- 1
Replace special characters in a file with their unicode code (Python)
- 2
Replace special characters in a file with their unicode code (Python)
- 3
How to pad and align unicode strings with special characters in python?
- 4
Replace unicode formatted special characters
- 5
Convert a string to unicode with special characters
- 6
Convert special Characters to Unicode Escape Characters Scala
- 7
to_json not converting special characters to unicode style
- 8
Julia: How to deal with special unicode characters
- 9
Swift Special Unicode Characters in String Literals
- 10
How to convert special characters in a string to unicode?
- 11
Swift Special Unicode Characters in String Literals
- 12
How to escape special characters in Groovy, excluding unicode
- 13
How to convert special characters in dictionary to unicode in swift
- 14
Storing Unicode and special characters in MySQL tables
- 15
Python - regex - special characters and ñ
- 16
Python printing special Characters
- 17
Python: CLI special characters
- 18
Python sorting by special characters (+/-)
- 19
Special characters in python
- 20
Python2.7, what does the special characters mean in the utf-32 encoding output of a unicode string?
- 21
Unpack fixed width unicode file line with special characters. Python UnicodeDecodeError
- 22
Big Data free text with special characters- search via Python and giving unicode errors
- 23
Python to convert special unicode characters (like ♡) to their hexadecimal literals in a string (like 0x2661)
- 24
How to define function names with special characters in Python 3
- 25
Python Latin Characters and Unicode
- 26
Converting funny, special Latin characters to unicode (foreign characters)
- 27
Parse.com/Nodejs http-request special characters removed
- 28
Parse.com/Nodejs http-request special characters removed
- 29
Python encode url with special characters
Comments