如何从NLTK树中获取令牌？

Robbie Barrat 发表于 Dev

罗比·巴拉特（Robbie Barrat）

所以我把这棵树还给我

Tree('S', [('This', 'DT'), ('is', 'VBZ'), ('a', 'DT'), ('test', 'NN'), (',', ','), Tree('PERSON', [('Stackoverflow', 'NNP'), ('Users', 'NNP')]), ('.', '.')])

我可以将它变成一个像这样的漂亮的python列表

sentence = "This is a test, Stackoverflow Users."
tokens = nltk.word_tokenize(sentence)
tagged = nltk.pos_tag(tokens)
entities = nltk.chunk.ne_chunk(tagged)
tree = repr(entities) # THIS VARIABLE IS THE TREE THAT IS RETURNED TO ME
# below this point it's about turning the tree into a python list
tree = (("[" + tree[5:-1] + "]")).replace("Tree", "").replace(")", "]").replace("(", "[")
tree = ast.literal_eval(tree) #you'll need to import ast (included with python)

现在，树变量是这样的：

['S', [['This', 'DT'], ['is', 'VBZ'], ['a', 'DT'], ['test', 'NN'], [',', ','], ['ORGANIZATION', [['Stackoverflow', 'NNP']]], ['users', 'NNS'], ['.', '.']]]

当我尝试遍历并得到一串句子时，我得到

"This is a test, ORGANIZATION."

而不是期望的

"This is a test, Stackoverflow users."

我不能简单地使用句子变量，我需要能够从列表列表中获取句子。任何代码片段或建议，将不胜感激。

睡觉

>>> from nltk import Tree
>>> yourtree = Tree('S', [('This', 'DT'), ('is', 'VBZ'), ('a', 'DT'), ('test', 'NN'), (',', ','), Tree('PERSON', [('Stackoverflow', 'NNP'), ('Users', 'NNP')]), ('.', '.')])
>>> yourtree.leaves()
[('This', 'DT'), ('is', 'VBZ'), ('a', 'DT'), ('test', 'NN'), (',', ','), ('Stackoverflow', 'NNP'), ('Users', 'NNP'), ('.', '.')]
>>> tokens, pos = zip(*yourtree.leaves())
>>> tokens
('This', 'is', 'a', 'test', ',', 'Stackoverflow', 'Users', '.')
>>> pos
('DT', 'VBZ', 'DT', 'NN', ',', 'NNP', 'NNP', '.')

另请参阅：如何遍历NLTK树对象？

本文收集自互联网，转载请注明来源。

如有侵权，请联系[email protected] 删除。