我正在尝试此代码
from sklearn.feature_extraction.text import TfidfVectorizer
import numpy as np
train_data = ["football is the sport","gravity is the movie", "education is imporatant"]
vectorizer = TfidfVectorizer(sublinear_tf=True, max_df=0.5,
stop_words='english')
print "Applying first train data"
X_train = vectorizer.fit_transform(train_data)
print vectorizer.get_feature_names()
print "\n\nApplying second train data"
train_data = ["cricket", "Transformers is a film","AIMS is a college"]
X_train = vectorizer.transform(train_data)
print vectorizer.get_feature_names()
print "\n\nApplying fit transform onto second train data"
X_train = vectorizer.fit_transform(train_data)
print vectorizer.get_feature_names()
这个的输出是
Applying first train data
[u'education', u'football', u'gravity', u'imporatant', u'movie', u'sport']
Applying second train data
[u'education', u'football', u'gravity', u'imporatant', u'movie', u'sport']
Applying fit transform onto second train data
[u'aims', u'college', u'cricket', u'film', u'transformers']
我使用fit_transform给矢量化器提供了第一组数据，因此它给了我一个功能名称，就像[u'education', u'football', u'gravity', u'imporatant', u'movie', u'sport']之后我将另一个训练集应用于相同的矢量化器一样，但是它给了我与不使用fit或fit_transform相同的功能名称。但是我想知道如何在不覆盖以前的oncs的情况下更新矢量化器的功能。如果再次使用fit_transform，以前的功能将被覆盖。因此，我想更新矢量化器的功能列表。所以我想要类似的方法[u'education', u'football', u'gravity', u'imporatant', u'movie', u'sport',u'aims', u'college', u'cricket', u'film', u'transformers']。
在sklearn术语中，这称为部分拟合，而您不能使用来实现TfidfVectorizer。有两种解决方法：
HashingVectorizer支持局部拟合的。但是，get_feature_names由于存在散列特征，因此没有方法，因此不会保留原始特征。另一个优点是，这将大大提高内存效率。第一种方法的示例：
from sklearn.feature_extraction.text import TfidfVectorizer
import numpy as np
train_data1 = ["football is the sport", "gravity is the movie", "education is important"]
vectorizer = TfidfVectorizer(stop_words='english')
print("Applying first train data")
X_train = vectorizer.fit_transform(train_data1)
print(vectorizer.get_feature_names())
print("\n\nApplying second train data")
train_data2 = ["cricket", "Transformers is a film", "AIMS is a college"]
X_train = vectorizer.transform(train_data2)
print(vectorizer.get_feature_names())
print("\n\nApplying fit transform onto second train data")
X_train = vectorizer.fit_transform(train_data1 + train_data2)
print(vectorizer.get_feature_names())
输出：
Applying first train data
['education', 'football', 'gravity', 'important', 'movie', 'sport']
Applying second train data
['education', 'football', 'gravity', 'important', 'movie', 'sport']
Applying fit transform onto second train data
['aims', 'college', 'cricket', 'education', 'film', 'football', 'gravity', 'important', 'movie', 'sport', 'transformers']
本文收集自互联网，转载请注明来源。
如有侵权，请联系[email protected] 删除。
我来说两句