我有一些看起来像的植物数据(但我最多有7个属性):
Unnamed: 0 plant att_1 att_2 ...
0 0 plant_a sunlover tall
1 1 plant_b waterlover sunlover
2 2 plant_c fast growing sunlover
我试图像这样使用pandas get_dummies:
df = pd.DataFrame({'A': ['a', 'b', 'a'], 'B': ['b', 'a', 'c'],'C': [1, 2, 3]})
pd.get_dummies(df, prefix=['col1', 'col2']):
。
C col1_a col1_b col2_a col2_b col2_c
0 1 1 0 0 1 0
1 2 0 1 1 0 0
2 3 1 0 0 0 1
但是,sunlover应该在att_1或att_2中编码为1。然后,我将得到大约30个虚拟变量,而不是7 * 30 = 210个变量。我试图遍历整个集合并为每个假人添加值:
for count, plants in enumerate(data_plants.iterrows()):
print("First", count, plants)
for attribute in plants:
print("Second", count, attribute)
该代码只是打印,因为我看到了浪费时间的问题。那样的工作,但是它不够快,无法用于100k和更多行。我考虑过使用.value_counts()来获取属性,然后访问数据帧虚拟变量以将其更新为1,但随后我将覆盖该属性。此刻,我有点迷茫,失去了主意。也许我不得不使用其他软件包?
目标将是这样的:
Unnamed: 0 plant att_1 att_2 sunlover waterlover tall ...
0 0 plant_a sunlover tall 1 0 1
1 1 plant_b waterlover sunlover 1 1 0
2 2 plant_c fast growing sunlover 1 0 0
使用get_dummies
有max
:
c = ['att_1', 'att_2']
df1 = df.join(pd.get_dummies(df[c], prefix='', prefix_sep='').max(axis=1, level=0))
print (df1)
plant att_1 att_2 fast growing sunlover waterlover tall
0 plant_a sunlover tall 0 1 0 1
1 plant_b waterlover sunlover 0 1 1 0
2 plant_c fast growing sunlover 1 1 0 0
3k
实际数据中行的性能应该有所不同:
df = pd.concat([df] * 1000, ignore_index=True)
In [339]: %%timeit
...:
...: c = ['att_1', 'att_2']
...: df1 = df.join(pd.get_dummies(df[c], prefix='', prefix_sep='').max(axis=1, level=0))
...:
...:
10.7 ms ± 1.11 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)
In [340]: %%timeit
...: attCols = df[['att_1', 'att_2']]
...: colVals = pd.Index(np.sort(attCols.stack().unique()))
...: def myDummies(row):
...: return pd.Series(colVals.isin(row).astype(int), index=colVals)
...:
...: df1 = df.join(attCols.apply(myDummies, axis=1))
...:
...:
1.03 s ± 22 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
另一个解决方案:
In [133]: %%timeit
...: c = ['att_1', 'att_2']
...: df1 = (df.join(pd.DataFrame([dict.fromkeys(x, 1) for x in df[c].to_numpy()])
...: .fillna(0)
...: .astype(np.int8)))
...:
13.1 ms ± 723 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句