从熊猫数据帧的每一行中的单词中删除多个字符组合

debugcn 发表于 Dev

I_m_LeMarque

我有一个类似于以下示例数据的大型数据集：

import pandas as pd

raw_data = {'ID': [1,2,3,4,5,6,7,8,9,10], 
        'body': ['FITrnXS$100', '$1000rnReason', 'rnIf', 'bevlauedrnrnnext', 'obccrnrnnoncrnrnactionrn', 'rnrnnotification', 'insdrnrnnon', 'rnrnupdated', 'rnreason', 'rnrnrnLOR']}
df = pd.DataFrame(raw_data, columns = ['ID', 'body'])
df

我想要做的是使用我在下面的代码中定义的单词列表：

remove_string = ['rn', 'rnr', 'rnrn', 'rnrnrn']

然后使用上面的 remove_string 从文本（数据框的“body”列）中的单词中删除这些（如果找到）。

下表将是预期的结果

ID  body            cleaned_txt    Removed_string
1   FITrnXS$100     FIT XS$100     rn
2   $1000rnReason   $1000 Reason      rn
3   rnIf            IF               rn
4   bevlauedrnrnnext    bevalue next    rnrn
5   obccrnrnnoncrnrnactionrn    obcc nonc actionrn  rnrn
6   rnrnnotification    notification    rnrn
7   insdrnrnnon insd non    rnrn
8   rnrnupdated updated rnrn
9   rnreason    reason  rn
10  rnrnrnLOR   LOR rnrnrn

不幸的是，我试图将数据转换为如下列表：

text = df['body'].tolist()

然后像这样应用函数：

def clnTxt(text):
    txt = [item.replace('rnrn', '\n') for item in text]
    txt = [item.replace('nrn', '\n') for item in txt]
    return txt

text = clnTxt(text)

这不是正确的方法。我应该能够直接在数据帧上应用函数，因此对于每一行，执行清理操作并创建其他列。

只是为我的问题寻找更好的解决方案。

耶斯列

因为较长的字符串包含较短的字符串，所以顺序很重要。因此，通过反向列表循环[::-1]并使用Series.str.extract新列的值，然后Series.str.replace与同一列一起使用。

最后DataFrame.dot用于将所有提取的值与分隔符组合到新列：

remove_string = ['rn', 'rnr', 'rnrn', 'rnrnrn']

df['cleaned_txt'] = df['body']
for i in remove_string[::-1]:
    df[i] = df['cleaned_txt'].str.extract('({})'.format(i)) 
    df['cleaned_txt'] = df['cleaned_txt'].str.replace(i, '')

df['Removed_string'] = (df[remove_string].notna()
                                         .dot(pd.Index(remove_string) + ',')
                                         .str.strip(','))
df = df.drop(remove_string, axis=1)
print (df)
   ID                      body     cleaned_txt Removed_string
0   1               FITrnXS$100       FITXS$100             rn
1   2             $1000rnReason     $1000Reason             rn
2   3                      rnIf              If             rn
3   4          bevlauedrnrnnext    bevlauednext           rnrn
4   5  obccrnrnnoncrnrnactionrn  obccnoncaction        rn,rnrn
5   6          rnrnnotification    notification           rnrn
6   7               insdrnrnnon         insdnon           rnrn
7   8               rnrnupdated         updated           rnrn
8   9                  rnreason           eason            rnr
9  10                 rnrnrnLOR             LOR         rnrnrn

如果需要用空格替换：

remove_string = ['rn', 'rnr', 'rnrn', 'rnrnrn']

df['cleaned_txt'] = df['body']
for i in remove_string[::-1]:
    df[i] = df['cleaned_txt'].str.extract('({})'.format(i)) 
    df['cleaned_txt'] = df['cleaned_txt'].str.replace(i, ' ')

df['Removed_string'] = (df[remove_string].notna()
                                         .dot(pd.Index(remove_string) + ',')
                                         .str.strip(','))
df = df.drop(remove_string, axis=1)
print (df)

   ID                      body        cleaned_txt Removed_string
0   1               FITrnXS$100         FIT XS$100             rn
1   2             $1000rnReason       $1000 Reason             rn
2   3                      rnIf                 If             rn
3   4          bevlauedrnrnnext      bevlaued next           rnrn
4   5  obccrnrnnoncrnrnactionrn  obcc nonc action         rn,rnrn
5   6          rnrnnotification       notification           rnrn
6   7               insdrnrnnon           insd non           rnrn
7   8               rnrnupdated            updated           rnrn
8   9                  rnreason              eason            rnr
9  10                 rnrnrnLOR                LOR         rnrnrn

编辑1：

#dictioanry for replace
remove_string = {"rn":" ", "rnr":"\n", "rnrn":"\n", "rnrnrn":"\n"}

#sorting by keys for list of tuples 
rem = sorted(remove_string.items(), key=lambda s: len(s[0]), reverse=True)
print (rem)
[('rnrnrn', '\n'), ('rnrn', '\n'), ('rnr', '\n'), ('rn', ' ')]

df['cleaned_txt'] = df['body']
for i, j in rem:
    df[i] = df['cleaned_txt'].str.extract('({})'.format(i)) 
    df['cleaned_txt'] = df['cleaned_txt'].str.replace(i, j)

cols = list(remove_string.keys())
df['Removed_string'] = (df[cols].notna().dot(pd.Index(cols) + ',')
                                        .str.strip(','))
df = df.drop(remove_string, axis=1)
print (df)
   ID                      body          cleaned_txt Removed_string
0   1               FITrnXS$100           FIT XS$100             rn
1   2             $1000rnReason         $1000 Reason             rn
2   3                      rnIf                   If             rn
3   4          bevlauedrnrnnext       bevlaued\nnext           rnrn
4   5  obccrnrnnoncrnrnactionrn  obcc\nnonc\naction         rn,rnrn
5   6          rnrnnotification       \nnotification           rnrn
6   7               insdrnrnnon            insd\nnon           rnrn
7   8               rnrnupdated            \nupdated           rnrn
8   9                  rnreason              \neason            rnr
9  10                 rnrnrnLOR                \nLOR         rnrnrn

本文收集自互联网，转载请注明来源。

如有侵权，请联系[email protected] 删除。