한 데이터 프레임의 행을 Pandas의 다른 데이터 프레임의 열로 전치하는 최적의 방법은 무엇입니까?

Thdoan

주어진 df_people :

   Name
0  Tom
1  Jerry

df_colors (헤더 행 없음) :

0  Red
1  Green
2  Blue

어떤이의 데이터 취할 수있는 최적의 방법으로 간주됩니다 df_colors를 하고 추가 df_people 하도록 df_people이 결합 될 때 같을 것이다 :

   Name   Color_0  Color_1  Color_2
0  Tom    Red      Green    Blue
1  Jerry  Red      Green    Blue

아래는 지금까지 내가 가지고있는 것인데, 작동하지만 더 나은 또는 더 간결한 방법이 있는지 궁금합니다.

# Store data for new columns in a dictionary
new_columns = {}
for index_people, row_people in df_people.iterrows():
    for index_colors, row_colors in df_colors.iterrows():
        key = 'Color_' + str(index_colors)
        if (key in new_columns):
            new_columns[key].append(row_colors[0])
        else:
            new_columns[key] = [row_colors[0]]

# Add dictionary data as new columns
for key, value in new_columns.items():
    df_people[key] = value

최신 정보

답변 해주셔서 감사합니다. 실제 데이터 프레임은 GB 크기이므로 속도가 중요했기 때문에 가장 빠른 방법을 사용했습니다. 다음은 테스트 케이스에 대한 코드입니다.

# Import required modules
import pandas as pd
import timeit

# Original
def method_1():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    # Store data for new columns in a dictionary
    new_columns = {}
    for index_people, row_people in df_people.iterrows():
        for index_colors, row_colors in df_colors.iterrows():
            key = 'Color_' + str(index_colors)
            if (key in new_columns):
                new_columns[key].append(row_colors[0])
            else:
                new_columns[key] = [row_colors[0]]
    # Add dictionary data as new columns
    for key, value in new_columns.items():
        df_people[key] = value

# YOBEN_S - https://stackoverflow.com/a/60805881/452587
def method_2():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    _s = pd.concat([df_colors]*len(df_people), axis=1)
    _s.columns = df_people.index
    df_people = df_people.join(_s.T.add_prefix('Color_'))

# Dani Mesejo - https://stackoverflow.com/a/60805898/452587
def method_3():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    # Create mock key
    _m1 = df_people.assign(key=1)
    # Set new column names, transpose, and create mock key
    _m2 = df_colors.set_index('Color_' + df_colors.index.astype(str)).T.assign(key=1)
    df_people =  _m1.merge(_m2, on='key').drop('key', axis=1)

# Erfan - https://stackoverflow.com/a/60806018/452587
def method_4():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    df_colors = df_colors.T.reindex(df_people.index).ffill().add_prefix('Color_')
    df_people = df_people.join(df_colors)

print('Method 1:', timeit.timeit(method_1, number=10000))
print('Method 2:', timeit.timeit(method_2, number=10000))
print('Method 3:', timeit.timeit(method_3, number=10000))
print('Method 4:', timeit.timeit(method_4, number=10000))

산출:

Method 1: 36.029883089
Method 2: 27.042384837999997
Method 3: 68.22421793800001
Method 4: 32.94155895

시나리오를 단순화하려는 노력으로 안타깝게도 지나치게 단순화했습니다. 지금 질문을 다시 말하기에는 너무 늦었으므로 나중에 관련 질문을 게시 할 것입니다. 실제 시나리오에서뿐만 아니라 수학, 그래서 대신에 단순히 추가 열을 포함 df_colorsdf_people나는 또한 추가 된 각 셀에 대해 해당 행의 열에 대한 몇 가지 계산을 수행해야합니다.

업데이트 2

샘플 데이터 프레임을 더 크게 만들고 (jezrael에게 감사드립니다) 두 가지 새로운 방법을 추가했습니다.

# Import required modules
import numpy as np
import pandas as pd
import timeit

# Original
def method_1():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    # Store data for new columns in a dictionary
    new_columns = {}
    for index_people, row_people in df_people.iterrows():
        for index_colors, row_colors in df_colors.iterrows():
            key = 'Color_' + str(index_colors)
            if (key in new_columns):
                new_columns[key].append(row_colors[0])
            else:
                new_columns[key] = [row_colors[0]]
    # Add dictionary data as new columns
    for key, value in new_columns.items():
        df_people[key] = value

# YOBEN_S - https://stackoverflow.com/a/60805881/452587
def method_2():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    _s = pd.concat([df_colors]*len(df_people), axis=1)
    _s.columns = df_people.index
    df_people = df_people.join(_s.T.add_prefix('Color_'))

# sammywemmy - https://stackoverflow.com/a/60805964/452587
def method_3():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    # Create a new column in df_people with aggregate of df_colors;
    df_people['Colors'] = df_colors[0].str.cat(sep=',')
    # Concatenate df_people['Name'] and df_people['Colors'];
    # split column, expand into a dataframe, and add prefix
    df_people = pd.concat([df_people.Name, df_people.Colors.str.split(',', expand=True).add_prefix('Color_')], axis=1)

# Dani Mesejo - https://stackoverflow.com/a/60805898/452587
def method_4():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    # Create mock key
    _m1 = df_people.assign(key=1)
    # Set new column names, transpose, and create mock key
    _m2 = df_colors.set_index('Color_' + df_colors.index.astype(str)).T.assign(key=1)
    df_people =  _m1.merge(_m2, on='key').drop('key', axis=1)

# Erfan - https://stackoverflow.com/a/60806018/452587
def method_5():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    df_colors = df_colors.T.reindex(df_people.index).ffill().add_prefix('Color_')
    df_people = df_people.join(df_colors)

# jezrael - https://stackoverflow.com/a/60826723/452587
def method_6():
    df_people = pd.DataFrame(['Tom', 'Jerry', 'Bob', 'John', 'Bill', 'Tim', 'Harry', 'Rick'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red', 'Green', 'Blue'] * 10, columns=None)
    _a = np.broadcast_to(df_colors[0], (len(df_people), len(df_colors)))
    df_people = df_people.join(pd.DataFrame(_a, index=df_people.index).add_prefix('Color_'))

print('Method 1:', timeit.timeit(method_1, number=3))
print('Method 2:', timeit.timeit(method_2, number=3))
print('Method 3:', timeit.timeit(method_3, number=3))
print('Method 4:', timeit.timeit(method_4, number=3))
print('Method 5:', timeit.timeit(method_5, number=3))
print('Method 6:', timeit.timeit(method_6, number=3))

산출:

Method 1: 74.512771493
Method 2: 1.0007798979999905
Method 3: 0.40823360299999933
Method 4: 0.08115736700000298
Method 5: 0.11704620100000795
Method 6: 0.04700596800000767

업데이트 3

실제 데이터 세트를 더 정확하게 반영하는 전치 계산 관련 질문을 게시했습니다 .

Pandas에서 전치하고 계산하는 가장 빠른 방법은 무엇입니까?

이스 르엘

numpy.broadcast_to방법 별로 성능을 향상시킬 수 있습니다 .

df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)

a = np.broadcast_to(df_colors[0], (len(df_people), len(df_colors)))
df = df_people.join(pd.DataFrame(a, index=df_people.index).add_prefix('Color_'))
print (df)
    Name Color_0 Color_1 Color_2
0    Tom     Red   Green    Blue
1  Jerry     Red   Green    Blue

import timeit

def method_2():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    _s = pd.concat([df_colors]*len(df_people), axis=1)
    _s.columns = df_people.index
    df_people = df_people.join(_s.T.add_prefix('Color_'))

def method_5():
    df_people = pd.DataFrame([['Tom'], ['Jerry']], columns=['Name'])
    df_colors = pd.DataFrame([['Red'], ['Green'], ['Blue']], columns=None)
    a = np.broadcast_to(df_colors[0], (len(df_people), len(df_colors)))
    df_people = df_people.join(pd.DataFrame(a, index=df_people.index).add_prefix('Color_'))

print('Method 2:', timeit.timeit(method_2, number=10000))
Method 2: 27.919169027998578

print('Method 5:', timeit.timeit(method_5, number=10000))
Method 5: 21.452649746001043

그러나 나는 DataFrame3k 행과 30 열에 대해 큰 테스트가 더 낫다고 생각합니다 . 그러면 타이밍이 다릅니다.

# Import required modules
import pandas as pd
import timeit

# Original
def method_1():
    df_people = pd.DataFrame(['Tom','Jerry','Bob'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red','Green', 'Blue'] * 10, columns=None)
    # Store data for new columns in a dictionary
    new_columns = {}
    for index_people, row_people in df_people.iterrows():
        for index_colors, row_colors in df_colors.iterrows():
            key = 'Color_' + str(index_colors)
            if (key in new_columns):
                new_columns[key].append(row_colors[0])
            else:
                new_columns[key] = [row_colors[0]]
    # Add dictionary data as new columns
    for key, value in new_columns.items():
        df_people[key] = value

# YOBEN_S - https://stackoverflow.com/a/60805881/452587
def method_2():
    df_people = pd.DataFrame(['Tom','Jerry','Bob'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red','Green', 'Blue'] * 10, columns=None)
    _s = pd.concat([df_colors]*len(df_people), axis=1)
    _s.columns = df_people.index
    df_people = df_people.join(_s.T.add_prefix('Color_'))

# Dani Mesejo - https://stackoverflow.com/a/60805898/452587
def method_3():
    df_people = pd.DataFrame(['Tom','Jerry','Bob'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red','Green', 'Blue'] * 10, columns=None)
    # Create mock key
    _m1 = df_people.assign(key=1)
    # Set new column names, transpose, and create mock key
    _m2 = df_colors.set_index('Color_' + df_colors.index.astype(str)).T.assign(key=1)
    df_people =  _m1.merge(_m2, on='key').drop('key', axis=1)

# Erfan - https://stackoverflow.com/a/60806018/452587
def method_4():
    df_people = pd.DataFrame(['Tom','Jerry','Bob'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red','Green', 'Blue'] * 10, columns=None)
    df_colors = df_colors.T.reindex(df_people.index).ffill().add_prefix('Color_')
    df_people = df_people.join(df_colors)

def method_5():
    df_people = pd.DataFrame(['Tom','Jerry','Bob'] * 1000, columns=['Name'])
    df_colors = pd.DataFrame(['Red','Green', 'Blue'] * 10, columns=None)
    a = np.broadcast_to(df_colors[0], (len(df_people), len(df_colors)))
    df_people = df_people.join(pd.DataFrame(a, index=df_people.index).add_prefix('Color_'))

print('Method 1:', timeit.timeit(method_1, number=3))
print('Method 2:', timeit.timeit(method_2, number=3))
print('Method 3:', timeit.timeit(method_3, number=3))
print('Method 4:', timeit.timeit(method_4, number=3))
print('Method 5:', timeit.timeit(method_5, number=3))

Method 1: 34.91457201199955
Method 2: 0.7901797180002177
Method 3: 0.05690281799979857
Method 4: 0.05774562500118918
Method 5: 0.026483284000278218

이 기사는 인터넷에서 수집됩니다. 재 인쇄 할 때 출처를 알려주십시오.

침해가 발생한 경우 연락 주시기 바랍니다[email protected] 삭제

에서 수정
0

몇 마디 만하겠습니다

0리뷰
로그인참여 후 검토

관련 기사

Related 관련 기사

뜨겁다태그

보관