Is there a pythonic way of shifting pandas dataframe cells to the left, while pushing out or overwriting any nan?

Jakob

I have a pandas dataframe (starting_df) with nan values in the left-hand columns. I'd like to shift all values over to the left for a left-aligned dataframe. My Dataframe is 24x24, but for argument's sake, I'm just posting a 4x4 version. After some cool initial answers here, I modified the dataframe to also include a non-leading nan, who's position I'd like to preserve. I have a piece of code that accomplishes what I want, but it relies on nested for-loops and suppressing an IndexError, which does not feel very pythonic. I have no experience with error handling in general, but simply suppressing an error does not seem to be the right strategy.

Starting dataframe and desired final dataframe:

enter image description here

Here is the code that (poorly) accomplishes the goal.

import pandas as pd
import numpy as np

def get_left_aligned(starting_df):
    """take a starting df with right-aligned numbers and nan, and
    turn it into a left aligned table."""
    left_aligned_df = pd.DataFrame()
    for temp_index_1 in range(0, starting_df.shape[0]):
        temp_series = []
        for temp_index_2 in range(0, starting_df.shape[0]):
            try:
                temp_series.append(starting_df.iloc[temp_index_2, temp_index_2 + temp_index_1])
                temp_index_2 += 1
            except IndexError:
                pass
        temp_series = pd.DataFrame(temp_series, columns=['col'+str(temp_index_1 + 1)])
        left_aligned_df = pd.concat([left_aligned_df, temp_series], axis=1)
    return left_aligned_df


df = pd.DataFrame(dict(col1=[1, np.nan, np.nan, np.nan],
                       col2=[5, 2, np.nan, np.nan],
                       col3=[7, np.nan, 3, np.nan],
                       col4=[9, 8, 6, 4]))

df_expected = pd.DataFrame(dict(col1=[1, 2, 3, 4],
                                col2=[5, np.nan, 6, np.nan],
                                col3=[7, 8, np.nan, np.nan],
                                col4=[9, np.nan, np.nan, np.nan]))

df_left = get_left_aligned(df)

I appreciate any help with this. Thanks!

Ferris

or transpose the df and use shift to shift by column, when the NA num is increasing 1 by 1.

dfn = df.T.copy()
for i, col in enumerate(dfn.columns):
    dfn[col] = dfn[col].shift(-i)
dfn = dfn.T

print(dfn)

   col1  col2  col3  col4
0   1.0   5.0   7.0   9.0
1   2.0   NaN   8.0   NaN
2   3.0   6.0   NaN   NaN
3   4.0   NaN   NaN   NaN

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Shifting Column to the left Pandas Dataframe

From Dev

Fill NaN values in a pandas DataFrame depending on values of cells to its left

From Dev

Pandas: shifting columns in grouped dataframe if NaN

From Dev

Shifting certain rows to the left in pandas dataframe

From Dev

Most pythonic way to concatenate pandas cells with conditions

From Dev

Pythonic way to get rows AND columns in Pandas Dataframe which contains NaN values

From Dev

pythonic way to parse/split URLs in a pandas dataframe

From Dev

more pythonic way - pandas dataframe manipulation

From Dev

Pythonic way to calculate streaks in pandas dataframe

From Dev

A Pythonic way to reshape Pandas.DataFrame's

From Dev

pandas - Pythonic way to slicing DataFrame with DateTimeIndex

From Dev

How to join two column in dataframe by overwriting only NaN values in pandas?

From Dev

Move cells to the left in pandas DataFrame bsed on groups

From Dev

Pandas: shifting columns depending on if NaN or not

From Dev

Shifting non-NA cells to the left

From Java

How to check if any value is NaN in a Pandas DataFrame

From Dev

Problems with pasting while shifting down cells

From Dev

Overwriting Nan values with .loc in Pandas

From Dev

Pandas easy API to find out all inf or nan cells?

From Java

What is the purpose of left shifting zero by any amount?

From Dev

Shifting columns in grouped pandas dataframe

From Dev

Shifting values in datetimeindex of pandas dataframe

From Dev

Adding row shifting in pandas dataframe

From Dev

Efficient/Pythonic way to Filter pandas DataFrame based on priority

From Dev

Efficient/Pythonic way to create lists from pandas Dataframe column

From Java

Pythonic way for calculating length of lists in pandas dataframe column

From Dev

Pythonic way to use an 'slicer' and a 'where'-equivalent on a pandas dataframe

From Dev

Pythonic way to convert Pandas dataframe from wide to long

From Dev

Pythonic way of obtaining serial correlation of elements in pandas dataframe

Related Related

  1. 1

    Shifting Column to the left Pandas Dataframe

  2. 2

    Fill NaN values in a pandas DataFrame depending on values of cells to its left

  3. 3

    Pandas: shifting columns in grouped dataframe if NaN

  4. 4

    Shifting certain rows to the left in pandas dataframe

  5. 5

    Most pythonic way to concatenate pandas cells with conditions

  6. 6

    Pythonic way to get rows AND columns in Pandas Dataframe which contains NaN values

  7. 7

    pythonic way to parse/split URLs in a pandas dataframe

  8. 8

    more pythonic way - pandas dataframe manipulation

  9. 9

    Pythonic way to calculate streaks in pandas dataframe

  10. 10

    A Pythonic way to reshape Pandas.DataFrame's

  11. 11

    pandas - Pythonic way to slicing DataFrame with DateTimeIndex

  12. 12

    How to join two column in dataframe by overwriting only NaN values in pandas?

  13. 13

    Move cells to the left in pandas DataFrame bsed on groups

  14. 14

    Pandas: shifting columns depending on if NaN or not

  15. 15

    Shifting non-NA cells to the left

  16. 16

    How to check if any value is NaN in a Pandas DataFrame

  17. 17

    Problems with pasting while shifting down cells

  18. 18

    Overwriting Nan values with .loc in Pandas

  19. 19

    Pandas easy API to find out all inf or nan cells?

  20. 20

    What is the purpose of left shifting zero by any amount?

  21. 21

    Shifting columns in grouped pandas dataframe

  22. 22

    Shifting values in datetimeindex of pandas dataframe

  23. 23

    Adding row shifting in pandas dataframe

  24. 24

    Efficient/Pythonic way to Filter pandas DataFrame based on priority

  25. 25

    Efficient/Pythonic way to create lists from pandas Dataframe column

  26. 26

    Pythonic way for calculating length of lists in pandas dataframe column

  27. 27

    Pythonic way to use an 'slicer' and a 'where'-equivalent on a pandas dataframe

  28. 28

    Pythonic way to convert Pandas dataframe from wide to long

  29. 29

    Pythonic way of obtaining serial correlation of elements in pandas dataframe

HotTag

Archive