Pandas DataFrame groupby two columns and get first and last

Nilani Algiriyage

I have a DataFrame Like following.

df = pd.DataFrame({'id' : [1,1,2,3,2],
                'value'  : ["a","b","a","a","c"], 'Time' : ['6/Nov/2012 23:59:59 -0600','6/Nov/2012 00:00:05 -0600','7/Nov/2012 00:00:09 -0600','27/Nov/2012 00:00:13 -0600','27/Nov/2012 00:00:17 -0600']})

I need to get an output like following.

combined_id | enter time | exit time | time difference

combined_id should be created by grouping 'id' and 'value'

g = df.groupby(['id', 'value'])

Following doesn’t work with grouping by two columns. (How to use first() and last() here as enter and exit times?)

df['enter'] = g.apply(lambda x: x.first())

To get difference would following work?

df['delta'] = (df['exit']-df['enter'].shift()).fillna(0)
Andy Hayden

First ensure you're column is a proper datetime column:

In [11]: df['Time'] = pd.to_datetime(df['Time'])

Now, you can do the groupby and use agg with the first and last groupby methods:

In [12]: g = df.groupby(['id', 'value'])

In [13]: res = g['Time'].agg({'first': 'first', 'last': 'last'})

In [14]: res = g['Time'].agg({'enter': 'first', 'exit': 'last'})

In [15]: res['time_diff'] = res['exit'] - res['enter']

In [16]: res
Out[16]:
                        exit               enter  time_diff
id value
1  a     2012-11-06 23:59:59 2012-11-06 23:59:59     0 days
   b     2012-11-06 00:00:05 2012-11-06 00:00:05     0 days
2  a     2012-11-07 00:00:09 2012-11-07 00:00:09     0 days
   c     2012-11-27 00:00:17 2012-11-27 00:00:17     0 days
3  a     2012-11-27 00:00:13 2012-11-27 00:00:13     0 days

Note: this is a bit of a boring example since there is only one item in each group...

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Java

Python pandas dataframe - daily data - get first and last day for every year

From Java

Pandas DataFrame Groupby two columns and get counts

From Java

How to GroupBy a Dataframe in Pandas and keep Columns

From Java

Pandas groupby two columns and plot

From Dev

summing two columns in a pandas dataframe

From Dev

How do I get the first timestamp (index) of a group when applying groupby to a python pandas dataframe?

From Dev

pandas: Divide DataFrame last row by first row

From Dev

How to get last group in Pandas' groupBy?

From Dev

Pandas dataframe first x columns

From Dev

Get first and last month in pandas dataframe where a specific value occurs

From Dev

Extract first and last row of a dataframe in pandas

From Dev

Remove last two characters from column names of all the columns in Dataframe - Pandas

From Dev

Subtracting Two Columns with a Groupby in Pandas

From Dev

get first and last values in a groupby

From Dev

pandas: get rows by comparing two columns of dataframe to list of tuples

From Dev

Pandas dataframe get all rows between zero(0) of mask column and get first and last row of each group

From Dev

sorting in pandas groupby with two columns

From Dev

get first and last values in a groupby

From Dev

pandas - how to get last n groups of a groupby object and combine them as a dataframe

From Dev

Pandas Dataframe: Multiplying Two Columns

From Dev

Pandas groupby two columns and plot

From Dev

Get the first cell from last row pandas dataframe

From Dev

Pandas DataFrame groupby two columns and get first and last

From Dev

pandas: groupby two columns nunique

From Dev

Remove last two characters from column names of all the columns in Dataframe - Pandas

From Dev

Return groupby columns as new dataframe in Python Pandas

From Dev

concatenate two columns in pandas dataframe

From Dev

Pandas Dataframe Groupby multiple columns

From Dev

Filter dataframe by two columns in Pandas

Related Related

  1. 1

    Python pandas dataframe - daily data - get first and last day for every year

  2. 2

    Pandas DataFrame Groupby two columns and get counts

  3. 3

    How to GroupBy a Dataframe in Pandas and keep Columns

  4. 4

    Pandas groupby two columns and plot

  5. 5

    summing two columns in a pandas dataframe

  6. 6

    How do I get the first timestamp (index) of a group when applying groupby to a python pandas dataframe?

  7. 7

    pandas: Divide DataFrame last row by first row

  8. 8

    How to get last group in Pandas' groupBy?

  9. 9

    Pandas dataframe first x columns

  10. 10

    Get first and last month in pandas dataframe where a specific value occurs

  11. 11

    Extract first and last row of a dataframe in pandas

  12. 12

    Remove last two characters from column names of all the columns in Dataframe - Pandas

  13. 13

    Subtracting Two Columns with a Groupby in Pandas

  14. 14

    get first and last values in a groupby

  15. 15

    pandas: get rows by comparing two columns of dataframe to list of tuples

  16. 16

    Pandas dataframe get all rows between zero(0) of mask column and get first and last row of each group

  17. 17

    sorting in pandas groupby with two columns

  18. 18

    get first and last values in a groupby

  19. 19

    pandas - how to get last n groups of a groupby object and combine them as a dataframe

  20. 20

    Pandas Dataframe: Multiplying Two Columns

  21. 21

    Pandas groupby two columns and plot

  22. 22

    Get the first cell from last row pandas dataframe

  23. 23

    Pandas DataFrame groupby two columns and get first and last

  24. 24

    pandas: groupby two columns nunique

  25. 25

    Remove last two characters from column names of all the columns in Dataframe - Pandas

  26. 26

    Return groupby columns as new dataframe in Python Pandas

  27. 27

    concatenate two columns in pandas dataframe

  28. 28

    Pandas Dataframe Groupby multiple columns

  29. 29

    Filter dataframe by two columns in Pandas

HotTag

Archive