StopIteration error when using groupby method of Pandas DataFrame

Shafique Jamal

I'm having a similar problem with the groupby method that this person has posted on StackOverflow:

pandas group StopIteration error

What I am trying to do with the grouby method is simpler, but I am getting a similar StopIteration error:

Traceback (most recent call last):
  File "prepare_data_TJ2012_v1p0.py", line 107, in <module>
    grouped = df.groupby('hh').apply(f)
  File "/Users/shafiquejamal/allfiles/htdocs/venvs/easyframes-py3/lib/python3.4/site-packages/pandas/core/groupby.py", line 637, in apply
    return self._python_apply_general(f)
  File "/Users/shafiquejamal/allfiles/htdocs/venvs/easyframes-py3/lib/python3.4/site-packages/pandas/core/groupby.py", line 644, in _python_apply_general
    not_indexed_same=mutated)
  File "/Users/shafiquejamal/allfiles/htdocs/venvs/easyframes-py3/lib/python3.4/site-packages/pandas/core/groupby.py", line 2657, in _wrap_applied_output
    v = next(v for v in values if v is not None)
StopIteration

Here is the code that produces it:

df = pd.DataFrame(
            {'educ': {0: 'pri', 1: 'bach', 2: 'pri', 3: 'hi', 4: 'bach', 5: 'sec', 
                6: 'hi', 7: 'hi', 8: 'pri', 9: 'pri'}, 
             'hh': {0: 1, 1: 1, 2: 1, 3: 2, 4: 3, 5: 3, 6: 4, 7: 4, 8: 4, 9: 4}, 
             'id': {0: 1, 1: 2, 2: 3, 3: 1, 4: 1, 5: 2, 6: 1, 7: 2, 8: 3, 9: 4}, 
             'has_car': {0: 1, 1: 1, 2: 1, 3: 1, 4: 0, 5: 0, 6: 1, 7: 1, 8: 1, 9: 1}, 
             'weighthh': {0: 2, 1: 2, 2: 2, 3: 3, 4: 2, 5: 2, 6: 3, 7: 3, 8: 3, 9: 3}, 
             'house_rooms': {0: 3, 1: 3, 2: 3, 3: 2, 4: 1, 5: 1, 6: 3, 7: 3, 8: 3, 9: 3}, 
             'prov': {0: 'BC', 1: 'BC', 2: 'BC', 3: 'Alberta', 4: 'BC', 5: 'BC', 6: 'Alberta', 
                7: 'Alberta', 8: 'Alberta', 9: 'Alberta'}, 
             'age': {0: 44, 1: 43, 2: 13, 3: 70, 4: 23, 5: 20, 6: 37, 7: 35, 8: 8, 9: 15}, 
             'fridge': {0: 'yes', 1: 'yes', 2: 'yes', 3: 'no', 4: 'yes', 5: 'yes', 6: 'no', 
                7: 'no', 8: 'no', 9: 'no'}, 
             'male': {0: 1, 1: 0, 2: 1, 3: 1, 4: 1, 5: 0, 6: 1, 7: 0, 8: 0, 9: 0}})
print(df)
print('-- groupby dataframes ---')
def f(df):
    print('-------------------------')
    print('DataFrame' )
    print(df)
    s = df['age']
    print(s)
    print('----> Not nulls:')
    s_notnulls = ~s.isnull()
    print(s_notnulls)
    print('----> Number of non-nulls: %d' % len(s_notnulls[s_notnulls==True]))
df.groupby('hh').apply(f)

I want to perform an operation on a column, by group, if there is at least one non-null value in another column.

I'm using pandas==0.14.1. It seems that the loop over the groups goes too long. Is this a bug? (or maybe I'm using the groupby method wrong...)

chrisb

You are getting this error because the function you are passing to apply doesn't return anything. If all you care about is the printed output, you could just return the df back, like this.

def f(df):
    print('-------------------------')
    print('DataFrame' )
    print(df)
    s = df['age']
    print(s)
    print('----> Not nulls:')
    s_notnulls = ~s.isnull()
    print(s_notnulls)
    print('----> Number of non-nulls: %d' % len(s_notnulls[s_notnulls==True]))

    return df

Then the apply will run through without error.

In [295]: df.groupby('hh').apply(f)
-------------------------
DataFrame
   age  educ fridge  has_car  hh  house_rooms  id  male prov  weighthh
0   44   pri    yes        1   1            3   1     1   BC         2
1   43  bach    yes        1   1            3   2     0   BC         2
2   13   pri    yes        1   1            3   3     1   BC         2
.....

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Java

Pandas transform dataframe using groupby when count of a string in a column is maximum

From Dev

how to concat sets when using groupby in pandas dataframe?

From Dev

using groupby on pandas dataframe to group by financial year

From Dev

Calculate STD manually using Groupby Pandas DataFrame

From Dev

Pandas: DataFrame filtering using groupby and a function

From Dev

using groupby on pandas dataframe to group by financial year

From Dev

Calculate STD manually using Groupby Pandas DataFrame

From Dev

pandas groupby of DataFrame using Series of substrings

From Dev

Error when using pandas dataframe map function in ipython notebook

From Dev

Error when using variables to filter dataframe with multiple conditions (Pandas)

From Dev

Any method to avoid creating individual files when using groupby and sortvalues in pandas

From Dev

Error when unpickling pandas dataframe

From Dev

Find the majority in dataframe using pandas.DataFrame.mode and groupby

From Dev

pandas dataframe groupby summation

From Dev

Reorder pandas groupby dataframe

From Dev

pandas, dataframe, groupby, std

From Dev

groupby - python pandas dataframe

From Dev

Pandas Groupby back to DataFrame

From Dev

groupby week - pandas dataframe

From Dev

Pandas Dataframe groupby Display

From Dev

pandas dataframe groupby summation

From Dev

Reorder pandas groupby dataframe

From Dev

Error when using the mousePress() method

From Dev

UnicodeEncodeError when using pandas method to_sql on a dataframe with unicode column names

From Dev

Using pandas.Dataframe.groupby without alphabetical ordering

From Dev

Transforming outliers in Pandas DataFrame using .apply, .applymap, .groupby

From Dev

Using Pandas GroupBy and size()/count() to generate an aggregated DataFrame

From Dev

Using print to access individual cells of pandas dataframe after groupby

From Dev

Using first row in Pandas groupby dataframe to calculate cumulative difference

Related Related

  1. 1

    Pandas transform dataframe using groupby when count of a string in a column is maximum

  2. 2

    how to concat sets when using groupby in pandas dataframe?

  3. 3

    using groupby on pandas dataframe to group by financial year

  4. 4

    Calculate STD manually using Groupby Pandas DataFrame

  5. 5

    Pandas: DataFrame filtering using groupby and a function

  6. 6

    using groupby on pandas dataframe to group by financial year

  7. 7

    Calculate STD manually using Groupby Pandas DataFrame

  8. 8

    pandas groupby of DataFrame using Series of substrings

  9. 9

    Error when using pandas dataframe map function in ipython notebook

  10. 10

    Error when using variables to filter dataframe with multiple conditions (Pandas)

  11. 11

    Any method to avoid creating individual files when using groupby and sortvalues in pandas

  12. 12

    Error when unpickling pandas dataframe

  13. 13

    Find the majority in dataframe using pandas.DataFrame.mode and groupby

  14. 14

    pandas dataframe groupby summation

  15. 15

    Reorder pandas groupby dataframe

  16. 16

    pandas, dataframe, groupby, std

  17. 17

    groupby - python pandas dataframe

  18. 18

    Pandas Groupby back to DataFrame

  19. 19

    groupby week - pandas dataframe

  20. 20

    Pandas Dataframe groupby Display

  21. 21

    pandas dataframe groupby summation

  22. 22

    Reorder pandas groupby dataframe

  23. 23

    Error when using the mousePress() method

  24. 24

    UnicodeEncodeError when using pandas method to_sql on a dataframe with unicode column names

  25. 25

    Using pandas.Dataframe.groupby without alphabetical ordering

  26. 26

    Transforming outliers in Pandas DataFrame using .apply, .applymap, .groupby

  27. 27

    Using Pandas GroupBy and size()/count() to generate an aggregated DataFrame

  28. 28

    Using print to access individual cells of pandas dataframe after groupby

  29. 29

    Using first row in Pandas groupby dataframe to calculate cumulative difference

HotTag

Archive