how to calculate mutual information of entire dataset in python
I have 50 variables in my dataframe. 46 are dependant variables and 4 are independandt variables (precipitation, temperature, dew, snow). I want to calculate the mutual information of my dependant variables agaisnt my independant.
So in the end i want a dataframe like this 
Right now i am calculating it using the following but it's taking so long because i have to change my y each time
X = df[['Temperature', 'Precipitation','Dew','Snow']] # Features
y = df[['N0037']] #target
from sklearn.feature_selection import mutual_info_regression
mi = mutual_info_regression(X, y)
mi /= np.max(mi)
mi = pd.Series(mi)
mi.index = X.columns
mi.sort_values(ascending=False)
mi
Using list comprehension:
indep_vars = ['Temperature', 'Precipitation', 'Dew', 'Snow'] # set independent vars
dep_vars = df.columns.difference(indep_vars).tolist() # set dependent vars
from sklearn.feature_selection import mutual_info_regression as mi_reg
df_mi = pd.DataFrame([mi_reg(df[indep_vars], df[dep_var]) for dep_var in dep_vars], index = dep_vars, columns = indep_vars).apply(lambda x: x / x.max(), axis = 1)
Collected from the Internet
Please contact [email protected] to delete if infringement.
edited at
Related
Related Related
- 1
Python's implementation of Mutual Information
- 2
Display Entire Dataset,Python
- 3
Calculating mutual information between two classifiers for a fixed dataset
- 4
Can Pandas DataFrame efficiently calculate PMI (Pointwise Mutual Information)?
- 5
Calculating mutual information in python returns nan
- 6
How to use Normalized Mutual Information to evaluate overlapping community from igraph in python
- 7
How to merge two data frames based on mutual information with python-pandas?
- 8
How to merge two data frames based on mutual information with python-pandas?
- 9
how to calculate the median on grouped dataset?
- 10
how to calculate medians for unsorted dataset
- 11
Shannon entropy to mutual information
- 12
How does python implement mutual recursion?
- 13
How does we calculate PSD of a dataset in R
- 14
Python: How to label this dataset
- 15
Computing Pointwise Mutual Information in Spark
- 16
How to calculate the information gain of a continuous feature
- 17
How to download the entire scored dataset from Azure machine studio?
- 18
How to invert the entire screen in python
- 19
How to calculate mutual friend using cypher query in Neo4j as per facebook representation?
- 20
How to calculate same substring from given entire NSString
- 21
How to calculate mean in python?
- 22
how to calculate percentage in python
- 23
How to calculate mean in python?
- 24
How calculate Diff in python
- 25
How to summarise useful information from existing dataset and combine in a new one?
- 26
How to calculate Quarter Over Quarter %change when the dataset is monthly
- 27
How to calculate daily averages from a dataset with hourly values in JavaScript
- 28
Mutual information and joint entropy of two images - MATLAB
- 29
Adjusted Mutual Information (scikit-learn)
Comments