我正在尝试将 .csv 表中的数据缩放到 0 到 1 之间的范围。我已经多次收到输入数据包含 NaN、无穷大或值太大的错误。
“ValueError:输入包含 NaN、无穷大或对于 dtype('float64') 来说太大的值。”
到现在为止,我总是能够弄清楚错误来自哪里,例如一个空单元格,有时表格中的空白或与 UTF-8 不兼容的字符。直到现在,我总能让它发挥作用。
这次我再次收到错误,但我无法找到错误。有没有办法找出哪个数据点是“NaN、无穷大或值太大”?由于我有很多数据点,我无法手动通过它。如果您有建议,我会非常高兴 - 即使这只是Excel中查找导致错误的值的一个技巧。您可以在下面找到我的代码和错误。不幸的是,我无法提供数据集,因为它包含机密信息。
代码:
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
# Load training data set from CSV file
training_data_df = pd.read_csv("mtth_train.csv")
# Load testing data set from CSV file
test_data_df = pd.read_csv("mtth_test.csv")
# Data needs to be scaled to a small range like 0 to 1
scaler = MinMaxScaler(feature_range= (0, 1))
# Scale both the training inputs and outputs
scaled_training = scaler.fit_transform(training_data_df)
scaled_testing = scaler.transform(test_data_df)
# Print out the adjustment that the scaler applied to the total_earnings column of data
print("Note: Parameters were scaled by multiplying by {:.10f} and adding {:.6f}".format(scaler.scale_[8], scaler.min_[8]))
# Create new pandas DataFrame objects from the scaled data
scaled_training_df = pd.DataFrame(scaled_training, columns=training_data_df.columns.values)
scaled_testing_df = pd.DataFrame(scaled_testing, columns=test_data_df.columns.values)
# Save scaled data dataframes to new CSV files
scaled_training_df.to_csv("mtth_train_scaled", index=False)
scaled_testing_df.to_csv("mtth_test_scaled.csv", index=False)
错误:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-2-4e3503c96698> in <module>()
14 # Scale both the training inputs and outputs
15 scaled_training = scaler.fit_transform(training_data_df)
---> 16 scaled_testing = scaler.transform(test_data_df)
17
18 # Print out the adjustment that the scaler applied to the total_earnings column of data
~/anaconda3_501/lib/python3.6/site-packages/sklearn/preprocessing/data.py in transform(self, X)
365 check_is_fitted(self, 'scale_')
366
--> 367 X = check_array(X, copy=self.copy, dtype=FLOAT_DTYPES)
368
369 X *= self.scale_
~/anaconda3_501/lib/python3.6/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
451 % (array.ndim, estimator_name))
452 if force_all_finite:
--> 453 _assert_all_finite(array)
454
455 shape_repr = _shape_repr(array.shape)
~/anaconda3_501/lib/python3.6/site-packages/sklearn/utils/validation.py in _assert_all_finite(X)
42 and not np.isfinite(X).all()):
43 raise ValueError("Input contains NaN, infinity"
---> 44 " or a value too large for %r." % X.dtype)
45
46
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
用
df.isnull().sum()
知道每列中缺失值的总数
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句