如何在Keras LSTM预测中为多变量输入和相关系列问题塑造测试数据

我有这个抢断的数据帧：

          y     x1  x2
0   0.198382    15  1.84227
1   0.195289    16  1.88341
2   0.195089    16  1.92455
3   0.201794    16  1.96569
4   0.208498    16  2.00683

我想使用Keras RNN LSTM模型使用X1和X2来预测y。每行都是特定日期的样本，我最终希望使用251天的新测试集进行预测。我创建了训练和测试集：

y_col = 'y'
train_size = len(df3)-251
train, test = df3.iloc[0:train_size].copy(), df3.iloc[train_size:len(df)].copy()
X_train = train.drop(y_col,axis=1)
X_test = test.drop(y_col,axis=1)
y_train = train[y_col]
y_test = test[y_col]
print(len(train), len(test))
31877 251

整个数据如下所示：

并放大：

然后，我将数据归一化：

Xscaler = MinMaxScaler(feature_range=(0, 1))
Xscaler.fit(X_train)
scaled_X_train = Xscaler.transform(X_train)
scaled_X_test = Xscaler.transform(X_test)

然后根据这个很棒的博客修改“ y”的形状：

y_train = np.array(y_train)
y_train = np.insert(y_train, 0, 0)
y_train = np.delete(y_train, -1)
y_train.shape
(31877,)

然后建立并训练模型：

n_input = 20
n_features=2
generator = TimeseriesGenerator(scaled_X_train, y_train, length=n_input, batch_size=32)
model = Sequential()
model.add(LSTM(150, activation='relu', input_shape=(n_input, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit_generator(generator,epochs=10)
loss_per_epoch = model.history.history['loss']
plt.plot(range(len(loss_per_epoch)),loss_per_epoch)

到现在为止还挺好。现在，我试图对测试集进行预测，但出现一个错误，该错误与X_test的形状有关：

model.predict(scaled_X_test)

ValueError: Error when checking input: expected lstm_6_input to have 3 dimensions, but got array with shape (251, 2)

在我的示例中，我认为训练集和测试集应具有相同的尺寸：

scaled_X_train.shape
(31877, 2)
type(scaled_X_train)
numpy.ndarray
scaled_X_test.shape
(251, 2)
type(scaled_X_test)
numpy.ndarray

我知道我应该将测试集更改为3D，但是应该在哪里插入第三个D，它的值应该是多少？

更新：尝试实现@Marco Cerliani解决方案时，测试生成器出现错误：

n_input = 20 
n_features=2
generator = TimeseriesGenerator(scaled_X_train, y_train, length=n_input, batch_size=32)
test_generator = TimeseriesGenerator(scaled_X_test, y_test, length=n_input, batch_size=32)
generator[0][0].shape
(32, 20, 2)
test_generator[0][0].shape
KeyError                                  Traceback (most recent call last)
<ipython-input-55-6feb1cf23e96> in <module>
----> 1 test_generator[0][0].shape

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras_preprocessing\sequence.py in __getitem__(self, index)
    370         samples = np.array([self.data[row - self.length:row:self.sampling_rate]
    371                             for row in rows])
--> 372         targets = np.array([self.targets[row] for row in rows])
    373 
    374         if self.reverse:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras_preprocessing\sequence.py in <listcomp>(.0)
    370         samples = np.array([self.data[row - self.length:row:self.sampling_rate]
    371                             for row in rows])
--> 372         targets = np.array([self.targets[row] for row in rows])
    373 
    374         if self.reverse:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\pandas\core\series.py in __getitem__(self, key)
    869         key = com.apply_if_callable(key, self)
    870         try:
--> 871             result = self.index.get_value(self, key)
    872 
    873             if not is_scalar(result):

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\pandas\core\indexes\base.py in get_value(self, series, key)
   4403         k = self._convert_scalar_indexer(k, kind="getitem")
   4404         try:
-> 4405             return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))
   4406         except KeyError as e1:
   4407             if len(self) > 0 and (self.holds_integer() or self.is_boolean()):

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: 20

更新2：

model = Sequential()
model.add(LSTM(150, activation='relu', input_shape=(n_input, n_features)))
model.add(Dense(1))
model.compile(optimizer='adam', loss='mse')
model.fit_generator(generator,epochs=3)
model.predict(test_generator)


---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-31-81ef70218432> in <module>
      4 model.compile(optimizer='adam', loss='mse')
      5 model.fit_generator(generator,epochs=3)
----> 6 model.predict(test_generator)

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\engine\training.py in predict(self, x, batch_size, verbose, steps, callbacks, max_queue_size, workers, use_multiprocessing)
   1431                 max_queue_size=max_queue_size,
   1432                 workers=workers,
-> 1433                 use_multiprocessing=use_multiprocessing)
   1434 
   1435         if x is None and steps is None:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\legacy\interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\engine\training.py in predict_generator(self, generator, steps, callbacks, max_queue_size, workers, use_multiprocessing, verbose)
   1844             workers=workers,
   1845             use_multiprocessing=use_multiprocessing,
-> 1846             verbose=verbose)
   1847 
   1848 

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\engine\training_generator.py in predict_generator(model, generator, steps, callbacks, max_queue_size, workers, use_multiprocessing, verbose)
    489 
    490         while steps_done < steps:
--> 491             generator_output = next(output_generator)
    492             if isinstance(generator_output, tuple):
    493                 # Compatibility with the generators

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\utils\data_utils.py in get(self)
    623         except Exception:
    624             self.stop()
--> 625             six.reraise(*sys.exc_info())
    626 
    627 

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\six.py in reraise(tp, value, tb)
    701             if value.__traceback__ is not tb:
    702                 raise value.with_traceback(tb)
--> 703             raise value
    704         finally:
    705             value = None

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\utils\data_utils.py in get(self)
    608                 try:
    609                     future = self.queue.get(block=True)
--> 610                     inputs = future.get(timeout=30)
    611                 except mp.TimeoutError:
    612                     idx = future.idx

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\multiprocessing\pool.py in get(self, timeout)
    655             return self._value
    656         else:
--> 657             raise self._value
    658 
    659     def _set(self, i, obj):

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\multiprocessing\pool.py in worker(inqueue, outqueue, initializer, initargs, maxtasks, wrap_exception)
    119         job, i, func, args, kwds = task
    120         try:
--> 121             result = (True, func(*args, **kwds))
    122         except Exception as e:
    123             if wrap_exception and func is not _helper_reraises_exception:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras\utils\data_utils.py in get_index(uid, i)
    404         The value at index `i`.
    405     """
--> 406     return _SHARED_SEQUENCES[uid][i]
    407 
    408 

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras_preprocessing\sequence.py in __getitem__(self, index)
    370         samples = np.array([self.data[row - self.length:row:self.sampling_rate]
    371                             for row in rows])
--> 372         targets = np.array([self.targets[row] for row in rows])
    373 
    374         if self.reverse:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\keras_preprocessing\sequence.py in <listcomp>(.0)
    370         samples = np.array([self.data[row - self.length:row:self.sampling_rate]
    371                             for row in rows])
--> 372         targets = np.array([self.targets[row] for row in rows])
    373 
    374         if self.reverse:

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\pandas\core\series.py in __getitem__(self, key)
    869         key = com.apply_if_callable(key, self)
    870         try:
--> 871             result = self.index.get_value(self, key)
    872 
    873             if not is_scalar(result):

~\AppData\Local\Continuum\anaconda3\envs\keras\lib\site-packages\pandas\core\indexes\base.py in get_value(self, series, key)
   4403         k = self._convert_scalar_indexer(k, kind="getitem")
   4404         try:
-> 4405             return self._engine.get_value(s, k, tz=getattr(series.dtype, "tz", None))
   4406         except KeyError as e1:
   4407             if len(self) > 0 and (self.holds_integer() or self.is_boolean()):

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_value()

pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

pandas\_libs\hashtable_class_helper.pxi in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: 20

更新3：感谢Marco Cerliani的大力帮助，我发现了一个错误，那就是我的y_test是一个熊猫系列，因此我将其转换为一个numpy数组，np.y_test = array(y_test)并且可以正常工作