What dimension is the LSTM model considers the data sequence?

E.G. Cortes

I know that an LSTM layer expects a 3 dimension input (samples, timesteps, features). But which of it dimension the data is considered as a sequence. Reading some sites I understood that is the timestep, so I tried to create a simple problem to test. In this problem, the LSTM model needs to sum the values in timesteps dimension. Then, assuming that the model will consider the previous values of the timestep, it should return as an output the sum of the values.

I tried to fit with 4 samples and the result was not good. Does my reasoning make sense?

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LSTM

X = np.array([
        [5.,0.,-4.,3.,2.],
        [2.,-12.,1.,0.,0.],
        [0.,0.,13.,0.,-13.],
        [87.,-40.,2.,1.,0.]
    ])
X = X.reshape(4, 5, 1)
y = np.array([[6.],[-9.],[0.],[50.]])

model = Sequential()
model.add(LSTM(5, input_shape=(5, 1)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(X, y, epochs=1000, batch_size=4, verbose=0)

print(model.predict(np.array([[[0.],[0.],[0.],[0.],[0.]]])))
print(model.predict(np.array([[[10.],[-10.],[10.],[-10.],[0.]]])))
print(model.predict(np.array([[[10.],[20.],[30.],[40.],[50.]]])))

output:

[[-2.2417212]]
[[7.384143]]
[[0.17088854]]
mnis

First of all, yes you're right that timestep is the dimension take as data sequence.

Next, I think there is some confusion about what you mean by this line

"assuming that the model will consider the previous values of the timestep"

In any case, LSTM doesn't take previous values of time step, but rather, it takes the output activation function of the last time step.

Also, the reason that your output is wrong is because you're using a very small dataset to train the model. Recall that, no matter what algorithm you use in machine learning, it'll need many data points. In your case, 4 data points are not enough to train the model. I used slightly more number of parameters and here's the sample results.enter image description here

However, remember that there is a small problem here. I initialised the training data between 0 and 50. So if you make predictions on any number outside of this range, this won't be accurate anymore. Farther the number from this range, lesser the accuracy. This is because, it has become more of a function mapping problem than addition. By function mapping, I mean that your model will learn to map all values that are in training set(provided it's trained on enough number of epochs) to outputs. You can learn more about it here.

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

What is the arithmetic mean of an empty sequence?

分類Dev

What is the arithmetic mean of an empty sequence?

分類Dev

What is the arithmetic mean of an empty sequence?

分類Dev

Shuffling training data with LSTM RNN

分類Dev

Shuffling training data with LSTM RNN

分類Dev

What is sequence file in hadoop?

分類Dev

What is exact sequence of operations made here?

分類Dev

xarray: reshape data, split dimension

分類Dev

Train a model using lstm and keras

分類Dev

How do I fit the model of two concatenate LSTM in keras?

分類Dev

manipulating ndarray dimension with function and adding new data to it

分類Dev

Avoiding duplicating graph in tensorflow (LSTM model)

分類Dev

What is the input to LSTM exactly?

分類Dev

Train Keras LSTM model with a variable number of features

分類Dev

What is the "0G" terminal sequence

分類Dev

Pytorch LSTM: Target Dimension in Calculating Cross Entropy Loss

分類Dev

LSTM autoencoder with Keras data shape issue

分類Dev

How to create input samples from pandas dataframe for a LSTM model?

分類Dev

Keras LSTM Model for text-generation purpose

分類Dev

How to build an LSTM time-series forecasting model in python?

分類Dev

How to write a function to translate data to higher dimension

分類Dev

how to train a model for item sequence segmentation

分類Dev

Encoder-Decoder LSTM model gives 'nan' loss and predictions

分類Dev

LSTM, Keras : How many layers should the inference model have?

分類Dev

TensorFlow different input and output shapes for stateful LSTM model

分類Dev

On what parameters boot sequence varies?

分類Dev

What is the purpose of the Model in MVC?

分類Dev

Django: Store multiple & dynamically ordered model sequence

分類Dev

Mapping Data Model to Domain Model

Related 関連記事

  1. 1

    What is the arithmetic mean of an empty sequence?

  2. 2

    What is the arithmetic mean of an empty sequence?

  3. 3

    What is the arithmetic mean of an empty sequence?

  4. 4

    Shuffling training data with LSTM RNN

  5. 5

    Shuffling training data with LSTM RNN

  6. 6

    What is sequence file in hadoop?

  7. 7

    What is exact sequence of operations made here?

  8. 8

    xarray: reshape data, split dimension

  9. 9

    Train a model using lstm and keras

  10. 10

    How do I fit the model of two concatenate LSTM in keras?

  11. 11

    manipulating ndarray dimension with function and adding new data to it

  12. 12

    Avoiding duplicating graph in tensorflow (LSTM model)

  13. 13

    What is the input to LSTM exactly?

  14. 14

    Train Keras LSTM model with a variable number of features

  15. 15

    What is the "0G" terminal sequence

  16. 16

    Pytorch LSTM: Target Dimension in Calculating Cross Entropy Loss

  17. 17

    LSTM autoencoder with Keras data shape issue

  18. 18

    How to create input samples from pandas dataframe for a LSTM model?

  19. 19

    Keras LSTM Model for text-generation purpose

  20. 20

    How to build an LSTM time-series forecasting model in python?

  21. 21

    How to write a function to translate data to higher dimension

  22. 22

    how to train a model for item sequence segmentation

  23. 23

    Encoder-Decoder LSTM model gives 'nan' loss and predictions

  24. 24

    LSTM, Keras : How many layers should the inference model have?

  25. 25

    TensorFlow different input and output shapes for stateful LSTM model

  26. 26

    On what parameters boot sequence varies?

  27. 27

    What is the purpose of the Model in MVC?

  28. 28

    Django: Store multiple & dynamically ordered model sequence

  29. 29

    Mapping Data Model to Domain Model

ホットタグ

アーカイブ