我是tensorflow的初学者。我已经建立了简单的模型,但是还没有尝试过多层LSTM之类的东西,因此非常感谢任何反馈:)
我目前正在尝试从头开始重新编码由sherjilozair构建的字符级模型,仅仅是因为我想知道如何使用tensorflow(我以前已经建立了自己的很小的DL库,由cs231n分配)。现在,我目前正在努力构建一个简单的2层LSTM模型,并且不确定有什么问题。这是我到目前为止编写的代码:
class Model():
def __init__(self, batch_size, seq_length, lstm_size, num_layers, grad_clip, vocab_size):
self.lr = tf.Variable(0.0, trainable=False)
#Define input and output
self.input_data = tf.placeholder(tf.float32, [batch_size, seq_length])
self.output_data = tf.placeholder(tf.float32, [batch_size, seq_length]) #although int would be better for character level..
#Define the model
cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=lstm_size) #can choose if basic or otherwise later on...
self.cell = cell = rnn_cell.MultiRNNCell([cell] * num_layers)
self.initial_state = cell.zero_state(batch_size, tf.float32)
with tf.variable_scope("lstm"):
softmax_w = tf.get_variable("softmax_w", [lstm_size, vocab_size])
softmax_b = tf.get_variable("softmax_b", [vocab_size])
#_, enc_state = rnn.rnn(cell, encoder_inputs, dtype=dtype)
#outputs, states = rnn_decoder(decoder_inputs, enc_state, cell)
outputs, states = seq2seq.basic_rnn_seq2seq(
[self.input_data],
[self.output_data],
cell,
scope='lstm'
)
#see how attention helps improving this model state...
#was told that we should actually use samples softmax loss
self.loss = tf.nn.sampled_softmax_loss(
softmax_w,
softmax_b,
outputs,
self.output_data,
batch_size,
vocab_size
)
而且我目前在tf.nn.sampled_softmax_loss上遇到问题。我在调试方面已经走了很长一段路,并且不了解Tensorflow的输入约定。每次都需要输入张量列表吗?
我收到以下错误:
Traceback (most recent call last):
File "Model.py", line 76, in <module>
vocab_size=82
File "Model.py", line 52, in __init__
vocab_size
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/nn.py", line 1104, in sampled_softmax_loss
name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/nn.py", line 913, in _compute_sampled_logits
array_ops.expand_dims(inputs, 1),
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 506, in expand_dims
return _op_def_lib.apply_op("ExpandDims", input=input, dim=dim, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 411, in apply_op
as_ref=input_arg.is_ref)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 566, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/constant_op.py", line 179, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/ops/constant_op.py", line 162, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape))
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 332, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/tensor_util.py", line 269, in _AssertCompatible
raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected
我不确定输入或变量的生成在做什么错。问题-如前所述-似乎在sampled_softmax_loss函数中,但是我真的不确定。具有以下参数(仅作为占位符,仅用于测试模型是否“可运行”):
Model = Model(batch_size=32,
seq_length=128,
lstm_size=512,
num_layers=2,
grad_clip=5,
vocab_size=82
)
另外,如果我犯了其他错误等,请在评论中告诉我!这是我的第一个在tensorflow中使用seq2seq模型的模型,因此任何建议都将不胜感激!
outputs
当tf.nn.sampled_softmax_loss需要单个张量时,此特定错误与通过列表有关。
该seq2seq.basic_rnn_seq2seq函数返回的大小张量清单[batch_size x output_size]
作为第一输出。假设每个输出都是一维的,则要使用tf.concat(创建一个张量size [seq_len x batch_size x 1]
),tf.squeeze最后一个维度(结果[seq_len x batch_size]
)和tf.transpose来合并输出列表,使其output
具有size [batch_size x seq_len]
,与相同self.output_data
。
要调试该问题,请使用来打印张量大小print(output.get_shape())
。
本文收集自互联网,转载请注明来源。
如有侵权,请联系[email protected] 删除。
我来说两句