Category: Misc

I’m working with tf.keras on a Machine Learning project, and I’d like to implement an Attention pooling layer.

The equation which describes it is in Table I of this paper (it’s indicated at the last row of the table, at the column “Pooling function”).

The paper also says:

in the attention pooling function, the weights for each frame w_i are learned with a dedicated layer in the network.

I tried to implement the Attention pooling layer, in tf.keras, by (after reading this Keras documentation page) subclassing the Keras Layer class, such as:

from tensorflow.keras import backend as K from tensorflow.python.keras.engine.base_layer import Layer from tensorflow.python.keras.engine.input_spec import InputSpec class AttentionPooling1D(Layer): def __init__(self, axis=0, **kwargs): super(AttentionPooling1D, self).__init__(**kwargs) self.axis = axis def build(self, input_shape): input_dim = input_shape[-1] self.w= self.add_weight(shape=(1, input_dim), name='w') def get_config(self): config = {'axis': self.axis} base_config = super(AttentionPooling1D, self).get_config() return dict(list(base_config.items()) + list(config.items())) def call(self, x, mask=None): product = x * self.w numerator = K.sum(product, axis=self.axis, keepdims=True) denominator = K.sum(x, axis=self.axis, keepdims=True) attention_output = numerator / denominator return attention_output 

I don’t if it is correct or not, so I post it here in order to have feedbacks, especially if there are any errors and/or I’m missing something.

I was doing an exercise by google dev’s ml tensorflow course. Im getting this error:

File “c:UsersshivaDocumentsAI_ML_TensorflowTensorflowEx2MNISTComputerVision.py“, line 24, in <module>

model.fit(x_train, y_train, epochs=5)

TypeError: Expected uint8, but got 1e-07 of type ‘float’.

———————————————————————————————————————————————

Here is the code:

# YOUR CODE SHOULD START HERE
# YOUR CODE SHOULD END HERE
import tensorflow as tf
mnist = tf.keras.datasets.mnist
(x_train, y_train),(x_test, y_test) = mnist.load_data()
# YOUR CODE SHOULD START HERE
model = tf.keras.models.Sequential([tf.keras.layers.Flatten(),
tf.keras.layers.Dense(128, activation=tf.nn.relu),

tf.keras.layers.Dense(10, activation=tf.nn.softmax)])
# YOUR CODE SHOULD END HERE
model = tf.keras.models.Sequential([
# YOUR CODE SHOULD START HERE

# YOUR CODE SHOULD END HERE
])
model.compile(optimizer = tf.keras.optimizers.Adam(),
loss = ‘sparse_categorical_crossentropy’,
metrics=[‘accuracy’])
model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)
# YOUR CODE SHOULD START HERE
# YOUR CODE SHOULD END HERE

———————————————————————————————————————————————

I dunno what shud I do? I checked my code but can’t find anything that might cause the error. I asked the same question on the r/learnmachinelearning but got no response, Pls help!

hello r/tensorflow. As a backend engineer, I am very unfamiliar with tensorflow and ML in general, so please forgive me if this question seems unreasonable to you.

Because of the need of my lab, I’ve been looking for a solution for tensorflow orchestration. We have one server with a powerful GPU, and several users who want to run their tensorflow jobs on that powerful GPU. Instead of making schedules offline and individually log in to the server, is there any open source project I can deploy to the server that serves as an orchestrator?

For example, it provides a simple WebUI to let the user upload their job and all necessary files. Then the user submits the job to add it to a queue, which will run when it’s the first in the line. It will also report the progress and the result of the job.

I think there should be some kind of open-sourced project out there that fits this need, but I haven’t found it yet. So please help.

I’m experimenting with some logic before creating a custom keras layer, but my Lambda layer isn’t allowing me to check the output shape with model.summary(). It says:

​

ValueError: Exception encountered when calling layer “Lambda_1” (type Lambda).

The following Variables were created within a Lambda layer (Lambda_1)

but are not tracked by said layer:

<tf.Variable ‘Lambda_1/map/while/RGAT_1/edge_type_0/kernel:0’ shape=(7, 10) dtype=float32>

<tf.Variable ‘Lambda_1/map/while/RGAT_1/edge_type_0/Edge_attention_parameters_0:0’ shape=(5, 4) dtype=float32>

The layer cannot safely ensure proper Variable reuse across multiple

calls, and consquently this behavior is disallowed for safety. Lambda

layers are not well suited to stateful computation; instead, writing a

subclassed Layer is the recommend way to define layers with

Variables.

​

Is there a way to temporally disable this behavior? 🤔

Here you can discuss anything that doesn’t require its own post

Hello, I’ve been searching/reading for a fair amount of hours, but I’m pretty much stuck with this problem.

This is my code:

from typing import NamedTuple class MessagePassingInput(NamedTuple): node_embeddings: tf.Tensor adjacency_lists: Tuple[tf.Tensor, ...] from keras import Model, layers import tensorflow as tf inputLayer_X = layers.Input(shape=tf.TensorShape(dims=(None, 7)),name="Input_X") inputLayer_A1 = layers.Input(shape=tf.TensorShape(dims=(None, 2)),name="Input_A1", dtype=tf.int32) inputLayer_A2 = layers.Input(shape=tf.TensorShape(dims=(None, 2)),name="Input_A2", dtype=tf.int32) inputLayer_A3 = layers.Input(shape=tf.TensorShape(dims=(None, 2)),name="Input_A3", dtype=tf.int32) 

And I would like that every entry in those inputs ends up in a next layer more or less like this: newLayer = [MessagePassingInput(inputLayer_X[i], [inputLayer_A1[i], inputLayer_A2[i], inputLayer_A3[i]]) for i in range(len(inputLayer_X))]. However, I’m just not being able to find how (I have tried with tf.map_fn and layers.Lambda, but wasn’t able to feed all those input layers and use the function in order)

If you could help me, I would be very grateful 🙏