Clash Royale CLAN TAG#URR8PPP

How to apply sigmoid function for each outputs in Keras?

This is part of my codes.

model = Sequential()
model.add(Dense(3, input_shape=(4,), activation='softmax'))
model.compile(Adam(lr=0.1),
loss='categorical_crossentropy',
metrics=['accuracy'])


with this code, it will apply softmax to all the outputs at once. So the output indicates probability among all. However, I am working on non-exclusive classifire, which means I want the outputs to have independent probability.
Sorry my English is bad...
But what I want to do is to apply sigmoid function to each outputs so that they will have independent probabilities.

2 Answers
2

You can try using Functional API to create a model with n outputs where each output is activated with sigmoid.

sigmoid

You can do it like this

in = Input(shape=(4, ))

dense_1 = Dense(units=4, activation='relu')(in)

out_1 = Dense(units=1, activation='sigmoid')(dense_1)
out_2 = Dense(units=1, activation='sigmoid')(dense_1)
out_3 = Dense(units=1, activation='sigmoid')(dense_1)

model = Model(inputs=[in], outputs=[out_1, out_2, out_3])


Why so complicated so create 3 separate outputs like suggested by the other answer?

You can just use 'sigmoid' activation for the last layer:

'sigmoid'

from tensorflow.keras.layers import GRU
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Activation
import numpy as np

from tensorflow.keras.optimizers import Adam

model = Sequential()
model.add(Dense(3, input_shape=(4,), activation='sigmoid'))
model.compile(Adam(lr=0.1),
loss='categorical_crossentropy',
metrics=['accuracy'])

pred = model.predict(np.random.rand(5, 4))
print(pred)


Output:

[[0.58463055 0.53531045 0.51800555]
[0.56402034 0.51676977 0.506389 ]
[0.665879 0.58982867 0.5555959 ]
[0.66690147 0.57951677 0.5439698 ]
[0.56204814 0.54893976 0.5488999 ]]


As you can see the classes probabilities are independent from each other. The sigmoid is applied to every class separately.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.