Keras / Tensorflow: Loss function with subtraction -

Max :

I not completly new to keras or tensorflow, but it's my first deep dive. I try to wirte my own loss function, which is a slight variation of mean_absolute_percentage_error from keras. I am able to write it with numpy:

def np_mean_relative_percentage_error(y_true, y_pred):
    err = np.abs((y_true - y_pred) / np.abs(y_true))
    diff = np.subtract(np.ones(err.shape, dtype=float), err)
    return 100. * np.mean(diff, axis=-1)

But I am not able to write it with keras / tensorflow, my current (not working) versions look like the following snippets. I am very grateful if someone completes the implementation or shows me how to subtract the tensors element by element with a constant.

Version 1:

def mean_relative_percentage_error(y_true, y_pred):
    err = K.abs((y_true - y_pred) / K.clip(K.abs(y_true), K.epsilon(), None))
    ones = K.ones_like(err)
    diff = K.update_sub(ones, err)
    return 100. * K.mean(diff, axis=-1)

Traceback (most recent call last):
  File "E:/Projekte/*ai/train.py", line 66, in <module>
    train(epochs=20, prefix='test_new_loss_fn')
  File "E:/Projekte/i*/ai/train.py", line 46, in train
    model = create_model((shape[0], shape[1], 3), backbone=backbone, loss_function=loss_fn, freeze_backbone=backbone_freeze, lr=learning_rate)
  File "E:\Projekte\*\ai\model\__init__.py", line 48, in create_model
    loss=loss_function, metrics=[mean_relative_percentage_error, metrics.mean_absolute_error])
  File "C:\Users\**\.conda\envs\tfGPU2\lib\site-packages\keras\engine\training.py", line 342, in compile
    sample_weight, mask)
  File "C:\Users\***\.conda\envs\tfGPU2\lib\site-packages\keras\engine\training_utils.py", line 404, in weighted
    score_array = fn(y_true, y_pred)
  File "E:\Projekte\ai_p\ai\utils\losses.py", line 8, in mean_relative_percentage_error
    diff = K.update_sub(ones, e)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\keras\backend\tensorflow_backend.py", line 999, in update_sub
    return tf.assign_sub(x, decrement)
  File "C:\Users\***f\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\state_ops.py", line 160, in assign_sub
    return ref.assign_sub(value)
AttributeError: 'Tensor' object has no attribute 'assign_sub'

Version 2:

def mean_relative_percentage_error(y_true, y_pred):
    err = K.abs((y_true - y_pred) / K.clip(K.abs(y_true), K.epsilon(), None))
    ones = K.variable(K.ones_like(err))
    diff = K.update_sub(ones, err)
    return 100. * K.mean(diff, axis=-1)

Traceback (most recent call last):
  File "E:/Projekte/*/ai/train.py", line 66, in <module>
    train(epochs=20, prefix='test_new_loss_fn')
  File "E:/Projekte/*/ai/train.py", line 46, in train
    model = create_model((shape[0], shape[1], 3), backbone=backbone, loss_function=loss_fn, freeze_backbone=backbone_freeze, lr=learning_rate)
  File "E:\Projekte\*\ai\model\__init__.py", line 48, in create_model
    loss=loss_function, metrics=[mean_relative_percentage_error, metrics.mean_absolute_error])
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\keras\engine\training.py", line 342, in compile
    sample_weight, mask)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\keras\engine\training_utils.py", line 404, in weighted
    score_array = fn(y_true, y_pred)
  File "E:\Projekte\*\ai\utils\losses.py", line 7, in mean_relative_percentage_error
    ones = K.variable(K.ones_like(err))
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\keras\backend\tensorflow_backend.py", line 402, in variable
    v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 183, in __call__
    return cls._variable_v1_call(*args, **kwargs)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 146, in _variable_v1_call
    aggregation=aggregation)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 125, in <lambda>
    previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variable_scope.py", line 2444, in default_variable_creator
    expected_shape=expected_shape, import_scope=import_scope)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 187, in __call__
    return super(VariableMetaclass, cls).__call__(*args, **kwargs)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 1329, in __init__
    constraint=constraint)
  File "C:\Users\*\.conda\envs\tfGPU2\lib\site-packages\tensorflow\python\ops\variables.py", line 1472, in _init_from_args
    self._initial_value)
ValueError: initial_value must have a shape specified: Tensor("loss/dense_3_loss/ones_like:0", shape=(?, ?), dtype=float32)

Matias Valdenegro :

There is no need for complicated tricks, your loss can be implemented with:

def mean_relative_percentage_error(y_true, y_pred):
    err = K.abs((y_true - y_pred) / K.clip(K.abs(y_true), K.epsilon(), None))
    diff = 1.0 - err
    return 100. * K.mean(diff, axis=-1)

This uses broadcasting in the 1.0 - err computation.