If you use the last of the simple linear activation function is still linear, multi-layered advantage is not working.