What do you mean? Each layer is selecting features and abstracting previous layers. A cat neuron abstracts all possible pixels that form cats.
He's just saying that we need better ways of composing this `deep` abstractions.
Tensor2Tensor[0], from the Google Brain team, has some strong recent results in that direction.
Related paper: "One Model To Learn Them All"[1].
[0]: https://github.com/tensorflow/tensor2tensor
[1]: https://arxiv.org/pdf/1706.05137.pdf
What do you mean? Each layer is selecting features and abstracting previous layers. A cat neuron abstracts all possible pixels that form cats.
He's just saying that we need better ways of composing this `deep` abstractions.