> Why would we not use those finely honed biases of ours and all the knowledge we've collected to kickstart a new form of intelligence?
Because priors are hard to embed in a model. For example, CNNs are great for translation invariance, but rotation and scaling don't come out of the box. Why don't they simply add the rotation invariance to the model? Because it's hard to express.
Also, human priors are limited. If AlphaGo was to be limited to human priors it would never have surpassed us.
The best approach so far is to make a network as free from priors as possible (like the transformers) and let it learn from the data, in essence let it rediscover the convolution or other efficient operations from mountains of data.
That's a limitation of neural networks, where inductive biases are very difficult to represent. Other approaches don't have any such problem, e.g. all the other approaches I listed above have well-defined, clean and tidy representations for inductive bias.
As to AlphaGo, this may come as a shock, but AlphaGo was not the first system to surpass humans in anything. It was the first system to surpass humans _in Go_ but e.g. the first computer system to beat a human grandmaster in chess was Deep Blue [1], in 1997, the first computer system to win a world championship against human players was the checkers (draughts) player Chinook [2], in 1990, the first computer system to outperform humans in medical diagnosis was MYCIN in the 1970's and so on. All those were systems that used strong inductive biases.
And of course, AlphaGo itself was limited by human priors- e.g. piece moves, checkerboard dimensions and structure were hard-coded into its architecture and a search for moves was performed by MCTS.
In any case, the lack of good enough priors is not a reason to not use any priors- it's a reason to look for better priors.
Edit: Has any end-to-end approach rediscovered an entire state-of-the-art architecture, like CNNs or LSTMs?
Yes, on self play, but not the supervised model. The supervised model was decent but nowhere near super human. It was used as a starting point for the self play model.
Because priors are hard to embed in a model. For example, CNNs are great for translation invariance, but rotation and scaling don't come out of the box. Why don't they simply add the rotation invariance to the model? Because it's hard to express.
Also, human priors are limited. If AlphaGo was to be limited to human priors it would never have surpassed us.
The best approach so far is to make a network as free from priors as possible (like the transformers) and let it learn from the data, in essence let it rediscover the convolution or other efficient operations from mountains of data.