Juergen Schmidhuber's RepLearn 2013 Talk Overview
My lab has been working on Deep Learning Neural Networks (NN) since
1991 [1-9]. Back then computers were a million times slower than
today. With today's hardware, however, both our recurrent [3,4] and
our feedforward [5,7-9] deep NN now outperform traditional methods on
a routine basis [4-9]. In particular, in 2009, they became the first
Deep Learners to win official international pattern recognition
competitions [4]; by 2012 they had won eight of them [6] (far more
than methods by any other group). In 2011, our GPU-based NN achieved
the first superhuman visual pattern recognition results [5].
[9] 22 Sept 2013: our deep neural networks won the MICCAI 2013 Grand Challenge on Mitosis Detection (important for cancer prognosis etc.)
[8] 1 Sept 2013: our deep NN now also are the best artificial offline recognisers of Chinese characters from the ICDAR 2013 competition (3755 classes), approaching human performance
[7] 2012: Deep NN win brain image segmentation contest (relevant for the starting huge brain projects in EU and US)
[6] 2012: 8th international pattern recognition contest won since 2009
[5] 2011: First superhuman visual pattern recognition
[4] 2009: First official international pattern recognition contests won by Deep Learning
[3] 1997: First purely supervised Deep Learner
[2] My first Deep Learner of 1991 + Deep Learning timeline 1962-2013 (also summarises the origins of backpropagation, still the central algorithm of Deep Learning). An experiment in massive open online peer review.
[1] 1991: Fundamental Deep Learning Problem discovered and analysed
Recent G+ posts on Deep Learning
Juergen Schmidhuber