APPLIED MATH SEMINAR

Speaker: Yann LeCun, Courant Institute, NYU

Title: Learning Sparse Feature Hierarchies

When/where: Tuesday, January 26th, 4:15 PM, AKW 200

Abstract:
Intelligent tasks such object recognition, auditory scene analysis, or
language understanding require the construction of good internal
representations of the world. Internal representations (or "features")
must be invariant (or robust) to irrelevant variations of the input,
but must preserve the information relevant to the task. An important
goal of our research, and an important challenge for Machine Learning
over the next few years, is to devise methods that can automatically
learn good internal representations from labeled and unlabeled data.
Theoretical and empirical evidence suggest that the visual world is
best represented by a multi-stage hierarchy, in which features in
successive stages are increasingly global, invariant, and
abstract. The main question is how can one train such deep
architectures from unlabeled data and limited amounts of labeled data.

We describe a class of methods to train multi-stage system in which
each stage performs a series of convolutions followed by simple
non-linearities. The unsupervised learning phase is based on sparse
coding methods, but includes a feed-forward predictor that gives a
quick approximation of the sparse code. A number of such stages are
stacked and trained sequentially.

An application to category-level object recognition with invariance to
pose and illumination will be described.  By stacking multiple stages
of sparse features, and refining the whole system with supervised
training, state-the-art accuracy can be achieved on standard datasets
with very few labeled samples. A real-time demo will be shown. Another
application to vision-based navigation for off-road mobile robots will
be shown. After a phase of off-line unsupervised learning, the system
autonomously learns to discriminate obstacles from traversable areas
at long range using labels produced with stereo vision for nearby
areas.

This is joint work with Y-Lan Boureau, Karol Gregor, Raia Hadsell,
Koray Kavakcuoglu, and Marc'Aurelio Ranzato.