Vision Group

Impact Areas

Transportation

Members

Ted Adelson

Fredo Durand

William Freeman

Polina Golland

Berthold Horn

Aude Oliva

Antonio Torralba

Eric Grimson

Projects

Project

Imaging Privacy Threats from an Ambient Light Sensor

Group

Human-Computer Interaction

First-ever imaging privacy threats from non-imaging components (ambient light sensors) of smart devices. An idea inspired by George Orwell's dystopian novel 1984 -- "Big Brother is watching you" through a two-way TV-like telescreen.

Leads

Yang Liu

Fredo Durand

William Freeman

Gregory Wornell

Research Areas

Impact Areas

Cybersecurity

Internet of Things

Project

Imaging Privacy Threats from an Ambient Light Sensor

Group

Human-Computer Interaction

Leads

Yang Liu

Fredo Durand

William Freeman

Gregory Wornell

Research Areas

Impact Areas

Cybersecurity

Internet of Things

Project

Quantification of Pulmonary Edema in Chest Radiographs

Group

Human-Computer Interaction

We develop machine learning algorithms to automatically and quantitatively assess the severity of pulmonary edema from chest x-ray images.

Polina Golland

Leads

Polina Golland

Research Areas

Impact Areas

Polina Golland

Project

Clinical Neuroimaging

Group

Human-Computer Interaction

We develop novel algorithms for neuroimage segmentation, registration, generative modeling, and joint modeling of brain structure with clinical and genetic data.

Polina Golland

Leads

Polina Golland

Research Areas

Impact Areas

Polina Golland

Project

Driver-Friendly Bilateral Control for Suppressing Traffic Instabilities

Group

Human-Computer Interaction

Self-driving cars themselves can solve traffic problems even without global control.

Leads

Research Areas

Algorithms & Theory

Systems & Networking

Project

Places Database for Scene Recognition

Group

Human-Computer Interaction

Places is a 10 million image database for scene recognition. It contains images from more than 400 scene categories. Places-CNNs are trained to recognize scene context in human-level accuracy.

Agata Lapedriza

Leads

Research Areas

Agata Lapedriza

Project

Semantic Scene Understanding through ADE20K dataset

Group

Human-Computer Interaction

We build up a pixel-wise annotated image dataset for scene parsing. Scene parsing network are also proposed to detect and segment visual concepts from any input images.

Sanja Fidler

Adela Barriuso

Leads

Research Areas

Sanja Fidler

Adela Barriuso

Project

Understandable Deep Networks

Group

Human-Computer Interaction

Our aim is to improve the interpretability of deep neural networks to make it possible to understand their decisions, debug their errors, and make systematic improvements.

Antonio Torralba

Leads

Antonio Torralba

Research Areas

Antonio Torralba

Project

Understanding Food images

Group

Human-Computer Interaction

We focus on learning combined modalities (cooking recipes and food images).

Antonio Torralba

Ingmar Weber

Leads

Antonio Torralba

Ingmar Weber

Research Areas

Antonio Torralba

Ingmar Weber

Project

VirtualHome: Representing Activities as Programs

Group

Human-Computer Interaction

We aim to create a virtual environment where agents learn to perform human tasks by executing programs. Furthermore, we aim to develop models that can generate such programs from video or text, enabling agents to understand and imitate such activities.

Sanja Fidler

Kevin Ra

Leads

Research Areas

Sanja Fidler

Kevin Ra

Project

Blood Pressure Imager

Vision Group

Development of affordable wearable continuous blood pressure monitor based on radial arterial pulse imaged from the skin surface of human wrist

Mumin Jin

Aya G. Halawi

Jianhua Li

Leads

Abhijit Biswas

Ted Adelson

Mandayam A. Srinivasan

Mohan Thanikachalam

Research Areas

Computational Biology

Mumin Jin

Aya G. Halawi

Jianhua Li

Project

Slip Detection with Deep Neural Network by Using GelSight Touch Sensor

Group

Human-Computer Interaction

Train a neural network by using video data from GelSight touch sensor to detect slip when a robot lift an object

Jianhua Li

Leads

Jianhua Li

Research Areas

Jianhua Li

Project

Peripheral Vision and Driver Behavior

Group

Human-Computer Interaction

Our goal is to understand what drivers can perceive when they are not attending to the road ahead and the consequences of their perceptions for safety.

Leads

Research Areas

Impact Areas

Transportation

Project

Learning Vision in Synthetic Environments

Group

Human-Computer Interaction

We aim to create a virtual environment to improve models for action recognition

Leads

Research Areas

Human-Computer Interaction

Project

Wearable Ultrasonic Echolocation

Group

Human-Computer Interaction

The goal of this project is to develop and test a wearable ultrasonic echolocation aid for people who are blind and visually impaired. We combine concepts from engineering, acoustic physics, and neuroscience to make echolocation accessible as a research tool and mobility aid.

Leads

Research Areas

Impact Areas

Project

Driver-Friendly Bilateral Control for Suppressing Traffic Instabilities

Group

Human-Computer Interaction

We are studying how best to implement bilateral control and make it acceptable to drivers

Berthold Horn

Leads

Berthold Horn

Research Areas

Systems & Networking

Impact Areas

Transportation

Berthold Horn

Project

Predictive Vision

Group

Human-Computer Interaction

We've developed an algorithm to anticipate visual events that may happen in the future

Leads

Research Areas

Project

SoundNet

Group

Human-Computer Interaction

This project focuses on learning sound representations from unlabeled video

Leads

Research Areas

Project

Where Are They Looking?

Group

Human-Computer Interaction

Our goal is to build a system that predicts where people are looking in images. Given an image and the location of a head, our approach follows the gaze of the person and identifies the object being looked at.

Leads

Research Areas

Impact Areas

Education

Project

Single-Image 3D Interpreter Network

Group

Human-Computer Interaction

We aim to understand 3D object structure from a single image. We propose an end-to-end framework which sequentially estimates 2D keypoint heatmaps and 3D object structure, by training it on both real 2D-annotated images and synthetic 3D data and by integrating a 3D-to-2D projection layer.

Leads

Research Areas

Impact Areas

Manufacturing

Project

Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks

Group

Human-Computer Interaction

In our work we developed a model that is able to synthesize many probable future frames with just a single image as input.

Leads

Research Areas

Project

Deep Inverse Planning

Group

Human-Computer Interaction

Deep inverse planning for learning from high-dimensional demonstrations

Leads

Research Areas

Algorithms & Theory

Project

Monitoring Fetal Health

Group

Human-Computer Interaction

We develop algorithms for fetal MRI interpretation, to enable noninvasive fetal monitoring from MRI.

Polina Golland

Leads

Polina Golland

Research Areas

Impact Areas

Polina Golland

Project

Learning the Arrow of Time from Videos

Group

Human-Computer Interaction

We train machines to visually detect the arrow of time, if a video is playing forward or backward, by watching millions of online clips.

Leads

Research Areas

Project

Understanding Light via Deep Neural Networks

Group

Human-Computer Interaction

Our goal is to understand the illumination of an environment. By disentangling the illumination effect from other intrinsic properties (e.g. geometry, texture, color), we can better understand how human perceive the world. It also has several applications such as single image relighting, color editing, etc.

Antonio Torralba

Leads

Antonio Torralba

Research Areas