University of California, Berkeley

Electrical Engineering and Computer Sciences Department

Announcements | Readings | Lecture Notes | Resources | Homework | Course Handouts
 

EE290T, Fall 2018
Advanced Topics in Signal Processing:
3D image processing and Computer Vision

Monday: 1:00 pm to 4:00 pm
531 Cory Hall

Prerequisite:  Signals and Systems at the level of EE120, basic linear algebra.

Course Announcement

Academic Dishonesty Policy

Piazza
Grading:
10% class participation
50% paper presentation and/or homework
40% Class project; proposals due October 15th.

Lecturer:
Professor Avideh Zakhor
507 Cory Hall
Phone: (510) 643-6777
avz@berkeley

Office Hours:
Monday, 4:00 pm - 5:00 pm 507 Cory Hall

Course Reader
Matthew Waliman
mwaliman@berkeley


Texts:

(a) ″Computer Vision: Algorithms and Applications″ by Richard Szeliski; Springer    PDF

(b) ″Computer Vision: A Modern Approach″ by Ponce and Forsyth    PDF

(c) ″ Multiple View Geometry in Computer Vision″ Hartley and Zisserman   PDF Tutorial Presentation

(d) ″Deep Learning″ Ian Goodfellow, Yoshua Bengio and Aaron Courville   Book

(e) ″Probabilistic Robotics″ by Sebastian Thrun, Wolfram Burgard, and Dieter Fox   PDF


Announcements:

  • 9/30/18
    Sign up for Piazza.
  • Welcome to EE290T!


Lectures

  1. Aug. 27, 2018
    Introduction to 3D Reconstruction and Recognition 
    3D Acquisition
  2. September 10, 2018
    SLAM Slides 
    Reading 
  3. September 17, 2018
    Reading: Chapter 2 of "Probablistic Robotics 
    SLAM Overview 
    Baye's Filter and SLAM 
    Intro to Deep Learning and Convolutional Neural Networks 
  4. September 24, 2018
    Reading 
    Camera Models 
    Camera Calibration 
  5. October 1, 2018
    Single View Geometry 
    Projective Geometry 
    Reading: Camera Models, Affine Projection 
    Reading: Single View Geometry 
  6. October 8, 2018
    Vanishing Points and Lines 
  7. October 15, 2018
    Reading: Chapter 8 Hartley and Zimmerman 
    Vanishing Points and Lines 
    How to Compute Camera Orientation 
    Importance of Camera Center 
  8. October 22, 2018
    Learning Methods for Single View Geometry 
    Intuition Behind Epipolar Geometry 
    Epipolar Geometry 1 
    SVD 
    Epipolar Geometry 2 
  9. October 29, 2018
    RANSAC 
    Feature Matching 
    Epipolar Geometry 1 
    Stereo Systems 
  10. November 5, 2018
    Factorization approach to structure from motion 
    Sparse bundle adjustment for structure from motion 
    Perspective-n-point 
  11. November 26, 2018
    Detectors and Descriptors 
    The Procustes Problem 
    Perspective-n-point 

Back to top


Readings

    Relevant CVPR 2018 Papers
  1. ″SurfConv: Bridging 3D and 2D Convolution for RGBD Images″, by Hang Chu, Wei-Chiu Ma, Kaustav Kundu, Raquel Urtasun and Sanja Fidler, 2018

  2. ″Pointwise Convolutional Neural Networks,″ by Binh-Son, Hua Minh-Khoi Tran, and Sai-Kit Yeung, 2018

  3. ″Recurrent Slice Networks for 3D Segmentation of Point Clouds,″ by Qiangui Huang, Weiyue Wang, and Ulrich Neumann, 2018

  4. ″Surface Networks,″ by Ilya Kostrikov1, Zhongshi Jiang1, Daniele Panozzo, Denis Zorin, and Joan Bruna, 2018

  5. ″CLEAR: Cumulative LEARning for One-Shot One-Class Image Recognition,″ by Jedrzej Kozerawski and Matthew Turk, 2018

  6. ″Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs,″ by Loic Landrieu and Martin Simonovsky, 2018

  7. ″Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net,″ by Wenjie Luo, Bin Yang and Raquel Urtasun, 2018

  8. ″Analytical Modeling of Vanishing Points and Curves in Catadioptric Cameras,″ by Pedro Miraldo, Francisco Eiras, and Srikumar Ramalingam, 2018

  9. ″3D Object Detection with Latent Support Surfaces,″ by Zhile Ren and Erik B. Sudderth, 2018

  10. ″A Network Architecture for Point Cloud Classification via Automatic Depth Images Generation,″ by RiccardoRoveri,LukasRahmann,A.Cengiz Oztireli, and Markus Gross, 2018

  11. ″Pixels, voxels, and views: A study of shape representations for single view 3D object shape prediction,″ by Daeyun Shin, Charless C. Fowlkes, Derek Hoiem, 2018

  12. ″Im2Pano3D: Extrapolating 360 ◦ Structure and Semantics Beyond the Field of View ,″ by Shuran Song, Andy Zeng, Angel X. Chang, Manolis Savva, Silvio Savarese, and Thomas Funkhouser, 2018

  13. ″SPLATNet: Sparse Lattice Networks for Point Cloud Processing,″ by Hang Su, Varun Jampani, Deqing Sun, and Subhransu Maji, 2018

  14. ″Tangent Convolutions for Dense Prediction in 3D,″ by Maxim Tatarchenko, Jaesik Park, Vladlen Koltun, and Qian-Yi Zhou, 2018

  15. ″Deep Parametric Continuous Convolutional Neural Networks ,″ by Shenlong Wang, Simon Suo, Wei-Chiu Ma, Andrei Pokrovsky, and Raquel Urtasun, 2018

  16. ″SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation,″ by Weiyue Wang, Ronald Yu, Qiangui Huang, and Ulrich Neumann, 2018

  17. ″Attentional ShapeContextNet for Point Cloud Recognition,″ by Saining Xie, Sainan Liu, Zeyu Chen, and Zhuowen Tu, 2018

  18. ″PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation,″ by Danfei Xu, Dragomir Anguelov, , 2018

  19. ″Automatic 3D Indoor Scene Modeling from Single Panorama,″ by Yang Yang, Shi Jin, Ruiyang Liu, Sing Bing Kang, and Jingyi Yu, 2018

  20. ″PIXOR: Real-time 3D Object Detection from Point Clouds,″ by Bin Yang, Wenjie Luo, and Raquel Urtasun, 2018

  21. ″Deep Depth Completion of a Single RGB-D Image,″ by Yinda Zhang and Thomas Funkhouser, 2018

  22. ″LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image,″by Chuhang Zou, Alex Colburn, Qi Shan, and Derek Hoiem, 2018

  23. ″Frustum PointNets for 3D Object Detection from RGB-D Data,″by Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, Leonidas J. Guibas, 2018

  24. Other Papers
  25. ″KinectFusion: Real-Time Dense Surface Mapping and Tracking∗,″ by Richard A. Newcombe, Andrew J. Davison, Shahram Izadi, Pushmeet Kohli, Otmar Hilliges, Jamie Shotton, David Molyneaux, Steve Hodges, David Kim, and Andrew Fitzgibbon, 2011

  26. ″Combining Inertial Navigation and ICP for Real-time 3D Surface Reconstruction,″ by Matthias Niessner, Angela Dai and M. , 2014

  27. ″VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition,″ by Daniel Maturana and Sebastian Scherer, 2015

  28. ″ElasticFusion: Dense SLAM Without A Pose Graph,″ by Thomas Whelan, Stefan Leutenegger, Renato F. Salas-Moreno, Ben Glocker and Andrew J. Davison, 2015

  29. ″3D Semantic Parsing of Large-Scale Indoor Spaces,″ by Iro Armeni1 Ozan Sener, Amir R. Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese, 2016

  30. ″FPNN: Field Probing Neural Networks for 3D Data,″ by Yangyan Li, Sören Pirk, Hao Su, Charles R. Qi, Leonidas J. Guibas, 2016

  31. ″Fine-To-Coarse Global Registration of RGB-D Scans,″ by Maciej Halber and Thomas Funkhouser, 2016

  32. ″SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks,″ by John McCormac, Ankur Handa, Andrew Davison, and Stefan Leutenegger,2016

  33. ″Semantic Scene Completion from a Single Depth Image,″ by Shuran Song, Fisher Yu, Andy Zeng, Angel X. Chang, Manolis Savva, and Thomas Funkhouser, 2016

  34. ″DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks,″ by Yu Xiang and Dieter Fox, 2017

  35. ″Matterport3D: Learning from RGB-D Data in Indoor Environments,″ by Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Nießner, Manolis Savva,Shuran Song, Andy Zeng, and Yinda Zhang, 2017

  36. ″3DLite: Towards Commodity 3D Scanning for Content Creation,″ by Jingwei Huang, Angela Dai, Leonidas Guibas, and Matthias Niessner, 2017

  37. ″BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration,″ by Angela Dai Matthias Niessner, Michael Zollhoer, Shahram Izadi, Christian Theobalt, 2017

  38. ″Joint 2D-3D-Semantic Data for Indoor Scene Understanding,″ by Iro Armeni, Alexander Sax, Amir R. Zamir, and Silvio Savarese, 2017

  39. ″PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation,″ by Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J. Guibas, 2017

  40. ″PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space ,″ by Charles R. Qi Li Yi Hao Su Leonidas J. Guibas, 2017

  41. ″ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes,″ by Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner, 2017

  42. ″Predicting Complete 3D Models of Indoor Scenes,″ by Ruiqi Guo, Chuhang Zou, and Derek Hoiem, 2017

  43. ″Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling ,″ by Jiajun Wu, Chengkai Zhang, and Tianfan Xue, 2017

  44. ″Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks,″ by Yinda Zhang†, Shuran Song†, Ersin Yumer, Manolis Savva, Joon-Young Lee, Hailin Jin and Thomas Funkhouser, 2017

  45. ″3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions,″ by Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao and Thomas Funkhouser,2017

  46. ″2D-Driven 3D Object Detection in RGB-D Images,″ by Jean Lahoud, Bernard Ghanem,2017

  47. ″3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation,″ by Angela Dai and Matthias Niessner, 2018

  48. ″ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans,″ by Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jugen Sturm, Matthias Nießner, 2018

  49. ″Deep Depth Completion of a Single RGB-D Image,″ by Yinda Zhang and Thomas Funkhouser, 2018

  50. ″Fusion++: Volumetric Object-Level SLAM,″ by John McCormac, Ronald Clark, Michael Bloesch, Andrew J. Davison, and Stefan Leutenegger, 2018

  51. ″Dynamic Graph CNN for Learning on Point Clouds,″ by Yue Wang, Yongbin Suan, Ziewi Liu, Sanjay Sarma, 2018

Resources

Open Access page for CVPR papers from 2013 to 2018
Time-of-Flight Cameras and Microsoft Kinect: A user perspective on technology and applications
Time of Flight Cameras: Principles, Methods, and Applications

Homework

Back to top


Handouts

Back to top

 Last updated 11/28/2011 by LB