Workshop 6 | Symposium on VLSI Technology and Circuits

* If you encounter menus do not work upon clicking, delete your browser's cache.

3D Image Sensor

Organizer : Dongjae Shin (Samsung Advanced Institute of Technology)

In recent years, 3D image sensors based on various architectures have been developed and presented, driven by ever increasing demands in various applications. From autonomous driving to augmented/virtual reality(AR/VR), 3D image sensors not only add new functionality to existing products, but also open up new applications and products. CMOS technology remains at the core of 3D image sensors, as with 2D image sensors, but with distinct technological differences in architecture and materials/processes. In the 3D sensor architecture, active illumination technology is as important as sensing technology. The combination of active illumination and associated sensing, presents various architectures that have not been adopted or required by 2D sensors and greatly expands the technology demands to realize such systems. Regarding materials and processes, laser/LED-based active illumination inevitably introduces exotic III/V challenges. Single-photon avalanche diodes (SPADs), acting as a receiver pixel, can present process challenges and, at both the transmit and receiver elements, 3D stacking presents its own unique set of challenges. Technical discussions are required to introduce, assemble and integrate exotic materials and processes into the framework of existing CMOS technology. Owing to this technological diversity, emerging approaches to the 3D image sensors are largely outside the scope of the 2D image sensors community, requiring comprehensive and dedicated technical discussions at industry and academic levels. This 3D image sensor workshop invites experts from related industries and academia to discuss the challenges and different proposed solutions that enable 3D sensing.|This workshop invites three groups of experts from the communities of users, developers and academia. The user group for autonomous vehicles and AR/VR discusses application-specific sensor utilization and deployment scenarios, technology requirements and application evolution vision. The developer group from Indirect Time-of-Flight(iTOF), Direct TOF(dTOF), and Frequency Modulated Continuous-Wave(FMCW) discusses the architecture, up-to-date status and future evolution of each technology. The academic group discusses overall technology trends and long-term vision for key technologies. This workshop features a line-up of speakers representing key technologies in the field of 3D image sensors, aiming to share the latest developments across the community and spark debate on pathfinding for the field which faces significant technology and device architecture diversity.

About Dongjae Shin

Dongjae Shin is a principal researcher at Samsung Advanced Institute of Technology(SAIT), and is currently working on silicon photonics to leverage the silicon infrastructure of Samsung for emerging applications including 3D image sensors and optical interconnects. Since 2002, he has been with Samsung for silicon photonics and optical communication R&D. Prior to Samsung, he was with Bell Labs, NJ for optical crossconnect R&D. He has two decades of industrial R&D experiences with 1 book, 75+ papers, and 125+ patents on LiDAR, DRAM I/O, WDM-PON, VLC, and near-field optics. Dongjae received the B.S., M.S., and Ph.D. degrees in physics from Korea Advanced Institute of Science and Technology(KAIST), Korea, in 1995, 1997, and 2001 respectively. He has been an IEEE senior member since 2012.

Presentations

1. Role and Function of LiDAR Sensor for Autonomous Driving: Beyond Autonomous Driving Level 3: Abstract:
In accordance with the evolution of electronic components in autonomous, automobiles could realize HDP (Highway Driving Pilot) functions beyond HDA (Highway Driving Assist), which are driver assistance functions. Therefore, it is necessary to secure lane maintenance performance that allows hand-off function and improve short-distance cut-in performance. Automotive lidar sensors play an important role in obtaining data necessary for recognizing and discriminating objects required for autonomous driving. In particular, since higher sensing performance is required to implement autonomous driving level 4 or higher, coverage and sensing distance according to the number and mounting location of lidar are all the more important. In order to realize the ultimate Level 4 or higher autonomous driving function in the coming future, lidar should increase the freedom of the mounting location through miniaturization. Also at the same time manufacturing process and material It should be possible to achieve cost reduction across all divisions such as assembly. Through this advance, we expect increase the number of lidar installations for the high quality cocoon sensing. To this end, technological innovation in the device unit, such as integration and high-efficiency technology, and silicon photonics technology at the semiconductor level including transmitting and receiving devices, is required.

About Kyoungchun Kweon

Kyoungchun Kweon was born in Seoul, Republic of Korea, in 1985. He received the B.S. degree in electronic engineering from the University of Kookmin, Seoul, Republic of Korea, in 2011, and the M.S. degree in nanobio materials and electronics from the Gwangju Institute of Science and Technology (GIST), Gwangju, South Korea, in 2013. He is currently a Senior Research Engineer of Hyundai Motor Company, Hwaseong, South Korea. His current research interests include sensor materials for autonomous driving.

2. 3D Sensing Technologies for Immersive Experiences in the Metaverse: Abstract:
Virtual Reality (VR) and Augmented Reality (AR) are fast-evolving fields which are expected to change the ways humans interact with each other and the ways they will access information in the coming years. Market forecasts estimate that the volumes of VR and later of AR systems will grow exponentially in the coming decade and beyond. The immersive experiences at the center of these applications require an accurate sensing of the environment, while consuming ultralow power and occupying a tiny footprint. This talk will begin with an overview of Virtual Reality and Augmented Reality systems, and the special challenges they present to the imaging architect. Next, we will provide an overview of four 3D sensing modalities – structured light, stereo imaging, indirect Time-of-Flight (iToF), and direct Time-of-Flight (dToF). We will describe the basic operating principles of each modality with an emphasis on how these principles affect the fit of these technologies for future VR and AR systems. We will conclude with a deeper discussion of the device- and package-level challenges.

About Hod Finkelstein

Hod Finkelstein heads Camera and Depth development at Meta’s Reality Labs, managing a global team of architects, technologists, software engineers, and algorithm developers, who are helping define the future of VR and AR. He previously held executive R&D positions at AEye and Sense Photonics (Ouster) where he led lidar development, including SPAD-based solid-state automotive lidars. Dr. Finkelstein’s prior roles included leading the development of handheld low-power hyperspectral cameras at TruTag/HinaLea, as well as the development of SPAD- and CIS-based DNA sequencers at Illumina. During his PhD, he invented the first generic-CMOS Single-Photon-Avalanche-Diodes (SPADs), which are widely used in cellphones and lidars, and first demonstrated arrays of these devices.

3. Integrated LiDAR Sensors for L4 Autonomous Vehicles: Abstract:
Currently long-range direct time-of-flight lidar for fully (L4-L5) autonomous vehicles are optomechanical systems containing custom discrete sensor components and using one or more axes of mechanical scanning. Solid state 3D lidar sensing has recently been introduced into mobile consumer electronics and some L2-L3 driver assistance systems. The small size, low cost, and mass production of these sensors was enabled by integration of sub-ns time-resolved photon counting arrays with integrated digital aggregation and readout ICs. With large markets, these sensors are expected to evolve rapidly, as we’ve seen in the past with consumer sensor products such as CMOS cameras. At what point could the integrated sensors be used for L4 autonomous vehicles, and what are the performance gaps that need to be addressed? Through our development of the Waymo Driver, Waymo has developed a deep understanding of the requirements for the Lidar component of our sensor system. We use a short range wide field of view lidar for maneuvering in dense city environments, and a long range lidar for high speed driving. The requirements of both these systems are currently beyond the capabilities of existing digital photon counting sensors. We will discuss these gaps and the path we see to transitioning to integrated photon counting sensors as they evolve, beginning with the short range perimeter lidar system.

About James Dunphy

James Dunphy has a PhD in Physics from UC Berkeley. He has worked on optoelectronics, MEMS, display and sensor technology at various startups in the Bay Area. He joined Google in 2013, and there led a group developing optics and displays for Augmented Reality. Since transferring to Google’s self-driving car project (now known as Waymo) 5 years ago, he has been engaged in developing the next generation of long range lidar sensors used on Waymo’s vehicles.
4. Indirect-ToF System for Non-Mobile Application: Abstract:
Recently, indirect time-of-flight is widely used for standalone sensing systems with high resolution for AR/VR and Robotics systems such as gesture detection, space modeling, and autonomous driving applications. These devices require weight reduction through miniaturization, heat reduction and operating time increase through low power consumption, and motion artifact minimization as the most important features. In this presentation, we introduce the development direction of i-ToF through the development of pixel technology, design technology, and system implementation technology. Pixel is the most important factor that determines the operating distance and resolution. Pixel pitch is shrinking from 7um to 3.5um and 38% QE @ 940nm was achieved to improve performance. From the circuit design, motion artifact was minimized by enhancing the readout FPS, and user convenience was expanded by embedding an interference compensation algorithm for multiple devices. The high power consumption due to the increase in computation was reduced by embedding the ISP inside the sensor, and as a result, power was reduced by about 70%, and depth output at 60 fps was made possible. 3D information will continue to be essential information for discovering new device types and applications. In order to realize this, the sensor will take charge of many technical areas, and it will be used as a more realistic application for users through system optimization.

About Seung-Chul Shin

Seung-Chul Shin received the B.S. and M.S. degrees in semiconductor science from Dongguk University, Seoul, South Korea, in 2005 and 2007, respectively, and the Ph.D. degree in electrical and electronic engineering from Yonsei University, Seoul, in 2020. He joined Samsung Electronics, Hwaseong, South Korea, in 2007, where he is currently working on advanced sensor circuit design and 3D image signal processing. His research interests are focused on low-noise circuit design, power management analog circuit design, Time-of-Flight sensing circuit and system design, and bio-electrical signal sensing system.
5. Time of Flight 3D-Sensing Architectures: Abstract:
This talk will introduce the basic concept of both direct and indirect time-of-flight before detailing the single-photon avalanche diode and fast-photodiode pixels that underpin these techniques, respectively. Pixel details will include a review of operation, key performance indicators and details of the underlying CMOS technologies that enable the state-of-the-art (TCAD, BSI, 3D-wafer stacking…). Moving to system level, details of range extraction techniques (histogramming, phase binning) will be presented along with examples of full system integration and full ranging performance, based on STMicroelectronics SPAD-based multi-zone DTOF products and our recently presented fast-photodiode-based, high-resolution iTOF solution.

About Bruce Rae

Bruce R. Rae is a Senior Member of Technical Staff at STMicroelectronics, Imaging Division, Edinburgh UK and he is manager of the Device, Process and Optical Simulation Group. In this role, he works closely with the silicon, process, module and system R&D teams to support the development of ST’s pixel and technology roadmap and specification, while supporting development of advanced simulation techniques. He also works with external collaborators on advanced R&D projects. He has extensive experience in the development of image and depth sensors and single photon counting systems. Bruce received the MEng and Ph.D. degrees in Electrical and Electronic Engineering from The University of Edinburgh, UK, in 2005 and 2009 respectively. He is a member of the IEEE VLSI Symposium and ISSCC Technical Program Committees and European Regional Chair for ISSCC 2023.
6. Silicon-Based FMCW Imaging for Human-Like Vision: Abstract:
FMCW technology enables a new generation of imaging solutions that can directly capture images of motion to create 4 dimensional pictures and movies. Integration of this technology in a compact silicon chip and its co-packaging with a conventional CMOS image sensor can enable a generation of cameras that work like the human eye. This can fuel the next industrial age by bringing AI and Robotics technologies together to enable the deployment of machines in unpredictable and unstructured environments to allow their participation in our society and economic growth.

About Mehdi Asghari

Mehdi is a serial entrepreneur with a track record of success. He is currently CEO and co-founder of SiLC, his third Silicon Photonics start up focusing on advanced imaging solutions. His previous start ups, Kotura and Bookham, both enabled successful exits. Mehdi is one of the early pioneers of silicon photonics industry with over 25 years of commercialization experience in this space. Prior to that he spent 10 years in the III/V industry. Mehdi holds an Electrical Engineering degree from Cambridge University, a Masters degree in Optoelectronics from Heriot-Watt University and a Ph.D in III/V Integrated Photonics from the University of Bath, all in the UK. He has authored and co-authored over 150 journal and conferenced publications and holds more than 50 active patents within the fields of silicon photonics and optoelectronics.