Three significant figures in the history of digital image sensor development collaborated on a new research paper that reviews nearly six decades of sensor evolution, detailing pivotal moments and technologies along the way, while also looking ahead to the future of camera tech.
As highlighted by DPReview, the collaborators, Eric R. Fossum, Nobukazu Teranishi, and Albert J.P. Theuwissen, detail across nearly 30 pages the progression of digital image sensors, their vital role in capturing human culture and experience, and where digital cameras go from here.
“Capturing images has been a human activity since prehistoric times, and camera capture has been a part of human culture for almost 200 years,” the researchers explain. “Image sensors are the microelectronic silicon chips that sit at the heart of every digital camera and convert light into electrical signals suitable for transmission, storage, and processing by computers, for use by machines and humans alike.”
Many modern-day photographers know about charge-coupled device (CCD) image sensors. This early digital sensor technology powered many early digital cameras and even persisted in consumer models until well into the DSLR era. The authors discuss the earliest implementations of camera-image pickup, including through cathode ray tube (CRT) tech (yes, like the super heavy television of old), and the major CCD milestones that followed.
“The 1960s saw the real emergence of integrated semiconductor devices, and the light sensitivity of semiconductors was well-known by then,” the authors write. A Honeywell photosensitive junction device in 1963, an IBM scanistor array the following year, and a 1966 Westinghouse 50-element phototransistor array were among the early examples. These devices had an output signal “proportional to the instantaneous optical input signal without intentional integration, and thus, the signal was weak and required gain inside of the pixel for amplification. In essence, these were the first active-pixel sensors.”
While these early devices were fairly described as extremely noisy, clunky, and inefficient photosensitive semiconductors, it did not take long for significant progress to arrive.
In 1969, Willard S. Boyle and George E. Smith created the charge-coupled device (CCD) sensor at Bell Labs, work for which they won the Nobel Prize in Physics in 2009. Not all were pleased about the award, but that’s mostly a story for a different day.
Whoever should be considered responsible for the 1969 creation of the CCD sensor, the invention set off a flurry of research and development in solid-state image sensors. Fossum, Teranishi, and Theuwissen detail the key advancements, including the frame-transfer CCD image sensor in 1971 (created in part by Mike Tompsett, who claimed he should have won the Nobel prize for the original CCD in 2009), the interline-transfer CCD in 1973, and pinned photodiode technology in 1982.
Pinned photodiode technology, created by one of the paper’s authors, Teranishi, in 1982 and further advanced by Fossum and Hondongwa in 2014, significantly improved CCD image sensor technology and laid the groundwork for major components also used in CMOS image sensor technology, the types of sensors used in all consumer digital cameras today.
Complementary technologies were developed adjacent to the digital image sensors themselves, including the Bayer filter (Bayer, 1976), sensor stitching (Rominger 1988, Monma and Yuzurihara 1993, Kreider et al. 1995, Monma and Yuzurihara 1998), and large sensor wafer fabrication (e.g., Lesser et al. 1997, Ay and Fossum 2006, Zacharias et al. 2007, Yamashita et al. 2011).
Despite continuous advancements, CCD sensors demonstrated numerous limitations. No sensor technology is perfect for everything, after all. CCD sensors struggle with charge transfer efficiency, readout rate, power, manufacturing yield, and integration.
With these limitations in mind, CCD sensor improvement nonetheless led to widespread adoption. By 1990, nearly all digital cameras used CCDs, including models made by Sony, Matsushita (Panasonic), Toshiba, Sharp, and NEC in Japan and companies like Philips, Thomson CSF, Kodak, and Texas Instruments elsewhere. There were other smaller CCD sensor players, too, just as with CMOS image sensor technology today.
CCD cameras were still huge at this time — some of the digital cameras sent into space for scientific work were “the size of a small refrigerator.” Unsurprisingly, researchers were looking at alternative technologies, including complementary metal-oxide semiconductor image sensors, or CMOS.
There was research into CMOS before the 1990s, but the need for more efficient, smaller, and easier-to-produce digital sensors spun the wheels of progress into high gear at this time. The authors discuss two primary CMOS research efforts, one focused on efficiency, low-cost single-chip imaging systems, and the other developing compact, very high-performance cameras.
The high-performance CMOS development was led by NASA’s Jet Propulsion Laboratory at Caltech in the United States and laid the groundwork for significant advancements to come in digital cameras.
“This effort resulted in the invention of the CMOS active-pixel image sensor with intrapixel charge transfer and represented the opposite end of the performance spectrum compared to the focus of VVL and IVP,” the researchers write.
With the aid of microlenses and backside illumination (yes, backside CMOS tech is not that new, it was created by Fossum in 1994, more than a decade before it was widely adopted in consumer cameras), CMOS sensors addressed many of the shortcomings of CCD. Not long after, CMOS would become the dominant digital camera sensor technology in the marketplace.
The authors then discuss 21st-century improvements, including mass production of backside-illuminated image sensors, continued shrinking of pixel size, 3D stacking technology, pill cameras, and much more.
Although digital cameras today are incredible and getting better all the time, that doesn’t mean that research into brand-new digital imaging technologies has stopped. Scientists continue to push the frontiers of solid-state image sensor technology, including through quanta image sensors, sensors that can detect single photons, and Single Photon Avalanche Diode (SPAD) image sensors.
“Solid-state image sensors have evolved continuously and are ubiquitous in our daily lives,” the researchers conclude, extolling how digital image sensors impact people’s lives, cultures, and science.
However, they also note how image sensor technology is used to cause harm by facilitating criminal activity and violating privacy.
“For a technology intended to benefit personal well-being and society at large with light and truth, it is indeed unfortunate that we must also reckon with, and control, the dark edge of the sword.”
The lengthy and detailed paper, “Digital Image Sensor Evolution and New Frontiers,” was published in the Annual Review of Vision Science. Once again, the paper was written by Eric R. Fossum, Nobukazu Teranishi, and Albert J.
Image credits: Featured image licensed via Depositphotos. The research paper, “Digital Image Sensor Evolution and New Frontiers,” by Fossum, Teranishi, and Theuwissen, has been published in Annual Review of Vision Science under Creative Commons Attribution 4.0 International License.