Description: Dome ports act as spherical windows in underwater housings through which a camera can observe objects in the water. As compared to flat glass interfaces, they do not limit the field of view, and they do not cause refraction of light observed by a pinhole camera positioned exactly in the center of the dome. Mechanically adjusting a real lens to this position is a challenging task, in particular for those integrated in deep sea housings. In this contribution a mechanical adjustment procedure based on straight line observations above and below water is proposed that allows for accurate alignments. Additionally, we show a chessboard-based method employing an underwater/above-water image pair to estimate potentially remaining offsets from the dome center to allow refraction correction in photogrammetric applications. Besides providing intuition about the severity of refraction in certain settings, we demonstrate the methods on real data for acrylic and glass domes in the water.

Description: In deep water conditions, vision systems mounted on underwater robotic platforms require artificial light sources to illuminate the scene. The particular lighting configurations significantly influence the quality of the captured underwater images and can make their analysis much harder or easier. Nowadays, classical monolithic Xenon flashes are gradually being replaced by more flexible setups of multiple powerful LEDs. However, this raises the question of how to arrange these light sources, given different types of seawater and-depending-on different flying altitudes of the capture platforms. Hence, this paper presents a rendering based coarse-to-fine approach to optimize recent multi-light setups for underwater vehicles. It uses physical underwater light transport models and target ocean and mission parameters to simulate the underwater images as would be observed by a camera system with particular lighting setups. This paper proposes to systematically vary certain design parameters such as each LED’s orientation and analyses the rendered image properties (such as illuminated image area and light uniformity) to find optimal light configurations. We report first results on a real, ongoing AUV light design process for deep sea mission conditions.

Description: Nowadays underwater vision systems are being widely applied in ocean research. However, the largest portion of the ocean - the deep sea - still remains mostly unexplored. Only relatively few image sets have been taken from the deep sea due to the physical limitations caused by technical challenges and enormous costs. Deep sea images are very different from the images taken in shallow waters and this area did not get much attention from the community. The shortage of deep sea images and the corresponding ground truth data for evaluation and training is becoming a bottleneck for the development of underwater computer vision methods. Thus, this paper presents a physical model-based image simulation solution, which uses an in-air texture and depth information as inputs, to generate underwater image sequences taken by robots in deep ocean scenarios. Different from shallow water conditions, artificial illumination plays a vital role in deep sea image formation as it strongly affects the scene appearance. Our radiometric image formation model considers both attenuation and scattering effects with co-moving spotlights in the dark. By detailed analysis and evaluation of the underwater image formation model, we propose a 3D lookup table structure in combination with a novel rendering strategy to improve simulation performance. This enables us to integrate an interactive deep sea robotic vision simulation in the Unmanned Underwater Vehicles simulator. To inspire further deep sea vision research by the community, we release the source code of our deep sea image converter to the public (https://www.geomar.de/en/omv-research/robotic-imaging-simulator).

Description: Nowadays various methods and sensors are available for 3D reconstruction tasks; however, it is still necessary to integrate advantages of different technologies for optimizing the quality 3D models. Computed tomography (CT) is an imaging technique which takes a large number of radiographic measurements from different angles, in order to generate slices of the object, however, without colour information. The aim of this study is to put forward a framework to extract colour information from photogrammetric images for corresponding Computed Tomography (CT) surface data with high precision. The 3D models of the same object from CT and photogrammetry methods are generated respectively, and a transformation matrix is determined to align the extracted CT surface to the photogrammetric point cloud through a coarse-to-fine registration process. The estimated pose information of images to the photogrammetric point clouds, which can be obtained from the standard image alignment procedure, also applies to the aligned CT surface data. For each camera pose, a depth image of CT data is calculated by projecting all the CT points to the image plane. The depth image is in principle should agree with the corresponding photogrammetric image. The points, which cannot be seen from the pose, but are also projected on the depth image, are excluded from the colouring process. This is realized by comparing the range values of neighbouring pixels and finding the corresponding 3D points with larger range values. The same procedure is implemented for all the image poses to obtain the coloured CT surface. Thus, by using photogrammetric images, we achieve a coloured CT dataset with high precision, which combines the advantages from both methods. Rather than simply stitching different data, we deep-dive into the photogrammetric 3D reconstruction process and optimize the CT data with colour information. This process can also provide an initial route and more options for other data fusion processes.

Description: With the rapid development and availability of underwater imaging technologies, underwater visual recording is widely used for a variety of tasks. However, quantitative imaging and photogrammetry in the underwater case has a lot of challenges (strong geometry distortion and radiometry issues) that limit the traditional photogrammetric workflow in underwater applications. This paper presents an iterative refinement approach to cope with refraction induced distortion while building on top of a standard photogrammetry pipeline. The approach uses approximate geometry to compensate for water refraction effects in images and then brings the new images into the next iteration of 3D reconstruction until the update of resulting depth maps becomes neglectable. Afterwards, the corrected depth map can also be used to compensate the attenuation effect in order to get a more realistic color for the 3D model. To verify the geometry improvement of the proposed approach, a set of images with air-water refraction effect were rendered from a ground truth model and the iterative refinement approach was applied to improve the 3D reconstruction. At the end, this paper also shows its application results for 3D reconstruction of a dump site for underwater munition in the Baltic Sea for which a visual monitoring approach is desired.

Description: Underwater cameras are typically placed behind glass windows to protect them from the water. Spherical glass, a dome port, is well suited for high water pressures at great depth, allows for a large field of view, and avoids refraction if a pinhole camera is positioned exactly at the sphere's center. Adjusting a real lens perfectly to the dome center is a challenging task, both in terms of how to actually guide the centering process (e.g. visual servoing) and how to measure the alignment quality, but also, how to mechanically perform the alignment. Consequently, such systems are prone to being decentered by some offset, leading to challenging refraction patterns at the sphere that invalidate the pinhole camera model. We show that the overall camera system becomes an axial camera, even for thick domes as used for deep sea exploration and provide a non-iterative way to compute the center of refraction without requiring knowledge of exact air, glass or water properties. We also analyze the refractive geometry at the sphere, looking at effects such as forward- vs. backward decentering, iso-refraction curves and obtain a 6th-degree polynomial equation for forward projection of 3D points in thin domes. We then propose a pure underwater calibration procedure to estimate the decentering from multiple images. This estimate can either be used during adjustment to guide the mechanical position of the lens, or can be considered in photogrammetric underwater applications.

Description: Underwater cameras are typically placed behind glass windows to protect them from the water. Spherical glass, a dome port, is well suited for high water pressures at great depth, allows for a large field of view, and avoids refraction if a pinhole camera is positioned exactly at the sphere’s center. Adjusting a real lens perfectly to the dome center is a challenging task, both in terms of how to actually guide the centering process (e.g. visual servoing) and how to measure the alignment quality, but also, how to mechanically perform the alignment. Consequently, such systems are prone to being decentered by some offset, leading to challenging refraction patterns at the sphere that invalidate the pinhole camera model. We show that the overall camera system becomes an axial camera, even for thick domes as used for deep sea exploration and provide a non-iterative way to compute the center of refraction without requiring knowledge of exact air, glass or water properties. We also analyze the refractive geometry at the sphere, looking at effects such as forward- vs. backward decentering, iso-refraction curves and obtain a 6th-degree polynomial equation for forward projection of 3D points in thin domes. We then propose a pure underwater calibration procedure to estimate the decentering from multiple images. This estimate can either be used during adjustment to guide the mechanical position of the lens, or can be considered in photogrammetric underwater applications.

Description: Reliable quantification of natural and anthropogenic gas release (e.g. CO2, methane) from the seafloor into the water column, and potentially to the atmosphere, is a challenging task. While ship-based echo sounders such as single beam and multibeam systems allow detection of free gas, bubbles, in the water even from a great distance, exact quantification utilizing the hydroacoustic data requires additional parameters such as rise speed and bubble size distribution. Optical methods are complementary in the sense that they can provide high temporal and spatial resolution of single bubbles or bubble streams from close distance. In this contribution we introduce a complete instrument and evaluation method for optical bubble stream characterization targeted at flows of up to 100 ml/min and bubbles with a few millimeters radius. The dedicated instrument employs a high-speed deep sea capable stereo camera system that can record terabytes of bubble imagery when deployed at a seep site for later automated analysis. Bubble characteristics can be obtained for short sequences, then relocating the instrument to other locations, or in autonomous mode of definable intervals up to several days, in order to capture bubble flow variations due to e.g. tide dependent pressure changes or reservoir depletion. Beside reporting the steps to make bubble characterization robust and autonomous, we carefully evaluate the reachable accuracy to be in the range of 1–2% of the bubble radius and propose a novel auto-calibration procedure that, due to the lack of point correspondences, uses only the silhouettes of bubbles. The system has been operated successfully in 1000 m water depth at the Cascadia margin offshore Oregon to assess methane fluxes from various seep locations. Besides sample results we also report failure cases and lessons learnt during deployment and method development.

Description: Most parts of the Earth’s surface are situated in the deep ocean. To explore this visually rather adversarial environment with cameras, they have to be protected by pressure housings. These housings, in turn, need interfaces to the world, enduring extreme pressures within the water column. Commonly, a flat window or a half-sphere of glass, called flat-port or dome-port, respectively is used to implement such kind of interface. Hence, multi-media interfaces, between water, glass and air are introduced, entailing refraction effects in the images taken through them. To obtain unbiased 3D measurements and to yield a geometrically faithful reconstruction of the scene, it is mandatory to deal with the effects in a proper manner. Hence, we propose an optical digital twin of an underwater environment, which has been geometrically verified to resemble a real water lab tank that features the two most common optical interfaces. It can be used to develop, evaluate, train, test and tune refractive algorithms. Alongside this paper, we publish the model for further extension, jointly with code to dynamically generate samples from the dataset. Finally, we also publish a pre-rendered dataset ready for use at https://git.geomar.de/david-nakath/geodt.