Research Article |
Corresponding author: Michael Heethoff ( heethoff@bio.tu-darmstadt.de ) Academic editor: Nesrine Akkari
© 2018 Bernhard Ströbel, Sebastian Schmelzle, Nico Blüthgen, Michael Heethoff.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Citation:
Ströbel B, Schmelzle S, Blüthgen N, Heethoff M (2018) An automated device for the digitization and 3D modelling of insects, combining extended-depth-of-field and all-side multi-view imaging. ZooKeys 759: 1-27. https://doi.org/10.3897/zookeys.759.24584
|
Digitization of natural history collections is a major challenge in archiving biodiversity. In recent years, several approaches have emerged, allowing either automated digitization, extended depth of field (EDOF) or multi-view imaging of insects. Here, we present DISC3D: a new digitization device for pinned insects and other small objects that combines all these aspects. A PC and a microcontroller board control the device. It features a sample holder on a motorized two-axis gimbal, allowing the specimens to be imaged from virtually any view. Ambient, mostly reflection-free illumination is ascertained by two LED-stripes circularly installed in two hemispherical white-coated domes (front-light and back-light). The device is equipped with an industrial camera and a compact macro lens, mounted on a motorized macro rail. EDOF images are calculated from an image stack using a novel calibrated scaling algorithm that meets the requirements of the pinhole camera model (a unique central perspective). The images can be used to generate a calibrated and real color texturized 3Dmodel by ‘structure from motion’ with a visibility consistent mesh generation. Such models are ideal for obtaining morphometric measurement data in 1D, 2D and 3D, thereby opening new opportunities for trait-based research in taxonomy, phylogeny, eco-physiology, and functional ecology.
Focus stacking, morphometry, structure from motion, photogrammetry, 3D modelling, DISC3D
Digitization has become a major challenge in the curation of natural history collections (
Traditionally, species descriptions and other scientific or popular documentations focus on 1-D characters, e.g., body length, the length of single body parts or relative length ratios between body parts. However, since organisms have 3D shapes, 2D or 3D traits are meaningful complementary information to characterize body shape and variation more completely (
Established tomographic 3D techniques have some shortcomings when it comes to pinned insects. Although delivering landmark data with a high precision (e.g.,
Visible light is to some extent reflected or remitted at or near the surface of the insect. Light returning from the specimen bears information about position, reflectivity, and color, and forms the basis for optical 3D surface scanning. Triangulation techniques determine the spatial position of surface points by the intersection of light rays. In the case of passive triangulation, ambient illumination is applied, and all rays used for measuring are viewing rays of different cameras or of a camera in different positions. Two such techniques, ‘Structure from Motion’ (SfM) and ‘Shape from Silhouette’ (SfS), are suitable for 3D insect scanning (
SfS (
SfM (
Here, a new imaging device (the Darmstadt Insect SCanner: DISC3D) is presented that overcomes the restrictions of the above-mentioned EDOF multi-view imaging, and provides data suitable for both, digitally archiving insects (and other small objects) and generating 3D models. We developed DISC3D with the aim of affordability, clonability, and minimization of manual processing steps. DISC3D allows specific configurations for different requirements (e.g., high resolution for archiving, high number of views for 3D modelling of complex structures, or fast imaging for mass digitalization).
DISC3D is published under the Creative Commons license CC BY-SA (https://creativecommons.org). The total costs of the device range between 4,000€ and 8,000€ (depending on the camera and availability of computers and educational software licenses). In the following an overview is given of the components of DISC3D (Fig.
For demonstration of DISC3D, a set of pinned insects and snail shells from our collection with a representative shape range and sizes between 1.5–30 mm were chosen. The following species were included:
Coleoptera:
Prosopocoilus savagei Hope (23 mm)
Anoplotrupes stercorosus Scriba (21 mm)
Stenocorus meridianus L. (19 mm)
Typhaeus typhoeus L. (18 mm)
Rutpela maculata Poda (17 mm)
Valgus hemipterus L. (8 mm)
Cryptocephalus sericeus L. (7 mm)
Pogonocherus hispidus L. (6 mm)
Phyllobius pyri L. (6 mm)
Tytthaspis sedecimpunctata L. (3 mm)
Lepidoptera:
Zygaena filipendulae L. (15 mm body size, 30 mm wingspan)
Hymenoptera:
Paraponera clavata Fabricus (20 mm)
Osmia adunca Panzer (12 mm)
Sphecodes ephippius L. (8 mm)
Diptera:
Thricops sp. Róndani (8 mm)
Culex pipiens L. (5 mm)
Oscinella frit L. (1.5 mm)
Gastropoda:
Helicodonta obvoluta O.F. Müller (9 mm)
Aegopinella nitens Michaud (8 mm)
Discus rotundatus O.F. Müller (5 mm)
A black foam plastic adapter is used to connect the insect pin to a ca. 100 mm long and 1 mm thick supporting steel pin. The mounting is adjusted to the center of rotation of a two-axis motorized gimbal and allows a free view on the specimen from virtually all sides. During an insect scan, the specimen undergoes a preassigned ‘pose program’ with an approximately constant angular distance between neighboring poses. Pose programs can be adjusted for specific needs (e.g., shape and complexity of the object). In this study, we mostly used a standard pose program with a mean angular distance of 10° of two neighboring poses, and a total number of 412 poses, 14 of which are not accessible due to geometric and optical constraints of the sample holder (see Suppl. material
Many insect surfaces are glossy and show bright reflections (‘hotspots’) when illuminated directly by a point-shaped light source. For that reason, we use two separately addressable hemispherical illumination domes: a ‘front-light dome’ that is adjacent to the camera but averted, and a ‘back-light dome’ on the far side facing the camera (Fig.
The indirect front-light illumination of the sample is optimal for curved or bumpy glossy insect surfaces. The back-light dome produces an even background illumination, and with the front-light switched off, a silhouette image of the insect can be captured. This silhouette image can be used to mask the background in the images taken with front-light illumination.
DISC3D is equipped with an industrial camera and a compact macro lens (Fig.
The complete process of image acquisition and processing is controlled with MATLAB (Mathworks, Natick, USA). Camera configuration and start, as well as the readout of the images are accomplished via the USB3 Vision interface standard. An Arduino Mega 2560 microcontroller board, connected to the PC via a serial interface controls the motion of the gimbal motors, switches the LEDs, and triggers the video stream of the camera. To synchronize the macro rail controller with the video stream, the StackShot motion is triggered by the ‘exposure active’ camera output signal of the first video frame.
For the calculation of the EDOF image from the focus stack, several proven software products are available (
After focus stack processing of the front-light and back-light images, a binary mask for the segmentation of the background is calculated and saved in the alpha-channel of the front-light image (Fig.
The MATLAB code is menu-based and includes (i) several calibration functions, (ii) the control of insect positioning, front- and backlight illumination, camera exposure and gain parameters, (iii) the interactive adjustment of several EDOF calculation and masking parameters on the base of the quality of obtained images, and (iv) the start of the scan with the chosen pose program and evaluation range of the image stacks. The interactive adjustment allows customizing all parameters to the specific characteristics of the particular specimen. Alternatively, proven parameter sets from earlier scans of the same or similar specimens can be used. The scan itself is completely automatic.
After the scan, the masked EDOF images of all poses are transferred to the SfM software PhotoScan Pro 1.4 (Agisoft LLC, St. Petersburg, Russia). Depending on shape and surface texture of the insects, images of some poses may not contain a sufficient number of matching feature points to allow the simultaneous calibration of the respective camera positions. Such weak poses are discarded by PhotoScan Pro, which impairs the quality of the resulting model. To cope with this problem, and to accelerate the SfM calculation, a special 3D target with a high number of features (textured sphere, see Fig.
For animation (see Suppl. material
Some of our 3D models were used to demonstrate the ease of obtaining 3D morphometric data of the scanned specimens in PhotoScan Pro. Surface area and volume were measured and plotted against each other.
Additionally, the reliability of measurements taken on 3D models was tested by comparing morphometric data obtained from the model and on the specimen itself using the statistical analysis software PAST (
The quality of the images and the 3D models of DISC3D is demonstrated here by some illustrative examples of insects and snail shells. We present some models as animated videos and some as textured or non-textured polygon-models, implemented as interactive 3D content.
DISC3D allows automatic acquisition of multi-view EDOF images for digitization and digital archiving of pinned insects. There is a trade-off between the digital resolution (size of the pixels on the side of the insects), and the number of views on the one hand, and the scanning time on the other hand. The quality and digital resolution of the EDOF images created by our stacking algorithm is well suitable for the inspection of many relevant morphological features and avoids artifacts of other approaches (
Comparison of images taken with a Keyence VHX 5000 digital microscope (lens: Z20, A–C) and DISC3D (D). The whole specimen of Pogonocherus hispidus can be imaged at once with the VHX 5000 with a X30-magnification (A). To compare the digital resolution, we focus on the pronotum of the beetle (B: VHX 5000, ×30; C: VHX 5000, ×100, D: DISC3D, ×1.26). Scale bars: 1 mm.
All 3D models shown here have been generated with Agisoft PhotoScan Pro with visibility-consistent mesh generation enabled. We show the results of the main steps, exemplified by a scan of a 6 mm long specimen of the Lesser Thorn-tipped Longhorn Beetle Pogonocherus hispidus (Fig.
Workflow of model generation of Pogonocherus hispidus with PhotoScan Pro. In total, 398 EDOF-images are taken with DISC3D (one example is shown in A). Using the image data, masks and camera positions estimated with the calibration sphere (see Fig.
Comparison of mesh quality and number of polygons, exemplified by a model of the shell of Discus rotundatus. The model with 1 million faces (A) has a file size (3D-PDF) of 35 MB and shows more detail, but the reduced model with 75.000 faces (B) still well resembles the structure with a file size (3D-PDF) of only 3 MB.
A set of insect and snail specimens were chosen with a representative shape and size range of 1.5–30 mm for visualization (Fig.
Overview and size comparison of the specimens used in this study. Coleoptera: a Prosopocoilus savagei b Anoplotrupes stercorosus, *: specimen was broken during comparative measurements c Stenocorus meridianus d Typhaeus typhoeus e Rutpela maculata f Valgus hemipterus g Cryptocephalus sericeus h Pogonocherus hispidus i Phyllobius pyri j Tytthaspis sedecimpunctata; Lepidoptera: k Zygaena filipendulae; Hymenoptera: l Paraponera clavata m Osmia adunca n Sphecodes ephippius; Diptera: o Thricops sp., p Culex pipiens q Oscinella frit; Gastropoda: r Helicodonta obvoluta s Aegopinella nitens t Discus rotundatus.; Scale bar: 1 cm (keep in mind that not all specimens are equidistant to the lens; i.e., at the same height of the needle).
With 1.5 mm, Oscinella frit is probably the smallest object that has ever been 3D modeled by SfM techniques (Fig.
We further provide interactive 3D-models of a mid-sized (Pogonocherus hispidus, 6 mm, Fig.
The models generated with EDOF images obtained by DISC3D have several advantages when compared to the models published by (
Surface area and volume of the 3D models were measured in PhotoScan Pro and plotted against each other (Fig.
3D models are not only helpful to obtain data which are impossible to be measured on the specimen itself (like surface area), they may also help to obtain more reliable data even of 1D measurements by avoiding parallax errors. We asked 22 people (laypersons, students and skilled entomologists) to measure two simple distances between easily observable landmarks that differed in their susceptibility to parallax errors (width of scutellum (sc) and length of the tibia (ti) of the left hind leg) on a specimen of the dung beetle Anoplotrupes stercorosus, using both, a Keyence VHX-5000 digital microscope (=2D), and a 3D-model (=3D) of the same specimen (Fig.
Due to observer errors, morphometric data can vary by more than 14% of the mean measured value (
To the best of our knowledge, DISC3D is the first and only system for automated multi-view EDOF imaging. The system allows the digitization of natural history collections and effective exchange of information about a specimen, avoiding physical handling or transfer of the specimen itself. The 3D-models facilitate accurate reproducible 1D, 2D, and 3D measurements to characterize the specimen, including functionally relevant traits such as surfaces or volumes, a promising perspective for functional ecology, comparative zoology, and physiology. For large scale digitization projects, several scanners could be used simultaneously. Due to the high degree of automatization, one person should be able to operate up to five devices in parallel for scanning and 3D-modelling. We encourage the community to copy our device and to join us in further developing DISC3D for archiving and 3D-modelling purposes. We will be happy to provide relevant information and share our experience.
We would like to thank Ken Justice and Stephan Neser (HDA) for providing ideas and technical support. The workshop team (Faculty of Biology, TUDA) is acknowledged for technical support. Numerous students from both institutions (HDA and TUDA) assisted in developing DISC3D.
Technical information
SV1: EDOF imaging
SV2: Illustrative examples