The figure below shows a taxonomic tree (genus and species within family; labels give genus), with samples from the corresponding colour palettes on the right. Some broad patterns are apparent: most of the greenish-bluish species lie within the Carabus genus, whereas those with yellow-orange tones are typically from Bembidion. Note that some of the yellowish-white colours may come from the heads of the specimen pins, which are present in most photographs (although these comprise a relatively small fraction of the image area).

Taxonomic tree of ground beetles (genus and species within the family Carabidae; labels show genus), with samples from the corresponding colour palettes on the right. Click for a higher-resolution version and zoom in.

A relatively simple method for depicting colour patterns in geographic space is to start by gridding up the region of interest. For each grid cell, occurrence data are used to find the species that are present, and then the colours in those species' colour palettes are sampled (with representative sampling weights, so that the more common colours are more prevalent in the sample).

Most of the occurrence data are from Europe and North America. For Europe, there appears to be a latitudinal gradient in colours, with green more prevalent in the south (e.g. France, Germany) and brown/dark orange through Scandinavia. Other, smaller scale patterns are also present, such as the region of blue, yellow, and brighter orange in the far north.

Ground beetle colouration across Europe. Each tile in the image shows a representative sample of the colours of species present in that grid cell. Click for a higher-resolution version and zoom in.

The occurrence data are sparser in North America, but similarly seem to have a more pronounced tendency towards orange hues at higher latitudes (through Canada and Alaska).

Ground beetle colouration across North America. Each tile in the image shows a representative sample of the colours of species present in that grid cell. Click for a higher-resolution version and zoom in.

Maps derived directly from occurrence data are limited to the spatial coverage of that data (see "Further work", below). Spatial regression models can help to fill in gaps, by modelling a response variable as a function of environmental covariates. The example map below shows this idea: taxa with red colouration were identified, and then the probability of finding those taxa was modelled as a function of altitude, temperature, and rainfall conditions using a random forest model (see source code for details).

Modelled probability of finding ground beetle taxa with red colouration, across Europe. Click for a higher-resolution version and zoom in.

This map does bear some resemblance to the direct colour results, which is encouraging. Note, though, that the actual results in this example are probably not overly insightful. This is — at least in part — because the predictor variables were just conveniently-available bioclimatic variables, and not selected on the basis of any particularly good ecological reasoning. The partial dependence plots from the regression model could in principle be examined to deduce the environmental conditions associated with red beetles — these are not presented here for this reason (but see the source code for how to do this).

The within-species colour variability results (below) suggest that beetles in North America might generally have more variable colouration than in Europe. Areas of particularly low and particularly high colour-diversity may reflect sampling effort and data availability: see discussion below.

Mean colour variability within ground beetle species across geographic space. Click for a higher-resolution version and zoom in.

The images used here were specifically chosen to be specimen images with consistent white or near-white backgrounds, so as to make the demonstration code more tractable. Undesirable images (e.g. of labels) were excluded automatically where possible (based on the file name of the image and the image colour palette characteristics) but manual exclusion of some such images was still required.

Relaxing the constraint on specimen-only images would lead to a much larger library of target images, broader taxonomic coverage, and potentially more robust results, but would in turn require more advanced processing techniques. The extra difficulties predominantly stem from the more-variable nature of the composition of such images — for example, the species in question is unlikely to be the only subject in the image, and will not be photographed from a consistent aspect (pose). Other sources of variability include lighting, weather conditions (rain, fog, bright sunshine), and camera equipment. However, if these difficulties can be overcome, the result will likely be a more general and more flexible set of tools, with a higher degree of automation, and ultimately greater benefit to the scientific community.

A relatively-recent but highly successful family of image processing algorithms are based on deep learning techniques such as convolutional neural networks. Software for these approaches — while still requiring considerable domain expertise to use effectively — is available (see e.g. Caffe and R-CNN). One of the difficulties is effectively training the networks. Worth noting is that pre-trained models are available online, and the Caffe web demonstration includes a "ground beetle" class — see this example screenshot:

Thus — for the ground beetle application at least, and possibly others — there might be existing, trained models that can be used directly.

Object detection and image segmentation are also important, so that objects of interest can be automatically identified and delineated in photographs. See below for an example of bird detection from Zhang et al.² (which actually goes further, detecting parts of objects, such as the head in this example).

Other software libraries such as OpenCV are also candidates for improved image processing.

The maps shown above generally use the species occurrence records to directly infer the presence or absence of a species at a given location. Some of the patterns in the results are therefore likely to be reflections of patterns in sampling effort and data availability (e.g. spatial and taxonomic biases in occurrence records and images) rather than genuine biological patterns. An obvious solution to this issue to to use spatial modelling to better account for distribution of effort. One possible approach would be to construct a distribution model for each of the species of interest (i.e. estimate probability of presence as a function of environmental and other covariates). Given a location of interest, the species composition would then be estimated from the models, rather than inferred directly from occurrence data.

Alternatively, rather than modelling the species distributions (which would be a lot of work for many species), the modelling could be applied directly to properties of the colour palettes — e.g. modelling red/green/blue colour channel intensities as a function of environment, or modelling the colour composition directly (e.g. treating it as a multinomial distribution, as can be done for community-level modelling of species abundances³). The "probability of finding beetles with red colouration" is an example of this idea.

The within-palette variability example shows the use of a dissimilarity-style analysis. There is an enormous body of ecological research based on the concept of dissimilarities, and so many of these techniques (clustering, ordination, etc) could potentially be applied here. Some subtleties would need to be investigated: for example, the within-palette variability used a dissimilarity measure based on human perception of colour difference⁴. Models using colour perception of different animal groups might yield very different results.

Finally, colouration in beetles reflects various ecological phenomena, including camouflage, mimicry, or warnings to predators. Analyses that draw on other biological data (predator occurrences, for example) might be worth pursuing.