Computational forensic facial reconstruction

Evison, M.P., Davy, S.L., March, J. and Schofield, D. (2004). Computational forensic

facial reconstruction. Facial Reconstruction. Conference Publication 1. International

Conference on Reconstruction of Soft Facial Parts in Potsdam/Germany from 10 to 12

November 2003. Germany: Bundeskriminalamt, pp. 29-34.

This text is the Accepted Manuscript only.

Evison, M.P., Davy, S.L., March, J. and Schofield, D. (2004). Computational forensic facial reconstruction. Facial Reconstruction. Conference Publication 1. International Conference on Reconstruction of Soft Facial Parts in Potsdam/Germany from 10 to 12 November 2003. Germany: Bundeskriminalamt, pp. 29-34.

Computational forensic facial reconstruction

Evison, M.P.1, Davy, S.L.

1, March, J.

2 and Schofield, D.

2

1 Deprtment of Forensic Pathology, The Medico-Legal Centre, The University of Sheffield, Watery

Street, Sheffield S3 7ES, UK. Tel. +44 114 2738721, Fax. +44 114 2798942, Email.

[email protected].

2 SChEME, University of Nottingham, Nottingham, NG7 2RD, UK. Tel. +44 115 9514084, Fax.

+44 115 9514115. Email. [email protected].

Introduction

Rapid developments in three-dimensional (3D) digitised image capture, computer visualisation

modelling and animation have begun make inroads into many areas of the forensic sciences,

including the rather conservative specialisation of forensic pathology. This has been the result of an

interdisciplinary collaboration between forensic pathologists, biological anthropologists and

computer scientists. For example (Figure 1), we have modelled the track of the blade in a sharp

force trauma or stabbing incident and been able to exclude certain body postures as having

occurred at the time of the injury (March et al. 2003).

Forensic facial reconstruction serves to illustrate one aspect of the results of these collaborations,

but surveying and reconstruction and modelling and animation of accident (Figure 2) or crime

scenes (Figure 3) are other fields in which 3D computerised methods are gradually being adopted

(Noond et al. 2002). Again, visualisation can be used as an investigative tool—in comparing

scenarios based on conflicting witness statements for example.

I would like to briefly review 3D forensic facial reconstruction, describing the traditional method,

and outlining the novel approaches being adopted in computerised 3D forensic facial reconstruction

by several research groups.

Traditional forensic facial reconstruction

The purpose of forensic facial reconstruction is to produce an image from the skull, which offers a

sufficient likeness of the living individual that it will facilitate identification of skeletal remains

when there are no other means available. Although facial reconstruction had begun in the

nineteenth century, the method gained notoriety with the work of Gerasimov (1968), depicted in the

1983 film in Gorky Park (Figure 4) directed by Michael Apted and based on a novel by Martin


Cruz Smith (1981), who had become familiar with Gerasimov’s work—his reconstruction of

Tamerlane features in the film.

These traditional 'plastic' methods (Isçan and Helmer 1993, Snow et al. 1970) use modelling clay or

plasticine to build up the depth of tissue on the skull (or a cast of the skull) to that of a living

individual. Tissue depths are known for 'landmark' sites on the skull and the face surface can be

recreated in using two approaches, which are not mutually exclusive. With the American method

the depths elsewhere on the face are interpolated between the landmark points (Figure 5) and then

into the interstices (Figure 6). The shape of the eyes, nose and mouth cannot be accurately

determined and are partly guesswork (Figure 7). The Russian method involves an attempt to

reconstruct the soft tissue anatomy of the face, but it is important to note—from a scientific

perspective—that the muscle or other soft tissue depths being applied are unquantified and, again,

therefore, largely subjective. It is now common to combine the use of tissue depth data with

reconstruction of the soft tissue anatomy (Figure 8), but the added value of this approach has not

been established empirically. Even for skilled practitioners, plastic reconstructions take one or two

days. The results obtained will differ between reconstructions and between practitioners.

There are typically twenty-one landmark sites for which tissue depth information is available—in

practice this means about thirty are used because some landmarks appear on the left and right sides

of the skull. This represents extremely sparse coverage of the facial surface.

It has also become increasingly apparent that the ‘canons of sculpture’ used to guide the

practitioner in the placement of facial features such as the eye balls and eye lids, tip of the nose and

lips may be flattering to the subject, but does not reflect common anatomical reality (Macho 1986,

1989; Stephan 2002) and—as discussed above—the shape of these features anyway is something of

a partial guess.

A final issue is the question of “how valuable is artistic quality in achieving the purpose of facial

reconstruction?” Clearly, facial reconstructions can be immensely variable in artistic quality, but as

it transpires, there is no evidence that an artistically accomplished facial sculpture—beautiful

though it may be—is any more effective in eliciting recognition in a forensic case than lacklustre

reconstructions produced by simple means: these seem to work just as well.

The tissue depth measurements used in facial reconstruction have often been those collected from

cadavers in the early part of the twentieth century, or before. These measurements are biased

because they come from small samples, because a dead person's tissues are not the same as in life,


and because they take only limited account of the average differences known to occur between

people of different age, build and sex, and between the major geographical areas.

For over a century, forensic artists and scientists have been attempting to improve the accuracy of

facial reconstructions from the skull, efforts that seem to have met with very limited success. There

have been no empirical studies in which improved success in outcomes in facial reconstruction

have been demonstrated, let alone shown to correlate with quantifiable developments in the

methodology or data applied.

Although some new tissue depth datasets have been collected using ultrasound (Hodson et al.

1985), computed tomography (Nelson 1995, Phillips and Smuts 1996) and magnetic resonance

imaging (Sahni et al. 2002), it is not surprising that Stephan is prepared to challenge the basic

efficacy of facial reconstruction (2002b, Stephan and Henneberg 2001). Ultimately, however,

practitioners do agree that forensic facial reconstruction can lead to identification when no other

options are available—that identification ultimately being confirmed by DNA profiling or forensic

odontology, for example.

Computerised facial reconstruction

A variety of computerised methods for 3D facial reconstruction have been developed since the

1980’s (Vanezis et al. 1989, Ubelaker and O'Donnell 1992, Miyasaka et al. 1995; Tu et al. 2000,

Nelson and Michael 1998; Kähler et al. 2003).

One approach (Vanezis et al. 1989) involves placing the tissue depth landmark positions on a 3D

laser scanned image of a skull manually using a mouse ‘click and drag’ operation. A computer

program is then executed that automatically attaches the selected set of tissue depths at the

appropriate landmarks and renders a smoothed composite facial image over the newly generated

surface (Figure 9). Although some subjectivity can remain in the 'pegging' of a composite facial

image onto the digitised skull matrix, the results are far more rapid and reproducible than sculpted

reconstructions.

A related approach (Kähler et al. 2003) involves manual location of landmark sites and tissue depth

markers—losing some of the advantage of automation—combined with a partial recreation of the

craniofacial musculature (Figure 10). Giving the musculature properties of motion even permits the

reconstruction to exhibit some facial expression.

As can be seen in examples of both plastic and computerised facial reconstruction, however, the

rudimentary finish of the reconstruction leads to a rather mannequin like appearance. Furthermore,


whilst computerised methods may be repeatable, fast and precise, as long as they employ the old

data, the quantitative basis of the reconstruction will be remain limited.

A second approach that represents a considerable quantitative advance on the use of the traditional

landmarks involves warping or deforming a volume tissue depth data set derived from medical

imaging of a living person’s head (Tu et al. 2000, Nelson and Michael 1998, Attardi et al. 1999).

Computed tomography (CT) is commonly employed to capture the volume tissue depth data set.

This approach clearly has the quantitative advantage of offering comprehensive tissue depth

coverage of the skull, but it is important to note that CT relies on the use of X-rays and is therefore

not an appropriate means of collecting tissue depth data from a sample population of healthy

individuals. Attardi et al. (1999) have presented their approach to deformation of volume CT

measurements in some detail, as applied to an Egyptian mummy (Figure 11).

Although deformation based methods may be statistically parsimonious, a lack of correspondence

with anatomical actuality leads these reconstructions to have a consistently cadaverous appearance.

Whilst the somewhat unsatisfactory appearance of the basic reconstruction can be overcome by

texture mapping, the use of a photograph—or composite—can result in a reconstruction that

resembles the photographic image, somewhat irrespective of the shape of the facial surface

At Sheffield we have taken a two-pronged approach to addressing some of the outstanding

problems in forensic facial reconstruction—improving the quantitative basis, the accuracy of

reconstruction and the quality of the finished reconstruction. We have turned to magnetic resonance

imaging (MRI) as an alternative to CT that is more suitable for population studies and have

completed a pilot study (Ratnam 1999) on the collection of volume tissue depth data from MRI

(Figure 12). We have also completed preliminary research on how 3D visualisation models—in this

case constructed in VRML (Evison and Green 1999)—can be used to model variables of facial

reconstruction such as obesity, age and ancestry (Green and Evison 1999)—see Figures 13—15.

Secondly, a fruitful collaboration with our colleagues at the University of Nottingham has allowed

us to carry out facial reconstructions using both the American and Russian—supported by tissue

depths—methods (Figures 16 and 17). We have been able to derive 3D skull surfaces from a series

of two-dimensional digital X-rays taken from Egyptian mummies, for example. Most recently we

have applied our method to the reconstruction of the face of a mummy believed by experts to be

that of Queen Nefertiti (Figure 18). In this case the basic reconstruction was finished by a graphic

artist to give a fine quality finish to the 3D visualisation model that included the clothing and

jewellery featured in the bust of Nefertiti in the Berlin Museum.


Conclusions

There has been quite rapid development in the field of computerised forensic facial reconstruction

since our review of 1997 (Tyrell et al.). This development has featured enhancements of computer

systems developed during the 1980’s and further application of volume data—chiefly from

computed tomography scanning. It has also encompassed novel approaches, such as computer

visualisation modelling of the craniofacial musculature and the development of interpolation

models for ageing and fattening of the face, and ancestry. The Internet has continued to provide a

powerful medium for presenting facial reconstructions and for communicating new research

findings.

The key areas of research that appear to need to be pursued are, firstly, the collection of volume

datasets from good sized samples of living individuals—and MRI seems preferable to CT for this

purpose; secondly, the development of “knowledge based” computational methods of applying that

data—automating the knowledge and skill of the anatomist and sculptor, and thirdly, rendering of

facial surfaces that appear lifelike without resorting to facial or composite facial images. Finally,

the tools derived from these research findings will ideally need to be incorporated into an integrated

and easy to use computer system.

Bibliography

Attardi. G., Betrò, M., Forte, M, Gori, R., Guidazzoli, A., Imboden, S. and Mallegni, F. (1999). 3D

facial reconstruction and visualization of ancient Egyptian mummies using spiral CT data: Soft

tissues reconstruction and textures application. ACM/SIGGRAPH 1999. ACM Press / ACM

SIGGRAPH, Computer Graphics Proceedings, Annual Conference Series

Cruz Smith, M. (1981). Gorky Park. NY: Random House.

Evison, M.P, and Green, M.A. (1999). Presenting Three-Dimensional Forensic Facial Simulations

on the Internet Using VRML. Journal of Forensic Sciences, 44(6), 1216-20.

Gerasimov, M.M. (1968). The Face Finder. London: Hutchinson and Co.

Green, M.A. & Evison, M.P. (1999). Interpolating Between Computerised Three-Dimensional

Forensic Facial Simulations. Journal of Forensic Sciences, 44(6), 1221-5.

Hodson, G., Lieberman, L.S., & Wright, P. (1985). In Vivo Measurements of Facial Tissue

Thicknesses in American Caucasoid Children. Journal of Forensic Sciences, 30(4), 1100-12.


Isçan M.Y. and Helmer, R.P. (eds.) (1993). Forensic Analysis of the Skull. New York: Wiley Liss.

Kähler, Haber, J. and Seidel, H-P. (2003). Reanimating the dead. Reconstruction of expressive

faces from skull data. ACM/SIGGRAPH 2003. ACM Press / ACM SIGGRAPH, Computer

Graphics Proceedings, Annual Conference Series.

Macho, G. (1986). An appraisal of plastic reconstruction of the external nose. Journal of Forensic

Sciences, 31, 1391-403.

Macho, G. (1989). Descriptive morphological features of the nose - an assessment of their

importance for plastic reconstruction. Journal of Forensic Sciences, 34, 1021-36.

March, J., Schofield, D., Evison, M. and Woodford, N. (2003), Forensic Pathology Evidence

Visualisation. American Journal of Forensic Medicine and Pathology (in press).

Miyasaka, S., Yoshino, M., Imaizumi, K. and Seta, S. (1995). The computer-aided facial

reconstruction system. Forensic Science International, 74(1-2), 155-65.

Nelson, L.A. (1995). The Potential Use of Computed Tomography Scans for the Collection of

Cranial Soft Tissue Depth Data [M.Sc. dissertation, unpublished]. Sheffield: University of

Sheffield.

Nelson, L.A. and Michael, S.D. (1998). The application of volume deformation to threedimensional

facial reconstruction: A comparison with previous techniques. Forensic Science International,

94(3), 167-81.

Noond, J., Schofield, D., March, J. and Evison, M. (2002). Visualising the Scene: Computer

Graphics and Evidence Presentation, Science and Justice, 42(2), 89-96.

Phillips, V.M. & Smuts, N.A. (1996) Facial Reconstruction: Utilization of computerized

tomography to measure facial tissue thickness in a mixed racial population. Forensic Science

International, 83, 51-9.

Ratnam, J. (1999). Magnetic resonance biometry [Unpublished M.Sc. dissertation]. Sheffield:

University of Sheffield.

Sahni, D., Jit, I., Sanjeev, D., Singh, P., Suri, S., and Kaur, H. (2002). Preliminary Study on

Facial Soft Tissue Thickness by Magnetic Resonance Imaging in Northwest Indians. Forensic

Science Communications [Internet], 4(1) www.fbi.gov/hq/lab/fsc/current/sahni.htm.


Snow, C.C., Gatliff, B., and McWilliams, K.R. (1970). Reconstruction of facial features from the

skull; an evaluation of its usefulness in forensic anthropology. American Journal of Physical

Anthropology, 33, 221-8.

Stephan, C.N. (2002a). Facial Approximation: Globe Projection Guideline Falsified by

Exophthalmometry Literature. Journal of Forensic Sciences, 47(4), 730-5.

Stephan, C. (2002b) Do Resemblance Ratings Measure the Accuracy of Facial Approximations?

Journal of Forensic Sciences, 47(2), 239-243.

Stephan, C.N. and Henneberg, M. (2001) Building Faces from Dry Skulls: Are They Recognized

above Chance Rates? Journal of Forensic Sciences, 46(3), 432-40.

Tu, P., Hartley, R.I., Lorensen, W.E., Allyassin, A., Gupta, R. and Heier, L. (2000). Face

Reconstructions using Flesh Deformation Modes. Paper presented to the International Association

for Craniofacial Identification, Washington DC, July 2000.

Tyrrell, A.J., Evison, M.P., Chamberlain, A.T. & Green, M.A. (1997). Forensic three-dimensional

facial reconstruction: historical review and contemporary developments. Journal of Forensic

Sciences, 42(4), 653-61.

Ubelaker, D.H. and O'Donnell, G. (1992). Computer assisted facial reconstruction. Journal of

Forensic Sciences, 3 , 155-62.

Vanezis, P., Blowes, R.W., Linney, A.D., Tan, A.C., Richards, R. and Neave, R. 1989. Application

of 3-D computer graphics for facial reconstruction and comparison with sculpting techniques.

Forensic Science International, 4, 69-84.


Figure 1. Modelling of a knife blade track as indicated by injury to the vertebrae and heart

observed at autopsy.

Figure 2. Animation modelling of the scene of a road traffic accident.

Figure 3. An interactive Virtual Reality environment of a crime scene showing smoke

movement (based on Computational Fluid Dynamics calculations).


Figure 4. Promotional material from Michael Apted’s 1983 film Gorky Park. A film inspired

by the work of Gerasimov.

Figure 5. Facial reconstruction using the American method showing the landmark points.


Figure 6. Facial reconstruction using the American method showing filling in of the interstitial

spaces.

Figure 7. Facial reconstruction using the American method showing the modelling of facial

features.


Figure 8. Facial reconstruction using the Russian method combined with landmarks (Sculptor:

Emily Evans).

Figure 9. Facial reconstruction of an older male constructed by morphing a composite

photographic image over a polygon mesh generated by computer using traditional

landmark data (University of Glasgow).

Figure 10. Facial reconstruction using manual placement of tissue depths at landmark sites

combined with partial reconstruction of craniofacial musculature (Kähler et al. 2003).


Figure 11. Facial reconstruction of an Egyptian mummy (left), rendered (right) using a

photographic image derived from a living individual (centre) (Attardi et al. 1999).

Figure 12. Schematic illustrating an approach to tissue depth data collection from MRI (Ratnam

1999).


Figure 13. Interpolation modelling of obesity.

Figure 14. Interpolation modelling of ageing.

Figure 15. Interpolation modelling of ancestry.


Figure 18. Facial reconstruction from 2D digital X-rays using the American method.

Figure 19. Facial reconstruction in Virtual Reality using the Russian method (Modeller: Tim

Gilbert)

Figure 20. Basic reconstruction (left) from 2D digital X-rays of an Egyptian mummy believed to

be that of Queen Nefertiti. Model finished by a graphic artist (right) modelling

jewellery and clothing from a contemporary bust of Nefertiti in the Berlin Museum.


Computational forensic facial reconstruction

Documents

Transcript of Computational forensic facial reconstruction