ENFHEX Collaborative Exercise No. 11

The material of the 11th Collaborative Exercise (CE11) has been collected. A CD with digitized images of the documents to be examined were sent to the participants (over 40 now). The original documents are now circulating among members according to a fixed timetable.

CE11 consists of the examination and comparison of six questioned signatures on six withdrawal forms. These signatures are to be compared with known handwriting from three individuals.

Registered members can download these reports from the CE Reports page in the Working Groups portal site (log in to the ENFSI Member Portal, then click "Handwriting" in the box at the right hand side of the page).

Having run a number of Collaborative Exercises - at no cost to the participants - the Steering Committee of ENFHEX felt it prudent to introduce a charge for each Collaborative Exercise. After some discussion it was felt that it should be recommended to ENFHEX members that a sum of € 100 should be charged for each participating organisation. The funds collected were to assist in preparing, running and reporting on each Collaborative Exercise. It was suggested that any excess revenue from the exercise fee could be used to subsidise the attendance at a Steering Committee Meeting of the hosts of the next ENFHEX Business Meeting.

Following on from this discussion, each ENFHEX member was sent a proposal form in October 2004, asking whether or not the members accepted the proposal. Some 48 letters were sent and the Secretary of ENFHEX received 33 responses, all in favour of introducing the fee. This number equates very closely to the number of organisations that currently take part in the Collaborative Exercises.

Kirsti Lehtonen, representative from the NBI in Finland and member of the ENFHEX Steering Committee, has agreed to act as "Treasurer" for these funds.

One of the objectives of the EU funded ENFHEX project was to work out a procedure for running European quality assessment trials (proficiency tests or collaborative exercises).

During the project five trials or "Collaborative Exercises" have been organised by the Swedish SKL for the ENFHEX project. The first trial case consisted of one photograph of an "extortion letter" with questioned writing and four photographs with comparison material from four different persons. The language used was English. The question was if any one of the suspect persons had written the letter. The case was distributed to the other participants in the project and the answers were to be sent to the Swedish representative by the 1st of March 1999.

The purpose of this trial was to see if this kind of material could be used for the wider WG. All participants were informed that the writing on the photos was plain, ordinary writing without any elements of traces of carbon or similar features that could affect the work of the examiners.

The results were evaluated before and discussed at the project committee meeting in Hamburg, June 1999. Each participant pointed out the problems of working with photographs, as it is impossible to examine e.g. the pen pressure on a photo. However, this limitation did not stop five laboratories from expressing fairly strong conclusions. Two laboratories were somewhat more cautious in their answers, but the context in their reports suggested that they were in agreement with the other laboratories. No participant gave an incorrect answer.

Reviewing the respon­ses on the trial, it was suggested that the level of difficulty could be somewhat higher in a future case meant for the wider group. The overall conclusion was that the trial was a successful exercise. In the opinion of most delegates, the test was better than the latest CTS test and no one mentioned the English language as a problem, though some mentioned caution at dealing with foreign writers.

Other issues discussed at the Hamburg meeting were the idea to involve a control group of lay persons (to test the level of difficulty of the test), the use of original material versus reproductions, the different approaches of laboratories related to the examination of photocopies and other reproductions (limi­tations in material are often reason to formulate weaker conclusions) and the distinction between a "right" answer in the sense of pointing out correctly the writer of a questioned handwriting versus a "right" answer in the sense of formulating a methodologically sound conclusion.

Usually in trial cases a handwriting sample of the author of the questioned writing is part of the comparison material. This might provoke bolder conclusions than would have been made in real life situations. One way to deal with this is to include tests with comparison material that does not contain handwriting samples of the author of the questioned writing (i.e. target absent tests).

It was decided to try out a second trial case using original material. Because of the complicated organisational aspects of circulating a test with original material and the lack of experience in doing this, it was also decided to try out this test with the institutes participating in the project first

The second trial case was produced by the Swedish SKL in co-operat­ion with a mem­ber of the Dutch NFI. In this case, originals of both the questioned material and the comparison material were sent to the participating labs. The degree of difficulty was increased compared to the first case and the amount of both the questioned and the comparison material was limited. This second trial case was sent to the first participant in August 1999. Each participating lab was allowed two weeks to examine the material. The test had to be completed by the end of the year. The participants were asked to include in their reports the line of reasoning behind the conclusions.

The results of the second trial case were evaluated and presented at the Conference in Cracow (Poland), September 2000. The conclusions in this case differed by two steps on the specially constructed scale of degrees. No participant gave an incorrect answer. The level of difficulty was considered sufficient but the material too limited. Some participants also reported that they were more careful with foreign handwriting in general.

In the meantime, Rolf Berzell of the SKL produced the third trial case, this time on signatures. To speed up the procedure he sent out two sets of practically similar signatures. Set A was sent to Portugal, Germany, Norway, England and The Netherlands. Set B was sent to Finland, Sweden, France, Scotland and Northern Ireland. Each participant was allowed two weeks time to study the documents, after which the material had to be sent to the next participant on the list. The participants were asked to draft a full report on their findings using both their own scale of conclusions and the specially constructed scale.

The re­sults of this third trial case were evaluated in the project committee meeting in Lin­köping, Sweden, January 2001. As this case was found suitable by the committee to be sent out to a wider group, it was decided to offer the case to the other ENFSI institutes. All ENFSI institutes were invited to participate in a "Collaborative Exercise" involving handwriting examination. Eighteen institutes accepted the invitation. In co-operation with Jonathan Morris, special instruction and answering sheets were designed for this test, and a timetable was drawn up, allowing each institute 8 working days time for the test. A full report was written on the test and its results, including those of the institutes participating in the test before. The results were presented by Rolf Berzell and Jonathan Morris at the Business Meeting in Rosny-sous-Bois, Paris, 19 October 2001.

For the fourth trial Patricia Pérali was asked to collect French handwriting. She dictated a text to forty people. From the resulting handwriting samples a selection was made to put together two sets of tests. Rolf Berzell and Jonathan Morris tried to make the sets as equal as possible and not too easy. These two sets circulated amongst the institutes of the members of the Project Committee, starting with Finland and Northern Ireland. Enclosed with the test were an information sheet, an instruction sheet and an answer sheet. Again, each institute was allowed two weeks time for the test and participants were requested to report their results using the specific scale mentioned on the answer sheet as well as their institutes' own scale of conclusions.

The results of this trial were presented to the project committee in their April 2002 meeting in Oslo. Concerns were raised about the preparation of the test. After discussing these results there was overall agreement that this test did not contain material based on a normally occurring casework scenario and therefore did not meet the requirements as stated in the guidelines of the ENFSI Quality and Competence Committee. The fourth test was therefore not used for a wider WG exercise.

It was decided to set up a new test with other material collected by Patricia Pérali. After circulating this fifth trial among the participating organisations in the project, it was found suitable to offer it to the wider group.

This "Fifth Collaborative Exercise" circulated between April and September 2002 through fifteen ENFSI organisations that wished to participate. This trial consisted of two, almost identical, sets of material which were circulated through two separate groups of ENFSI organisations. Each group of organisations received one questioned document (identified as Q1) and three sets of specimen handwriting from three individuals (identified as K1, K2 and K3). All of the material consisted of "original" handwriting. Each organisation (or examiner) was asked to compare each set of specimen handwriting with the questioned item and to answer the question "Is there any evidence that the author of each of the sets of known handwriting is responsible for the questioned letter?" The participants were asked to give a full report on their findings using their own conclusion scale and also the five-point scale as indicated on the answer form. The laboratories were given free range to determine whether the reports were to be written by one or more examiners. Each organisation was allowed to keep the originals for two weeks.

Results were obtained from fourteen organisations. These results were presented to the project committee at their September 2002 meeting in Rijswijk. A full report of this trial will be presented in the first 2003 bulletin.

Both Rolf Berzell and Jonathan Morris want to continue to co-ordinate the Collaborative Exercises after the end of the project because a lot of organisations want to participate in it. They will try to invite more organisations from Eastern and Southern European countries. The ENFHEX committee has agreed to support the continuation of the trials programme.

Bryan Found, who is working for the Document Examination Team of the Victoria Police as well as for the Forensic Expertise Profiling Laboratory of La Trobe University, both in Australia, was invited to present a Workshop at the Conference in Cracow. Found et al developed a very interesting programme for characterising and profiling individual and collective expertise in forensic handwriting examination. Police and government handwriting examiners in Australia and New Zealand have participated in developing this testing programme over the past 6 years. Also Manfred Hecker ran a different workshop at Cracow using American workshop material.

Issues related to QA trials (how proficiency tests should be designed, for what purpose they are used, how to handle the results of proficiency tests etc.) are also being discus­sed in the ENFSI Quality and Competence Committee. This committee had meetings in Birmingham (March 1999), Cracow (September 2000), and Paris (March 2002). Rolf Berzell attended these meetings as representative of the ENFHEX WG. The Quality and Competence Committee drafted a Guidance on the conduct of Proficiency Tests and Collaborative Exercises within ENFSI. Future ENFHEX trials will be organised in accordance with this Guidance.