- contents -- entities -- signals -- coreference -- relationships -- general guidelines -- recipe -- histopath -- radiology -

The guidelines document history

Revision 2.06

Revision number 2.06
Date 16/05/2007
Description Minor guideline changes; points to emphasise to annotators in "news and updates" page
Source of changes Consensus sets

Section Change
Chemotherpay Added a guideline to deal with annotaiton of chemotherapy
Staging codes Added a guideline to deal with annotation of tumour staging codes
Updates and news section Above points and others added to Updates and news section

Revision 2.05

Revision number 2.05
Date 09/05/2007
Description Points to emphasise to annotators; changes to "news and updates" page
Source of changes Consensus sets

Section Change
Investigation, Intervention Added a guideline to say that where the distinction between an investigation and an intervention is blurred (e.g. biopsy, staging), then an investigation should be annotated.
Typing mistakes Added a guideline to deal with the case where a typist fails to place a space between two mentions
Result Added a guideline to deal with tests that have multiple sub-tests or parameters, such as FBC
Updates and news section Above points and others added to Updates and news section

Revision 2.04

Revision number 2.04
Date 29/03/2007
Description Added item to updates and news: reminder on setting the annotator name in Knowtator
Source of changes n/a

Revision 2.03

Revision number 2.03
Date 27/03/2007
Description Added an updates and news section
Source of changes n/a

Revision 2.02

Revision number 2.02
Date 09/03/2007
Description Points to emphasis to trainee annotators; small number of changes
Source of changes Discussions with annotators; debug batch 5; training batches 3 and 4

Section Change
Intervention Please be sure to check what is and what isn't an Intervention. In particular "seeing" a patient is not an intervention, neither do we want general, non-specific words like "treated" to be annotated as Interventions.
Signals must always modify something Signals (Negation, Laterality, and Sub-location) must always modify something. They cannot stand on their own. In the case of Negation, this is a condition. For the other two, usually a locus. Please use Step 3 of the annotation recipe when you mark signals, add their "modifies" relation. Do not wait until you are annotating other relations to do this.
Annotating relationships Please read the general guidelines on annotating relationships. In particular, please try to only annotate those relationships that the text is telling you about. Often, such relationships are clearly stated. Sometimes, the text is saying something, but it needs some clinical knowledge interpret this and to decide on the relationship. This should not mean, however, that you try to deduce every single relationship between every single entity, regardless of whether the text is saying something about it. We are only interested in what the text is telling us. Please read the general guideline on this.
Recipe: co-reference Annotators are asked to take particular care with co-reference. Please look for co-reference as a separate step, before dealing with other relationships. The guidelines recipe has been updated to include this extra step.
Negation values, Laterality values Please make sure that you fill in the values for Laterality (right, left or bilateral) and Negation (absent or uncertain)
Sub-location Several more stock phrase examples have been added for Sub-location
Co-reference of sets Coreference annotations should not be created between things and sets that they are members of. See guideline for examples.
General guidelines: sets Note about annotating sets of things added.
Drug or device: general drug classes Note about coreference of general drug class terms added.

Revision 2.01

Revision number 2.01
Date 27/02/2007
Description Small number of changes post debug run 4. Used for debug run 5. Issued to annotators during annotation of training runs.
Source of changes Debug run 4

Section Change
Complex loci Annotating complex loci and sub-locations can be confusing. A tabulated recipe with examples has been added to help clarify this. Please try to use this recipe when faced with complex loci and sb-locations
Negation Note that "?" (meaning query) is a valid negation signal, with value uncertain. Added example to guidelines
Result in histopathology There are various ways of expressing positive histopathology stain and other test results, as well as the word "positive". Examples added.
Condition "Abnormality" is a valid Condition. See the example added in the Conditions section, sub-section "General terms for problems and diseases"
Condition, Condition in histopathology Cellular processes such as "transformation" and "prolifeation" should be marked as Conditions. See the relevant conditon section, and also Histopathology reports.
Histopathology reports: Locus Added further examples of cellular and tissue loci.
Sublocation modifies relationship Note that sub-location may modify more than one Locus. This is not currently possible in the Knowtator tool, but will be added shortly
Signals: modifying entities Signals modify nearby entities in the same phrase. They are often miss-annotated. The general guidelines section on signals has been expanded to discuss this
Sub-location Sub-location is about general language area words such as "upper" and "inner" that modify anatomy, and should not be used to annotate other anatomical terms. Further examples added to the relevant Sub-location section.
Coreference Please note that this section has been rewritten (and hopefully clarified) since Revision 1.43. This especially affects those annotators working prior to 16 February. Please re-read this section, if you have not read it recently.
Negation Please annotate the entire negating phrase. See the example in this section. Please note that there has been a small clarification of this section since Revision 1.43. This especially affects those annotators working prior to 16 February. Please re-read this section, if you have not read it recently.

Revision 2.00

Revision number 2.00
Date 15/02/2007
Description First multiple page version. Used for Manchester annotator training
Source of changes Guideline training with Archana; initial Sheffield debugging

Section Change
TheGuidelines Some rarely used technical sections removed
TheGuidelinesDrugOrDevice Add an example to body substance section
TheGuidelinesCoreference Add a note about co-references to conjunctions
TheGuidelinesModifiesLaterality Add a note about which of several following entities a laterality should modify
TheGuidelinesCondition Add a note about other people's conditions, mentioned in a patient's document
TheGuidelinesRecipe Brought more in line with training video
TheGuidelinesLocus Add an example of a physiological function as a locus
TheGuidelinesGeneral Note on using paper dictionaries

Revisions 1.00 to 1.43

Revision Date Description
Revision 1.43 08 Feb 2007 - 09:42 Last revision prior to splitting across several files
Revision 1.41 06 Feb 2007 - 17:40 Changes in response to debug run 3. Version sent out for debug run 4. Also used for Sheffield internal debug run.
Revision 1.38 19 Jan 2007 - 14:08 A few incidental changes. "Developing the corpus" section removed: now documented elsewhere.
Revision 1.31 02 Jan 2007 - 11:59 Further changes made in response to debug iteration 2: further comments from annotators
Revision 1.30 30 Dec 2006 - 17:31 Changes made in response to debug iteration 2, difference analysis and initial comments. Version sent to annotators for 3rd iteration.
Revision 1.26 18 Dec 2006 - 18:08 Small number of outstanding issues from dry run error analysis, that were not caught in previous discussions and comments. No substantial changes, mainly the addition of examples to clarify.
Revision 1.24 18 Dec 2006 - 10:57 Issues from dry run comments incorporated. Version sent to annotators for second iteration.
Revision 1.22 15 Dec 2006 - 15:50 Decisions and discussions from December 2006 consortium meeting incorporated. Many draft comment boxes that related to these discussions also removed.
Revision 1.18 07 Dec 2006 - 16:45 Version circulated prior to December 2006 consortium meeting. Minor changes plus some introductory paragraphs.
Revision 1.14 01 Dec 2006 - 16:49 Version sent to Manchester for annotation dry run. All initial Sheffield and Manchester comments incorporated.
Revision 1.11 22 Nov 2006 - 21:32 Draft prior to adding in initial Sheffield and Manchester comments

Notes on revision numbers and change history

Prior to revision 2.00 the guidelines existed as a single monolithic wiki page. This could be tracked via the wiki revision system. The revision numbers used are the same as the wiki revision numbers.

After revision 1.43, the guidelines moved to multiple wiki pages. Each page has its own wiki revision numbering, and so it is not possible to keep track of a single revision for the the main guidelines topic and all of its sub-topics in this way. Therefore, a new revision number has been adopted covering the whole of the wiki guideline site in its multiple pages. This revision number starts at 2.00 and is independent of any wiki revision number.

For the single page revisions 1.00 to 1.43, changes can easily be tracked via the wiki diff feature. For revisions 2.00 onwards, notes on the main changes are given above. Minor changes and clarifications are not included.