Annotation of conceptual co-reference and text Mining the Qur'an

This research contributes to the area of corpus annotation and text mining by developing novel domain specific language resources. Most practical text mining applications restrict their domain. This research restricts the domain to the Qur'anic Text.

In this thesis, a number of pre-processing steps were undertaken and annotation information were added to the Qur'an. The raw Arabic Qur'an was pre-processed into morphological units using the Qur'anic Arabic Corpus (QAC). Qur'anic terms were indexed and converted into a vector space model using techniques in Information Retrieval (IR). In parallel, nearly 24,000 Qur'anic personal pronouns were annotated with information on their referents. These referents are consolidated and organized into a total of over 1,000 ontological concepts. Moreover, a dataset of nearly 8,000 pairs of related Qur'anic verses are compiled from books of scholarly commentary on the Qur'an. This vector space model, the pronoun tagging, the verse relatedness dataset, and the part-of-speech tags available in QAC all together served for a number of Qur'anic text mining applications which were rendered online for public use. Among these applications: lemma concordance, collocation, POS search of the Qur'an, verse similarity measures, concept clouds of a given verse, pronominal anaphora and Qur'anic chapter similarity.

Furthermore, machine learning experiments were conducted on automatic detection of verse similarity/relatedness as well as categorization of Qur'anic chapters based on their chronology of revelation. Domain specific linguistic features were investigated to induct learning algorithms. Results show that deep linguistic and world knowledge is needed to reach the human upper bound in certain computational tasks such as detecting text relatedness, question answering and textual entailment. However, many useful queries can be addressed using text mining techniques and layers of annotations made available through this research. The works presented here can be extended to include other similar texts like Hadith (i.e., saying of Prophet Muhammad), or other scriptures like the Gospels.