ADCAIJ: Advances in Distributed Computing and Artificial Intelligence Journalb
Regular Issue, Vol. 13 (2024), e31602
eISSN: 2255-2863
DOI: https://doi.org/10.14201/adcaij.31602

A Systematic Analysis of Various Word Sense Disambiguation Approaches

Ganesh Chandra^a, Sanjay K. Dwivedi^b, Satya Bhushan Verma^c, Manish Dixit^a

^a Department of Computer Science & Engineering, Madhav Institute of Technology & Science (Deemed University), Gwalior, M.P, India

^b Department of Computer Science, Babasaheb Bhimrao Ambedkar (A Central) University, Lucknow, UP, India

^c Computer Science & Engineering, Shri Ramswaroop Memorial University Lucknow Deva Road, Barabanki, India, 225003

✉ ganesh.iiscgate@gmail.com, skd200@yahoo.com, satyabverma1@gmail.com, dixitmits@mitsgwalior.in

ABSTRACT

KEYWORDS

1. Introduction

Language is a powerful communication medium for sharing of information. The word or phrases in any language having more than one meaning is known as ambiguous. For example, in the sentence «what is apple rate» here the word apple has two meanings, apple fruit or apple phone. Nowadays majority of people used web for searching of information. Searching may suffer if ambiguity occurs. Ambiguity is the problem of natural language processing (NLP) which can change the sense of word used in the sentence or query in different ways (Kaplan, 1955; Zipf, 2013; Chandra et al., 2019; Chandra et al., 2020).

Word sense disambiguation (WSD) is the technique of identifying the correct sense (meaning) of word, if word have more than one meaning or sense in a sentence (Weiss, 1973; Madhu et al., 1965; Resnik et al., 1999; Brown, 2005; Chandra et al., 2014). WSD is highly demanding technology in various applications such as information retrieval (IR), information extraction, machine translation (MT) (Masterman, 1961), question answering system, cross lingual applications, and document classification. WSD uses corporate, training, and lexical databases to provide the correct meaning of particular word in a given context (Edmonds et al., 2007).

The general structure of WSD is shown in figure 1 where several meanings are obtained from various resources for ambiguous word. The disambiguation algorithm is applied on obtained results to fetch the correct meaning. WSD research began in the early days of machine translation in the late 1940s as a distinct computational task. In 1949, the problem of WSD in computational context was firstly introduced by Warren Weaver (1949). Later, Bar-Hillel (1960) stated that the problem of WSD cannot be easily solved by computer (Bar-Hillel, 1960). The problem of WSD was associated with AI in 1970s as subtask of semantic interpretation system. By the 1980s, WSD research reached at their turning point due to the development of lexical resources such as Oxford Advanced Learner’s Dictionary of Current English (OALD). Lexical resources are used to extract knowledge using automatic method. In the 1990s, the development of Word Net, sensval and statistical revolution in NLP moves in the direction of computational linguistics. During this period, WSD research started to use the supervised learning technique. During 2000s, research moved towards coarse-grained senses, domain adaptation, semi-supervised, unsupervised, a combination of different methods or approaches, and knowledge based on graph-based methods.

The paper is organized as follows: Section 2 describes WSD approaches, Section 3 presents the resources of WSD, Section 4 describes the importance and applications of WSD, Section 5 describes the challenges of WSD, and finally the paper is concluded in Section 6.

2. WSD Approaches

In the field of information retrieval, WSD plays an important role. Firstly, improves the retrieval precision by identifying the correct meaning against an ambiguous word in both query & document. Secondly improves the recall results by improving the relationship between the query words and which are not in query. WSD is highly useful in improving the quality of retrieval results. In WSD, the matching is performed between sense knowledge vector and context vector for word disambiguation.

In order to solve the issue of ambiguity, the various approaches used for WSD are knowledge-based, supervised (Niu et al., 2005), unsupervised (Leacock et al., 1998) and semi-supervised. These are described in the subsections below. Table 1 shows the previous research work on WSD.

2.1. Knowledge Based Approach

In this approach, the various resources such as dictionary, thesaurus, WordNet, SemCor, Wikipedia etc. are used to identify the appropriate meaning of word in a context. This approach came into existence in between 1970s to 1980s (McInnes, 2009).

Figure 2 describes the general structure of knowledge-based approach. For disambiguation of words, this approach uses hand coded rule or grammar rule. The algorithms used in this approach are selectional preference, Overlap of Sense Definitions.

This approach either uses hand coded rule or grammar rule for the disambiguation of words and uses the sense whose definition is most similar to the context of the ambiguous word. The various knowledge-based approaches are described as follows (Jin and Chen, 2013).

2.1.1. Selectional Preferences

The knowledge-based method of WSD frequently uses selectional preferences as a source of linguistic information (McCarthy et al., 2003). This approach finds the information related to the relationship between different types of word categories. Although, it is important process of disambiguation but its restricted semantic significance makes it less successful. This is not about the individual word form for disambiguation, rather it is specific to verb or adjective classes. The basic assumption made by selectional preferences is that the distinct senses of the same predicating word (verb or adjective) will have different selectional preferences with respect their arguments (noun). The simplest way of understanding the selectional preferences is to determine the semantic association provided by word-to-word relationship Verma, S. B., & Saravanan, C. (2018, September).

In Figure 3, selectional preferences are obtained from automatically preprocessed and parse text. The parser is applied to data to identify the grammatical relationship between nouns, verbs, and adjectives. Selectional preferences are acquired for the grammatical relationship between nouns, verbs, and adjectives using WordNet synsets to define the sense inventory Verma, S. B., Yadav, A. K. (2021).

2.1.2. Overlap of Sense Definitions

Overlap of sense definition is a knowledge-based approach depends on the calculation of word overlap between the sense definition of target words (two or more). This is also called as Lesk algorithm or Gloss overlap (Lesk, 1986) which utilizes two types of bags: sense and target bag. Sense bag contains the words in the definition of a candidate sense of the ambiguous word and context bag contains the words in the definition of each context word. A machine-readable dictionary (MRD) is used in this algorithm and its basic function is to identify an overlap between the features of different sense of an ambiguous word and the word found in its context. Here, weight is assigned to each sense and the sense that has the maximum overlap value is nominated as the contextually appropriate word (Pedersen, 2007).

2.1.3. Structural Approaches

The structural or syntactic approach is used to represent the hierarchical (tree-like) structure (as shown in Figure 4). Here, structural information is used for the recognition and measurements of patterns, both in a local and global context (Bunke et al.,1990). Its aim is to classify data based on the structural interrelationships of features.

2.1.4. Discussion

An advantage of the knowledge-based approach over the other approaches of WSD is that data is not required for each word that needs to be disambiguated. Another advantage of this approach is that it allows the systems to disambiguate the words in running text, which is referred to as all-words disambiguation. An all-words disambiguation method is more important than lexical-sample disambiguation methods because lexical-sample methods can only disambiguate words for which there is a large set of training data. All-word disambiguation methods are scalable and can be used in real-word practical applications. The disadvantage of the knowledge-based approach is that it is language- and domain-dependent because a knowledge source is required in the appropriate language and domain (Sammut and Webb, 2010).

2.2. Supervised Approach

The supervised learning approach of WSD uses a machine learning technique to induce a classifier from manually sense-annotated datasets. Classifiers (word experts) are used to assign the correct sense of each word for classification in the supervised approach. Supervised approaches have two phases: the training phase and the testing phase (Grozea, 2004). In the training phase, a sense annotated training corpus is used by classifiers to extract the semantic and syntactic features. In the testing phase, the classifiers aim to find the most suitable sense of each word based on surrounding words in the sentence. The supervised approach includes decision lists, naïve Bayes, decision tree and neural networks (Verma et al., 2023).

In Figure 5, the instance is the context in which the target word is used. The ambiguous word that occurs in a sentence is known as the target word. The different meanings of a target word are referred to as senses or concepts which are obtained from a sense inventory or concept inventory. Manually annotated training data is used as input for an evaluation module that divides the data into training and testing data. The vector creation module extracts the features from training data and produces vectors (training vector and test vector). The training vector is used by the supervised learning algorithm as input. The supervised learning model is created by the supervised learning algorithms which take the test vector as input to assign a suitable concept to each of the vectors. The result obtained from supervised learning becomes an input for the evaluation module, where the accuracy of the model is calculated (Chandra and Dwivedi, 2020a).

2.2.1. Decision Lists

The decision lists, first described by Rivest (1987), are an ordered set of rules in a list format which is based on a set of weighted if-then-else rules. In 1994, a decision list was first used by Yarowsky (1994) on the senseval corpus. Decision lists are based on a probabilistic measure where features are extracted from the set of the training corpus. The features that are extracted in the training phase of the WSD algorithm are: part-of-speech, semantic and syntactic features, collocation vector and co-occurrence vector. Once the features are selected, different kinds of rules (features-value, sense, score) are created, and applied in the decreasing order of score to form a decision list.

2.2.2. Naïve Bayes Method

The naïve Bayes method is considered under the probabilistic approach based on the application of Bayes theorem (Berrar, 2019). The conditional probability is calculated for each sense of word for the features given in context. The naïve Bayes classifier uses a large context and tests each word around the target word where each word provides some information for the identification of the sense of the target word. The naïve Bayes classifier mostly assigns the maximum probability to the correct class for accurate results. In this model, texts such as sentences or documents are represented in the form of bag (multiset) of words without using grammar rule and the order of words depends on multiplicity.

2.2.3. Decision Trees

Based on prediction model, decision trees were introduced by Quinlan (1987). In decision tree, a sense-tagged corpus is used as resource on which training is performed. In decision tree, a classification rule is applied in the form of a «yes-no» rule. With the help of these rules, the training dataset is partitioned recursively. Each internal node of a decision tree represents a feature on which experiments are conducted, each branch of a decision tree represents a feature value, and an external node represents the sense. The feature vectors of a decision tree are the same as the feature vectors of a decision list, for the disambiguation of a word in the testing phase, each feature vector is traversed through the tree to reach the leaf node. The sense contained in the leaf node is considered as the correct sense of word. The most popular algorithms used for the generation of a tree are: ID3, ID4, ID5 or ID5R and C4.5 algorithm (Quinlan, 1987).

An example of a decision tree for WSD is reported in Figure 6. For instance, if the noun bank must be classified in the sentence «we sat on a bank of sand», the tree is traversed and, after following the no-yes-no path, the choice of sense bank/RIVER is made. The leaf with empty value (-) indicates that no choice can be made on the basis of those specific feature values.

2.2.4. Neural Networks

A neural network model was proposed by McCulloch and Pitts (1943). It is an interconnected group of artificial neurons that uses a computational model, based on a connectionist approach, for the processing of data. In this approach, pairs of input features and desired responses are used as input to the learning program. The different types of neural network models include the perceptron network model, the feed-forward network model and the recurrent network model. Out of these models, the perceptron model is used for the purpose of WSD. The perceptron model is based on the hidden Markov model (HMM) and the backpropagation feed-forward networks. The purpose of this approach is to partition the training contexts into non-overlapping sets for the desired responses using input features. Major problems occur in neural networks, namely, difficulties in interpreting the results, the need for a large quantity of training data, and the tuning of parameters such as thresholds, decay, etc.

Figure 7 represents the multilayer perceptron neural network (input layer, hidden layer, and output layer) which has three features and outputs the two senses of a target word in context.

2.2.5. Support Vector Machines

Support vector machine (SVM) was proposed by Boser et al., (1992) and it plays an important role in classification problems. SVM are based on the structural risk minimization principle. SVMs are a linear classifier that produces the hyperplane for separating the positive and negative training examples (called support vector) with largest margin. Nowadays, SVM is also used to solve the problems of pattern recognition, image recognition and NLP-related problems such as chunking, parsing and WSD etc.

2.2.6. Discussion

Word sense ambiguity is considered a central problem among NLP researchers across different languages. To solve this problem, many international research groups are actively working on WSD approaches. The supervised approach uses semantically annotated corpora to train machine learning algorithms to decide which sense of word fits which contexts. The words in these annotated corpora are manually tagged with semantic classes taken from a particular lexical semantic resource. To improve the performance of current supervised WSD systems, the set of features with linguistic knowledge must be enriched, however, this is currently not available in wide-coverage lexical knowledge bases. In general, supervised learning methods produce very high disambiguation accuracy in comparison to other WSD methods. The disadvantage of supervised learning is that it requires manually annotated training data for each word that needs to be disambiguated. This is a labor intensive and time-consuming process. The major challenges found in this approach are data quality and data selection (Ng, H.T, 1997).

2.3. Unsupervised Approach

In the supervised approach, the training data must be manually created, which is very expensive (Wang, 2006). This problem is known as knowledge acquisition bottleneck. The unsupervised approach solves this problem by introducing the concept that the sense of a particular word, depends on its neighboring words. The unsupervised WSD approach divides the occurrence of a specific word into a number of classes in order to decide whether the occurrence of that word has the same sense or not. The important task of this approach is to identify sense clusters. The different types of methods used in the unsupervised approach are context clustering, word clustering and co-occurrence graph (Chen et al., 2012). In the context clustering method, clusters are created by finding the co-occurrence of a word with the target word. In the word clustering method, words are created according to their semantic nature (e.g., subject-verb, adjective-noun etc.). In the graph-based method, a graph is created with the help of a grammatical relationship (Navigli, 2010).

2.3.1. Context Clustering

In the context clustering method, every occurrence of a target word is represented as a context vector in corpus. These vectors are grouped into clusters for the identification of the sense of the target word. The most famous context clustering algorithm was proposed by Schutze, where the occurrences of an ambiguous word are grouped into clusters of senses based on the contextual similarity of their occurrences.

Figure 8 represents the general model of the clustering method. An evaluation module takes the test data as input for evaluation and then sends this data to the vector creation module. The test data and unannotated training data is used by the vector creation module as input. A vector is created by the vector creation module for each instance of testing data and training data. The training vector is grouped together into clusters using clustering algorithms in discrimination module. The test vectors and clusters are used by the disambiguation module as input to assign a concept for each cluster using assignment algorithms. The concept whose vector is closest to the test vector is assigned to the target word. After that, the accuracy of the system is calculated by evaluation module SBV, BP, and BKG (2022).

Advantages: