Sarcasm Text Detection on News Headlines Using Novel Hybrid Machine Learning Techniques

The experiment conducted this research demonstrates the efficiency of all seven machine learning techniques, utilizing two vectorization models. The accuracy and f1-score outcomes for the proposed hybrid technique using the bag-of-words vectorization model are 0.85% and 0.77%, respectively, whereas the outcomes for the proposed hybrid model using the TF-IDF vectorization model are lower at 0.53% and 0.60%. Both accuracy and f1-score for random forest models employing the vectorization models bag-of-words and TF-IDF are 0.84% and 0.76%, respectively. The two machine learning techniques likewise produce results with excellent accuracy, ranging between 80% and 81%. The accuracy of the three machine learning techniques is just 0.73% with adaboost, 65% with xgboost, and 0.56% with gradient boosting, respectively. Naive Bayes and logistic regression both have F1 scores of 76%. The F1 scores for xgboost approaches are lower at 0.62%, 0.61% for adaboost, and 0.60% for gradient boosting.

The rest of this article is organised as follows: A review of the earlier research, upon which this study is built, is provided in Section 2. The collection of the data, dataset preprocessing techniques, the vectorization technique, machine learning models, and performance parameter metrics that convert texts into numerical vectors are all covered in Section 3. Section 4 provides a description of the model results. In Section 5, conclusions are listed, and the future lines of research are considered.

2. Literature Review

To categorise material that is sarcastic, researchers employ a wide range of strategies. Tables 1 and 2 contain information about various relevant studies in more depth.

Farha & Magdy (2020) introduced ArSarcasm, a dataset for detecting sarcasm in Arabic that was produced by reannotating existing datasets for sentiment analysis in Arabic. There are a total of 10,547 tweets in the sample, 16% of which are sarcastic. Modern sentiment detectors degrade when presented with ironic content, according to studies. Finally, they use BiLSTM to train a model for sarcasm recognition.

Kumar et al. (2020) presented a multi-attention-based bidirectional long-short memory connectivity method for detecting irony in sentences. The research findings demonstrate that the effectiveness of BiLSTM is improved by a multi-head attention mechanism, and that it outperforms feature rich SVM classifiers.

Razali et al. (2021) focused on sarcasm detection in tweets utilizing handcrafted contextual cues and characteristics retrieved via deep learning. The feature representation of the CNN architecture was retrieved before it was mixed with meticulously constructed feature sets. These individually designed attribute sets were developed on the basis of their corresponding contextual justifications. Each attribute set has been presented expressly with the goal of detecting irony in mind. The goal is to identify the best attributes. Some sets function properly even when used independently. The performance outcomes of all parameters in the study were favorable. For this assignment, it was discovered that logistic regression is the best classification technique.

The study of Pawar & Bhingarkar (2020) proposed a pattern-based method for detecting sarcasm, using data from Twitter. The analysis of tweets as irony or non-irony employed four sets of attributes that contained several instances of precise sarcasm. The proposed attributes were examined, and new cost classes were evaluated.

Abulaish & Kamal (2018) provided a unique strategy for the identification of self-irony. While approaches were utilised for attribute identification and classification, rule-based strategies focus on identifying potential self-irony tweets. The proposed method was assessed using a Twitter dataset and contrasted with other cutting-edge sarcasm detection techniques.

Liu et al. (2019) simulated face-to-face conversation in their research. The researchers used a method that included auxiliary factors such as syntax, numeric, emoji, etc., to increase classification results. The findings of the research showed that their method performed better in terms of efficiency.

Goel et al. (2022) recognized and comprehended sarcastic behaviour and patterns. This research intended to bridge the gap between the intelligence of humans and machines. The authors used different neural methods to recognize irony on the internet. The dataset was built using various previously trained word-embedding methods, and their prediction accuracy was evaluated to enhance the proposed model's accuracy. The goal was to be able to categories the writer's general attitude as sardonic or non-sarcastic in order to make sure that the intended audience receives the right information. The final analysis demonstrated that the proposed ensemble approach performed better than the other cutting-edge methods.

Nayak & Bolla (2022) examined different computation and machine learning techniques to identify ironic headlines in this study. Experimental tests demonstrated that the recall, accuracy, and F1-score were improved when pre-trained transformer-based embeddings were integrated with LSTM networks.

The study of Yin et al. (2021) proposed a unique network for sentiment classification and irony identification. The network’s modules were used by MT_SS to jointly train for both objectives, acquiring local attribute map and global attribute description using convolutional neural networks. The tests demonstrated that the authors’ proposed approach was superior to cutting-edge methods.

Kumar et al. (2019) presented an approach which was built on a novel hybrid of sAtt-BLSTM and convNet employing GLoVe for creating semantic word embeddings. Using balanced and unbalanced datasets, the proposed model's robustness was examined.

Misra (2022) researched sarcasm detection. The accessibility of contextual tweets was also necessary because many tweets are in reply to other tweets, making it difficult to discern sarcasm without context. The author compiled the news headline dataset. Unlike HuffPost, which provides accurate news, The Onion seeks to produce satirical representations of current events. The author added links to the news items' original sources to make it more useful and enable further data extraction if required. Along with other use cases, besides sarcasm detection, the author discussed several dataset characteristics in this research.

Bharti, Sathya Babu, et al. (2017) identified sarcasm in Hindi tweets. In this manuscript, the dataset was considered when determining the level of sarcasm. The proposed scheme achieved a 79.4% accuracy rate.

Mandal & Mahto (2019) described a novel CNN and LSTM-based sarcasm detection method and considered the sarcasm in news headlines.

The study of Jariwala (2020) proposed that the best features should be chosen before the data is sent for classification. Therefore, data pre-processing cleans up the information to improve the outcomes. The effectiveness of irony detection utilizing the best feature sets was demonstrated by the findings.

Kumar & Katiyar (2019) described the datasets, feature extraction, and method employed in earlier sarcasm detection techniques. The study demonstrated that SVM is the best method for sarcasm detection, the learning algorithm was a crucial step in building a model, and the News Headline dataset was the best one to use for research because it was created by experts.

In the study of Mohammed et al. (2020), effective short text classification techniques were put to the test for the first time in identifying sarcasm. Additionally, detailed output for each technique was shown, based on various research performance criteria.

Ying et al. (2021) explained the precision of categorization for news texts. This research employed a quantitative methodology established by the author. The accuracy scores acquired using the Naive Bayes approach were compared to those achieved using other techniques in this study to assess the method's efficacy. The results showed that the accuracy value did not reach its optimum and could still be improved upon by restructuring and reevaluating the data.

Table 1 presents the findings from these investigations and lists the different methods they had employed. A wide range of methods can be applied to evaluate sentiment on various types of feedback, including Naïve Bayes (NB), support vector machines (SVM), k-nearest neighbor (KNN), logistic regression (LR), decision trees (DT), discriminant analysis, random forests (RF), deep neural networks, long-short-term memory (LSTM), gated recurrent units (GRU), A2Text-Net, convolutional neural networks (CNN), bidirectional LSTM (BiLSTM), and CNN-LSTM. When compared with other methods, all strategies provide optimal results, with SVM, Naive Bayes, Random Forest, and hybrid models providing the best findings for sarcasm recognition in text. The objective of this research is to determine the most effective method for sarcasm detection in texts.

Explain Table 2's methods and the effectiveness matrix that were utilised to get the predictions. Models such as naive bayes (NB), support vector machine (SVM), logistic regression (LR), random forest (RF), decision tree (DT), k-nearest neighbor (KNN), xgboost, and adaboost are used to predict the results, and efficiency matrices, such as accuracy (A), precision (P), recall (R), and F1-score, are also used (F1).

3. Methodology

The primary objective of this study is to identify sarcasm in text using a dataset of news headlines, and to contrast the proposed hybrid strategy with existing methods. The Jupyter Notebook platform, Microsoft Visio for creating diagrams, and Originpro for plotting graphs were used to test the proposed method for detecting sarcasm in text in news headlines. Figure 2 displays the proposed approach. The following six subsections make up this section: data source, dataset preparation, feature extraction vectorization model, machine learning techniques, performance parameters, and proposed algorithm. Below is a brief summary of each subsection.

Figure 2. Proposed strategy

3.1. Dataset Sourcing

On 5 September 2022, the news headlines dataset for sarcasm detection from Kaggle (https://www.kaggle.com/datasets/rmisra/news-headlines-dataset-for-sarcasm-detection) was used in this study. A JSON file containing the dataset was provided. Table 3 shows the dataset containing three attribute.

3.2. Dataset Preprocessing

Preprocessing data sets is crucial to sentiment analysis. The majority of user-generated material is unorganized (Nguyen et al., 2018). The following are the primary actions taken during text normalization:

• Remove special characters and stopwords: Stopwords and many other special characters should be removed in this phase since the accent sign character causes a lot of problems in preprocessing procedures.

• Lemmatize text: The process of getting a word's grammatical root is called lemmatization. Lemmatizing frequently enables computers to distinguish between various tenses of the same word.

• Perform tokenization: Tokenization is a feature extraction method that involves breaking down a sentence’s words into smaller pieces. For this task, the NLTK library's tokenize module has been utilised (Nguyen et al., 2018).

A dataset word cloud was created after the dataset preprocessing phase. Figure 3 shows the dataset's word clouds. The two-word clouds depict words that are both sarcastic and non-sarcastic.

Figure 3. Shows the word clouds of datasets

3.3. Feature Extraction Vectorization Model

To provide statistical or machine learning algorithms with useful features, raw textual data must first be vectorized. Because the procedure produces a collection of numerical vectors as its outcome, it is also referred to as "vectorization." The bag-of-words (BoW) and term frequency inverse document frequency (TF-IDF) models were used in this study.

• Bag-of-Words (BoW): The number of times a word appears in a text is measured using the BoW metric (Thavareesan & Mahesan, 2019).

$$BoW=occurrences of the word "w"$$

• Term Frequency Inverse Document Frequency (TF-IDF): It is determined by multiplying the TF and IDF values, where TF represents the frequency of occurrence in the current document and IDF represents the overall rarity of a term.

$$IDF\left(w\right)=log\left[\frac{Quantity of documents}{Number of documents that include Word w}\right]$$

3.4. Machine Learning Techniques

The methods used in this study are described in detail below.

• Random Forest: It is employed in machine learning to address classification and regression problems. However, organizational contexts represent the main application. Similar to this, machine learning's random forest method creates decision trees from the examples provided before predicting the result that fit the situation the best. By combining the outcomes of several decision trees, the overfitting scenario is typically eliminated. Unlike the single decision tree method, the random forest machine learning technique also processes a wide variety of data outputs (Sentamilselvan et al., 2021).

• AdaBoost: This approach is regarded as the best schema in the machine learning paradigm. Alternative names for this include adaptive boosting. Adaboost is used by researchers.

• Gradient Boosting: For classification and regression problems, it is one of the most widely used machine learning approaches. Using this method, weak estimation techniques can be ensembled (Sentamilselvan et al., 2021).

• Logistic Regression: Similar to the majority of keyword-based algorithms, logistic regression comes in two flavors: classification and regression. In a binary classification framework, logistic regression simply shows the probability that the input will result in the desired output. When using a classifier model, a cutoff value is selected, and the class is determined by whether the likelihood of the inputs is higher than the threshold, in which case they will be placed in one class, or lower than the threshold, in which case they will be placed in another class. The standard threshold value of 10 is employed for this investigation (Razali et al., 2021).

• XGBoost: Today's most popular boosting method is called XGBoost. It is a method of implementing gradient-boosting engines. Other gradient-boosting methods cannot match XGBoost’s speed or accuracy (Bagate & Suguna, 2021). It is one of the ensemble strategies that makes use of the second partial differential equation of the error function, which provides more details about the gradient's direction and how to minimize it.

• Naïve Bayes: It is an improvement of the Bayes theorem and a conditional probabilistic prediction method. The main benefit of a Naive Bayes classifier is that it can categorise a dataset more rapidly and with fewer training examples. The efficiency rate of the classifier is higher than that of other supervised learning methods. In this method, each attribute pair operates independently of the others (Kanakam et al., 2022).

• Proposed Model: The proposed approach is a hybrid one that uses two vectorization models to provide a solution to the sarcasm detection problem. The model uses BoW and TF-IDF to determine whether or not the headline text is ironic. The machine learning technique receives the classification result from both models and learns the appended sentence for categorization likelihood. The vectors are combined and then transmitted identify whether the input is sarcastic.

• In a hybrid approach, different classifiers are used to handle a number of problems and boost accuracy. The method for detecting sarcasm in text is based on random forest and naive bayes techniques. The naive bayes-random forest method is a novel optimal method for categorization issues. It fixes the problems with both strategies. A statistical learning approach called "naive bayes" is based on the bayes rule, and the assumption of independence for detecting sarcasm in text is based on the random forest and naive bayes techniques. The naive bayes-random forest method is a novel and optimal method for categorization issues. It fixes the problems with both strategies. A statistical learning approach called "naive bayes" is based on the bayes rule and the assumption of independence. According to the conditional assumption of independence, the qualities are conditionally independent given the class. As a result, the method is effective and scales effectively. The supervised learning method known as "random forest" uses the bagging operation and the random methodology of subspace. As the fundamental classification model, decision trees were used in this method. To offer variability among the base learners, a random selection of attributes was used. In consequence, on datasets with messy or unimportant data, the model can produce promising learning approaches (Onan & Tocoglu, 2020). And lastly, our proposed solution is more accurate than the other strategies we reviewed. The testing of the proposal demonstrated that it performs better than other state-of-the-art approaches in terms of accuracy.