Fake Review Detection Using ML

Procedures and Functions (Postgresql)
October 24, 2023

 

This model is designed to identify fraudulent reviews based on the dataset it was trained on. Due to limitations in computational resources, the model was trained using the dataset provided below. Having tested this model on different platforms,  it performs well with Amazon reviews. Simply input a review, and the model will assess its authenticity, determining whether the review is real or fake.

How to test this model.

  1. Go to this huggingface page to copy a review from the dataset. Astrobd Fake Review Dataset.
  2. Locate the 'text' column and copy any of the reviews there. For example, you can choose any of these reviews:
    1. Real Review: "I really like this thermal. I don't put it in the microwave, though. I heat the water and then pour it in the thermal. I take it with me out in the cold and tea stays hot for hours. It won't spill in my bag at all! Just love it."
    2. Fake Review: "I enjoyed this book, kept reading it and hoping it would continue. I have read all the books in"
    3. Real Review: "This crate has been great for our 3 month old Corgi. He takes up about one-third to one-half of the crate right now, so we have been using the divider. It's great that we will still be able to use this crate when he is full size."
    4. Fake Review: "Bought as a gift. We have had the wooden trains for a few months now and this one is a favorite. The plastic trains are very sturdy, and the wood is very sturdy. The wooden train is a good"
  3. Paste the review and then "click to submit".  Validate the model's classification using the 'label' column on the dataset.

Label:  Identifies the review as real/fake.

Text: The review itself.

Please write a review and click submit. If you encounter an error, please try again as the server may be booting up


    










 How is this different from other fake review detection models ?


 Most fake review detection models that I've come across online,  rely on traditional machine learning techniques such as logistic regression, random forest, SVM, and k-NN. Although these methods are effective, this project explores the use of transformers to compare performance with  these conventional approaches. In this project, I experimented with several transformer models including GPT-2, BERT, Electra, RoBERTa, DistilBERT, ALBERT, XLNet, BART, DeBERTa, GPT-2, and FunnelTransformer.


 


Approach


Detecting fake reviews does not follow a one-rule approach, as algorithms can always be fooled. However, psychological insights into fake review patterns can improve the accuracy of fake review detection systems by considering factors such as writing style, language patterns, and inconsistent sentiments, which are grounded in psychological research.


 


Importance


This system is of special interest due to its significance for businesses. These days, virtually every business has an online presence, and reviews play a significant role in determining the integrity, consumer trust, and future behavior of these platforms.


The choice of this system is motivated by several factors. Firstly, the reliance on reviews has made fake reviews a pervasive problem, undermining consumer confidence and distorting market dynamics. Secondly, fake online reviews influence over $791 billion in annual eCommerce spending in the US (Marciano, 2021). Thirdly, advances in machine learning and natural language processing present an opportunity to develop algorithms capable of distinguishing between real and fake reviews with high accuracy.


This detection system aims to inform the development of feasible machine learning solutions that will aid in the detection and mitigation of fake reviews. By integrating these solutions, the practical deliverables can significantly enhance the integrity of online platforms, foster transparency and trust, and protect consumers from misguided reviews.
“You rarely come across a fake 3-star review” (Das, 2023).


We can get more insights into the authenticity of reviews by examining reviewer’s behaviour.








Real Reviews




Fake Reviews








Reviews are posted at regular intervals 




Reviews are posted at high frequency in a short period of time, and sometimes at odd hours








Reviewers post reviews on a variety of products spanning different categories




Reviews are often posted in similar categories of similar products and similar brands. 








Balanced distribution of ratings. 




Extreme rating behaviour. Often 1 Star or 5 Star.






Hypothesis


Long reviews are usually more trustworthy than short ones. They have a higher probability of providing more details. A clue to detecting fake reviews is that they tend to be short and generic, without offering many specific details (Fairbank, 2024).


The issue of detecting fake reviews is multifaceted as it involves natural language processing, machine learning and analysing behaviours. 


Mapping reviews to geolocation


Mapping reviews to geolocations and IP’s can help produce better classification results. Texts are easy to manipulate but it is more expensive to exploit IPs and geolocations. For instance, similar fake reviews can be tagged with flagged proxies from virtual IP providers, real reviews often have distinct geolocations and IP’s. By having a pool of fake and real IPs, our model can find patterns to differentiate fake reviews from real ones. So, let’s say a reviewer decides to buy fake reviews, there is a high probability that these reviewers will use IPs that have already been flagged or those with similar subnets within a fllagged pool of fake ip's. 


  








Reviewer’s IP




Country




Type of Review








100.943.548.583 (Virtual)




United Kingdom




Fake








100.943.548.643 (Virtual)




United Kingdom




Fake








100.943.548.688 (Virtual)




United Kingdom




Fake








85.456.092.688 (Distinct)




United Kingdom




Real








134.765.726.094 (Distinct)




United Kingdom




Real








 


 


 


 


 


 


In the table above, we can see that the first three IP’s have a similar pattern (100.943.548…) while the last two are distinct (85.456.092.688, 134.765.726.094). Please note: The Ip's above are for illustration purpose(s).


 
  • Logistic Regression, Multinomial Naïve Bayes, Random forest Preprocessing For Traditional Machine Learning Algorithms 
    Removal of punctuations and other irrelevant characters, Removal of Stop Words, Lemmatization, and Tokenization

  • Review FeaturesFeature Engineering
    Bag of Words, Sentiment Analysis and determining other lexicon features
  • Train-test split, no. of epochs, loss functions and other relevant metrics  for fine-tuning the modelsTraining and Fine-tuning Models
    Training multiple models on the dataset.
  • Cross Validation ApproachTesting and Validation
    Train Overall Accuracy, Test Overall Accuracy, Test Positive Accuracy, Test Negative Accuracy, Precision, Recall, F1, ROC-AUC


Traditional ML


For completeness, I started this project by experimenting with traditional ML models to compare accuracy values. These models include Logistic Regression, Multinomial Naïve Bayes, KNN, and Random Forest.


The accuracy values ranged between 85% and 90%.
Neural Networks


Neural networks had a better performance. I achieved a test accuracy of 91% with ANN.  The process involves using Adam optimizer as the optimizer, binary cross-entropy as the loss function, sigmoid for activation, and two dropout values of 0.5.
Transformers


 After experimenting with different transformers, FunnelTransformer had the best performance, achieving a 96% test accuracy and a 94% train accuracy.
If you're interested in accessing the complete code used to train this model, please feel free to reach out via the Contact Me form on this website.