Multi-label Text Classification Approach for User Profiling in Chat Messages

Social media and Instant Messaging applications have become popular among teenagers, making them vulnerable to pedophiles. They approach their victims using fake identities, posing as teenagers to facilitate grooming them. An algorithm to detect offender profiles might help Law Enforcement Agencies (LEAs) to discover potential pedophiles by analyzing chat messages.

Several researchers have addressed the problem of user profiling, trying to identify different profile features, including age and gender. Initial solutions employed Support Vector Machine and Naïve Bayes classifiers with n-gram for feature extraction. More recent techniques use Transformers architecture like BERT. However, analyzing short and low-quality text (i.e., spelling mistakes, abbreviations, emojis) as grooming chat messages to profile users make the problem even more challenging.

In this work, we use a BERT model as a multi-label text classifier to recognize three user profiles from the chat messages. The model digests a number of chat messages and predicts the age group, gender, and potential offenders. For training and testing the model, we used the ChatCoder dataset that consists of 57 chat transcripts collected from the Perverted Justice website in April 2010, with a chat length between 83 and 12704 messages. Besides, we analyze the relation between the size of the chat message and the classifier’s performance by feeding it with 1, 5, 10, 15, and 20 concatenated messages for training and prediction.

Our experiments show that when the text is short, i.e., one message, the performance drops significantly to a 0.65 F1 score. However, when the model is fed with more chat messages, i.e., 5, 10, 15, and 20 concatenated messages, the F1 score increases to 0.70, 0.87, 0.93, and 0.94, respectively.

Our proposal for recognizing user profiles from the chat messages could be a valuable asset for helping LEAs profile users via text analysis.