Pacific Asia Knowledge Discovery and Data Mining Conference (PAKDD) 2016 Contest

The Results

Thank you to all those who made submissions to the task and a big congratulations to the winning teams.

 

 

The Task (provided by Parrot Analytics)

“Sarcasm detection on Reddit comments”

Sentiment analysis is a fairly well-developed field, but on the Internet, people often don’t say exactly what they mean. One of the toughest modes of communication for both people and machines to identify is sarcasm. Sarcastic statements often sound positive if interpreted literally, but through context and other cues the speaker indicates that they mean the opposite of what they say. In English, sarcasm is primarily communicated through verbal cues, meaning that it is difficult, even for native speakers, to determine it in text.

Sarcasm detection is a subtask of opinion mining. It aims at correctly identifying the user opinions expressed in the written text. Sarcasm detection plays a critical role in sentiment analysis by correctly identifying sarcastic sentences which can incorrectly flip the polarity of the sentence otherwise. Understanding sarcasm, which is often a difficult task even for humans, is a challenging task for machines. Common approaches for sarcasm detection are based on machine learning classifiers trained on simple lexical or dictionary based features. To date, some research in sarcasm detection has been done on collections of tweets from Twitter, and reviews on Amazon.com. For this task, we are interested in looking at a more conversational medium—comments on Reddit—in order to develop an algorithm that can use the context of the surrounding text to help determine whether a specific comment is sarcastic or not.

Evaluation

An application will compare a solution file with reddit_test_labels.csv file containing the answers to the test set and results will be presented in an online score board.

Submissions

Please submit your task via email by the 15th of February, 2016.

Contest dates

Start date: 25th Nov 2015
End date: 15th Feb 2016

Prizes (sponsored by Qrious)

  • First Prize: Apple Mac Book + (subject to the conference budget) 1 free PAKDD’16 conference registration
  • Second Prize: New high end Smartphone (to be defined – either Samsung or Apple) + (subject to the conference budget) 1 free PAKDD’16 conference registration

Terms and Conditions

  • The competition is open for all interested researchers, specialists and students. Only members of the Contest Organizing Committee cannot participate.
  • Participants may submit solutions as teams made up of one or more persons.
  • Each team needs to designate a leader responsible for communication with the Organizers. A single person can be a leader of only one team.
  • One person may be incorporated in maximally 3 teams.
  • Each team needs to be composed of a different set of persons.
  • The total number of submissions for any single team is limited to 100 solutions.
  • A winner of the competition is chosen on the basis of the final evaluation results. In a case of draws in the evaluation scores, time of the submission will be taken into account.
  • Each team is obliged to provide a short report describing their final solution. Reports must contain information such as the name of a team, names of all team members, the last preliminary evaluation score and a brief overview of the used approach. The report length should not exceed 2000 words and it should be submitted in the PDF format by the 15th of February, 2016. Only submissions made by teams that provided the reports will qualify for the final evaluation.
  • By enrolling to this competition you grant the organizers rights to process your submissions for the purpose of evaluation and post-competition research.

Organising Committee

  • Muhammad Asif Naeem, Auckland University of Technology
  • Arturas Vedrickas, Parrot Analytics