System Architecture​

The system uses two separate classifiers, one for phrase-level sentiment and one for tweet-level.

Features

​

Both classifiers use the following features. For a text span (phrase or tweet)

  • presence or absence of:
    • raw bag-of-words (BOW) unigrams
    • normalized/stemmed BOW unigrams
    • stemmed segmented hashtag BOW
    • user mentions
    • URLs
    • hashtags;
  • number of question marks and number of exclamation points
  • number of positive, negative, and neutral emoticons
  • whether the text span contains and elongated words

The phrase-level classifier also uses the following additional features:

  • normalized BOW unigram features derived from 3 tokens preceding the target phrase;
  • normalized BOW unigram features derived from 3 tokens following the target phrase;
  • length 2, 3, and 4 character prefixes and suffixes for each token in the target phrase;
  • whether the phrase was in all caps;
  • whether phrase contained only stop words;
  • whether a phrase contained only punctuation;
  • whether the phrase contained a word whose length is eight or more

Lexicons

We used the features derived from the following lexicons: AFINN, Opinion Lexicon, Brown Clusters, Hashtag Emotion, Sentiment140, Hashtag Sentiment, Subjectivity, and General Inquirer.

We used the following features for lexicons that provided scores:

  • the average sentiment score for the text span
  • the total number of positively scored words in the span
  • the maximum score (or zero if no words had a sentiment score)
  • the score of the last positively scored word
  • three most influential (most positive or most negative) scores for the text span; this was only used by the phrase-level system

Spelling Correction

  • Non-prose
    • URL, mention, number, emoticon, proper noun
  • Abbreviations
    • “2moro”, “omg”
  • Elongated Words
    • “heyyy”
  • Colloquial
    • “lol”, “haha”, “hahaha”, ...
  • Else: PyEnchant spell checker