Sentiment API Market Comparison: Comparative Sentences

In Bitext we continue to make efforts to offer ways to compare the different Sentiment Analysis services in the market that provide a free online evaluation API demo whose terms of use do not prevent its use in comparisons.

With this idea in mind, we started comparing six services: Repustate, OpenDover, Lymbix, Skyttle, uClassify, Aiaioo Labs and Daedalus, along with Bitext.

Now we have prepared the second step of this comparison: this time we have analyzed the results these services return for a very common linguistic structure: comparative sentences. The example is

"Spider-Man is so much better than Superman".

This sentence contains two different opinions in the same part of the sentence: a very positive opinion about the Marvel superhero Spider-Man, and a very negative opinion about the Man of Steel, Superman. Same thing happens every time you find a comparative sentence: the comparison conveys a positive opinion to one of the members of the comparison, and a negative opinion to the other one.

What is the result of the analysis for this sentence in the different API Demos?

Some trends we have detected:

  • Most APIs are not able to identify the two different opinions about two subjects;  they just return a positive score for the whole sentence.
  • In fact, no API (other than Bitext) detects the negative opinion about Superman (implied, as we have seen, in the comparative sentence).
  • Additionally, most APIs don’t detect the intensifier structure "so much".

Therefore, a Sentiment Analysis service must be able to detect this kind of linguistic structures (such as comparative sentences) in order to extract all the relevant Sentiment information from a text. 

Subscribe to Email Updates