With this idea in mind, we started comparing six services: Repustate, OpenDover, Lymbix, Skyttle, uClassify, Aiaioo Labs and Daedalus, along with Bitext.
Now we have prepared the second step of this comparison: this time we have analyzed the results these services return for a very common linguistic structure: comparative sentences. The example is
"Spider-Man is so much better than Superman".
This sentence contains two different opinions in the same part of the sentence: a very positive opinion about the Marvel superhero Spider-Man, and a very negative opinion about the Man of Steel, Superman. Same thing happens every time you find a comparative sentence: the comparison conveys a positive opinion to one of the members of the comparison, and a negative opinion to the other one.
What is the result of the analysis for this sentence in the different API Demos?
Some trends we have detected:
- Most APIs are not able to identify the two different opinions about two subjects; they just return a positive score for the whole sentence.
- In fact, no API (other than Bitext) detects the negative opinion about Superman (implied, as we have seen, in the comparative sentence).
- Additionally, most APIs don’t detect the intensifier structure "so much".
Therefore, a Sentiment Analysis service must be able to detect this kind of linguistic structures (such as comparative sentences) in order to extract all the relevant Sentiment information from a text.