We had a conversation this week with a French multinational information technology services and consulting company. We explained to them what we do and we also had a little time to run a demo showcasing our natural language understanding technology Percipion. They were interested in one thing: be able to assess sentiments. They told us that for them this technology is not working and that they do not believe that machines are able to assess content provided in natural language to identify those specific psychological states. For them, this technology has no value at all. Many of their clients are asking after this but since they do not believe in this technology, they provide it for free!
And yes, they are right. Machines are not able to assess sentiments!
We came to the exact same conclusion as we tested those who today in 2021 are available on the market. But our technology is different. Our approach is different. The foundation of our Cognitive AI engines is different. We told them that our technology is working and they requested us to prove this.
It is important at this stage to explain what we want to achieve. Today, in organizations, many processes are executed by people. The reason is that those processes require human intelligence. To assess the voice of the customer material, reviews or feedback and to be able to understand the sentiments that are enclosed, a human eye is necessary.
We took from the Internet a customer review, written in French, and assessed sentiments. The original verbatim is the following. “J’ai enfin reçu mon nouveau SmartPhone. Quelle déception. J’suis furraxe! Ce qui m’agace le plus c’est sa lenteur. J’ai essayé d’installer des applications et franchement c’est cauchemardesque! Aucun plaisir à utiliser ce truc. Je vais le retourner dès aujourd’hui mais je reste inquiete sur la qualité du SAV. En résume, je ne suis pas du tout contente de cet achat! Une vraie merde !!!”
This could be translated as : “I finally received my new SmartPhone. What a disappointment. I’m pissed off! What bothers me the most is how slow it is. I have tried installing apps and frankly it’s a nightmare! No fun using this stuff. I will return it today but I am still worried about the quality of the after-sales service. In short, I am not at all happy with this purchase! A real shit !!!”
What is your opinion? How do you, as a human, understand this? Is it positive? Is it negative? We think that the content of this verbatim is all negative.
At MacAnima, our vision is that tomorrow, the same processes will be executed by people and by machines. To be able to do this, we will have to create a new generation of artificial intelligence able to replicate human intelligence. At MacAnima, this is what we do. We do Cognitive AI. If we are successful in what we do, our technology should be able to understand the verbatim that we took as an example the same way that humans do understand it. As easy as that.
The first thing that we did was to assess this French verbatim using natural language understanding technologies provided by Microsoft, Google and IBM. These companies are leading the competition.
Let’s talk about Microsoft. Their AI concluded that the content of this verbatim is neutral, neither positive or negative! Some words which do express “sentiments” have not been recognized as such (“déception”, “furraxe”). Please note that the word “furraxe” was misspelled within the original verbatim. Maybe this is the reason why this word has not been recognized. That said, the AI from Microsoft could have been smart enough to identify this word as misspelled. The last thing is that we had to take out of the verbatim the sentence “une vraie merde” (“a real shit”) since they refused to assess the provided content considering the presence of unwanted words!
Let’s talk about Google. Their AI concluded that the content of this verbatim is to 30% negative! The sentence “J’suis furraxe” (“I’m pissed off”) was considered as neutral as well as the sentence “Une vraie merde” (“A real shit”). We do believe that the machine here misunderstood the real meaning of those words!
Let’s talk about IBM. Their AI concluded that the content of this verbatim is 59% negative! The misspelled word “furraxe” was recognized as being negative which is correct and which is also a sign that Watson is able to process words even if they are misspelled. It is curious to see that the words “nouveau Smartphone” (“new Smartphone”) were considered as conveying a positive sentiment. Also, words such as “cauchemardesque” (“nightmare”) or “contente” (“happy”) were not associated with a sentiment.
When we saw those results, we could only agree with what this Consulting company told us. This technology is not working! The whole verbatim is negative and those natural language technologies should come up with the same conclusion. It is never the case. They all failed. It is not possible to trust these artificial intelligent solutions at all.
Now let’s see how Percipion is assessing this verbatim. To make it short, Percipion concluded that the sentiments within this verbatim are 100% negative!
Let us explain how Percipion is working to better understand this. Emotion is a mental state which can be seen as the result of a cognitive process. It is a positive or negative experience, producing different physiological, behavioral and cognitive changes. The role of emotions is to motivate adaptive behaviors. Consciously experiencing an emotion is exhibiting a mental representation of that emotion from a past or hypothetical experience, which is linked back to a content state of pleasure or displeasure. The content states are established by verbal explanations of experiences, describing an internal state. Percipion is able to assess those verbal explanations to identify which emotions have been triggered.
Percipion is based on the work done by Robert Plutchik, an American Emeritus Professor in Psychology who worked for the Albert Einstein College of Medicine. Plutchik considered there to be eight primary emotions (anger, fear, sadness, disgust, surprise, anticipation, trust, and joy). He also considered that basic emotions can be modified to form 24 additional complex emotions. The complex emotions could arise from cultural conditioning or association combined with the basic emotions. Alternatively, similar to the way primary colors combine, primary emotions could blend to form the full spectrum of human emotional experience.
Sentiments are a component of emotions. There are two kinds of sentiments, positive and negative. Positive sentiments are made up of emotions with a positive valence such as love or hope, for example, and negative sentiments are made up of emotions with a negative valence, such as fear or anger. Sentiments always have a pleasant or unpleasant valence and are the mental expression of our homeostasis. They inform us of our inner state.
In order to assess sentiments, Percipion will proceed with an emotional assessment first and then group assessed emotions in seven different categories to express those sentiments. Since Percipion does conform to the latest IEEE P7001 standards about AI transparency, we are able to follow what Percipion is doing while assessing the verbatim and see how the conclusion is computed. The output for this verbatim was the following:
Percipion v2.20.06 Semantic Parser for Natural Language Understanding
Copyright 2020-2021 MacAnima SAS
Start the engine ————————————————————
Current membership tier : PREMIUM
Verbatim to be assessed is smartphone01.txt
Assessment results written to smartphone01_SENTIMENTS.xml
Percipion will assess SENTIMENTS
The verbatim is not empty, assessment can begin
Unstructured text processing ————————————————————
Get verbatim binary size
Read the raw verbatim
Remove unwanted characters
Break sentences into shorter parts
Convert buffer to uppercase
Clean the raw buffer
Detect verbatim language ————————————————————
The verbatim to be assessed is written in French
Syntactic Analysis ————————————————————
Begin to simplify language & perform grammatical analysis. Please wait…
Verbatim language was simplified & grammatical analysis done
Assessment can begin
Frame of reference ————————————————————
Lexical and Relational Semantics ————————————————————
Assessment is starting. Please wait…
Verbatim word CONTENTE was associated with VB property CONTENT using Fuzzy Logic(Occ: 1 / Neg: 1 / Val: 0)
Verbatim word INQUIETE associated with VB property INQUIETE (Occ: 1 / Neg: 0 / Val: 0)
Verbatim word PLAISIR associated with VB property PLAISIR (Occ: 1 / Neg: 0 / Val: -1)
Verbatim word CAUCHEMARDESQUE associated with VB property CAUCHEMARDESQUE (Occ: 1 / Neg: 0 / Val: 0)
Verbatim word AGACE associated with VB property AGACE (Occ: 1 / Neg: 0 / Val: 0)
Verbatim word FURRAXE was associated with VB property FURAX using Fuzzy Logic (Occ: 1 / Neg: 0 / Val: 0)
Verbatim word DECEPTION associated with VB property DECEPTION (Occ: 1 / Neg: 0 / Val: 0)
Object AGACE is linked to object CONTRARIETE
Object DECEPTION is linked to object CONTRARIETE
Object FURAX is linked to object RAGE
Object CAUCHEMARDESQUE is linked to object TERREUR
Object INQUIETE is linked to object APPREHENSION
Object CONTENT is linked to object JOIE
Object PLAISIR is linked to object EXTASE
Object CONTRARIETE is linked to object COLERE
Object RAGE is linked to object COLERE
Object TERREUR is linked to object PEUR
Object APPREHENSION is linked to object PEUR
Object EXTASE is linked to object JOIE
Assessment done !
Reporting results ————————————————————
Results will be written to smartphone01_SENTIMENTS.xml
Before reporting sentiments, report emotions…
ANTICIPATION FREQ=0.00% VAL=0
JOIE FREQ=10.53% VAL=-1
CONFIANCE FREQ=0.00% VAL=0
PEUR FREQ=21.05% VAL=0
SURPRISE FREQ=0.00% VAL=0
TRISTESSE FREQ=0.00% VAL=0
DEGOUT FREQ=0.00% VAL=0
COLERE FREQ=31.58% VAL=0
AMOUR FREQ=0.00% VAL=0
SOUMISSION FREQ=0.00% VAL=0
CRAINTE FREQ=0.00% VAL=0
DESAPPOINTEMENT FREQ=0.00% VAL=0
REMORDS FREQ=0.00% VAL=0
MEPRIS FREQ=0.00% VAL=0
AGRESSIVITE FREQ=0.00% VAL=0
OPTIMISME FREQ=0.00% VAL=0
ESPOIR FREQ=0.00% VAL=0
CULPABILITE FREQ=15.79% VAL=-1
CURIOSITE FREQ=0.00% VAL=0
DESESPOIR FREQ=0.00% VAL=0
INCREDULITE FREQ=0.00% VAL=0
ENVIE FREQ=0.00% VAL=0
CYNISME FREQ=0.00% VAL=0
FIERTE FREQ=21.05% VAL=-1
ANXIETE FREQ=0.00% VAL=0
RAVISSEMENT FREQ=0.00% VAL=0
SENSIBLERIE FREQ=0.00% VAL=0
HONTE FREQ=0.00% VAL=0
INDIGNATION FREQ=0.00% VAL=0
PESSIMISME FREQ=0.00% VAL=0
MORBIDITE FREQ=0.00% VAL=0
DOMINATION FREQ=0.00% VAL=0
And now determine related sentiments…
Positive sentiments (PosMax) = Anticipation, Delight, Joy, Love
Positive sentiments (PosAvg) = Optimism, Pride, Trust
Positive sentiments (PosMin) = Curiosity, Hope
Average sentiments (Average) = Cynicism, Envy, Sentimentality, Surprise
Negative sentiments (NegMin) = Awe, Contempt, Disapproval, Dominance, Guilt, Sadness, Shame, Submission…
Negative sentiments (NegAvg) = Aggressiveness, Anxiety, Fear, Morbidness, Pessimism, Remorse
Negative sentiments (NegMax) = Anger, Despair, Disgust, Outrage
POSMAX F=10.53% V=-1
POSAVG F=21.05% V=-1
POSMIN F=0.00% V=0
AVERAGE F=0.00% V=0
NEGMIN F=15.79% V=-1
NEGAVG F=21.05% V=0
NEGMAX F=31.58% V=0
Let’s now have a look at the details of the assessment.
As you can see, to assess sentiments, the first thing that Percipion will do is to assess emotions and then, from this assessment, will assess sentiments.
Within this verbatim, a few words were positive, like the word “contente” (“happy”) but this positive word was within a negative grammatical context (“je ne suis pas du tout contente” / “I am not at all happy”). The meaning of this word is therefore the exact opposite! Percipion detected this.
The word “plaisir” (“fun”) is also a positive word but is within this sentence associated with a negative word “aucun” (“no”) and therefore the valence of this sentence is also different. This positive word, within the context of this sentence, has to be considered as being negative. Again, Percipion detected this.
Some other natural understanding technologies were not able to process the word “cauchemardesque” like IBM Watson. Here, because Percipion is based on ontologies and because we took great care to build the internal knowledge of our Cognitive AI, Percipion is able to recognize this word and is able to associate it with the correct emotion. Again, it is important that this word is taken into account since what we want to achieve is to understand the meaning of a verbatim!
As you can also see, the word “furraxe” which was misspelled is detected by Percipion and associated with the correct word, here “furax”. This recognition of misspelled words is not based on any kind of lexicon but is done by an analog fuzzy logic algorithm based on our innovation called the SmartNeuron.
Now that all meaningful words have been recognized, those words will be associated with their parent concepts (we use ontologies). The internal data structure of Percipion is analogous to the human long-term memory (a sort of VirtualBrain) so that those associations are made possible.
Once the assessment is done, Percipion will display results with their associated valences. The two meanings which have been associated with a positive emotion do have a negative valence and therefore can be considered as being negative. There is one negative sentiment level which is also associated with a negative valence so that it can be considered as even more negative. To conclude, Percipion came to the conclusion that 100% of the sentiments within this verbatim are negative.
One more thing which is important. If, what we want to assess, is just a sort of feeling people might have, to identify what people think about something is positive or negative, from a psychological point of view, this can be done by assessing affective states. This would be quicker and easier. Mainly based on their needs or intention, People perceive, filter, select and evaluate certain of those stimuli because those stimuli have the potential to affect them in a positive or negative way. Those stimuli are referred to as pressive while all others which are not relevant are referred to as inert. Pressive perception is how individuals interpret a press as either a positive or negative stimulus. In other words, this interpretation or evaluation is the result of how they experience a given interaction. In psychology, this is referred to as the affective state.
We decide how to feel about a situation after we have interpreted and explained the “phenomena”. Our appraisal of a situation causes an emotional, or affective, response that is going to be based on that appraisal. Many theorists consider affect to be post-cognitive: elicited only after a certain amount of cognitive processing of information has been accomplished. In this view, such affective reactions as liking, disliking, evaluation, or the experience of pleasure or displeasure each result from a different prior cognitive process that makes a variety of content discriminations and identifies features, examines them to find value, and weighs them according to their contributions. These can be found within a verbatim. If we use Percipion to assess affective states within the same verbatim, the results are the same as those which were found as Percipion assessed sentiments!
To conclude, Percipion was able to understand this verbatim like a human would have understood it and came up with the same conclusion.