Study notes: Fan Haiying’s micro-expression recognition

As a level 12 online surfer, one day a post on Xiaohongshu attracted attention and I became interested in micro-expression recognition, so I followed her recommendation and went to Station B to study Fan Haiying 's course.

It’s so funny. When I saw the Supernova logo, I remembered that I took an elective hydrology course when I was an undergraduate. Normally, I had to hang up and pass Supernova’s online classes. Now I have come here specifically to learn from him.

By the way, there is a conference discussion on deep . This should also be an interdisciplinary field that computers can participate in.

Micro-expressions are also a very niche field in the field of psychology. The only major research teams in China are Jiang Zhenyu and Fan Haiying. There is a blogger on Zhihu who has summarized some knowledge points and created a favorites , which is quite good.

1. Origin of microexpression

1.1 Basic emotions and secondary emotions

Contemporary psychology divides human emotions into basic emotions and secondary emotions.
There are five basic emotions: happiness, sadness, anger, fear and disgust, which correspond to specific physical states.
Secondary emotions are subtle variations or combinations of the above five emotions.

Ordinary expressions generally last for 0.5-5 seconds.
Macro expressions: occur naturally without covering up.
Micro expressions: expressions that last very short, less than 0.2 seconds.
Micro expressions are produced involuntarily when people are stimulated by the outside world. Instant real reactions of thought control.
Micro-expression means concealment, often accompanied by deception
(Fan Haiying recommends movies: "lie to me" and "People's Prosecutor")

Through the analysis of micro-expressions, a person's emotions can be read. From the hatred hidden deep in the heart, to sexual impulse, to jealousy.

1.2 Contribution of Paul Ekro

The contributions of Paul Ekro's books "The Analysis of Emotions" and "Lying"
are as follows:
① The emotions expressed by facial expressions are cross-culturally universal, and the meaning of gestures largely depends on the local culture
② Proposed the concept of micro-expressions
③Research and release facial expression coding system
④Research on interpersonal deception

Determine whether the person is lying and whether the other party is deceiving.

2. The meaning of micro-expressions

2.1 Principle of micro-expression generation

Expression is the emotional information shown on the face or posture. It is the attitude and inner experience of people towards external things in the process of cognition and will.
The principle of expression generation has a rich scientific basis, including the function of the brain and the coordination of the neuroendocrine system.
The generation of micro-expressions is an instantaneous and real reaction that people involuntarily show when they are stimulated by the outside world and are not controlled by thinking.

Possible mechanisms for the generation of micro-expressions:
① After receiving information stimulation, emotional biological behaviors appear first
② Most of the muscles on the face, limbs, and trunk involved in expression expression are skeletal muscles that can be controlled
③ Later, due to the intervention of social decision-making behavior, depression Even distorted
④The emotional expression only shows a short duration and a small form, which achieves "micro"

The expression form of expression determines the expression form of micro-expression.
Micro-expression also includes: facial expression, body posture, micro-semantics and other paralinguistic behaviors.

2.2 Morphological meaning of micro-expressions

Research on classification of micro-expression morphology and meaning
① Facial expressions
Surprise, disgust, contempt, anger, fear, sadness, joy

②Body micro-reactions:
freezing response, comfort response, escape response, looking up response, love-hate response, territorial response, fight response, victory or defeat response

③Non-verbal reactions
, reactive behaviors (paraverbal behaviors, head reactions)
actions (unintentional controlled actions)

Expressions originate from human instinct, and every expression has original meaning.
Humans are social animals, or social animals. One of the important sources of maintaining social stability and harmonious interpersonal relationships is precisely our various emotions. Express friendship and happiness through smile, express displeasure through jealousy and anger. This is the basic form of emotional expression that humans develop from the age of two and a half.

Emotions are the attitudes of likes and dislikes formed by human beings on the basis of their judgment of the interests and dangers of a situation, which affects their choice of avoidance behavior and thus affects their living conditions and interpersonal relationships.

The significance of micro-expression recognition: when you see the emotional expression, you see the cognitive evaluation of the person involved, and you see the true heart. This is the essence and analytical significance of emotion.

2.3 The mind-reading connotation of micro-expressions

(Fan Haiying recommends understanding: Freud)
Freud’s iceberg theory: The foundation and core of Freud’s psychoanalytic theory is the theory of the unconscious. He believed that human psychological structure is composed of three parts: consciousness, preconsciousness and subconsciousness.

Psychology is the subjective reflection of the objective material world by living
things. In the process of interacting with the outside world, people understand the things in the external world through various senses, think about the relationship between things and their own needs through the activities of the mind, and are accompanied by emotional experiences such as joy, anger, sorrow, and joy. A series of psychological activities of volitional behavior selection.

The manifestations of psychological activities are called psychological phenomena, including psychological processes and psychological characteristics (personality).
Psychological processes: cognition, emotion, will
. Personality: personality tendencies, personality psychological characteristics.

2.3.1 Psychological process
Cognition: refers to the process of people understanding external things, or the process of information processing of external things that act on people’s sensory organs (feeling, perception, memory, representational language, thinking, imagination)

• Feeling: It is the reflection of the human brain on the individual attributes of objective things that directly act on the sensory organs. Category (external/internal), susceptibility and sensory threshold, sensory phenomena (sensory adaptation, sensory afterimage, sensory contrast, synesthesia)
• Perception: It is the reflection in the human brain of the totality of objective things that directly act on the sensory organs. It is the result of the coordinated activities of various sensory organs and is restricted by people's knowledge, experience and attitude. Characteristics (wholeness, selectivity, constancy); types (space, time, movement, illusion)
• Memory: It is the reflection of past experiences in the mind. It is the process by which the human brain encodes, stores and retrieves input information. Past - perception - thinking - experience operation - image - extraction. Category (image, scene, emotion, semantic, action memory) process (memorization, retention, recall, recognition, forgetting). Representation (the process by which images of things perceived in the past are reproduced in the mind), three memory systems (instantaneous memory, short-term memory, long-term memory) (long-term memory-semantic/image encoding-then extracted to short-term memory) in memory)
• Thinking: It is the human brain's indirect and general reflection of objective things. It can understand the essence of things and the internal connections between things. Language and speech: tools for symbolic system thinking and communication; communication process - psychological activity process (carrier)
• Imagination: the thinking process of processing and transforming existing images to create new images
• Attention is the pointing and concentration of mental activities or conscious activities on a certain object. It is not a mental process, but just a state accompanying mental activities. Attention can place the selected object at the center of mental activities or conscious activities and maintain it, so that it can be processed effectively

Consciousness: It is the awareness (external things/internal state) in the awakened state. It has important psychological functions and plays a role in integrating, managing and regulating the human body and mind system.

Unconscious: As opposed to conscious, it refers to mental activities and processes that individuals are not aware of.

Intention: It is a general term for various requirements that people have in life. Will is the psychological process by which people's thinking and decision-making are reflected in their actions. It is the psychological process of consciously establishing goals, regulating and controlling actions, and achieving predetermined goals by overcoming difficulties and setbacks.

Will: It is a conscious and purposeful behavior, which is manifested by overcoming difficulties and setbacks. Behavior controlled by will is usually called volitional behavior. Establishing purpose is the prerequisite for volitional action, but in the process of establishing purpose, conflicts of motivation often occur. Motivation conflicts (game theory) generally have four forms: dual approach (you can’t have your cake and eat it too), double avoidance (the lesser of two evils), approach-avoidance (eating sugar to avoid getting fat), double approach-avoidance (two evils) Every job has its pros and cons and you have to make trade-offs).

2.3.2 Personality
Personality: refers to the sum of actual and potential psychological activities that affect individual behavior formed by an individual under the influence of heredity and environment. (a characteristic that distinguishes one person from another). The connotation of personality includes two aspects: personality tendency and personality characteristics.

Personality tendencies: including people's needs, motivations, interests, values ​​and beliefs (such as God rewards those who work hard), etc., are the dynamic system that promotes people to engage in various activities.

Personality psychological characteristics: including people's abilities, temperament and character, which determine a person's Personal characteristics and differences between individuals in behavioral patterns (human core)

• Ability is the psychological condition necessary to complete an activity smoothly and effectively.
• Temperament is the psychological characteristic of the dynamic nature of psychological activities in terms of intensity, speed, stability and flexibility. (temper, temperament, temperament)
• Character is a person's personality characteristics shown in his stable attitude towards reality and habitual behavior.

2.4 Application value of micro-expressions

Analyze a person's past behavior, current behavior, and predict future behavior. The
non-verbal content expressed by micro-expressions can better reveal a person's true heart than the verbal content that can be controlled by consciousness.
application:

• Event verification
• Improve emotional intelligence: manage emotions and inspire self-development; understand other people’s emotions and handle interpersonal relationships well

3.Microexpressions·Facial expressions

3.1 Baseline response

Baseline reaction refers to a person's habitual reaction, individual muscle shape and habitual expression reaction characteristics, which makes each person's expression slightly different in commonality.

Commonality: It is a muscle group common to all human faces. It has similar characteristics in the process of encountering stimuli and stimulating reactions.
Baseline: basic expression line, a person's muscle position and shape under normal conditions.
Baseline reaction: habitual expression reaction characteristics, emotional reaction characteristics formed due to muscle growth in expression habits. Baseline responses vary from person to person.

3.2 Surprise

Surprise is the first emotional state that occurs when faced with external stimuli. In order to make a comprehensive judgment on whether the sudden stimulus will harm you and what response strategy you should adopt.

Surprised expression meaning:

• Lift the upper eyelids, open your eyes wide, keep your eyes alert, and raise your eyebrows: In order to see more clearly, get as much visual information as possible to help determine the nature of the stimulus and potential dangers.
• Keep your mouth natural and relaxed, and inhale with your mouth open: in order to inhale more air and accumulate energy for the next step of fleeing or fighting.
• The expression only lasts for 1 second: for quick judgment, quick escape or response. (A cheetah running at full speed can approach 35 meters in one second)

The expression of surprise means surprise and attention to the source of the stimulus.
Surprise will also form some corresponding emotions during the transformation process. Common ones include surprise and panic.

3.3 Disgust, contempt

Disgust
: The essence of disgust is a rejection reaction of moving away from the stimulus when one does not want to see, smell or eat something that makes one feel disgusted.

Basic disgust is a comprehensive oral reaction caused by touching or imagining dirty or disgusting things. ( feel sick and vomit)

Interpersonal disgust is a reaction similar to nausea and vomiting that occurs when the body feels as if it is invaded by a rotten smell when encountering someone or something that is extremely annoying.

Meaning of disgust expression:

• The eyebrows are pressed down and frowned; the eyebrows and eyes are narrowed or closed, and the eyes and face are tense: in order to reduce the negative visual stimulation caused by the stimulus source. At the same time, it can also protect the eyeballs from damage caused by increased internal pressure caused by nausea and vomiting.
• Wrinkle the nose and lift the upper lip; form nasolabial grooves on both sides of the nose; in order to reduce negative olfactory stimulation caused by irritants.
• Shut up; in order to reduce the negative taste stimulation caused by the stimulus source.

The meaning of the expression of disgust is denial and distancing from the source of stimulation.

Contempt
Contempt, which can be understood as a modification of the expression of strong disgust, is a more restrained version. Usually comes from contempt and denial of ugly appearance, clumsy thinking or movement, and tasteless style.

Meaning of contempt expression:

• One corner of the mouth moves upward while unilaterally shrinking the corner of the mouth; one side of the lip raises: Contempt and denial of others from the heart will stimulate an inner sense of superiority, usually resulting in a slight smile (but incomplete).
• Eyebrow, eye, and nose movements without disgust: A comprehensive oral reaction to touching or imagining dirty or disgusting things without basic disgust.

Contempt expresses contempt, denial and self-superiority towards the source of stimulation.

3.4 Anger

Anger is an emotional state that is triggered by attacks to eliminate danger when faced with threats and harmful stimuli. It is a general mobilization of energy under stress. The essence of anger is aggression in the face of threats.

3.5 Fear

Fear is an emotional state of panic, fear, and uneasiness that occurs when faced with threats and harmful stimuli.

The essence of fear is feeling powerless and fearing that the consequences are about to happen.

When you fall into a fearful mood, the thought of being threatened may occupy a person's mind for a long time. The continuous and intense panic will exhaust the person and may eventually result in death from exhaustion.

The expression of fear means fear and uneasiness about the source of stimulation

3.6 Sadness

Sadness is a sad emotional state triggered by facing adverse consequences that threaten and hurt. It is an emotional expression of grief and sadness when people face a major loss and are unable to recover.

The body's weak state will cause the eyebrows, eyes, and corners of the mouth to droop. Unable to save oneself, regressing to the help-seeking mode of infancy (crying

Pay attention to the discomfort and even pain in the person's head, neck, cheeks, throat, back, shoulders, arms, stomach, and legs during the grieving process.

The meaning of the sad expression is the sorrow and sadness of irreparable loss.

3.7 Pleasure

Pleasure is an emotional state of self-satisfaction or joy that transcends satisfaction when the stimulus source is higher than subjective expectations.

There are dozens of pleasant emotions, including: sensual pleasure, happiness, satisfaction, excitement, relief, surprise, ecstasy, pride, relief, gratitude, and schadenfreude.

Laughter can also be expressed in combination with other emotions besides pleasure, and has complex characteristics.

4.Microexpressions·Body microreactions

Physical micro-reactions refer to the small physical movements that people involuntarily show that are not controlled by thinking and are instantaneous and real when they receive effective stimulation.

Common types: freezing, escape, comfort, looking up, love and hate, territory, fighting, victory or defeat

• Freeze reaction form: frozen body (restraint)
• Escape response pattern: face, body and stimulus source distanced
• Comforting response pattern: The perpetrator targets his/her own touch or masturbation movements of multiple parts
• Looking up reaction pattern: head raised, torso straight, chin slightly raised
• Negative looking up reaction pattern: varying degrees of trunk bending + lowering head

4.1 Freezing reaction

The freezing reaction is a person's first reaction when exposed to unexpected stimuli.
Unexpected stimuli are effective clues for detecting lies.

4.2 Escape response

The flight response is a reaction that occurs when a person is faced with a threatening stimulus such as danger or injury but feels unable to fight it. Staying away from the source of stimulation and storing energy for escape are the physical and mental basis of the escape response. Emotional experiences accompanied by uneasiness, disgust, fear, and even anger. (disgust, fear)

4.3 Comfort response

The comfort response is some self-settling and comforting reactions that are subconsciously displayed when people are subject to negative stimuli such as criticism, pressure, and denial in order to relieve their inner discomfort: visual comfort, auditory comfort, oral comfort, skin comfort... (discomfort, depressed)

4.4 Looking up reaction

The look-up reaction is a reaction after judging one's own abilities, status differences, predictions of victory or defeat, and positioning of advantages and disadvantages.

The instinct accumulated through evolution makes people look up to objects that are taller than themselves and look down on objects that are shorter than themselves. At the same time, people will also instinctively try to raise their bodies as much as possible in order to gain an advantage, or lower their bodies to show humility. and obedience.

4.5 Love-hate reaction

The love-hate reaction is a reaction dominated by the two extremes of psychological distance between people (love and hate).

When you love someone, you will take the initiative to get close to the other person; when you hate them, you will take the initiative to distance yourself, gnash your teeth, and even do more crazy things.

The distance between bodies can reflect the psychological distance between people

4.6 Territorial response

Territorial response is the leadership style displayed by people in their own "territory".

In their own territory, people will appear relaxed, comfortable, dignified, and can control effortlessly.

4.7 Combat reaction

The fight response is the strongest expression of anger.

The causes of anger and fighting, no matter how specific, can be attributed to threats to survival and reproduction.

How to fight, whether to adopt an offensive or defensive posture, or whether to break out into an offensive posture at any time when the defense starts, is a choice made after evaluating one's self-resistance and offensive capabilities.

4.8 Victory and defeat reactions

The victory-defeat reaction is what happens after a fight is over.

The winner will be joyful, showing off and relaxed, with the nervous system in a state of excitement and the body showing an upward reaction against gravity.

The loser's nervous system enters a state of depression, the whole body loses energy, and the body's center of gravity is downward and contracted, showing dejection, sadness, and discouragement.

5.Microexpression·Nonverbal behavior

5.1 Reactive behavior

Non-verbal behavior refers to the expressions, movements, colors, postures, paralinguistic behaviors, etc. that accompany the speaker's verbal behavior.

Reactive behavior: refers to a reflex physiological reaction, a physiological reaction that is difficult to control voluntarily.

• 1. Paralinguistic behavior: usually defined as "vocal language without fixed semantics" refers to language characteristics other than the verbal content of speech, such as the speed of reaction, the loudness of the voice, the length of the speech, etc.
• 2. Head reactions: including facial color, dry mouth, sweating, etc.

Action: includes unintentionally controlled actions and intentionally controlled actions.

• 1. Unintentionally controlled movements: head movements (head posture, facial expressions, eyebrows, eyes, mouth, etc.); arm movements (shoulders, elbows, arms, hands, etc.); leg and foot movements (legs, feet) and sitting postures.
• 2. Intentionally controlled movements: face (dramatic face, "stone face"); eyes (shifting gaze); hands (busy hands); legs (shaking legs); sitting posture (controlled posture)

5.2 Characteristics and significance of reactive behavior

Reactive behavior refers to reflex physiological reactions, physiological reactions that are difficult to control voluntarily.

1. Paralinguistic behavior, usually defined as "language" with sound but no fixed semantics, refers to language characteristics other than the verbal content of speech, such as the speed of reaction, the loudness of the voice, the length of the speech, etc.

Paralinguistic behaviors that suggest lying:

• higher pitch;
• Increased slips of the tongue and stuttering; slowed down speech rate;
• Response time is too long or too short.

2. Head reaction refers to the reflex physiological reaction of the head, which is difficult to control independently. Including facial color, dry mouth, sweating, etc.

• Face and neck reactions: becoming pale, nervous, flushing or even erythema, embarrassment (encountering a difficult problem);
• Oral reaction: frequent swallowing movements, clicking sounds in the mouth, constant licking of the lips and nervousness (dry mouth);
• Forehead and nose reaction: sweating and nervousness around the forehead and nose, especially under the nose and above the lips;
• Reactions in hands, feet, and armpits: Reactions in hands, feet, and armpits: Hands feel cold, wet, and tense (can be felt when shaking hands)

5.3 The significance of unintentional control of body reactions

1. Blinking (unintentional controlled action)
Under normal conditions, blinking occurs 6-8 times per minute, and each blinking action is only 0.1 seconds.
High frequency of blinking, nervousness, lying (anxiety, worry);
prolongation of the duration of a single blinking movement,
vision blockage (disgust or escape);

2. Eye movement,
eye movement indicators and selection.
Eye movement indicators with fixation point as the core: When and Where, how long.
Other indicators: pupil size, gaze trajectory, saccade angle, etc.
Experience in distinguishing true and false: the first movement after receiving stimulation. reaction

3. Mouth (unintentional control of movements)
just smiles showing upper teeth and gums - fake smile: biting lips, pursed mouth; covering mouth; biting fingers or nails or other objects - blocking mouth and saying inappropriate words

4. Shoulders (unintentional control movements)
One shoulder is high and the other is low. The higher shoulder is facing the side of the inquirer, angry, unprepared to cooperate or deceive;
a shrug has the meaning of denial or refusal;
the shoulders are drooped, arched or the shoulders are shrugged forward.
Frustration or acceptance before confession;
one or both shoulders back, neck and back straightened. Angry
or aggressive defensive state.

5. Arms (unintentionally controlled movement)
Crossing arms is a defensive behavior; (fear, resistance, anger)
The height of the arms when crossing the chest can be used as an indicator of negative emotions. The higher the position, the more resistance or anger the person being asked The greater the emotion.
Crossing the arms and surrounding oneself with the arms (hugging the torso tightly with the arms) can also be seen with a lowered head, arched back, no eye contact - frustration, refusal to cooperate.

6. Legs (unintentionally controlled movements)

Cross your legs and change the order of crossing your legs. Set up physical barriers to block verbal attacks from interrogators. Delay time (answer questions).

Keep shaking your knees to relieve tension.

7. Feet (unintentional controlled movements)

Tapping your feet on the ground or even stomping your feet can reflect two emotional states: a relatively gentle tap, indicating that you are experiencing general fear and nervousness, or indicating that you are losing patience with inquiries, using this method to convey a desire to leave.
Stomping his feet: First, it reflects his high excitement and hostility; second, it is to relieve tension.

8. Sitting posture (unintentional movement)
leaning toward the door or window
, or leaning away from the inquirer.
Nervousness and desire to escape from the current unfavorable environment;
one leg stretched straight forward or in the direction of the door
- this This posture is a bit like the preparatory movements before a race. There is an intention to escape.