AI predictive capabilities
AI predictive capabilities were the subject of the webinar “Understanding people with AI“, organized by Athics.
Professor Marco Piastra, Professor of Artificial Intelligence and Deep Learning at the Department of Industrial, Information and Bioengineering Engineering, University of Pavia and Professor Patrizia Catellani, Professor of Social Psychology at the Department of Psychology, Università Cattolica del Sacro Cuore in Milan presented the results of the recent publication about the use of Large Language Models in the estimation of personality traits.
Written by portrait team
Esploring AI predictive capabilites
The study “On the Emergent Capabilities of ChatGPT-4 to Estimate Personality Traits” by the Department of Industrial, Information and Bioengineering Engineering of the University of Pavia and the Department of Psychology of the Catholic University of the Sacred Heart in Milan analyzed the ability of models such as ChatGPT to identify the psychometric traits of a person based on the analysis of texts.
AI predictive capabilities
AI’s predictive capabilities offer added value compared to sentiment analysis. While the latter focuses on identifying the emotional polarity of a text (positive, negative or neutral), LLMs aim for a deeper analysis, trying to understand the psychometric traits of the writer.
In other words, LLMs do not limit themselves to identifying the emotional tone of a text, but try to infer more stable characteristics of the individual, such as their predisposition to open-mindedness or level of neuroticism.
As Professor Piastra explained:
“Trying to understand the personality traits of the writer is an additional step compared to sentiment analysis, because it is not just about analyzing a text, but going beyond, towards the person.”
Why ChatGPT-4?
The study focused on ChatGPT rather than specialized technologies like Portrait because the main goal was to evaluate the emerging capabilities of a large-scale linguistic model in recognizing personality traits, without having been trained specifically for this purpose.
Professor Marco Piastra clarified this aspect in the webinar:
“There are many systems that do this type of function and are trained ad hoc, including Portrait. We did not make comparisons with your system, we limited ourselves to what is available in the scientific literature.
On the other hand, we wanted to understand how much ChatGPT-4, which was not trained for this purpose, would have been able to capture these personality traits anyway.”
LLM Self-validation
Another aspect investigated was the ability of LLMs to self-validate their predictions.
The study revealed that, although ChatGPT-4 was not specifically trained for this function, it is able to estimate personality traits with a certain accuracy. However, the correlation between the results obtained by the AI and the self-assessments of the people was moderate.
A critical aspect that emerged is the difficulty of ChatGPT-4 in assessing the reliability of its own answers. Even with limited or unrepresentative inputs, the model tends to attribute excessive confidence to its own assessments.
“ChatGPT-4 still gives an estimate, an assessment that obviously is absolutely not correlated with the actual profile and does not realize that its assessment is actually an unreliable assessment.”
The Big Five model and its measurement
The analysis of AI’s predictive capabilities was based on the Big Five, the psychological model that divides personality into five dimensions: extroversion, agreeableness, conscientiousness, neuroticism and openness to experience.
The Big Five model, considered one of the most consolidated in personality psychology, divides human personality into five main dimensions. Professor Patrizia Catellani described how it works in the webinar:
“All our behaviors, our choices, everything we do depends on many psychological factors, emotions, motivations and context. However, there are relatively more stable personality dimensions, and the Big Five are certainly the most studied dimensions, since they have been analyzed in relation to many people’s behaviors, even in the workplace and in performance.”
The five dimensions of personality in the Big Five model are:
Extroversion: tendency to be sociable, energetic and assertive.
Agreeableness: inclination towards cooperation, trust and compassion.
Conscientiousness: degree of self-discipline, organization and reliability.
Neuroticism: predisposition to anxiety, insecurity and emotional instability.
Openness to experience: curiosity, imagination and availability to new experiences.
These traits can be measured through different methodologies, including self-assessment questionnaires or third-party assessments. Today, artificial intelligence allows these traits to be inferred also from written texts, facial expressions and digital behaviors, making personality analysis more sophisticated and automated.
AI predictive capabilities: the results
The study highlighted the need for more representative datasets to improve AI predictive capabilities.
LLMs therefore have promising potential in the field of AI predictive capabilities, but their use in personality assessment still needs significant improvement.
To fully exploit the potential of LLMs and AI predictive capabilities, further developments will be needed in three main directions:
Improving predictive accuracy
The use of more representative datasets and advanced optimization techniques could increase the ability to correlate predictions with reliable psychometric assessments.
Better model self-validation
LLMs should be able to estimate more accurately the level of confidence in their responses, avoiding providing assessments when the data are insufficient or unrepresentative.
Integration with specialized tools
The union between generalist LLMs and specific technologies for personality analysis (such as Portrait) could lead to more robust systems, capable of combining the generative capacity of linguistic models with methodologies validated in the psychological field.
AI predictive capabilities application
The integration of AI into psychology and behavioral sciences opens up new horizons in several sectors. Professor Catellani highlighted the importance of “happy communication”, that is, an interaction that is in tune with the needs and characteristics of the person:
“If we can find a communication key that is in tune with the personality of the person, communication becomes more effective. It is not just a question of content, but also of tone and style. Personalized communication can increase adherence to positive behaviors, such as taking care of one’s health or professional development.”
Some of the possible applications of AI’s predictive capabilities include:
- Personalization of communication to improve audience engagement.
- Support in human resources for the assessment of soft skills and talent development.
- Adaptation of messages in the health and wellness field to encourage positive behaviors.
AI predictive capabilities: awareness
One of the most fascinating aspects of generative AI is its ability to reveal hidden patterns in language. According to Prof. Piastra:
“These machines are telling us that there are deep patterns in language that we cannot grasp. It does not mean that they are sentient, but that we are discovering something new about our own ability to communicate.”
Prof. Catellani added a reflection on the need for a critical use of technology:
“A greater awareness in the use of AI can also help us to counter negative phenomena, such as the spread of fake news. If AI can be used to personalize communication, it can also be used to develop tools that make people more critical and aware in processing information.“
Conclusions
The webinar on the predictive capabilities of AI highlighted how artificial intelligence is an extremely powerful tool, but one that requires conscious use. As Prof. Piastra emphasized:
“There is no technology without risks, but the risk is never intrinsic to the technology, but rather to the use that is made of it. Knowing the potential of these tools helps to promote conscious and correct use, avoiding delegating decisions that are better managed directly, especially when interacting with people.”
Prof. Catellani concluded with an appeal for shared responsibility:
“AI is a tool in our hands. It is up to us to decide how to use it, ensuring that it is used for the good of people and not for manipulative or distorting purposes. Only through ethical use can we fully exploit its positive potential.”
AI can become an ally in understanding human language and personality, but only if it is used with judgment and attention. The goal is not to replace human intelligence, but to enhance it, making communication more effective and personalized without losing sight of ethics and transparency.
More articles from our blog
What are the differences between Emotion AI and psychometric profiling?
The main differences between Emotion AI and psychometric profiling according to European AI Act and an ethical view of artificial intelligence.
How AI and Psychology can promote positive changes in people
AI and psychology can work together to facilitate personalized communication aimed at promoting positive changes in people.
Marketing Segmentation with Psychometric Traits
Psychometric profiling for marketing allows you to identify the main character traits and expected behavior of prospects and customers to set the communication strategy and marketing campaigns.
Follow us on Linkedin
Curious to learn more about AI’s predictive capabilities?
Follow us on Linkedin.
Offices
Viale Fulvio Testi, 128 20092 – Cinisello Balsamo (MI)
Headquarter
Via Meuccio Ruini 10, 42124 - Reggio Emilia (RE)