Josep Curto
Academic Director of the Master's Degree in Business Intelligence and Big Data at the Open University of Catalonia (UOC) and Adjunct Professor at IE Business School
As noted by the MIT AI Risk Repository, one of the risks identified in the domain-based taxonomy is “disinformation, surveillance and large-scale influence”. Any article that helps to confirm and understand these risks is welcome. Above all, if it is done transparently (the code and dataset are available) for academic validation and to potentially extend to other LLMs. Considering the flattery issues presented by one of the latest versions of chatGPT, which, when discovered by users, forced OpenAI to perform a rollback, this study is extremely relevant. Flattery, combined with persuasion, increases the risk mentioned above.
It should be noted that this study should be extended to other LLMs such as Llama (from META), Claude (from Anthropic), Gwen (Alibaba) or others to determine whether this phenomenon is common and to analyse the scenario where the LLM collects information about the user and whether in that scenario (which is the usual one) persuasion is as marked as in the context of the study.