Building Job Seekers’ Profiles: Can LLMs Level the Playing Field?

This study investigates the impact of language complexity on the performance of an NLP-based recommender system that assists job seekers in adding relevant occupation labels and skills to their profiles. The system, deployed by Job Market Finland (JMF), was evaluated to determine whether it biases its recommendations towards more complex language inputs, potentially disadvantaging users who employ simpler language. Additionally, the study explores the effectiveness of using large language models (LLMs) to enhance simpler descriptions and mitigate potential biases. By utilizing a stratified sample of occupations and crafting varied descriptions (original, simple, complex, and LLM-improved), we analyzed the system’s recommendations against a ground truth. Results indicate that the system favored more complex language, improving occupation label suggestions (but not skill recommendations). This bias is not mitigated by the use of an LLM, suggesting potential unintended consequences for users who employ simpler language and highlighting the opacity in optimizing such systems.