MAS-TENER: a modified attention score transformer encoder for Indonesian skill entity recognition

Abstract

Skill entity recognition is a crucial task for aligning educational curricula with the evolving needs of the industry, particularly in multilingual job markets. This study introduces modified attention score transformer encoder (MAS-TENER), a novel transformer-based model designed to enhance the recognition of skill entities from Indonesian job descriptions. The proposed model modifies the attention mechanism by integrating relative positional embeddings and removing the scaling factor in self-attention. These improvements enhance the context of tokens, allowing for the accurate establishment of hard skills, soft skills, and technology skills. The MAS-TENER model was pre-trained and fine-tuned using a combinF.ation of job description datasets and additional corpora, achieving an F1-score of 90.46% at the entity level. The experimental results demonstrate the model's ability to handle unstructured, mixed-language job descriptions, with significant potential for curriculum reform and the development of new workforce capabilities. The study demonstrates the efficacy of the MAS-TENER model as an effective response for any natural language processing (NLP) task in low-resource languages. Moreover, the scope of long-term job market analytics in action research has been a key skill set in the education policy arena, demonstrating collaborative workforce capabilities.