Jakarta, INTI - The ability of Artificial Intelligence (AI) to generate fluent Indonesian text does not necessarily guarantee its accuracy in understanding language structures, meanings, and contexts. Regional expressions, informal language, and variations in everyday communication remain significant challenges for Indonesian language processing technologies.

Devi Ambarwati Puspitasari, a researcher at the BRIN Research Center for Language, Literature, and Communities (PR BSK), explained that AI capabilities should be evaluated based on their accuracy in performing linguistic tasks rather than simply their ability to produce fluent text.

Such evaluations need to measure the consistency of results while taking into account the quality and representativeness of the datasets used.

"We need to distinguish between the ability to generate natural-sounding text and the ability to perform linguistic tasks accurately. For example, two systems can be asked to provide labels or annotations for the same sentence. Will they produce identical and consistent results? What happens when the sentence structure changes or certain elements are replaced?" Devi explained during the 89th webinar session, titled "From the Youth Pledge to the Digital Space: Strengthening Indonesia's Language and Literature Ecosystem in the Digital Era," on Wednesday (October 28).

These challenges were demonstrated through tests involving regional language variations. When prompted to combine Indonesian with the East Javanese or Surabaya dialect, ChatGPT repeatedly used the word "rek" and included the word "piye."

Meanwhile, Google Translate translated the expression "kada papakah," taken from the 2023 IKN and PPU Corpus, as "is it okay?", treating the element "papa" as a noun or proper name.

These examples demonstrate that AI-generated responses that appear natural are not necessarily linguistically accurate. Therefore, AI models need to be tested using measurable datasets and methodologies to identify areas where they perform well and where errors persist.

Linguistic Corpora Provide a Foundation for Evaluating AI

Evaluating AI capabilities in language processing requires linguistic corpora, which are structured collections of texts used to study word patterns, language structures, usage variations, and linguistic changes over time.

Before analysis, these texts undergo curation and preprocessing, including data cleaning, segmentation, metadata organization, and linguistic annotation, a process of assigning labels based on specific language characteristics.

One resource supporting this research is the CILLCO Indonesian Corpora, developed within BRIN's Research Center for Language, Literature, and Communities. The corpus can be used to examine word usage patterns and evaluate AI models.

However, analytical results must still account for data quality, coverage, linguistic representativeness, and the methodologies applied. A corpus does not automatically represent the full diversity of the Indonesian language.

"Therefore, discussions about AI should not stop at asking whether machines can speak Indonesian. We need to understand which aspects machines can handle successfully, where they make mistakes, and what evidence supports those conclusions," Devi emphasized.

Broader perspectives on the development of digital technology in language and humanities research were also presented by Herry Jogaswara, Head of BRIN's Organization for Archaeology, Language, and Literature Research (Arbastra).

He noted that AI advancements offer new opportunities while also introducing challenges for the development of language and literature research.

"Digital technology developments, including Artificial Intelligence (AI), have generated various perspectives. We were also previously asked by the Head of BRIN to provide input on AI's role in language, humanities, and related research fields. Therefore, this discussion is strategically important for broadening perspectives and encouraging the use of digital technology in advancing language and literature research," Herry said during the opening of the webinar organized by BRIN's Research Center for Language, Literature, and Communities to commemorate Youth Pledge Day.

Linguistic Diversity in the Digital Space

At the same event, Ivan Lanin, Director of Narabahasa, emphasized that recognizing Indonesian as the national unifying language does not mean that all speakers must use it uniformly.

Conversational styles, regional expressions, slang, and code-switching reflect individual identities, social relationships, and communication contexts.

These variations need to be considered in language research and the development of language processing technologies. Understanding how language is used in different contexts is essential to ensuring that analysis goes beyond word forms and also considers meaning and communication situations.

Beyond AI development, computational technologies are also creating new opportunities in literary research.

Leon Woltermann, a doctoral student at the National University of Singapore, presented the use of a distant reading approach to identify patterns across large collections of literary works. This method enables researchers to uncover trends that may be difficult to identify through reading individual works one by one.

In his research, Leon used a corpus of approximately 10,000 Indonesian short stories collected since 2022 from newspapers and magazines.

The dataset was used to examine short story publication patterns, relationships between writers and media outlets, and geographical locations featured in literary narratives.

Although computational analysis helps identify patterns across large datasets, its findings still require critical interpretation. Data quality and research methodologies also play important roles in determining the accuracy of the conclusions.

The development of technology-driven language and literature research therefore requires high-quality datasets, validated methodologies, proper documentation, and interdisciplinary collaboration.

These foundations not only support AI development but also strengthen understanding of Indonesia's linguistic diversity and encourage the responsible use of technology.

Conclusion 

BRIN highlights that Indonesia's linguistic diversity presents significant challenges for AI development, particularly in understanding regional expressions, informal language, and communication contexts. Strengthening linguistic datasets, evaluation methodologies, and interdisciplinary research will be essential to improving AI accuracy while preserving Indonesia's diverse linguistic and cultural heritage.

Read more: Komdigi Encourages Eastern Indonesian Youth to Master AI to Boost Productivity