Abstract

This study aims to assess the latest developments in the capabilities of three of the most recent “chat-prompted” LLMs (Large Language Models), Gemini 2.5 Pro, ChatGPT 4o, and DeepThink-R1, as of July 2025 in understanding the pragmatics of the Arabic language and detecting speech acts via testing on segments derived from audiovisual media material, for example, the Egyptian Arabic talk show “Maa’koum Mona El-Shazly” “ ﻲﻟذﺎﺸﻟا ﻰﻨﻣ ﻢﻜﻌﻣ .” Unlike social media interactions and simple conversations, other media formats, as in talk shows, integrate more pragmatic context(s), and are expected to present a challenging dataset for LLMs to test their abilities. Investigating “pragmatic understanding” is limited in the field(s) of linguistics and Natural Language Processing (NLP), as focus is often on semantic understanding of LLMs only, as in sentiment analysis. The study aims to fill this gap by offering a new dimension of analysis via testing on “chat-prompted” LLMs. The study relied on a mixed-method approach for data analysis of LLMs’ outputs. A total of 23 excerpts selected from an audiovisual media episode were used to run the pragmatic analysis test on the three models (Gemini 2.5, ChatGPT 4, and DeepThink-R1) via the “chat-prompt” interface. Findings revealed that the first model was more adept in interpreting the pragmatic features in the utterances compared to the second and third models, both qualitatively and quantitatively, followed by the third model. The second model seems to be less performant in recognizing indirect speech acts, compared to the other two models. The study concluded with an example for an in-class AFL application that focuses on pragmatics (Arabic Pragmatic Analyzer) that integrates the first model for an enhanced learning and instruction experience, providing guidance for pragmatic instruction in AFL classes using LLMs and AI.

School

School of Humanities and Social Sciences

Department

Applied Linguistics Department

Degree Name

MA in Teaching Arabic as a Foreign Language

Graduation Date

Fall 2-15-2026

Submission Date

11-30-2025

First Advisor

Dr. Zeinab Taha

Second Advisor

Dr. Maha Bali

Committee Member 1

Dr. Zeinab Taha

Committee Member 2

Dr. Maha Bali

Committee Member 3

Dr. Dalal Abou ElSeoud

Extent

224p.

Document Type

Master's Thesis

Institutional Review Board (IRB) Approval

Not necessary for this item

Disclosure of AI Use

Code/algorithm generation and/or validation

Download

Available for download on Saturday, January 22, 2028

Share

COinS