Abstract：Bilingual individuals are competent to acquire, store and use two languages, demonstrating the cognitive competence of the human brain to control bilingualism. However, the language representation and processing process remain unclear. In recent years, the research has revealed that eye-tracking technology can identify and interpret linguistic cognitive processes and objectively express brain responses related to bilingual processing, which has become a new research direction and hotspot in the field of cognitive linguistics. Eye-tracking technology combining with brain imaging is the most promising research method for cognitive linguistics. This paper systematically reviewed the research on bilingual tasks and the related brain mechanism on account of eye-tracking technology and imaging technology as well as its application in the field of aphasia and dyslexia, meanwhile, clarified the positive effects of eye-tracking data analysis on the exploration of bilingual brain function, aiming to provide new thoughts for the investigation of bilingual processing brain mechanism.
马恒芬, 张长颉. 基于眼动追踪技术的认知语言学研究进展[J]. 中国生物医学工程学报, 2023, 42(6): 750-756.
Ma Hengfen, Zhang Changjie. Research Progress of Cognitive Linguistics Studies Based on Eye-Tracking Technology. Chinese Journal of Biomedical Engineering, 2023, 42(6): 750-756.
[1] Luk G, Green DW, Abutalebi J, et al. Cognitive control for language switching in bilinguals: A quantitative meta-analysis of functional neuroimaging studies [J]. Lang Cogn Process, 2011, 27(10): 1479-1488. [2] Sulpizio S, Del Maschio N, Fedeli D, et al. Bilingual language processing: a meta-analysis of functional neuroimaging studies [J]. Neurosci Biobehav Rev, 2020, 108: 834-853. [3] Brignoni-Pérez E, Matejko AA, Jamal NI, et al. Functional neuroanatomy of arithmetic in monolingual and bilingual adults and children [J]. Hum Brain Mapp, 2021, 42(15): 4880-4895. [4] Kovelman I, Baker SA, Petitto LA. Bilingual and monolingual brains compared: a functional magnetic resonance imaging investigation of syntactic processing and a possible "neural signature" of bilingualism [J]. J Cogn Neurosci, 2008, 20(1): 153-169. [5] Lim JZ, Mountstephens J, Teo J. Eye-tracking feature extraction for biometric machine learning [J]. Front Neurorobot, 2022, 15: 796895. [6] Kowler E. Eye movements: the past 25 years [J]. Vision Res, 2011, 51(13): 1457-1483. [7] Bolmont M, Bianchi-Demicheli F, Boisgontier MP, et al. The woman's body (not the man's one) is used to evaluate sexual desire: an eye-tracking study of automatic visual attention [J]. J Sex Med, 2019, 16(2): 195-202. [8] Tao Ling, Wang Quan, Liu Ding, et al. Eye tracking metrics to screen and assess cognitive impairment in patients with neurological disorders [J]. Neurol Sci, 2020, 41(7): 1697-1704. [9] Skaramagkas V, Giannakakis G, Ktistakis E, et al. Review of eye tracking metrics involved in emotional and cognitive processes [J]. IEEE Rev Biomed Eng, 2023, 16: 260-277. [10] Korda AI, Ventouras E, Asvestas P, et al. Convolutional neural network propagation on electroencephalographic scalograms for detection of schizophrenia [J]. Clin Neurophysiol, 2022, 139: 90-105. [11] Freeman MR, Marian V. Visual word recognition in bilinguals: Eye-tracking evidence that L2 proficiency impacts access of L1 phonotactics [J]. Stud Second Lang Acquis, 2022, 44(3): 759-787. [12] Marsman JB, Renken R, Velichkovsky BM, et al. Fixation based event-related fMRI analysis: using eye fixations as events in functional magnetic resonance imaging to reveal cortical processing during the free exploration of visual images [J]. Hum Brain Mapp, 2012, 33(2): 307-318. [13] 于秒,闫国利.《眼动追踪:应用语言学研究实用指南》述评 [J]. 当代语言学, 2020, 22(4): 627-630. [14] Clay V, König P, König S. Eye tracking in virtual reality [J]. J Eye Mov Res, 2019, 12(1):10.16910. [15] Rayner K. Eye movements and attention in reading, scene perception, and visual search [J]. Q J Exp Psychol (Hove), 2009, 62(8): 1457-1506. [16] Wu CC, Wolfe JM. Eye movements in medical image perception: a selective review of past, present and future [J]. Vision (Basel), 2019,3(2):32. [17] Armstrong RA. Visual problems associated with traumatic brain injury [J]. Clin Exp Optom, 2018, 101(6): 716-726. [18] Ehinger BV, Groβ K, Ibs I, et al. A new comprehensive eye-tracking test battery concurrently evaluating the Pupil Labs glasses and the EyeLink 1000 [J]. Peer J, 2019, 7: e7086. [19] Holmqvist K, Örbom SL, Hooge ITC, et al. Eye tracking: empirical foundations for a minimal reporting guideline [J]. Behav Res Methods, 2023, 55(1): 364-416. [20] Hooge ITC, Holleman GA, Haukes NC, et al. Gaze tracking accuracy in humans: one eye is sometimes better than two [J]. Behav Res Methods, 2019, 51(6): 2712-2721. [21] Niehorster DC, Zemblys R, Beelders T, et al. Characterizing gaze position signals and synthesizing noise during fixations in eye-tracking data [J]. Behav Res Methods, 2020, 52(6): 2515-2534. [22] Holmqvist K, Nyström M, Andersson R, et al. Eye tracking: a comprehensive guide to methods and measures [M]. New York: Oxford University Press, 2011:392-400. [23] Thurgood C, Whitfield TWA. Light emitting diodes as a rapid visual display for use in psychological experiments [J]. Opt Photon J, 2013, 3(1): 93-97. [24] Richlan F, Gagl B, Schuster S, et al. A new high-speed visual stimulation method for gaze-contingent eye movement and brain activity studies [J]. Front Syst Neurosci, 2013, 7: 24. [25] Overton WF. Relationism and relational developmental systems: a paradigm for developmental science in the post-Cartesian era [J]. Adv Child Dev Behav, 2013, 44: 21-64. [26] Odean R, Nazareth A, Pruden SM. Novel methodology to examine cognitive and experiential factors in language development: combining eye-tracking and LENA technology [J]. Front Psychol, 2015, 6: 1266. [27] Xu Dongxin, Richards JA, Gilkerson J. Automated analysis of child phonetic production using naturalistic recordings [J]. J Speech Lang Hear Res. 2014, 57(5):1638-1650. [28] Van Rooy B, Pretorius EJ. Is reading in an agglutinating language different from an analytic language? an analysis of isiZulu and English reading based on eye movements [J]. South Afr Linguist Appl Lang Studies, 2013, 31(3): 281-297. [29] 于秒, 孙会, 逄小婷. 拼音对汉语二语初学者汉语阅读作用的眼动追踪研究 [J]. 语言教学与研究, 2022 (4): 20-30. [30] Cong Fengjiao, Chen Baoguo. The letter position coding mechanism of second language words during sentence reading: Evidence from eye movements [J]. Q J Exp Psychol (Hove), 2022, 75(10): 1932-1947. [31] Pagán A, Blythe HI, Liversedge SP. Parafoveal preprocessing of word initial trigrams during reading in adults and children [J]. J Exp Psychol Learn Mem Cogn, 2016, 42(3): 411-432. [32] Milledge SV, Blythe HI, Liversedge SP. Parafoveal pre-processing in children reading English: the importance of external letters [J]. Psychon Bull Rev, 2021, 28(1): 197-208. [33] 于秒, 闫国利. 内隐韵律与语境影响歧义结构加工的眼动研究 [J]. 天津大学学报(自然科学与工程技术版), 2015, 48(8): 686-691. [34] Comishen KJ, Bialystok E, Adler SA. The impact of bilingual environments on selective attention in infancy [J]. Dev Sci, 2019, 22(4): e12797. [35] Repnik KM, Chondrogianni V, Sorace A. Linking disambiguation and retention in a developmental eye-tracking study with monolingual and multilingual children [J]. J Exp Child Psychol, 2021, 206: 105072. [36] Kalashnikova M, Escudero P, Kidd E. The development of fast-mapping and novel word retention strategies in monolingual and bilingual infants [J]. Dev Sci, 2018, 21(6): e12674. [37] Kandhadai P, Hall DG, Werker JF. Second label learning in bilingual and monolingual infants [J]. Dev Sci, 2017, 20(1): 10.1111. [38] 于秒, 闫国利, 石锋. 语境对汉语"V+N"式歧义结构消解作用的眼动研究 [J]. 心理科学, 2016, 39(1): 22-27. [39] Desmeules-Trudel F, Joanisse MF. Learning unfamiliar words and perceiving non-native vowels in a second language: insights from eye tracking [J]. Acta Psychol (Amst), 2022, 226: 103590. [40] Bialystok E. The bilingual adaptation: How minds accommodate experience [J]. Psychol Bull, 2017, 143(3): 233-262. [41] Langer N, Ho EJ, Alexander LM, et al. A resource for assessing information processing in the developing brain using EEG and eye tracking [J]. Sci Data, 2017, 4: 170040. [42] Dimigen O, Sommer W, Hohlfeld A. Coregistration of eye movements and EEG in natural reading: analyses and review [J]. J Exp Psychol Gen, 2011, 140(4):552-572. [43] Dirix N, Cop U, Drieghe D, et al. Cross-lingual neighborhood effects in generalized lexical decision and natural reading [J]. J Exp Psychol Learn Mem Cogn, 2017, 43(6): 887-915. [44] Francis WS, Gurrola BV, Martínez M. Comprehension exposures to words in sentence contexts impact spoken word production [J]. Mem Cognit, 2022, 50(1): 192-215. [45] van Assche E, Duyck W, Gollan TH. Linking recognition and production: Cross-modal transfer effects between picture naming and lexical decision during first and second language processing in bilinguals [J]. J Mem Lang, 2016, 89: 37-54. [46] Ahn D, Abbott MJ, Rayner K, et al. Minimal overlap in language control across production and comprehension: evidence from read-aloud versus eye-tracking tasks [J]. J Neurolinguist, 2020, 54:100885. [47] Holmqvist Olander M, Wennås Brante E, Nyström M. The effect of illustration on improving text comprehension in dyslexic adults [J]. Dyslexia, 2017, 23(1):42-65. [48] Jou YT, Mariñas KAA. Developing inclusive lateral layouts for students with dyslexia - Chinese reading materials as an example [J]. Res Dev Disabil, 2023, 132:104389. [49] Parshina O, Lopukhina A, Goldina S, et al. Global reading processes in children with high risk of dyslexia: a scanpath analysis [J]. Ann Dyslexia. 2022,72(3):403-425. [50] Nerušil B, Polec J, Škunda J, et al. Eye tracking based dyslexia detection using a holistic approach [J]. Sci Rep, 2021, 11(1):15687. [51] Christoforou C, Fella A, Leppänen PHT, et al. Fixation-related potentials in naming speed: a combined EEG and eye-tracking study on children with dyslexia [J]. Clin Neurophysiol, 2021, 132(11):2798-2807. [52] Christoforou C, Theodorou M, Fella A, et al. RAN-related neural-congruency: a machine learning approach toward the study of the neural underpinnings of naming speed [J]. Front Psychol, 2023, 14:1076501. [53] Christou S, Sanz-Torrent M, Coloma CJ, et al. Real-time comprehension of Spanish prepositions and prepositional locutions in bilingual children with developmental language disorder: a study based on eye-movement evidence [J]. Int J Lang Commun Disord, 2021, 56(1): 51-71. [54] Marchman VA, Fernald A. Speed of word recognition and vocabulary knowledge in infancy predict cognitive and language outcomes in later childhood [J]. Dev Sci, 2008, 11: F9-F16. [55] Mak M, Faber M, Willems RM. Different kinds of simulation during literary reading: insights from a combined fMRI and eye-tracking study [J]. Cortex, 2023, 162:115-135.
