Three- and four-dimensional mapping of speech and language in patients with epilepsy

Introduction

Invasive studies of patients with focal epilepsy have established and expanded many generally-accepted theoretical frameworks for the functional neuroanatomy of speech and language function; the generalization of such findings is possible because the presence of focal epilepsy does not imply the entire brain to be abnormal. Penfield and colleagues established functional brain maps based on the summary results of direct cortical stimulation of which locations varied across patients (Penfield and Boldrey, 1937; Penfield and Jasper, 1954; Penfield and Roberts, 1959). These maps, originally hand-drawn in 2D, still serve as important textbook materials for current students in medicine, psychology and neuroscience. During the past decade, investigators summarized the results of intraoperative cortical stimulation during awake craniotomy and generated a probabilistic map digitally delineated on the lateral surface of 3D MRI (Sanai et al., 2008; Tate et al., 2014; Chang et al., 2016). These probabilistic maps of critical speech and language function are clinically useful in predicting the language outcome following epilepsy and tumour surgery. Here, we expanded their concept and newly generated a more spatially comprehensive 3D map showing both lateral, medial and inferior surfaces of the bilateral cerebral hemispheres, using the results of extraoperative cortical stimulation via subdural electrodes performed in a large cohort of patients with focal epilepsy. The benefits of extraoperative over intraoperative stimulation include the capability to study young children and to test more widespread regions including the medial and inferior surfaces of the cerebral cortex (Perrine et al., 1994; Malow et al., 1996; Schäffler et al., 1996; Hamberger et al., 2001; Ojemann et al., 2003; Lee et al., 2009; Lesser et al., 2010).

The present study also newly generated 4D maps of auditory naming function based on event-related modulation of electrocorticography (ECoG) signals measured in the same patient cohorts. Taking into account ECoG signals from all patients in each age group, video movies delineated the dynamics of cortical activation and deactivation on a common 3D brain surface image with a frame duration of 10 ms (Supplementary Videos 1 and 2). Thereby, amplitude augmentation of high-gamma activity (70–110 Hz) was treated as a primary measure of in situ cortical activation during an auditory language task (Crone et al., 1998a; Towle et al., 2008; Lachaux et al., 2012); whereas high-gamma attenuation reflecting deactivation/inhibition (Shmuel et al., 2006). Previous studies, including ours, have reported the spatial concordance between speech/language areas defined by electrical stimulation and naming-related high-gamma augmentation (Leuthardt et al., 2007; Kojima et al., 2012; Ruescher et al., 2013; Arya et al., 2015; Wang et al., 2016). Resection of cortical regions showing naming-related high-gamma augmentation was reported to increase the risk of postoperative language deficits (Cervenka et al., 2013; Kojima et al., 2013a, b). Event-related high-gamma augmentation was reported to co-occur with beta-attenuation (15–30 Hz; Crone et al., 1998b; Miller et al., 2007; Lachaux et al., 2012); thus, we treated event-related beta-attenuation as a secondary measure of cortical activation in the present study.

We designed a group ECoG analysis on a large patient cohort to determine the rapid dynamics of cortical activation and deactivation across widespread brain regions in both hemispheres during an auditory naming task. Our central hypothesis, motivated by the theory of functional antagonism in the cortical networks (Fox et al., 2005; Anticevic et al., 2012), was that areas advantageous to function directed toward speech and language would be activated at a given moment, while those obtrusive to such function would be deactivated. A number of functional MRI studies of healthy individuals reported that cortical deactivation takes place in certain cortical regions during cognitive tasks and that those showing task-related deactivation are referred to as the default mode network (reviewed in Raichle, 2015). ECoG studies of small patient cohorts also reported that deactivation involved variable portions of the studied cerebral hemisphere during sensorimotor and cognitive tasks (Asano et al., 2009b; Jerbi et al., 2010; Ramot et al., 2012; Toyoda et al., 2014). Because of the large spatial coverage and excellent temporal resolution, we hypothesized that our ECoG analysis would reveal the rapidly alternating activation and deactivation taking place in both the same and different locations. Ultimately, we expected that the results of our study would expand or refine the current neurobiological models of speech and language (Hickok and Poeppel, 2007; Rauschecker and Scott, 2009).

Finally, we determined the effect of development on the dynamics of cortical activation/deactivation during the task. Specifically, we determined the temporal dynamics of left-hemispheric dominant activation present only in the older group (≥10 years of age). Healthy children younger than 10 years of age are still undergoing rapid development of vocabulary, naming skills, and verbal working memory (Gaulin and Campbell, 1994; Korkman et al., 2001). Neuropsychological studies of acquired left-hemispheric lesions and left hemispherectomy following Rasmussen’s encephalitis suggested that patients younger than 10 years can quickly recover from aphasia by efficiently transferring speech and language function to the right hemisphere, whereas such functional recovery, particularly in the expressive domain, is slower and less complete in older individuals (Vargha-Khadem et al., 1985; Boatman et al., 1999; Hertz-Pannier et al., 2002; Elger et al., 2004). Previous functional MRI studies of healthy children suggested that younger children more frequently have bilateral representations of language processing particularly in the frontal lobes (Gaillard et al., 2000; Holland et al., 2001). Healthy adults, regardless of handedness, can show such bilateral haemodynamic activation, but their locations of language-activated areas tend to be more consolidated compared to those of children (Pujol et al., 1999; Knecht et al., 2000; Szaflarski et al., 2002). Taken together, we tested our hypothesis that portions of the cortical structures, particularly the frontal lobe regions, would show left-hemispheric dominant activation only in the older group and that such activation would take place during utilization of expressive language function (i.e. around stimulus offset and after) rather than during listening to questions.

Materials and methods

Patients

The inclusion criteria consisted of: (i) two-stage surgery with chronic/extraoperative subdural ECoG recording as a part of presurgical evaluation of focal epilepsy at Children’s Hospital of Michigan or Harper University Hospital in Detroit between January 2007 and September 2015; (ii) age ≥ 4 years; and (iii) functional cortical mapping with direct cortical stimulation as well as measurement of naming-related ECoG signal changes as described below. The exclusion criteria consisted of: (i) presence of massive brain malformations (such as large perisylvian polymicrogyria or hemimegalencephaly) that confound the anatomical landmarks for the central sulcus and Sylvian fissure; (ii) severe cognitive dysfunction reflected by verbal comprehension index or verbal IQ < 70 (Bouchard et al., 2013); (iii) inability to complete the auditory-naming task due to the lack of adequate vocabulary, comprehension of task instructions, or cooperation; (iv) primary language other than English (Cervenka et al., 2011); (v) history of previous epilepsy surgery; and (vi) right-hemispheric language dominance as suggested by the result of Wada test or left-handedness associated with left-hemispheric congenital neocortical lesions (Rasmussen and Milner, 1977; Knecht et al., 2000; Akanuma et al., 2003; Möddel et al., 2009; Kojima et al., 2013a, b).

To determine the effect of age on the dynamics of task-related cortical modulations, we classified patients into two age groups: (i) an older group consisting of patients at least 10 years of age; and (ii) a younger group consisting of children younger than age 10. This age cut-off was chosen partly because electrical stimulation in children younger than 10 had been previously reported to be ineffective in localizing speech or language areas (Schevon et al., 2007). We determined if there was a difference in the distribution or proportion of aetiology, seizure-onset zone, handedness, or number of antiepileptic drugs taken by each patient between the two age groups (Mann-Whitney U-test and Fisher’s exact probability test). The study was approved by the Institutional Review Board at Wayne State University, and written informed consent was obtained from the patients or guardians of patients. Written assent was obtained from children older than 13. Oral assent was obtained from younger children.

Subdural electrode placement

Platinum macroelectrodes (10 mm inter-contact distance) were placed in the subdural space generously over the affected hemisphere, to satisfactorily determine the boundaries between the epileptogenic zone and eloquent areas (Asano et al., 2009a). Electrode plates were stitched to adjacent plates or the edge of dura mater to avoid movement of electrodes after placement. The location of electrode placement was determined purely based on the clinical needs, and we did not place electrodes more than clinically indicated (Nonoda et al., 2016).

Acquisition of 3D surface images

MRI, including a volumetric T₁-weighted spoiled gradient echo image as well as fluid-attenuated inversion recovery (FLAIR) image of the entire head, was obtained preoperatively (Nagasawa et al., 2011). A 3D surface image was created with the location of electrodes directly defined on the brain surface, and the spatial accuracy of the co-registration procedure at the individual level was reported to be 1.2 ± 0.7 mm with a maximal misregistration of 2.7 mm (Muzik et al., 2007; Alkonyi et al., 2009). The spatial accuracy of electrode display on the 3D brain surface image was confirmed by direct visualization of intraoperative pictures, which have been treated as the gold standard to determine the co-registration accuracy (Wellmer et al., 2002; Dalal et al., 2008; Pieters et al., 2013). Post-implant CT images were used, as needed, to confirm the co-registration accuracy in the medial and inferior surfaces of the cortex (Foster et al., 2013).

The spatial normalization of each individual electrode site was performed using FreeSurfer scripts (http://surfer.nmr.mgh.harvard.edu). All electrode sites on each individual’s FreeSurfer brain surface were transformed into Talairach coordinates, and finally plotted on the averaged FreeSurfer pial surface image, referred to as ‘fsaverage’ (Desikan et al., 2006; Chan et al., 2011). The surface-based group averaging using FreeSurfer has been validated in children, and the mean registration error was reported to be 1.56 mm in those ranging in age from 4 to 11 years old (Ghosh et al., 2010). Automatic parcellation of cortical gyri was performed at both individual and spatially normalized brain surfaces (Desikan et al., 2006; Matsuzaki et al., 2015). Each hemisphere contained 35 regions of interest (Fig. 1A) and all electrode sites were assigned anatomical labels. The number of available electrodes within each region of interest is provided in Supplementary Table 1.

Figure 1

Open in new tabDownload slide

Regions of interest and distribution of subdural electrodes included in the analysis. (A) aCC = anterior cingulate gyrus; aFG = anterior fusiform gyrus; aITG = anterior inferior temporal gyrus; aMTG = anterior middle temporal gyrus; aSFG = anterior superior frontal gyrus; aSTG anterior superior temporal gyrus; cMFG = caudal middle frontal gyrus; Cun = cuneus gyrus; Ent = entorhinal gyrus; FP = frontal pole; IPL = inferior parietal lobule; iPoCG = inferior postcentral gyrus; iPreCG = inferior precentral gyrus; LG = lingual gyrus; LOG = lateral occipital gyrus; MOF = medial orbitofrontal gyrus; pCC = posterior cingulate gyrus; PCL = paracentral lobule; PCun = precuneus gyrus; pFG = posterior inferior temporal gyrus; PHG = parahippocampal gyrus; pITG = posterior inferior temporal gyrus; pMTG = posterior middle temporal gyrus; Pop/PTr/POr = pars opercularis/pars triangularis/pars orbitalis within the inferior frontal gyrus. LOF = lateral orbitofrontal gyrus; pSFG = posterior superior frontal gyrus; pSTG = posterior superior temporal gyrus; rMFG = rostral middle frontal gyrus; SMG = supramarginal gyrus; SPL = superior parietal lobule; sPoCG superior postcentral gyrus; sPreCG = superior precentral gyrus; TP = temporal pole. The numbers of ≥10-year-old (B) and (C) <10-year-old patients whose ECoG data at a given site contributed to further analysis are presented.

Extraoperative ECoG recording

ECoG was continuously recorded at bedside for 3–7 days at a sampling frequency of 1000 Hz with simultaneous video recording, using a 192-channel Nihon Kohden Neurofax 1100A Digital System). To provide more generalizable results, we made the best effort to reduce the effect of epileptiform discharges on ECoG signals. Electrode sites classified as seizure onset zone (Asano et al., 2009a) or those affected by a structural lesion were excluded from further analysis. Likewise, sites showing interictal spikes (Jacobs et al., 2011; Zijlmans et al., 2011) or artefacts (Kojima et al., 2013b; Uematsu et al., 2013) during the task were excluded from analysis. Previous ECoG studies including ours have suggested that the detrimental effect of spikes on event-related high-gamma measures was confined to sites showing interictal spike discharges (Brown et al., 2012a; Zweiphenninga et al., 2016).

Direct cortical stimulation and generation of 3D probability maps

Cortical stimulation was performed as part of clinical management (Kojima et al., 2012; Kumar et al., 2012). A pulse-train of repetitive electrical stimuli was delivered to neighbouring electrode pairs, with stimulus frequency of 50 Hz, pulse duration of 300 μs, and train duration ranging up to 5 s. Stimulus intensity was initially set to 3 mA, and increased up to 9 mA in a stepwise manner until a clinical response or after-discharge was observed. During each period of stimulation, each patient was asked to answer brief auditory questions such as ‘What flies in the sky?’. Unlike a picture naming or verb association task (Hamberger, 2007), our task requires patients to process both syntax and semantics for comprehension of a whole sentence. Electrode sites at which stimulation reproducibly resulted in clinical symptoms, were determined by at least two investigators, including at least one neuropsychologist (Kojima et al., 2012). Each patient was asked for the reason, in case he/she failed to verbalize a relevant answer during stimulation. Additional tasks such as syllable repetition, humming, counting, and reciting ABC’s were performed to determine the specific role of each cortical site at which stimulation resulted in a given clinical symptom. Bipolar stimulation is less time-consuming compared to monopolar, and suggested to be more suitable for patients with limited attention span. Bipolar and monopolar neurostimulation is equally accurate for localization of eloquent areas (Kombos et al., 1999). To specify which electrode within a stimulated pair was responsible to induce a symptom of interest, we stimulated some electrodes with different neighbouring electrodes (Sinai et al., 2005). The probability of each symptom induced by stimulation was calculated in each electrode site within each patient, and subsequently presented at a given site on the averaged FreeSurfer pial surface image with a Gaussian half-width at half maximum of 7.5 mm, using MATLAB R2015b software (The MathWorks Inc., Natick, MA, USA). Grand-averaging of all available patients’ data finally yielded a 3D map showing the mean probability of each stimulation-induced symptom in each of the older and younger groups.

Auditory naming task

Measurement of auditory naming-related modulation of ECoG signals was performed using a method similar to those previously reported (Brown et al., 2008; Kojima et al., 2013b). None of the patients had a seizure within 2 h prior to or during task performance. While awake and comfortably seated on a bed in a room with unwanted noises minimized, patients received question-and-answer trials (Fig. 2). The duration of a question ranged from 1 to 2.5 s (median: 1.8 s). All questions, beginning with either ‘what’, ‘where’, ‘when’ or ‘who’, were designed to elicit one- or two-word answers with nouns (Brown et al., 2012b). The onset and offset of question stimuli as well as the onset of the patient’s vocalization of the response were marked for each trial. Patients were instructed to answer ‘I don’t know’ when they did not know the answer to or did not understand a stimulus. Trials not accompanied by correct answers were excluded from further analysis. The mean number of included trials was 79.4 (standard deviation: 19.9).

Figure 2

Open in new tabDownload slide

Task. Patients were asked to listen to a series of questions and to overtly verbalize a relevant answer during extraoperative ECoG recording. We then measured the per cent change of high-gamma amplitudes compared to the baseline period.

Time–frequency ECoG analysis and generation of 4D mapping of speech and language

Each ECoG trial was transformed into the time–frequency domain using a complex demodulation technique (Papp and Ktonas, 1977) incorporated in BESA® EEG V.5.1.8 software (BESA GmbH, Gräfelfing, Germany; Hoechstetter et al., 2004; Brown et al., 2008). A given ECoG signal was assigned an amplitude (a measure proportional to the square root of power) as a function of time and frequency at each trial. The time–frequency transform was obtained by multiplication of the time–domain signal with a complex exponential, followed by a low-pass filter. The low-pass filter used here was a finite impulse response filter of Gaussian shape, making the complex demodulation effectively equivalent to a Gabor transform. The filter had a full-width at half-maximum of 2 × 15.8 ms in the temporal domain and 2 × 7.1 Hz in the frequency domain. At each time–frequency bin at each electrode site, we measured the amplitude change at the bands, in steps of 5 Hz and 10 ms, relative to the mean amplitude in a resting period between 600 to 200 ms prior to stimulus onset. Time–frequency analysis was repeated with a time-lock to (i) stimulus onset; (ii) stimulus offset; and (iii) response onset. The per cent changes in high-gamma (70–110 Hz) and beta amplitudes (15–30 Hz) at each electrode site and each moment were spatially presented with a Gaussian half-width at half-maximum of 7.5 mm, and sequentially animated on the average FreeSurfer pial surface image as a function of time throughout the task. Grand-averaging of all available patients’ data finally yielded a 4D map showing ‘when’ and ‘where’ high-gamma and beta activities are augmented and attenuated during the auditory-naming task in each of the older and younger groups.

Region of interest analyses on task-related modulation of ECoG signals

This analysis was performed to determine the timing of statistically significant augmentation or attenuation of high-gamma and beta activities at each region of interest (Fig. 1A) in each hemisphere in each age group. We plotted the mean and standard error of amplitude change of high-gamma (and beta) amplitude across all available channels at each region of interest as a function of time. We then determined ‘when’ ECoG amplitudes were increased or decreased from the baseline during the aforementioned resting period, using studentized bootstrap statistics followed by Bonferroni correction (Davison and Hinkley, 1997; Terwee et al., 2010; Toyoda et al., 2014; Nishida et al., 2016). It should be noted that there was no statistical difference between ‘the mean across all channels within each region of interest’ and ‘the grand-mean across individual means within each region of interest’ (Supplementary Fig. 1); thus, the risk of a single patient’s data severely deviating the calculated mean from the truth is small in our study. Our simple analytic approach intrinsically benefits from a well maintained statistical power. The level of significance was set at corrected P = 0.05.

Results

Behavioural results

We studied 100 patients who satisfied both inclusion and exclusion criteria. The older group consisted of 84 patients at least 10 years of age, whereas the younger group included 16 children younger than age 10. Patient profiles are provided in Table 1. A total of 7367 and 1438 artefact-free non-epileptic electrodes were available for further analysis in the older and younger groups, respectively (Fig. 1B and C). On average, 105.2 electrode sites derived from 26.3 patients contributed to each region of interest in the older group, and 20.5 sites from 5.4 patients in the younger group (Supplementary Table 1).

All patients satisfying the aforementioned criteria satisfactorily completed the auditory naming task. Behavioural results are summarized in Table 1. The response time (duration between stimulus offset and response onset) was shorter in the older compared to the younger group (mean: 1.76 versus 2.17 s; P = 0.03 on Mann-Whitney U-test). Patients in the older group, compared to those in the younger, had more trials with correct answers (mean trials per patient: 81.5 versus 68.0; P = 0.002 on Mann-Whitney U-test), indicating that each electrode site in the older group benefitted from a better signal-to-noise ratio by 9% on average. There was no difference in aetiology, seizure-onset focus, handedness, or number of antiepileptic drugs taken by each patient between the two age groups (Table 1).

3D mapping based on results of cortical stimulation

The 3D maps for the older and younger groups are separately presented in Fig. 3 and Supplementary Fig. 3, respectively. In the older group, auditory hallucination was elicited by stimulation of posterior superior-temporal gyrus of either hemisphere. Receptive aphasia was elicited primarily by stimulation of the left posterior superior/middle-temporal gyri. Expressive aphasia was elicited by widespread regions in the left temporal and frontal lobes, including the posterior portions of superior/middle/inferior-temporal gyri as well as superior/middle/inferior-frontal gyri. Speech arrest was elicited by stimulation of bilateral inferior precentral gyri as well as left posterior superior-frontal gyrus. The extents of stimulation-defined eloquent cortices (including speech/language areas) in the younger group appeared smaller than and included within those in the older group. The younger group, compared to the older, had a smaller probability of revealing stimulation-induced speech/language symptoms including auditory hallucination, receptive aphasia, expressive aphasia, and speech arrest; [11/120 (9.2%) versus 105/544 (19.3%; P = 0.007 on Fisher’s exact probability test) in the left posterior frontal lobe structures and 4/40 (10.0%) versus 71/274 (25.9%; P = 0.02) in the left posterior temporal lobe structures]. Auditory hallucination was elicited in 15/78 (10/40, left; 5/38, right) older patients and 1/13 (0/7, left; 1/6, right) younger patients of whom electrode placement involved the posterior superior-temporal gyrus.

Figure 3

Open in new tabDownload slide

3D probabilistic mapping of cortical function based on the results of cortical stimulation in the older group. Auditory hallucination: perception of various pitches of sounds or alteration of the neuropsychologist’s voice’s pitch. Speech arrest: inability to initiate and continue vocalization not directly explained by the effect of forced jerking of tongue or lip. Expressive aphasia: dysnomia or inability to provide a relevant answer during a 5-s stimulation although the neuropsychologist (R.R.) confirmed that a given question was understood and the capability of vocalization was maintained. Receptive aphasia: inability to understand a question though being aware that a question was given by the neuropsychologist. To differentiate the nature of aphasia, the neuropsychologist asked each patient the reason, in case he/she failed to verbalize a relevant answer during stimulation. The locations of stimulation-induced sensory/motor symptoms are presented in Supplementary Fig. 2. The maps derived from the younger group are provided in Supplementary Fig. 3.

High-gamma modulation during listening to auditory stimulus

Figure 4 and Supplementary Video 1 present the dynamics of cortical activation and deactivation, in a 4D manner, reflected by high-gamma augmentation and attenuation in the older group. Immediately after stimulus onset, high-gamma activity began to be augmented and peaked at 100–200 ms in the bilateral superior-temporal gyri, and to a larger degree in the posterior portions (Fig. 5A). Superior-temporal high-gamma augmentation lasted while auditory questions were delivered. At 100–200 ms after stimulus onset, high-gamma activity began to be augmented in bilateral inferior precentral gyri and continued to be augmented with left hemispheric dominance. At 300–700 ms after stimulus onset, high-gamma activity was slightly but significantly attenuated in the pars orbitalis [Brodmann area 47 (BA47)] of the left inferior-frontal gyrus.

Figure 4

Open in new tabDownload slide

Snapshots of 4D mapping of speech and language based on the results of auditory naming-related modulation of high-gamma and beta activity in the older group. Event-related amplitude augmentation is reflected by red, whereas attenuation by blue. ‘+0.4’ indicates that the amplitude was augmented by 40% compared to the mean during the resting period between −600 and −200 ms relative to stimulus onset. (A) At +70 ms relative to stimulus onset, high-gamma and beta activities were both augmented in the superior-temporal gyri, bilaterally. (B) At +500 ms relative to stimulus onset, high-gamma activity was augmented, whereas beta activity was attenuated in the superior-temporal gyri, bilaterally. (C) At stimulus offset, widespread areas of the left temporal-frontal lobes as well as the right superior-temporal region showed high-gamma augmentation and beta attenuation. Simultaneously, widespread areas of the right frontal-parietal lobes showed high-gamma attenuation and beta augmentation. (D) Beta attenuation in widespread areas lingered between stimulus offset and response onset. Conversely, at response onset, high-gamma augmentation was confined to the posterior superior-frontal as well as pre-/postcentral gyri, bilaterally. Supplementary Videos 1 (older group) and 2 (younger group) delineate the spatiotemporal profiles of high-gamma and beta amplitude modulations in the lateral, medial, and inferior surfaces of the cerebral hemispheres relative to stimulus onset, stimulus offset, and response onset.

Figure 5

Open in new tabDownload slide

Open in new tabDownload slide

Open in new tabDownload slide

Open in new tabDownload slide

Region of interest analyses in the older group. (A) The changes of high-gamma (pink) and beta (blue) amplitudes within each region of interest are plotted relative to stimulus onset (bold line: mean; thin lines: standard error). Upper and lower horizontal bars indicate the timing when ECoG amplitudes were significantly augmented or attenuated compared to the baseline value during the resting period, respectively. ‘+0.2’ indicates 20% amplitude augmentation compared to the baseline value during the resting period. pSTG = posterior superior-temporal gyrus; iPreCG = inferior precentral gyrus; POr = pars orbitalis of the inferior-frontal gyrus. (B) The changes of ECoG amplitudes in the left hemisphere are plotted relative to stimulus offset. cMFG = caudal middle-frontal gyrus; ENT = entorhinal gyrus; IPL = inferior parietal lobule; pCC = posterior cingulate gyrus; PHG = parahippocampal gyrus; pMTG = posterior middle-temporal gyrus; pSFG = posterior superior-frontal gyrus; PTr = pars triangularis of the inferior-frontal gyrus. POp = pars opercularis of the inferior-frontal gyrus; rMFG = rostral middle-frontal gyrus; SMG = supramarginal gyrus. (C) The changes of ECoG amplitudes in the right hemisphere are plotted relative to stimulus offset. (D) The changes of ECoG amplitudes are plotted relative to response onset. iPoCG = inferior postcentral gyrus.

High-gamma modulation around the offset of auditory stimulus

Within the left temporal-parietal lobes, high-gamma augmentation began to involve the left posterior middle-/inferior-temporal regions at 600 ms prior to stimulus offset, while the left superior-temporal regions continued to be activated (Fig. 5B). High-gamma augmentation subsequently involved the surrounding temporal-parietal lobe structures around stimulus offset.

Within the left frontal lobe, high-gamma augmentation began to involve the pars opercularis (BA44) of the left inferior-frontal region at 500 ms prior to stimulus offset, while the left inferior precentral region continued to be activated (Fig. 5B). High-gamma augmentation gradually spread to the anterior and superior structures in the left frontal lobe around stimulus offset. In the pars orbitalis (BA47) of the left inferior-frontal gyrus, high-gamma attenuation following stimulus onset was eventually replaced by high-gamma augmentation at 200 ms after stimulus offset. High-gamma augmentation in these left frontal lobe regions of interest lasted longer than that in the aforementioned left temporal lobe regions of interest.

While widespread regions in the left hemisphere showed high-gamma augmentation, high-gamma augmentation in the right hemisphere was minimal except for that lingering in the posterior superior-temporal gyrus. Rather, high-gamma attenuation involved the rostral middle-frontal gyrus, pars orbitalis of inferior-frontal gyrus, and inferior-parietal lobule of the right hemisphere with the maximum degree around stimulus offset (Fig. 5C). Such high-gamma attenuation preceded high-gamma augmentation in the contralateral homotopic regions by 400–600 ms.

High-gamma modulation around the onset of overt response

High-gamma augmentation continued to involve the posterior superior-frontal regions as well as inferior precentral regions bilaterally with left hemispheric dominance following stimulus offset. Such high-gamma augmentation peaked around response onset and gradually subsided over the duration of overt response (Fig. 5D). High-gamma augmentation in the other regions of interest in the left frontal lobe subsided by response onset. During a short period prior to response onset, high-gamma augmentation in superior-temporal gyri completely subsided, and high-gamma activity in the right posterior superior-temporal region was rather attenuated.

Difference between high-gamma and beta modulations

In general, high-gamma augmentation was well correlated with beta attenuation, whereas high-gamma attenuation was associated with beta augmentation with exceptions (Fig. 4). The most notable exception was that both high-gamma and beta activities were simultaneously augmented in bilateral posterior and anterior superior-temporal regions immediately following stimulus onset (Fig. 5A). Superior-temporal beta augmentation was subsequently followed by beta attenuation during stimulus presentation. We also found that high-gamma modulation was more dynamic compared to beta modulation; as best reflected in bilateral inferior precentral and posterior superior-frontal regions, high-gamma augmentation showed a sharper peak around response onset compared to beta attenuation (Fig. 5D).

Effect of age on high-gamma modulations

Only two regions of interest showed a significant difference in the degree of left-to-right asymmetry in high-gamma modulation between the older and younger groups (Fig. 6). In the older group, both rostral middle-frontal gyrus and pars orbitalis of the inferior-frontal gyrus showed high-gamma augmentation in the left hemisphere but attenuation in the right around stimulus offset. In the younger group, conversely, the rostral middle-frontal gyrus of either hemisphere failed to show significant amplitude modulation around stimulus offset, though the power to detect a difference is likely sufficient (56 electrodes from the left and 76 from the right). We did not find the objective evidence that such failure to observe high-gamma augmentation in the younger group can be attributed to the effect of left frontal lobe epilepsy. There was no difference in left rostral middle-frontal high-gamma activity between 29 sites derived from patients with left frontal lobe epilepsy and 27 sites from the remaining patients in the younger group (P > 0.05 on studentized bootstrap statistics).

Figure 6

Open in new tabDownload slide

Difference in auditory naming-related high-gamma modulation in the inferior/middle frontal gyri between the older and younger groups. Orange line: mean amplitude change in the left hemisphere. Green line: mean amplitude change in the right hemisphere. ‘+0.1’ indicates 10% amplitude augmentation compared to the baseline during the resting period. Pink vertical bars denote the epochs when left-to-right asymmetry significantly differed between the older and younger groups (Bonferroni corrected P < 0.05). cMFG = caudal middle-frontal gyrus; POr = pars orbitalis of the inferior-frontal gyrus; PTr = pars triangularis of the inferior-frontal gyrus; POp = pars opercularis of the inferior-frontal gyrus; rMFG = rostral middle-frontal gyrus.

In the younger group, pars orbitalis of the right inferior-frontal gyrus did not show high-gamma attenuation but rather augmentation around stimulus offset (Fig. 6). This observation is difficult to attribute to the effect of right frontal lobe epilepsy. There was no difference in right pars orbitalis high-gamma activity between 10 sites derived from patients with right frontal lobe epilepsy and 14 sites from the remaining patients in the younger group (P > 0.05).

Discussion

Our collective evidence, summarized in 3D and 4D, explains when and where the cortex exerts (i) perception of speech sounds; (ii) auditory-motor transformation of sound representations; (iii) short-term maintenance of speech sounds; (iv) mapping sounds onto a meaningful concept; (v) retrieval of a word relevant to the concept of the question heard; (vi) articulation of the word; and (vii) auditory feedback. Below we discuss how our study has supported, expanded, or refined the current neurobiological models of speech and language.

Roles of the temporal neocortex in speech recognition

Our 3D probability map derived from direct cortical stimulation is highly consistent with the proposed roles of temporal lobes in the dual-stream model of auditory speech processing (Hickok and Poeppel, 2007). This model proposes that (i) the bilateral superior-temporal gyri exert perception of speech sounds; (ii) the gyrus along the left superior-temporal sulcus subsequently does critical phonological processing; and (iii) the left middle-/inferior-temporal gyri map phonological onto semantic representations. Our stimulation study demonstrated: (i) stimulation of posterior superior-temporal gyri on both hemispheres elicited auditory hallucination; (ii) that of left posterior superior-/middle-temporal gyri elicited transient inability to understand questions asked by our neuropsychologist; and (iii) that of left posterior middle-/inferior-temporal gyri elicited transient inability to generate a relevant answer though understanding given questions (Fig. 3). Our observations in the left superior-/middle-temporal gyri were consistent with those in previous studies of intraoperative electrical stimulation (Ojemann et al., 1989; Sanai et al., 2008; Tate et al., 2014; Chang et al., 2016). Our study using chronic subdural electrodes expanded the extent of the 3D map to the inferior surface of the temporal lobes bilaterally; thereby, we found that stimulation of the left posterior inferior-temporal gyrus elicited transient inability to generate an answer to auditory questions.

Our 4D mapping expands the understanding of the temporal dynamics of each of the temporal lobe gyri for a given function during speech. High-gamma augmentation in the posterior superior-temporal gyri of both hemispheres peaked immediately following stimulus onset, became moderate throughout stimulus presentation, and began to subside at stimulus offset (Supplementary Video 1); this time course is consistent with the notion that these structures constantly process acoustic/phonological information throughout stimulus presentation. On the other hand, high-gamma augmentation in the left posterior middle-/inferior-temporal gyrus began at 3/4 of the total duration of stimulus presentation and peaked at stimulus offset. This observation indicates that much of the high-order process mapping phonological onto semantic representations takes place at once after a series of words such as ‘What flies in the sky’ are heard rather than constantly for each word like ‘What’, ‘flies’, and so on. This interpretation is consistent with the model suggesting that the brain initially maintains a set of words as a single chunk for a short time, and ‘digests’ the concept not of each single word but rather the full series of words (Wickens, 1970; Baddeley, 2000).

Roles of the inferior precentral gyrus in sensory-motor transformations and short-term maintenance of speech sounds

The collective data of this and previous studies suggest that the inferior precentral gyrus plays important roles in short-term maintenance of a series of spoken words before the conceptual processing taking place in the left posterior middle-/inferior-temporal regions. Our ECoG analysis demonstrated that, following stimulus onset, high-gamma augmentation sequentially involved the superior-temporal and inferior precentral gyri of both hemispheres. Inferior precentral high-gamma augmentation was initially bilaterally symmetric, but became left-dominant during the latter half of stimulus presentation (Fig. 5A). Early bilateral inferior-precentral high-gamma augmentation can be attributed to the processing of sensory-motor transformations for speech sounds (Cogan et al., 2014) or automatic buffer of perceived acoustic representations (i.e. covert rehearsal; Baddeley, 2000). This attribution is certainly supported by our observation of speech arrest induced by stimulation of inferior precentral gyrus on either hemisphere (Fig. 3A). We want to point out that sensory-motor transformations for speech sounds is primarily exerted by the inferior precentral gyrus (Rauschecker and Scott, 2009; Cogan et al., 2014) rather than the caudal left middle-frontal gyrus as delineated in Hickok and Poeppel (2007).

We believe that inferior precentral high-gamma augmentation (Figs 4B and 5A) also accounts for active short-term working memory maintenance of speech stimuli for the following reasons. Our recent ECoG study of 19 patients showed that the effect of verbal memory load was identified specifically in the left inferior precentral gyrus during the working memory maintenance period and that high-gamma augmentation in the left inferior- and middle-frontal gyri took place only during the subsequent scanning/selection period to determine a match among previously encountered items (Kambara et al., 2017). Other ECoG and functional MRI studies of visual working memory also found high-gamma and haemodynamic activations during the maintenance period (Howard et al., 2003; Meltzer et al., 2008; Noy et al., 2015), whereas functional MRI studies were not designed to effectively dissect activation during the maintenance and subsequent scanning/selection periods (Narayanan et al., 2005; Michels et al., 2010). A lesion-deficit study of humans demonstrated that lesions to the precentral gyrus impaired spatial working memory maintenance, whereas lesions to the dorsolateral prefrontal cortex (including the inferior frontal gyrus) had no effect on working memory performance if the precentral gyrus was spared (Mackey et al., 2016). Recent studies using ECoG, diffusion-weighted tractography, and cortico-cortical evoked potentials have indicated that the left inferior precentral gyrus is anatomically and functionally connected to the temporal neocortex showing auditory naming-related high-gamma augmentation (Brown et al., 2014; Keller et al., 2014). Our next logical step is to determine, using event-related causality analysis (Flinker et al., 2015), the network dynamics between the left inferior precentral gyrus exerting working memory maintenance and left temporal neocortex exerting the semantic process.

Roles of the left inferior- and middle-frontal gyri in lexical retrieval

The left inferior-frontal gyrus is denoted as a part of the dorsal stream of the dual-stream model of speech processing, and suggested to exert lexical retrieval (Hickok and Poeppel, 2007; Rauschecker and Scott, 2009). Our results in this study suggest that lexical retrieval may be exerted primarily by the left inferior-frontal gyrus but also by the left middle-frontal gyrus. Stimulation of the left inferior- and middle-frontal regions, primarily in the posterior portions, elicited transient inability to name a word relevant to a question (Fig. 3). Previous studies of large patient cohorts reported that the location of essential naming sites are variable in a given individual and that stimulation of the posterior portion of the left middle-frontal gyrus elicited transient naming error in subsets of patients (Ojemann et al., 1989; Duffau et al., 2002; Sanai et al., 2008; Tate et al., 2014; Chang et al., 2016). Functional imaging studies have shown that tasks involving naming consistently elicit haemodynamic activation in the left inferior- and middle-frontal gyri (Binder et al., 1996; Gaillard et al., 2000). Lesion studies have suggested that both left inferior- and middle-frontal gyri play critical roles in lexical retrieval (Seghier et al., 2016). Both functional imaging and ECoG studies reported that the pars opercularis (BA44) is preferentially activated by phonological tasks, whereas the pars triangularis (BA45) and orbitalis (BA47) by semantic tasks (Bookheimer, 2002; Mainy et al., 2008).

The exact role of the left middle-frontal gyrus in lexical retrieval has been relatively understudied, and current evidence fails to conclude that the roles in lexical retrieval drastically differ between the structures above and below the inferior frontal sulcus. Functional imaging studies have suggested that the left middle-frontal gyrus exerts working memory (Barch et al., 1997; Gaillard et al., 2000), whereas our ECoG study showed that both middle- and inferior-frontal gyri both showed high-gamma augmentation not during the working memory maintenance but during the scanning/selection period (Kambara et al., 2017). Lesion, imaging and ECoG studies using Stroop tasks suggest that the left middle-frontal gyrus plays a role in top-down control including selection of verbal responses (MacDonald et al., 2000; Aron et al., 2004; Koga et al., 2011), but also suggest that top-down control function may also be present in the left inferior-frontal gyrus.

Our ECoG analysis showed that naming-related high-gamma augmentation sequentially involved the pars opercularis, triangularis, and orbitalis within the left inferior-frontal gyrus across stimulus offset (Fig. 5B). Likewise, high-gamma augmentation sequentially involved the caudal and rostral regions within the left middle-frontal gyrus. These ECoG observations do not support the hypothesis that neural information is unidirectionally transferred from an anterior-to-posterior direction within the inferior-frontal gyrus (Rauschecker and Scott, 2009). Taking into account the connectivity between the temporal and frontal lobes (Matsumoto et al., 2004; Brown et al., 2014; Dick et al., 2014), our results rather support the notion that the posterior frontal lobe structures receive the neural information from the left temporal lobe neocortex before the anterior ones do.

Roles of the posterior superior-frontal and inferior precentral gyri in motor execution

Our observations are highly consistent with the notion that motor execution in speech are exerted by the posterior superior-frontal and inferior precentral gyri (Price, 2012). High-gamma augmentation began to involve bilateral posterior superior-frontal gyri around stimulus offset and peaked at response onset, with left hemispheric responses earlier and larger (Fig. 5D). High-gamma augmentation in the inferior precentral gyri peaked immediately after response onset, when the degree of augmentation was bilaterally symmetric. Speech arrest was elicited primarily by stimulation of the inferior precentral gyrus of either hemisphere; this observation is most consistent with a probabilistic 3D language map among others (Tate et al., 2014). Our study expanded the extent of the 3D map and showed that speech arrest was also elicited by stimulation of the left posterior superior-frontal gyrus (Fig. 3). Damage of this cortical structure can lead to mutism or expressive aphasia requiring speech therapy (Fontaine et al., 2002). On the other hand, damage restricted to the inferior precentral gyrus in the right hemisphere can lead to anarthria (Sugishita et al., 1987; Tanji et al., 2001), a deficit in linguistic prosody (Kuschmann and Lowit, 2012), or even expressive aphasia (Trojanowski et al., 1980).

Roles of the superior-temporal gyri in auditory feedback

Our 4D mapping demonstrated that high-gamma augmentation involved the posterior superior-temporal gyri of both hemispheres immediately after response onset, when the degree of high-gamma augmentation was a fraction of that during stimulus presentation (Fig. 5). This finding does not indicate that the same electrode sites showed high-gamma augmentation with a reduced degree but that only subsets of superior-temporal sites responded to the patient’s own voice with a similar degree (Supplementary Video 1), as reported in detail by other groups (Flinker et al., 2010; Greenlee et al., 2011). Superior-temporal high-gamma augmentation immediately after response onset was not left-hemisphere dominant; thus, this process may reflect auditory feedback for monitoring the intensity and pitch of their own voice rather than the semantic contents of responses (Lane et al., 1970).

Roles of the left medial-temporal and posterior cingulate regions in verbal memory

High-gamma augmentation involved the left parahippocampal gyrus prior to stimulus offset and subsequently involved the left entorhinal and posterior cingulate gyri. This stream is not included in the dual stream model of speech processing (Hickok and Poeppel, 2007; Rauschecker and Scott, 2009). Yet, further discussion is worthwhile, particularly because verbal memory decline was induced by surgical damage of the left parahippocampal gyrus showing high-gamma augmentation during covert picture naming followed by successful recall (Kunii et al., 2014). Our ECoG observation of left-hemispheric dominance supports the hypothesis that high-gamma augmentation in this stream would reflect episodic memory encoding and/or memory retrieval in the verbal domain. Conversely, another ECoG study using picture stimuli reported that parahippocampal high-gamma augmentation during episodic memory encoding was bilaterally symmetric (Burke et al., 2014). Other lesion and imaging studies suggest that the posterior cingulate cortex plays a role in autobiographical memory retrieval (Valenstein et al., 1987; Maddock et al., 2001). Studies using diffusion-weighted tractography and cortico-cortical evoked potentials have indicated that the parahippocampal and posterior-cingulate gyri are anatomically and functionally connected (Khalsa et al., 2014; Enatsu et al., 2015). Additional ECoG studies combining a control memory task are warranted to determine the role of high-gamma augmentation involving this ‘limbic’ speech processing stream.

Significance of high-gamma attenuation during the naming task

Our proposed model of speech and language includes that the human brain rapidly and alternately activates and deactivates cortical areas advantageous or obtrusive to function directed toward speech and language at a given moment. Our ECoG analysis showed regional high-gamma attenuation at three different timings.

• The pars orbitalis (BA47) within the left inferior-frontal gyrus showed high-gamma attenuation at 300–600 ms after stimulus onset while the network between the superior-temporal and inferior precentral gyri was presumably exerting working memory maintenance of speech sounds (Fig. 5A; Baddeley, 2000; Kambara et al., 2017). After stimulus offset, the left BA47 later showed high-gamma augmentation, reflecting semantic processing for lexical retrieval (Bookheimer, 2002; Mainy et al., 2008). We hypothesize that initiation of the verbal working memory maintenance process would suppress internal thought or mind wandering exerted by the left BA47 during the resting period (Fox et al., 2015).

• Around stimulus offset, high-gamma attenuation involved the rostral middle-frontal gyrus, pars orbitalis of inferior-frontal gyrus, and inferior-parietal lobule of the right hemisphere (Fig. 5C), all of which are often denoted to be part of the default mode network (Fox et al., 2015). Our novel observation of such high-gamma attenuation preceding high-gamma augmentation in the contralateral homotopic regions by 400–600 ms (Fig. 6) raises the possibility that transient deactivation of the right hemisphere may facilitate or disinhibit the homotopic language network on an as-necessary basis. Previous studies using repetitive transcranial magnetic stimulation showed that transient inhibition of the primary motor cortex facilitated function of the contralateral homologue (Kobayashi et al., 2004).

• Immediately prior to response onset, posterior superior-temporal gyrus in the right hemisphere showed mild but significant high-gamma attenuation. Plausible explanations include that either motor or auditory networks generate corollary discharge signals to modulate the neural activity in the superior-temporal gyrus (Flinker et al., 2010; Greenlee et al., 2011).

Developmental consolidation of the language system

The younger group presented left-hemispheric dominant high-gamma augmentation in the posterior frontal lobe structures but not in the rostral middle-frontal gyrus and pars orbitalis of the inferior-frontal gyrus (Fig. 6). The rostral middle-frontal gyrus (including BA46) failed to show high-gamma augmentation or attenuation in either hemisphere, whereas the pars orbitalis (BA47) showed bilateral high-gamma augmentation around stimulus offset. Our observation indicates that the developmental consolidation of the language system exerting the semantic process and high-order executive control (MacDonald et al., 2000; Aron et al., 2004) had not been completed. This notion is consistent with the results of lesion studies reporting that patients younger than 10 years can quickly recover from aphasia resulting from left frontal lobe damage, by efficiently transferring speech and language function to the right hemisphere (Vargha-Khadem et al., 1985; Boatman et al., 1999; Hertz-Pannier et al., 2002; Elger et al., 2004). Taking into account the tight correlation between high-gamma activity and haemodynamic/metabolic activities on imaging (Nishida et al., 2008; Scheeringa et al., 2011), our ECoG observation very well explains the previous functional MRI observation of bilateral representations of language processing in the frontal lobes in young children (Gaillard et al., 2000; Holland et al., 2001).

Significance of task-related high-gamma augmentation and beta attenuation

Our results support the notion that measurement of naming-related high-gamma augmentation is useful to predict the locations of stimulation-defined speech/language areas (Leuthardt et al., 2007; Kojima et al., 2012; Ruescher et al., 2013; Wang et al., 2016). The present study also showed that beta-attenuation was spatially and temporally correlated with high-gamma augmentation in general but with exceptions as follows. Substantial proportions of the posterior and anterior superior-temporal sites of both hemispheres showed beta augmentation immediately following stimulus onset and this was followed by sustained beta attenuation (Supplementary Video 1). Similar temporal dynamics of event-related beta modulation have been reported by ECoG studies of the primary auditory, visual, and somatosensory cortices (Crone et al., 2001; Fukuda et al., 2010; Uematsu et al., 2013); thereby, simultaneous augmentation of beta and high-gamma activity is attributed to the wideband evoked potentials elicited by sensory stimuli. Thus, it is not feasible to assume that the degree of beta attenuation is linearly correlated to the underlying activation in the primary sensory cortices, which clinicians desire to accurately localize during presurgical evaluation.

Advantages and limitations of our 3D and 4D mapping techniques

The advantage of our ECoG study includes the capability of assessment of both temporal dynamics of neural modulation and stimulation-induced behavioural changes in the lateral, medial, and inferior surfaces of the bilateral cerebral hemispheres with an unprecedented large cohort. Our 4D map (Supplementary Video 1) may be useful for teaching purposes, since the spatial-temporal dynamics of cortical activation related to language processing is easy to digest. Our 3D maps (Fig. 3) provide reasoning for cortical activation at a given region and determined the causal roles of given cortical structures in speech, language and sensory-motor functions. We recommend assessment of region of interest analysis (Fig. 5) for readers to confirm the degree of high-gamma modulation at spatial voxels sampled by less than six patients (Fig. 1), where the mean amplitude changes were presented in a less robust manner. Not all patients underwent the same neuropsychological batteries; thus, we are unable to directly compare the intelligence quotients between the two age groups. Nonetheless, considering the tight association between a larger number of antiepileptic drugs and more severe seizure burden as well as related cognitive impairment (Kwan and Brodie, 2001), the present study failed to find objective evidence that the effect of epilepsy or antiepileptic medications on ECoG measurement differed between the two age groups. A prior study has shown that interictal spikes from a frontal lobe seizure onset zone may transiently alter ongoing language processing dynamics at nearby sites outside of the seizure onset (Brown et al., 2012a); however, we excluded any trials associated with interictal spikes and our averaging technique and statistics would be expected to further minimize any influence upon our final results.

Our study was not designed to localize the regions involved in mapping visual onto semantic representations. Previous studies, with patients assigned naming tasks in both auditory and picture domains, reported that picture naming error was induced by stimulation of more posterior and basal temporal sites whereas auditory naming error by more anterior and lateral sites (Malow et al., 1996; Hamberger et al., 2001; Shimotake et al., 2015). Likewise, an ECoG study reported that picture naming elicited high-gamma augmentation specifically in portions of the left basal temporal region, whereas auditory naming in the left anterior superior-temporal gyrus (Cervenka et al., 2013). We currently plan to generate a 4D map of picture naming function, and compare the language networks in auditory and visual domains. We expect that such studies will localize the regions involved in semantic analysis in domain-specific and independent manners.

Future research

Invasive studies using ECoG and stimulation techniques can further improve our understanding of the developmental ontogeny of cortical network dynamics responsible for speech and language processing from infancy to adulthood (Cho-Hisamoto et al., 2012). Currently available models, primarily based on the results of lesions, non-invasive neurophysiology, and neuroimaging, suggest that variable language-processing functions emerge at distinct regions at different ages, and gradually grow or disappear in a heterogeneous manner (Minagawa-Kawai et al., 2011; Benasich et al. 2014; Skeide and Friederici, 2016). Although the developmental timeline for the emergence of different language functions has been outlined, the neurobiological basis for this timeline is poorly understood. Highly desired is a neurobiological model of human language development that takes into account the temporal dynamics by which neocortical networks subserving language and domain-general functions are recruited and interact with each other as the human brain matures. Such studies would require another large cohort of patients; thus, multicentred studies may be warranted.

Abbreviations

Abbreviations
 
• BA

  Brodmann area

 
• ECoG

  electrocorticography

Acknowledgements

We are grateful to Alanna Carlson, MS, LLP, Carol Pawlak, REEG/EPT at Detroit Medical Center, Wayne State University for the collaboration and assistance in performing the studies described above.

Funding

This work was supported by NIH grants NS047550 (to E.A.), NS064033 (to E.A.), MH107512 (to N.O.), and NS089659 (to J.W.J.), as well as the intramural grant from Children's Hospital of Michigan Foundation (to E.A.).

Supplementary material

Supplementary material is available at Brain online.

References