CHAPTER 25
Big Data and Artificial Intelligence

Ali Alim‐Marvasti

MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE

Artificial intelligence (AI) has been in development for decades. Several factors have contributed to its increasing success:

Big Data: Data and Storage Capacity

Increased storage of large amounts of digital clinical data in various forms, including personal wearable devices, electronic health records (EHR), laboratory results, neuroimaging, and EEG.
Hardware: Graphics Processors

Development of efficient microprocessors capable of multiple parallel computations and cloud computing, optimising the training of sophisticated deep learning models using big data.
Software: Algorithms, Deep Learning, and Generative AI

Advances in algorithms that improve accuracy, generalisability, and significantly reduce model training times, including deep neural networks. The advent of attention mechanisms in transformers, utilised in large language models and diffusion models, enables the generation of images and videos.

MACHINE LEARNING

Machine learning (ML) is the foundational tool on which AI is built. ML is based on mathematical and statistical learning models, whereby an error function is defined, and this error is iteratively minimised through incremental data and variable adjustments. There are different ML forms (Table 25.1).⁴ Examples of individual ML units include logistic regression and support vector machines (Table 25.2).

DEEP LEARNING AND ARTIFICIAL NEURAL NETWORKS (DNN)

DNN architectures with three or more layers of interconnected ML units are grouped to learn complex associations. DNNs, also known as artificial neural networks, were inspired by biological neural networks found in the brain’s cortical layers and the retina. Each DNN layer extracts a different pattern of the underlying data. This is akin to how retinal photoreceptors project information onward to bipolar and retinal ganglion cell layers to process other aspects of visual stimuli (feature abstraction).

Table 25.1 Machine Learning types.

Learning type	Description	Applications in epilepsy	Other applications	Human labels used?	Level of supervision	References
Supervised	Learns from input–output pairs with full labels	Seizure detection and classification from EEG signals	Email spam detection, classification	✅ fully labelled data	🔵 Fully Supervised	Automated epilepsy recognition from 500 EEG of 25 people with focal epilepsy and controls labelled from the Bonn dataset, using supervised ML and combined CNN and LSTM²
Semi‐supervised	Small labelled set guides learning from large unlabelled set	EEG analysis with limited seizure labels; seizure prediction combining labelled and unlabelled data	Medical imaging, classification with sparse labels	⚠️ Partially Partially labelled data	🔵 Mostly Unsupervised	Semi‐supervised seizure detection from EEG, using VAE¹
Self‐supervised	Learns by solving pretext tasks with data‐generated labels to overcome the problem of insufficient annotated data	Learning EEG representations by predicting masked signal segments; anomalous channel detection	Image colourisation, audio‐visual correspondence	❌ No manual labels (labels created from data)	🔵 Self‐Supervised	Self‐supervised learning with attention mechanism for EEG‐based seizure detection from 23 cases, nearly 1000 hours of recordings of 173 seizures²
Unsupervised	Finds hidden patterns or structure in data without labels	Discovering EEG signal clusters; phenotyping; seizure identification without labelled data	Anomaly detection	❌ No labels at all	🔵 Fully Unsupervised	Unsupervised EEG seizure identification using VAE, which is trained on normal recordings only³

CNN: convolutional neural network. LSTM: long short‐term memory. VAE: Variational Autoencoder.

Table 25.2 Supervised learning: learning from and automating expertise.

Epilepsy data modality	Supervised ML model	Brief model description	Diagnostic example(s)	Therapeutic example(s)
Clinical history and seizure semiology text (EHR)	Logistic regression	A taxonomy text of equivalent words and phrases searches EHR to devise continuous and binary predictor variables, then used to predict a binary target variable of interest using the log‐odds formula	Temporal lobe or extra‐temporal epilepsy as target variable⁵	ASM suggestions, predictions of who may need epilepsy surgery
Clinical history and seizure semiology text (EHR)	Support Vector Machine (SVM)	A taxonomy of equivalent words and phrases is used to search EHR and devise continuous and binary predictor variables, which are then plotted in high‐dimensional space. SVMs determine hyperplanes that optimally differentiate the categories of the target variable
Neuroimaging data (MRI)	CNN	Convolutional neural networks are a type of DNN in which individual units in each layer only consider information from a limited set of units of the previous layer, akin to the visual system’s receptive fields. Multiple layers of units (‘neurones’) abstract features that, with each progressive layer receive inputs from a broader ‘receptive field’ of the original MRI or EEG data	Automated detection of focal cortical dysplasia, tumours, and postsurgical resection cavities⁶	Automated guidance of intracranial electrode implantation to avoid critical cortex and vasculature
EEG (digital readouts)	CNN		Automated spike detection, automated labelling of EEG records	Automated EEG spike detection, e.g. for automated delivery of responsive neurostimulation
Large single modality (EHR) or multimodal data (text, MRI, EEG)	CNN, RNN, LLM	Recurrent DNNs have connections that feed back outputs into their inputs or previous layers, providing memory or context. Such models are suited for language comprehension. To be effective in epilepsy, they need to be trained on large amounts of epilepsy‐specific data. Recurrent convolutional networks could analyse time‐sequence video, EEG, or other data	Analysis of video‐telemetry data integrating semiology and EEG findings to diagnose a seizure, and determine the time and electrodes of onset	Diagnosis of epilepsy, frequency, and determination of semiologies, prognostication

DNNs come in different architectural flavours based on how each layer connects to the next. Recurrent Neural Networks (RNN), specifically a subtype called Long Short‐Term Memory (LSTM), and Convolutional Neural Networks (CNN) are commonly used for language and image processing. DNNs can be used in supervised and unsupervised learning (Table 25.1).

ARTIFICIAL INTELLIGENCE (AI)

AI systems are high‐dimensional ML algorithms that perform exceedingly well in particular tasks and simulate higher cognitive functions, such as understanding, language, and problem‐solving. AI algorithms are usually based on DNN architectures. AI can mimic or exceed human‐level understanding or performance in the topic it was trained on. Examples of AI include self‐driving cars.

GENERATIVE AI (‘GEN AI’)

Generative AI is a subset of models that, rather than classify or analyse data, generate new data resembling real‐world data. These models simulate creativity and develop new, original content. Generative models are prone to ‘hallucinating’ images or data that are either impossible or incorrect. Two popular frameworks include transformers, such as large language models (LLM, e.g. ChatGPT, Gemini, DeepSeek, Llama), and diffusion models that can generate images and videos (e.g. DALL‐E, MidJourney, StableDiffusion).

SUPERVISED MACHINE LEARNING

ML and AI can be adapted for specific objectives. For example, to classify a seizure using an expert‐consensus devised seizure classification, an automated algorithm that can read clinical histories and EEG is required. The family of suitable algorithms would be supervised ML, as the algorithms for labelled cases need to be supervised (Table 25.2).

Supervised ML involves iterations of the following:

Data curation, processing, and splitting data into training, cross‐validation, and test sets
Initial choice of model
Training and cross‐validating the model on the data for parameter selection
Testing model on hold‐out test set
Further evaluations of the model, including external and prospective data.

With any ML model, including DNN, it is easy to overfit idiosyncrasies in data, which may make the model seem attractive when tested in‐house, but may yet have limited generalisability. Test sets and external/prospective validation (the final two iterations in supervised ML) are essential to avoid over‐fitting and ensure generalisability. When assessing a model’s performance, it is helpful to conduct a brief mental evaluation (Table 25.3).

Table 25.3 ML quick appraisal checklist.

AI topic	Appraisal question(s)	Comments and example(s)
Data integrity	How do we know the data are accurate?	What is the evidence the data are accurate? For small data: has it been independently verified? Or have random manual checks been performed for big data to compare with original data sources (e.g. EHR)? For example, if a natural language processing tool were used to extract features, would a random subset of the features have been manually checked against the clinical history?
Data representativeness and generalisability	How representative is the training data of the population of interest?	For example, were the selection criteria chosen such that they may boost performance (‘overfitting’)? Are the data and demographics representative of people with epilepsy across the world?
Data bias and generalisability	Are there any biases in the data (e.g. demographics)?	Are there explicit sex, ethnicity, or age biases? May also consider agents used to collect data and unconscious bias involved in data curation. For example, people with epilepsy and intellectual disabilities have worse outcomes than the general population. However, they are persistently underrepresented in research⁷
Model generalisability	Does the model perform significantly worse on specific (demographic) subsets of the data?	Biases in big data may not be easily noticeable. When the trained model is tested on specific subpopulations (e.g. individuals with learning disabilities), is its performance on par with that of the general population?
Model training and generalisability	Was a hold‐out test set used to give an unbiased estimate of the model’s performance?	Algorithms perform significantly better on data they have been trained on, and so if a study reports training or cross‐validated performance, this is likely inflated (‘overfitting’) compared with how it would perform in real life
Model training and generalisability	In studies without a test‐set, were cross‐validation folds used to report an average of the model’s performance (desirable), or was the model’s performance quoted on the ‘refit’ entire data (undesirable)?	With less data, e.g. infrequent epileptic phenotypes, it may be acceptable not to have a test set. In such cases, it is crucial for cross‐validation to serve as a substitute for the test set without requiring further training of the model. Otherwise, some standard algorithms use the cross‐validation folds to train the model further, further inflating performance metrics (‘overfitting’)
Model training and generalisability	In studies without a test‐set, were regularisation parameters used?	Regularisation reduces the performance metrics of a model on training data, in exchange for better generalisability. Regularisation can take many forms and names, depending on the exact model used
Model performance metrics	Which performance metric was used and why? Was the performance metric benchmarked? Were suitable confidence intervals provided?	Ideally, it should provide the full confusion matrix from which scores are derived. Accuracy, sensitivity, specificity, or even area under the receiver operating characteristic curve (AUC) alone may be appropriate depending on the data. PPV, NPV, balanced accuracy, and MCC are alternative metrics; the latter two are particularly useful for unbalanced data when the target variable is infrequent (<10%). Benchmarking should be available using both the proportion of target variables in the data and the best univariate predictor. This enables us to judge the value of the multimodal model compared to simply guessing or knowing the state of a single clinical variable. Confidence intervals could be provided using bootstrapping, making it easier to compare overlaps in performance with the benchmarks

UNSUPERVISED MACHINE LEARNING

Elucidating novel insights requires an algorithm to identify underlying data patterns independently, such as underlying factor structures. As an example, to understand the relationship between epilepsy comorbidities, other features, and treatment resistance, and to use such clusters to stratify people with epilepsy who respond to medication or surgery. In this approach, the answer is to supervise the algorithm, and thus the family of suitable algorithms includes unsupervised ML (Table 25.4). Finding patterns in unstructured data can help exceed performance beyond current expertise by providing novel insights.

Table 25.4 Unsupervised learning: exploring patterns in unstructured data.

Epilepsy data modality	Unsupervised machine learning method	Brief model description	Comments and example(s)
Video telemetry or multimodal continuous data	Principal components analysis (PCA)	PCA is a dimensionality reduction algorithm preserving variances (a measure of spread). All data must be on a similar scale. For example, when given data from multiple camera feeds, not all contain unique information PCA is akin to finding the 2D shadow that best resembles a 3D object	PCA is best suited for continuous and complex data when we lack insight into the main dimensions of the data. PCA eliminates autocorrelations between covariates, and the results of PCA can later be used in supervised learning for better training performance E.g. several cameras during video telemetry show an individual from different angles, but with correlated data. PCA would be a suitable method to reduce the data. PCA can also be used to ascertain the most useful underlying factors reported epilepsy questionnaires⁸
fMRI	Hierarchical clustering (HC)	HC groups individuals together that have the most shared attributes for a given variable, in a nested (hierarchical) tree manner	Individuals are grouped based on similarities (‘inverse distance’) in a given feature. Clustering can be highly sensitive to the ‘similarity measure’ (e.g. Euclidean distance, Spearman correlation). The similarity measure should always be explicitly stated and justified E.g. patterns of inferior frontal gyrus and Wernicke’s area activation are used for automated HC classification of atypical language activation patterns in focal epilepsy⁹
Decomposed EEG signals	K‐means clustering (KMC)	KMC clusters based on a plot of the data, and in 2 dimensions clusters in ‘circles’ around a mean, based on how close data are to the cluster’s mean	KMC is sensitive to the similarity measure (‘distance’) and requires a good estimate of ‘k’, the number of clusters into which the data should be grouped As the initial location of clusters is randomised, each run of KMC can yield different results even for the same value of k and similarity measure. In contrast, HC yields consistent results. An example would be using KMC on decomposed EEG signals as subsequent features to a DNN to diagnose epileptic seizures¹⁰
EEG or population‐level clinical data (e.g. from EHR)	t‐distributed stochastic neighbour embedding (t‐SNE)	t‐SNE allows visualisation of large high‐dimensional data in lower‐dimensional space and plots similar individuals next to each other. t‐SNE is an alternative to KMC and PCA	t‐SNE preserves similarity of co‐occurrence of underlying comorbidities between individuals (cf. PCA preserves variance). While PCA may collapse subtle differences between categories, t‐SNE preserves these For example, t‐SNE could be used to map processed EEG signals down to 2 dimensions, before a supervised algorithm detect epileptic seizures¹¹
EEG and MRI	(Variational) Autoencoder DNN (AE)	Autoencoders are DNN that have a middle layer with significantly less units than the data input dimensions. In contrast, the initial and last layers of the DNN have the same number of units. This middle layer is an accurate compressed (latent) representation of the data (encoded) that informs us of the underlying structure inherent in the data	AE can replace PCA or other dimensionality reduction techniques before supervised learning. VAE are a statistical variant that can be used to generate more data. This may be particularly useful when we have scarce real‐world data; or when we wish to improve generalisability on external data that may be slightly different and ‘noisy’ E.g. VAE to develop a latent representation of normal EEG and an anomaly detection algorithm to then diagnose seizures on EEG^1,3 VAE can also be used for MRI image reconstruction and removal of artefacts, aiding better imaging diagnostics¹²

AI‐ASSISTED UNDERSTANDING AND DIAGNOSIS OF EPILEPSY

Many AI algorithms can allegedly assist in the diagnosis and classification of epilepsies, using data from clinical history and investigations. Algorithms can aid in seizure prediction, localisation, and lateralisation, and provide clinical decision support and outcome predictions for surgery.¹³ The primary data sources are the history and semiology from EHR, imaging, and EEG.¹⁴

The rise of LLM has revolutionised natural language processing from EHR, which in the past required highly specialised algorithms for medical, neurological, and epilepsy text processing, including specialised ontologies.^5,15,16 The accuracy of GPT‐4 LLM’s diagnosis of epileptic vs dissociative seizure improves from 57% to 64% with only one example of dissociative seizures. This modest and below‐expert performance is achieved without fine‐tuning the LLM on a dedicated corpus of epilepsy semiology.¹⁷ The breadth and depth of LLM understanding can be applied and retrained on epilepsy specific data. LLMs will inevitably outperform pre‐LLM bespoke algorithms on semiology localisation and lateralisation.⁵

Algorithms can already help remove artefacts in neuroimaging by using autoencoders,¹² and detect anomalies on MRI that human experts may miss, including cortical dysplasia and hippocampal sclerosis, with increasing accuracy.^6,18–21

Seizure detection from EEG signals has traditionally required a large amount of labelled data and the use of semi‐supervised and self‐supervised methods (Table 25.1). Besides traditional signal processing methods, a multitude of techniques have been implemented, including SVM, PCA, KNN, CNNs, LSTM, AE, and others.^1–3,22 The best performances report accuracy metrics of more than 90%, or in the region of 95%.^1–4,22 Seizure forecasting has also demonstrated increasing success in small studies.²³ The current models have variable accuracies; however, they often have high false positives, may be sensitive to noise, and may be poorly generalisable. The performance metrics are also not usually directly comparable due to differences in datasets and target variables (seizure vs normal or seizure vs interictal). These have been barriers to their adoption in clinical practice.

Seizures can also be detected using wearable accelerometers and video telemetry with increasing accuracy; e.g. by employing a CNN to analyse video feeds and ML models to diagnose tonic, tonic–clonic, or focal motor seizures.²³

The strength and promise of AI lie in its ability to represent underlying features that require separate algorithms, such as semiology, graph theory, comorbidities, and imaging features, all within a single algorithm. Ideally, seizure diagnosis and prediction models should utilise multimodal data from digital wearables, investigations, and clinical history.^14,24,25

AI‐ASSISTED TREATMENT OF EPILEPSY

AI can assist epilepsy treatment by considering individual factors, comorbidities, and localisation and lateralisation of seizures for drug‐resistant focal epilepsy. Algorithms are well poised to help determine the choice of ASM and assist in epilepsy surgery and prognostication.^25–27

While multivariable models exist for predicting seizure recurrence and 1‐year seizure freedom, e.g. from a large meta‐analysis including over 1750 people,²⁸ their accuracy and general clinical applicability remain unproven.

ASM choice is a clinical decision which becomes less evidence‐based the greater the number of ASMs that have failed to control seizures. Comorbidities, medication interactions, genetics, and other factors may help determine the likelihood of responsiveness to a particular ASM and the risk of significant side effects. Precision medicine for ASM recommendations will require the use of each individual’s unique biological and clinical data. Challenges that remain to be addressed include data availability, heterogeneity, and the necessity for algorithms to be regularly updated with new data and undergo continuous clinical validation.¹⁹

Various algorithms have been developed for computer‐assisted resection for drug‐resistant focal epilepsy and surgical prognostication.^13,14,25 Precision epilepsy surgery, including surgical planning and prognostication, is arguably more advanced than AI‐assisted ASM recommendations, partly due to the lack of need for genetic and other data not routinely obtained for all individuals.^29,30

BIG DATA, AI, AND ETHICS

For AI, large amounts of data are required. Data integrity is paramount, encompassing privacy, bias, representability, and generalisability. Due consideration must be given to the lawful and ethical use of data at every stage, from collection to anonymisation, AI model training, clinical validation, and automated clinical decision support systems. The European Union has developed a ‘human‐centric’ approach to data and AI, endeavouring to protect fundamental rights and respect individual dignity.³¹ This Expert‐Group’s Assessment List for Trustworthy AI (ALTAI) was published in 2020.^31,32 ALTAI’s principles can be used as a checklist to ensure AI systems are ethical and robust:

Human Agency and Oversight: Evaluate the degree of human oversight
Technical robustness and safety: Evaluate the degree to which models have been tested and validated for vulnerabilities
Privacy and data governance: ensure systems comply with the law and that mechanisms for data governance and privacy exist
Transparency: ensure the model design and decision‐making processes are transparently documented
Diversity, non‐discrimination, and fairness: check whether data and algorithms were assessed for bias towards minorities or those with disabilities
Societal and environmental well‐being: evaluate whether the system has implications for employment, society, and democracy
Accountability: check that the system has processes in place for independent audits and/or an AI ethics review board to ensure accountability

Algorithms, including LLMs, can perpetuate racial and gender biases,³³ and generative AI can exhibit significant cognitive biases.³⁴ Data curation, study design, and algorithm choice may have ethical ramifications that need to be identified using structured frameworks.³⁵ Available tools should be used to ensure AI is implemented responsibly.^31,36

FUTURE APPLICATIONS OF AI IN EPILEPSY

Clinical decisions are often nuanced and take into account more than the EHR, imaging, and investigation findings. During decision‐making, clinicians usually consider the individual’s non‐verbal communication, their specific circumstances, and personal priorities. Different medications can be trialled until the benefits outweigh the risks for the individual. Reinforcement learning algorithms explore a vast space of rewards and consequences, enabling numerous individual diagnostic and treatment decisions to be made to maximise cumulative reward. This could be defined as a combination of objective and subjective health outcomes (e.g. seizure freedom and acceptable side effects). Acceptable side effects are difficult to predict without firsthand experience, and reinforcement learning can provide feedback at every stage of the treatment paradigm. Reinforcement learning has been successful in mastering computer games and digital environments where feedback can be perfectly simulated; however, its applications to diagnostics and therapeutics have not yet been successful.

Unsupervised methods may yet reclassify epilepsies based on comprehensive and integrated video telemetry, imaging, neurophysiology, and genetic panel data. AI applications to epilepsy include the prediction of epilepsy and seizures before onset with increasing accuracy, which may dramatically reduce seizure‐associated morbidity.^14,24 It will help elucidate the deep covariant structures of epileptic comorbidities, hinting at shared aetiology and pathophysiology, and personalise treatment, from tailoring medication plans to optimising surgical strategies.

The current bottlenecks in the clinical applications of AI are data integrity, sharing, and generalisability, rather than whether the algorithms are capable of performing the tasks. This has led to calls for ML and AI research to shift focus from model building to data curation and cleaning, known as data‐centric AI.

In recent years, the sophistication of algorithms and applications to extensive datasets has greatly enhanced generalizability. While earlier models aimed to reduce clinicians’ workload through clinical decision support, AI tools present a possible future in which epilepsy can be diagnosed and classified with minimal clinical oversight.³⁷ Ultimately, combinations of AI models with subspecialised architectures may be integrated as various components of a single AI agent for the comprehensive management of people with epilepsy.

Tags: Epilepsy a queen square textbook

Oct 17, 2026 | Posted by admin in NEUROLOGY | Comments Off

Neupsy Key

Fastest Neuropsychology Insight Engine

25: Big Data and Artificial Intelligence