Enhancing Precision Oncology: Deep Learning Models vs. Classical Machine Learning Models in Multi-Label Breast Cancer Classification

Advancements in single-cell RNA sequencing (scRNA-seq) provide critical insights into cancer heterogeneity, yet analyzing high-dimensional data remains challenging. This study compares GRU with Low-Rank Adaptation (LoRA), Transformer, and XGBoost for multi-label breast cancer classification. Using k-fold cross-validation and paired t-tests, results show GRU-LoRA and Transformer outperform XGBoost in accuracy, precision, recall, and F1-score, particularly for rare cancer subtypes. While XGBoost offers interpretability, deep learning models excel in capturing complex gene interactions. These findings underscore the potential of deep learning in precision oncology, enabling more scalable and accurate diagnostic tools.

Introduction

Single-cell RNA sequencing (scRNA-seq) has transformed cancer research by providing detailed insights into gene expression at the cellular level. However, its high dimensionality, sparsity, and complexity pose challenges for traditional machine learning models like XGBoost, which rely on manual feature selection and struggle with class imbalance. In contrast, deep learning approaches, such as Gated Recurrent Unit (GRU) with Low-Rank Adaptation (LoRA) and Transformer models, excel in learning complex gene interactions, making them highly effective for breast cancer subtype classification [1].

This study compares XGBoost, GRU-LoRA, and Transformers for multi-label breast cancer classification using scRNA-seq data. Performance is evaluated using k-fold cross-validation, precision, recall, and F1-score, with statistical significance assessed through paired t-tests. The findings highlight the advantages of deep learning in genomic analysis and its potential for enhancing precision oncology [2]. The datasets used in the study were obtained from publicly available repositories, specifically GEO datasets GSE167036, GSE176078, and GSE161529. These datasets include a variety of breast cancer subtypes—ER+, HER2+, TNBC, BRCA1—as well as non-cancer cell types. Appropriate sampling, preprocessing, and quality control enable robust model performance evaluation across high-dimensional and heterogeneous gene expression profiles.

The paper is structured as follows: Section 2 reviews related work, Section 3 outlines the research problem and methodology, Section 4 details model architectures, Section 5 presents results, and Section 6 concludes with future directions.

Related Works

Machine Learning in Genomic Data Analysis

Machine learning (ML) has revolutionized genomic research by enabling efficient classification, prediction, and feature extraction from high-dimensional biological datasets. Classical ML models such as Support Vector Machines (SVM), Random Forest, and XGBoost have been widely used in gene expression analysis, owing to their interpretability and robustness in structured data classification [3]. XGBoost, a tree-based ensemble learning method, is particularly effective in handling imbalanced datasets and nonlinear feature interactions, making it a preferred choice for biological classification tasks [4].

However, classical ML models often struggle with the high-dimensionality and sparsity of single-cell RNA sequencing (scRNA-seq) data, leading to performance limitations in feature selection and representation learning. These challenges have led to an increasing reliance on deep learning approaches, which automatically learn hierarchical representations from raw genomic sequences [5], [6].

GRU-LoRA in Genomic Research

Recurrent Neural Networks (RNNs) are well-suited for processing sequential biological data, such as gene expression time-series and cell differentiation pathways. Among them, Gated Recurrent Units (GRU) have demonstrated strong performance in modeling long-term dependencies and temporal dynamics in gene expression profiles. GRUs address vanishing gradient issues, enabling more efficient training on large genomic datasets [7].

The Low-Rank Adaptation (LoRA) technique, a parameter-efficient fine-tuning method, has further improved GRU’s performance in genomics by reducing memory usage and computational costs while maintaining classification accuracy. LoRA achieves this by injecting small, trainable matrices into pre-trained models, allowing efficient adaptation to new tasks without modifying the entire network [8]. This combination—GRU with LoRA—has been particularly useful for single-cell analysis, where training large models on limited annotated data is a challenge [8].

Transformer-Based Models for Genomic Applications

Transformers have emerged as state-of-the-art architectures for genomic data analysis, surpassing RNN-based models in both speed and accuracy. Unlike RNNs, Transformers process entire sequences in parallel, significantly improving computational efficiency for large-scale scRNA-seq data [9].

The key advantage of Transformers lies in their self-attention mechanism, which enables them to capture long-range dependencies between genes and identify complex gene regulatory networks. Pretrained models such as Pytorch’s Transformer-based architectures have been successfully applied to gene classification, sequence alignment, and cancer subtype identification, demonstrating high precision and recall in scRNA-seq tasks [10].

Additionally, Transformers have shown superior generalization capabilities, reducing overfitting through multi-head attention and positional embeddings. Their ability to integrate multi-modal genomic datasets makes them ideal for cancer subtype classification, where spatial and molecular interactions must be considered.

Problem Statement, Hypothesis Statement, and Research Question

Problem Statement

The problem is that traditional machine learning models like XGBoost and Random Forest struggle to capture complex gene interactions in single-cell RNA sequencing (scRNA-seq) data because of its high dimensionality, sparsity, and noise, necessitating the use of deep learning models such as GRU with LoRA and Transformer architectures for improved breast cancer subtype classification.

Hypothesis Statement

Deep learning models, specifically GRU-LoRA and Transformer architectures, improve accuracy, precision, recall, and F1-score for breast cancer subtype classification using multi-modal scRNA-seq data compared to traditional machine learning models like XGBoost.

Research Question

How do deep learning models, specifically GRU-LoRA and Transformer architectures, significantly improve performance in terms of accuracy, precision, recall, and F1-score for breast cancer subtype classification using multi-modal scRNA-seq data compared to traditional machine learning models like XGBoost?

Methodology

The study employs an experimental quantitative approach to evaluate the effectiveness of deep learning models (GRU-LoRA, Transformer) against a classical machine learning model (XGBoost) in classifying breast cancer subtypes using single-cell RNA sequencing (scRNA-seq) data. The study systematically manipulates and controls independent variables—model type (XGBoost, GRU-LoRA and Transformer)—to assess their impact on dependent variables—accuracy, precision, recall, and F1-score—through comparative analysis [10].

The methodology includes data collection, data preprocessing, feature selection, model training, hyperparameter tuning, and statistical analysis to determine which model performs best in classifying breast cancer subtypes. The study focusses on model evaluation on accuracy, precision, recall, and F1-score while also examining trade-offs in model interpretability and performance in multi-class handling and generalization [11].

Data Collection Procedure

The dataset utilized in this study consists of single-cell RNA sequencing (scRNA-seq) data obtained from publicly available genomic repositories, including the Gene Expression Omnibus (GEO), Cancer Genome Atlas (TCGA), and European Genome-Phenome Archive (EGA). These datasets provide high-dimensional gene expression profiles categorized into five breast cancer subtypes: ER+, HER2+, TNBC, BRCA1, and Normal. Data acquisition involved curating samples from multiple studies to ensure complete coverage of cells and genes information [12]. A systematic preprocessing approach was applied to obtain high quality data for the study. First, multiple scRNA-seq datasets were integrated while addressing batch effects and technical variations. Duplicates and redundant cell entries were removed to prevent data skewness. Cells with excessive missing values or ambiguous annotations were discarded to improve dataset reliability. A critical aspect of the preprocessing pipeline involved properly annotating the cancer and non-cancer labels, ensuring that the five subtypes—ER+, TNBC, HER2+, BRCA1, and Normal—were clearly defined for downstream analysis. The annotation process was essential for maintaining the biological relevance of the dataset, as inconsistencies in labeling could lead to misclassification errors in predictive analysis [13].

Data Analysis Procedure

A structured preprocessing pipeline used in the study ensured data integrity and consistency across models. Quality control involved filtering cells with high mitochondrial gene expression (>10%) and low gene counts (<200), while scrublet detected and removed doublets. Normalization was performed via log transformation and library size scaling, and the top 5000 most variable genes were selected using Scanpy’s variance thresholding method [14].

The dataset was split into 80–20 and 50–50 configurations to evaluate model performance and stability, with stratified sampling maintaining a balanced subtype distribution. XGBoost was tuned with a learning rate of 0.01, max depth of six, 100 estimators, and early stopping. GRU-LoRA used two bidirectional layers with 256 hidden units, dropout (0.3), and LoRA adaptation for efficiency. The Transformer model included four self-attention layers, 512 embedding dimensions, and ReLU activation with dropout (0.3).

XGBoost Model

The XGBoost classifier is a gradient boosting model optimized for multiclass classification. It uses the multi:softmax objective, applying a SoftMax transformation to assign probabilities across multiple classes and tree leaf weight is applied L2 regularization in a way that minimizes the loss function while maintaining model stability and preventing overfitting.

SoftMax function to convert the class probabilities:

$$\begin{array}{r}P(y=k|\mathit{x})=\frac{e^{f_k\left(\mathit{x}\right)}}{\sum _{j=1}^K{e}^{f_j\left(\mathit{x}\right)}}\end{array}$$

$$\begin{array}{r}\text{mlogloss}=L=-\frac{1}{N}\sum _{i=1}^N\sum _{k=1}^K{y}_{i,k}\log p_{i,k}\end{array}$$

$$\begin{array}{r}{L}_{\text{total}}=mlogloss+L2+L1==L+\lambda {\sum }_j{\beta }_j^2+\alpha {\sum }_j\left|{\beta }_j\right|\end{array}$$

Using Taylor approximation for tree optimization:

$$\begin{array}{r}{g}_i=\frac{\mathrm{\partial }f\left(x_i\right)}{\mathrm{\partial }L},h_i=\frac{{\mathrm{\partial }}^{2}f\left(x_i\right)}{{\mathrm{\partial }}^{2}L}\end{array}$$

Gradient base weight and tree optimization:

$$\begin{array}{r}{w}_j^{\ast }=-({\sum }_{i\in I_j}{h}_i+\lambda {\sum }_{i\in I_j}{g}_i)\end{array}$$

where $P(k|x)$ is the predicted probability that that the input features x belongs to class k, $f_k\left(x\right)$ is the unnormalized logit output by the model for class k, and K is the total number of classes. $f\left(x_i\right)$ and, $p_{i,k}$ are the raw prediction score and predicted probability for class k, $\lambda$ and α are L2 and L1 regularization terms, βj are the leaf weights, $g_i$ and $h_i$ are the first and second order derivatives for tree optimization.

The XGBoost model minimizes a regularized objective function that integrates prediction error and model complexity. It typically uses multiclass log loss (mlogloss) as the loss function to evaluate how closely the predicted class probabilities align with the true labels. To prevent overfitting and enhance generalization, the model includes regularization terms: L1 regularization encourages sparsity by reducing the influence of less relevant features, while L2 regularization penalizes large weight values to maintain model stability. Optimization is performed using Taylor expansion, which incorporates both the first-order derivative (gradient) and the second-order derivative (Hessian) to guide efficient tree structure updates. The final output is a set of additive decision trees that progressively correct errors from previous iterations and achieve accurate prediction.

GRU-LoRA Model

Gated Recurrent Unit (GRU) with Low-Rank Adaptation (LoRA) is an effective deep learning model for sequence-based genomic data analysis while optimizing computational efficiency and preserving model expressiveness. The GRU cell processes sequential inputs through gating mechanisms, where the update gate controls past information retention, the reset gate determines how much prior hidden state influence is retained, and the candidate activation generates new state information. The final hidden state is computed as a weighted combination of the past and candidate states [8].

The candidate hidden state is computed as:

$$\begin{array}{r}\widetilde{h_t}=\tanh (W_h{x}_t+U_h(r_t\odot h_{t-1})+b_h)\end{array}$$

The update hidden state is computed as:

$$\begin{array}{r}{h}_t=(1-z_t)\odot h_{t-1}+z_t\odot \widetilde{h_t}\end{array}$$

where $z_t$ is update state, $r_t$ is reset state, $W_h$ and $U_h$ are weight matrices and $b_h$ is offset parameter.

LoRA enhances this process by decomposing weight updates into low-rank matrices, effectively reducing the number of trainable parameters and maintaining performance. This modification replaces full-rank weight updates with a low-rank factorization, where the weight update ΔW = AB introduces minimal additional computation. The final weight update follows Wnew = W+ΔW ensuring a lightweight yet robust architecture suitable for high-dimensional genomic sequence modeling.

Transformer Model

The Transformer model is designed for high-dimensional genomic sequence classification using self-attention mechanisms to capture long-range dependencies within scRNA-seq data. Unlike recurrent models such as GRU, which process sequences sequentially, the Transformer operates in parallel, making it more computationally efficient for high dimensional features and sequence dataset. The model employed in the study begins with an input projection layer, where the input dimension is projected to a 768-dimensional space using a fully connected layer. This transformation is followed by Batch Normalization to stabilize training and improve convergence. A dropout layer is applied to prevent overfitting before feeding the processed input into the Transformer encoder. The encoder with six tacked layers and multi-head self-attention computes attention scores using query (Q), key (K), and value (V) matrices [10], [15].

The model assigns probability scores to each class using the SoftMax function, ensuring that all predicted outputs sum to one, enabling accurate classification:

$$\begin{array}{r}P(y=k|x)=\frac{e^{f_k\left(x\right)}}{\sum _{j=1}^K{e}^{f_j\left(x\right)}}\end{array}$$

where $P(y=k|x)$ is the predicted probability that that the input features x belongs to class k, $f_k\left(x\right)$ is the unnormalized logit output by the model for class k, and K is the total number of classes.

The attention outputs are passed through a two-layer feed-forward network, applying ReLU activation to introduce non-linearity and improve feature extraction:

$$\begin{array}{r}FFN\left(x\right)=max(0,x{W}_1+b_1)W_2+b_2\end{array}$$

where x is the input feature vector, $W_1$ and $W_2$ are weight matrices, $b_1$ and $b_2$ are bias parameters

Each head learns attention independently, and results are concatenated and projected to the final representation. Multihead attention allow the model to learn different attention representations for gene features by processing multiple attention heads in parallel:

$$\begin{array}{rl}MultiHead(Q,K,V)& =Concat\\ & (hea{d}_1,hea{d}_2,\dots ,hea{d}_h)W_O\end{array}$$

Optimization and Loss Computation

To address class imbalances, Weighted CrossEntropy Loss was used in the deep learning models train and test ensuring that minority classes contribute proportionally to the loss. Also, GRU-LoRA and Transformer models used the AdamW optimizer, with a learning rate of 1e-4 and weight decay to improve generalization and XGBoost used log loss optimization with L1 and L2 penalties to reduce overfitting. Fig. 1 represents data analysis workflow for the breast cancer and non-cancer labels classification [15].

Fig. 1. Data analysis workflow.

Performance Metrics

To evaluate the effectiveness of the models in breast cancer subtype classification using single-cell RNA sequencing (scRNA-seq) data, multiple performance metrics were employed. These metrics include accuracy, precision, recall, F1-score, to ensure a comprehensive assessment of model performance:

$$\begin{array}{r}Accuracy=\frac{TP+TN}{TP+TN+FP+FN}\end{array}$$

$$\begin{array}{r}Precision=\frac{TP}{TP+FP}\end{array}$$

$$\begin{array}{r}Recall=\frac{TP}{TP+FN}\end{array}$$

$$\begin{array}{r}\mathit{F}\mathit{1}\mathit{\text{-}}\mathit{S}\mathit{c}\mathit{o}\mathit{r}\mathit{e}=2\times \frac{Precision\times Recall}{Precision+Recall}\end{array}$$

Validation Metrics

To ensure the robustness and generalizability of the models in classifying breast cancer subtypes from single-cell RNA sequencing (scRNA-seq) data, a five-fold cross-validation was performed, followed by a paired t-test to compare accuracy and F1-score on the minor class like BRCA among XGBoost, GRU-LoRA, and Transformer models. Five-fold cross-validation is a widely used technique in machine learning to assess model performance while reducing the risk of overfitting. The dataset is divided into five equal folds, where each model is trained on four folds and tested on the remaining fold. This process is repeated five times, ensuring that each fold serves as a test set exactly once. The final accuracy is computed as the mean of the five validation results. The paired t-test is used to determine whether the differences in model performance between the deep learning models (GRU-LoRA, Transformer) and the classical model (XGBoost) are statistically significant [15].

Results

The classification reports for both the 80–20 and 50–50 data splits demonstrate the comparative performance of the XGBoost, GRU, and Transformer models in predicting breast cancer subtypes. Tables I and II represent comparison of the performance matrics of the deep learning models (GRU-LoRA and Transformer) and XGBoost, particularly in terms of F1-score for BRCA1 classification. In the 80–20 split, the Transformer model achieved the highest overall accuracy, followed closely by GRU with Low Rank Adaption (LoRA), while XGBoost displayed lower precision and recall, indicating challenges in handling high-dimensional scRNA-seq data. Similarly, in the 50–50 split, both deep learning models maintained superior predictive performance, reinforcing their robustness in breast cancer subtype classification [16].

Table I. MODEL Comparison (50–50)

Model	Class	Precision	Recall	F1-score
XGBoost	BRCA	0.82	0.67	0.75
XGBoost	ER	0.86	0.90	0.88
XGBoost	HER2	0.81	0.76	0.78
XGBoost	Normal	0.93	0.96	0.94
XGBoost	TNBC	0.84	0.80	0.82
Trans	BRCA	0.90	0.86	0.88
Trans	ER	0.93	0.93	0.93
Trans	HER2	0.88	0.88	0.88
Trans	Normal	0.97	0.97	0.97
Trans	TNBC	0.90	0.89	0.90
GRU	BRCA	0.91	0.88	0.89
GRU	ER	0.94	0.94	0.94
GRU	HER	0.89	0.90	0.89
GRU	Normal	0.97	0.98	0.98
GRU	TNBC	0.90	0.90	0.90

Table II. MODEL Comparison (50–50)

Model	Class	Precision	Recall	F1-score
XGBoost	BRCA	0.82	0.67	0.74
XGBoost	ER	0.86	0.90	0.88
XGBoost	HER2	0.80	0.76	0.78
XGBoost	Normal	0.92	0.96	0.94
XGBoost	TNBC	0.84	0.79	0.82
Trans	BRCA	0.88	0.81	0.85
Trans	ER	0.90	0.94	0.92
Trans	HER2	0.89	0.82	0.86
Trans	Normal	0.95	0.97	0.96
Trans	TNBC	0.89	0.88	0.88
GRU	BRCA	0.88	0.88	0.88
GRU	ER	0.92	0.95	0.94
GRU	HER	0.90	0.87	0.88
GRU	Normal	0.97	0.97	0.97
GRU	TNBC	0.91	0.88	0.89

The ROC-AUC scores in Fig. 2 compare the predictive performance of XGBoost, Transformer, and GRU-LoRA models across two data splits: 80–20 (training-testing) and 50–50 (balanced split). These scores measure how well each model distinguishes between different breast cancer subtypes and normal samples. The results indicate that both Transformer and GRU-LoRA models significantly outperform XGBoost in classifying breast cancer subtypes using scRNA-seq data. GRU-LoRA achieved the highest ROC-AUC scores across all subtypes, demonstrating its ability to effectively capture sequential gene expression patterns [17], [18].

Fig. 2. Bar plot of the models ROC-AUC score comparison.

Fig. 3 presents a line chart of accuracy comparing the performance of GRU with LoRA, Transformer, and XGBoost across multiple test runs. The chart illustrates the trend of accuracy improvements over epochs, highlighting how deep learning models, GRU and Transformer, achieve higher accuracy compared to the classical XGBoost model. While XGBoost shows stable performance, its accuracy plateaus earlier, whereas GRU with LoRA demonstrates steady improvement, benefiting from low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The Transformer model also achieves high accuracy, leveraging self-attention mechanisms for learning intricate relationships in scRNA-seq data. The visualization in Fig. 3 reinforces the study’s findings that deep learning models outperform classical methods in classifying breast cancer subtypes with greater precision and reliability [19].

Fig. 3. Line plot of the models comparison.

To statistically validate the performance differences, a paired independent t-test was conducted to compare the accuracy and F1-scores of the models. As shown in Table III, the p-values for accuracy comparisons between GRU-LoRA and XGBoost, as well as Transformer and XGBoost, were below 0.05, confirming that the observed differences were statistically significant. Additionally, the t-test evaluation of the model performance on underrepresented classes, particularly BRCA1, the F1-scores from both the 80–20 and 50–50 data splits were analyzed. The comparison between deep learning models (GRU-LoRA and Transformer) and the classical XGBoost model was conducted using a paired t-test to determine statistical significance.

Table III. T-Test Results for Accuracy Comparison

Comparison	T-statistic	P-value
XGBoost—GRU (20–80)	52.3022	1.9792e−11
XGBoost—Transformer (20–80)	38.2234	2.4102e−10
XGBoost—GRU (50–50)	55.7338	1.1919e−11
XGBoost—Transformer (50–50)	34.3837	5.5956e−10

Table III results show deep learning models (Transformer and GRU) is significantly outperform XGBoost in the five-fold test accuracy evaluation.

Table IV represent the GRU-LoRA model exhibited the highest t-statistic (18.8698, p-value = 2.7967e−03) across both data splits for F1-Score, indicating a substantial improvement over XGBoost in correctly classifying BRCA1 cases. This suggests that GRU-LoRA’s ability to model sequential dependencies and adapt low-rank parameter updates enhances its robustness, particularly in handling sparse scRNA-seq data offer superior predictive accuracy and reliability in breast cancer subtype classification [20].

Table IV. T-Test Results for Comparing Minor Class Comparison

Comparison (F1-Score)	T-statistic	P-value
XGBoost-GRU (BRCA1)	18.8698	2.7967e-03
XGBoost-Transformer (BRCA1)	16.6825	3.5739e-03

Conclusion

This experimental study encapsulates key insights into the comparative analysis of deep learning and classical machine learning methodologies for single-cell RNA sequencing (scRNA-seq) data analysis, specifically for breast cancer subtype classification. The study demonstrated the superior predictive performance of deep learning models in handling high-dimensional genomic data, outperforming classical machine learning approaches.

The study sought to determine whether deep learning architectures such as Gated Recurrent Units (GRU) and Transformers significantly improved predictive accuracy compared to traditional algorithms like XGBoost. The above findings confirmed this hypothesis, illustrating that deep learning models consistently yielded higher accuracy, precision, recall, and F1 scores across evaluation metrics. The results further indicated that models incorporating attention mechanisms, such as Transformers, were particularly effective in capturing long-range dependencies within gene expression data, while GRU-LoRA demonstrated efficiency in optimizing parameter updates [21].

Despite the evident advantages of deep learning models, certain limitations are acknowledged. These include increased computational requirements, model interpretability challenges, and the potential for overfitting in cases of limited data availability. Additionally, while class-balancing techniques such as weighted loss functions and dropout were applied, handling highly imbalanced datasets remains an ongoing challenge in this study.

The implications of this study extend beyond model performance comparisons, offering insights into the practical applications of deep learning in genomic research. The ability to accurately classify breast cancer subtypes holds great promise for precision oncology, facilitating improved diagnostic workflows, patient stratification, and targeted treatment planning. Future research should further focus on integrating multi-modal data sources, such as proteomics and epigenomics, to enhance predictive accuracy. Additionally, exploring hybrid models that combine classical approaches’ interpretability with deep networks’ learning efficiency may offer a balanced solution for real-world applications [22].

This experimental quantitative study highlights the transformative potential of deep learning in computational biology and cancer genomics. This study demonstrates its advantages over classical machine learning methods and contributes to the ongoing advancement of artificial intelligence applications in genomic research. Expanding these methodologies to larger and more diverse datasets, optimizing model interpretability, and integrating domain-specific knowledge will be crucial steps toward fully leveraging deep learning for future cancer diagnostics and precision medicine breakthroughs.