Papers › HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical...

HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis

8 Nov 2024arXiv:2411.11896archive 2025-07-28

Saedeh Tahery, Fatemeh Hamid Akhlaghi, Termeh Amirsoleimani

The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.

Paper PDF Code

Code

ecgresearch/heartbert officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Heartbeat Classification Self-Supervised Learning Sleep Stage Detection

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Adam Attention Attention Dropout BERT Dense Connections Dropout LSTM Layer Normalization Linear Layer Linear Warmup With Linear Decay Multi-Head Attention Residual Connection RoBERTa Sigmoid Activation Softmax Tanh Activation Weight Decay WordPiece

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections