Bernhard Bermeitinger
Title
Dr.
Last Name
Bermeitinger
First name
Bernhard
Email
bernhard.bermeitinger@unisg.ch
ORCID
Phone
+41 71 224 79 17
46 results
Now showing 1 - 10 of 46
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Interpretable type 2 diabetes incidence prediction with AutoScore: A model based on standard clinical parameters(2026-02-01)Scopus© Citations 1 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, 425-P: Machine Learning Identifies Glypican 4 as Key Predictor of Five-Year Mortality in Heart Failure Patients with Prediabetes or Diabetes(2025-06-20) - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Interpretable type 2 diabetes incidence prediction with AutoScore: A model based on standard clinical parameters(2025-10)Scopus© Citations 1 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, 1386-P: Machine Learning Predicts T2DM Incidence Using Basic Clinical and Laboratory Parameters(2025-06-20) - Some of the metrics are blocked by yourconsent settings
Item type:Publication, 426-P: Predicting Coronary Stenoses Using Machine Learning to Reduce Unnecessary Angiographies(2025-06-20) - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Is More Data Worth the Cost? Dataset Scaling Laws in a Tiny Attention-Only Decoder(IEEE, 2026-05-06); ; ; Training Transformer language models is expensive, as performance typically improves with increasing dataset size and computational budget. Although scaling laws describe this trend at large scale, their implications in controlled, smaller-scale settings remain less explored. In this work, we isolate dataset-size effects using a strongly reduced attention-only decoder architecture. By training on progressively larger power-of-two subsets, we observe smooth performance improvements accompanied by clear diminishing returns, consistent with scaling-law behavior. Using only about 30 % of the training data is sufficient to reach approximately 90 % of the full-data validation token-level accuracy. These results provide actionable insights into dataset scaling in a controlled, component-isolated setting and offer practical guidance for balancing dataset size and computational cost in compute- and data-restricted environments, such as small research labs and exploratory model development.Type:conference paperJournal:2026 IEEE Swiss Conference on Data Science and AI (SDS) - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Subset Pretraining for Enhancing Neural Network Training EfficiencyWe propose a novel alternative to traditional randomly sampled mini-batches for gradient computation: using a fixed subset for complete pretraining of a neural network model. This approach enables deterministic convergence instead of a merely probabilistic one, as proven by the stochastic approximation theory, whose prerequisites are frequently violated by popular optimization algorithms. The approach is justified by the hypothesis that the loss minimum of the training set can be expected to be well-approximated by the minima of its subsets. Such subset minima can be computed in a fraction of the time necessary for optimizing with the whole training set. They are also compatible with efficient second-order optimization methods, such as the conjugate gradient optimizer. These methods are particularly efficient in the convex environment of the loss minimum. The image classification datasets MNIST, CIFAR-10, and CIFAR-100, (optionally extended by augmentation of training data) test this hypothesis. The experiments confirm that the models achieve performance equivalent to that when trained with the conventional training scheme. In conclusion, if the overdetermination ratio for the given model and dataset sufficiently exceed unity, even small subsets are representative. This results in a possible reduction of the computing expense to a tenth or less. This paper is an extended version of Spörer et al. [13].Type:conference paperJournal:Communications in Computer and Information ScienceVolume:2703 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Is There an Optimal Depth of Residual Networks?(Springer Nature (Switzerland), 2025-05-03); ; Although deep neural networks have given the name to the domain of "deep learning", it is still an unresolved question of which depth is the best choice for a given task. A large depth brings problems with the convergence of gradientbased learning algorithms through the phenomenon of vanishing gradient. On the other hand, it is a widespread opinion that a small depth has insufficient representational power for many tasks. The discovery of the concept of residual connections-an identity mapping parallel to a conventional layer-has alleviated the convergence problem so that the discussion of optimum depth lost a part of its motivation, resulting in the assumption of "the more the better". The work presented here shows that a shallow architecture of parallel layers has comparable expressive power as a deep stack of residual layers. This is theoretically justified by expanding the residual layer stack analogical to the Taylor expansion, truncating the higher-order terms into a single broad layer composed of original layers in parallel. This hypothesis has been confirmed by computing experiments with the widespread computer vision benchmark datasets MNIST and CIFAR-10. The 6,912 runs have shown that the shallow and the deep architectures do not substantially differ in performance on both training and validation sets if the total number of parameters is equal. The rough equivalence of the two extreme (deep and shallow) architectures suggests the possibility that an intermediary architecture may be superior. Another series of computing experiments disclosed that the performance does not substantially differ even then. The conclusion is that the performance of an architecture depends more substantially on the total number of parameters than on the sequential or parallel connection of layers.Type:conference paperJournal:Communications in Computer and Information ScienceVolume:2454 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, BioMistral-Clinical: A Scalable Approach to Clinical LLMs via Incremental Learning and RAG(The Asian Federation of Natural Language Processing and The Association for Computational Linguistics, 2025-12); The integration of large language models (LLMs) into clinical medicine represents a major advancement in natural language processing (NLP). We introduce BioMistral-Clinical 7B, a clinical LLM built on BioMistral-7B (Labrak et al., 2024), designed to support continual learning from unstructured clinical notes for real-world tasks such as clinical decision support. Using the augmented-clinical notes dataset provided by Hugging Face (2024), we apply prompt engineering to transform unstructured text into structured JSON capturing key clinical information (symptoms, diagnoses, treatments, outcomes). We employ selfsupervised continual learning (SPeCiaL) (Caccia and Pineau, 2021) to achieve efficient incremental training. Evaluation on MedQA (Jin et al., 2021) and MedMCQA (Pal et al., 2022) shows that BioMistral-Clinical 7B improves accuracy on MedMCQA by nearly 10 points (37.4% vs. 28.0%) over the base model, while maintaining comparable performance on MedQA (34.8% vs. 36.5%). Building on this, we propose the BioMistral-Clinical System, which integrates Retrieval-Augmented Generation (RAG) (Lewis et al., 2020) to enrich responses with relevant clinical cases retrieved from a structured vector database. The full system enhances clinical reasoning by combining domain-specific adaptation with contextual retrieval.Type:conference paper - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Machine learning identifies glypican 4 as key predictor of 5-year mortality in heart failure patients with prediabetes or diabetes(Springer Science and Business Media LLC, 2025-08-26)Scopus© Citations 8