Sebastian Loeschcke

Ph.D. Student
Department of Computer Science, Machine Learning Section
University of Copenhagen and Pioneer Centre for Artificial Intelligence

I work on efficient training of deep learning models, focusing on low-rank methods and quantization. I am advised by Michael J. Kastoryano and Serge Belongie, and supported by the Danish Data Science Academy (DDSA). In 2025, I spent six months visiting Anima Anandkumar’s group at Caltech, and collaborated with Jean Kossaifi (NVIDIA Research). In 2025–2026, I completed a six-month internship at Qualcomm AI Research in Amsterdam, working with the Model Efficiency group under the supervision of Markus Nagel.

Portrait of Sebastian Loeschcke

Selected Publications

TensorGRaD

TensorGRaD: Tensor Gradient Robust Decomposition for Memory-Efficient Neural Operator Training

Sebastian Loeschcke, David Pitt, Robert J. George, Jiawei Zhao, Cheng Luo, Yuandong Tian, Jean Kossaifi, Anima Anandkumar

arXiv preprint, 2025.

 Code  Paper

TensorGRaD factorizes gradient tensors into complementary low-rank and sparse components, reducing optimizer-state memory by up to 75 % while matching the accuracy of Adam, even on turbulent Navier–Stokes at Re = 105.

LoQT

LoQT: Low-Rank Adapters for Quantized Pre-Training

Sebastian Loeschcke*, Mads Toftrup*, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson

NeurIPS 2024. *Equal contribution. Best Paper Award at WANT@ICML.

  Code   Paper

LoQT enables efficient quantized pre-training of LLMs with results close to full-rank non-quantized models. It enables pre-training of a 13B LLM on a 24GB GPU without model parallel, checkpointing, or offloading strategies during training.

PuTT visualization

Coarse-To-Fine Tensor Trains for Compact Visual Representations

Sebastian Loeschcke, Dan Wang, Christian Leth-Espensen, Serge Belongie, Michael J. Kastoryano, Sagie Benaim

ICML 2024.

  Project page   Code   Paper

PuTT (Prolongation Upsampling Tensor Train), a method for learning a coarse-to-fine tensor train representation of visual data, excelling in 2D/3D fitting and novel view synthesis, even with noisy or incomplete data.

Text-Driven Stylization of Video Objects

Sebastian Loeschcke, Sagie Benaim, Serge Belongie

ECCV Workshop on AI for Creative Video Editing and Understanding, 2022.
Best Paper Award Oral Presentation

  Project page   Paper   Video summary

A method for stylizing video objects in an intuitive and semantic manner following a user-specified text prompt.

Progressive Parameter Space Visualization for Task-Driven SAX Configuration

Sebastian Loeschcke, Marius Hogräfer, Hans-Jörg Schulz

EuroVA, 2020. Oral Presentation

  Paper

Tool that uses progressive visual analytics to guide users in hyperparameter search for SAX and PAA algorithms.

Other Work

Talks

  • Invited talk "Low-Rank and Quantized Learning" at UC Santa Barbara, hosted by Zheng Zhang, Mar. 2025.
  • Invited talk with Serge Belongie at NVIDIA Research, presenting LoQT, hosted by Georg Zitzlsberger, Nov. 2024.
  • Invited talk at the D3A 2.0 Conference (Danish Data Science, Digitalization, and AI), October 23, 2024, on Resource-Aware Machine Learning, hosted by Pinar Tozun and Raghavendra Selvan.
  • Oral presentation at the ICML Workshop on Advancing Neural Network Training (WANT@ICML), 2024.
  • Text-Driven Stylization of Video Objects, oral presentation at the ECCV Workshop on AI for Creative Video Editing and Understanding, Tel Aviv, 2022.
  • Progressive Parameter Space Visualization for Task-Driven SAX Configuration, presented in Norrköping, Sweden, 2020.

Awards