Sebastian Loeschcke

Ph.D. Student
Department of Computer Science
University of Copenhagen and Pioneer Centre for Artificial Intelligence

alternative

My research focuses on efficient training, using low-rank methods, sparsity, and quantization.
I'm advised by Michael J. Kastoryano and Serge Belongie, and funded by the Danish Data Science Academy (DDSA).
Currently, I am visiting Anima Anandkumar's group at Caltech and collaborating with Jean Kossaifi from NVIDIA Research.

  email   Google Scholar   LinkedIn   Github   Twitter Bluesky  Bluesky

Publications

LoQT

LoQT: Low-Rank Adapters for Quantized Pre-Training

Sebastian Loeschcke*, Mads Toftrup*, Michael J. Kastoryano, Serge Belongie, Vésteinn Snæbjarnarson

NeurIPS 2024. *Equal contribution. (Best Paper Award) at WANT@ICML.

  Code   Paper

LoQT enables efficient quantized pre-training of LLMs with results close to full-rank non-quantized models. It enables pre-training of a 13B LLM on a 24GB GPU without model parallel, checkpointing, or offloading strategies during training.


Putt Image

Coarse-To-Fine Tensor Trains for Compact Visual Representations

Sebastian Loeschcke, Dan Wang, Christian Leth-Espensen, Serge Belongie, Michael J. Kastoryano, Sagie Benaim

ICML 2024.

  Project page   Code   Paper

PuTT (Prolongation Upsampling Tensor Train), a method for learning a coarse-to-fine tensor train representation of visual data, excelling in 2D/3D fitting and novel view synthesis, even with noisy or incomplete data.


Text-Driven Stylization of Video Objects

Sebastian Loeschcke, Sagie Benaim, Serge Belongie

ECCV Workshop on AI for Creative Video Editing and Understanding, 2022.
(Best Paper Award) (Oral Presentation)

  Project page   Paper   Video summary

A method for stylizing video objects in an intuitive and semantic manner following a user-specified text prompt.


Progressive Parameter Space Visualization for Task-Driven SAX Configuration

Sebastian Loeschcke, Marius Hogräfer, Hans-Jörg Schulz

EuroVA, 2020. (Oral Presentation)

  Paper

Tool that uses progressive visual analytics to guide users in hyperparameter search for SAX and PAA algorithms.

Talks

Invited talk with Serge Belongie for at NVIDIA Research, presenting LoQT, hosted by Georg Zitzlsberger.


Invited talk at the D3A 2.0 Conference (Danish Data Science, Digitalization, and AI), October 23, 2024, on Resource-Aware Machine Learning, hosted by Pinar Tozun and Raghavendra Selvan.


Oral presentation at the ICML Workshop on Advancing Neural Network Training (WANT@ICML), 2024.


Text-Driven Stylization of Video Objects, oral presentation at the ECCV Workshop on AI for Creative Video Editing and Understanding, Tel Aviv, 2022.


Progressive Parameter Space Visualization for Task-Driven SAX Configuration, presented in Norrköping, Sweden, 2020.

Awards

Best Paper Award at ICML workshop on Advancing Neural Network Training WANT@ICML.


Best Paper Award, ECCV Workshop on AI for Creative Video Editing and Understanding, Tel Aviv 2022. Twitter post.


Queen Margrethe II's travel grant. September 2022. News article from Aarhus University.


Awarded a prestigious Ph.D. scholarship from the Danish Data Science Academy (DDSA), contributing to the vibrant landscape of data science research.