DiSK: Diffusion Model for Structured Knowledge

Abstract

We introduced a generative framework that can process and model diverse data types, from numbers to strings to complex hierarchical properties. This is especially valuable for tasks like filling in missing data in structured databases or tables. Notably, our experiments showcased the model’s proficiency in accurately predicting numerical values, which is vital for scientific applications. This approach can be used in Augmented Language Modeling to improve information retrieval and knowledge manipulation.

Ouail Kitouni

Ph.D. Student

Working on the Science of Deep Learning.