Informed Machine Learning: Excess Risk and Generalization

Machine Learning (ML) based predictive models are impacting research, industry, and society at large thanks to their ability to model or surrogate real systems. Two of the main current limitations of ML are the need for large amounts of high quality data and low performance far away from the observed data. For this reason, in certain applications where prior knowledge is available, researchers have developed Informed ML (IML) to decrease ML high quality data voracity and increase ML extrapolation abilities. In this work we study the differences between ML and IML excess risk and generalization using also some examples to elucidate the theoretical discussions. Our findings shed some light on the mechanisms and the conditions under which IML outperforms ML.