Themes

Expressivity

Kutyniok, G. (2022). The Mathematics of Artificial Intelligence. arXiv preprint arXiv:2203.08890 – section 3.1

Given a function class/space C and a class of deep neural networks NNθ, how does the approximation accuracy when approximating elements of C by networks Φ ∈ NNθ relate to the complexity of such Φ?

Optimization

Kutyniok, G. (2022). The Mathematics of Artificial Intelligence. arXiv preprint arXiv:2203.08890 – section 3.2

in the neural network setting, the optimization problem is non-convex, which makes it—even when using a non-random version of gradient descent—very hard to analyze.

Generalization

Kutyniok, G. (2022). The Mathematics of Artificial Intelligence. arXiv preprint arXiv:2203.08890 – section 3.3

One of the mysteries of deep neural networks is the observation that highly overparameterized deep neural networks in the sense of high complexity of the network do not overfit with overfitting referring to the problem of fitting the training data too tightly and consequently endangering correct classification of new data.

Explainability

Kutyniok, G. (2022). The Mathematics of Artificial Intelligence. arXiv preprint arXiv:2203.08890 – section 3.4

The area of explainability aims to “open the black box” of deep neural networks in the sense as to explain decisions of trained neural networks. These explanations typically consist of providing relevance scores for features of the input data. … We … describe in more detail an approach which is based on information theory …