All research items

Introduction

In systems biology, both mechanistic differential equation models and machine learning are used to elucidate biological processes and make predictions. An advantage of mechanistic models is their high level of interpretability, moreover these models can be built with little to no data. Highly plastic data-driven models offer a high predictive capability, but often require large volumes of data and provide little direct interpretability.

A promising method to integrate these techniques, is the use of universal differential equations (UDEs)¹. In this method, a neural network is introduced into a mechanistic model, representing less well-known relationships between the described variables. In this way, flexible neural networks can be used to learn unknown relationships, while retaining the strong inductive bias of biological mechanistic models. In this study, we demonstrate the potential of UDEs to learn unknown reactions using experimental data. Moreover, we introduce a framework of regularisation to further support the training of UDEs on sparse data.

Approach

To benchmark the ability of the UDE to recover unknown interactions, simulated data of an enzymatic conversion using Michaelis-Menten kinetics was generated. The Michaelis-Menten term was replaced with a neural network and this new model was fit on the generated data. Model predictive capabilities were evaluated for varying durations of data collection.

Next, to demonstrate the generalisability of the approach, the rate of glucose appearance in the Bergman glucose-minimal model² was replaced with a neural network, and the model was fit on human meal response data, investigating the ability of the model to learn more complex functions while retaining biological plausibility of the results.

Finally, to determine if this approach can accommodate inter-individual variation in data the glucose-dependent insulin release mechanism in the EDeS model³ was replaced with a neural network, the model was then fit on both the average response, as well as on the indvidiual response to an oral glucose tolerance test.

Results

Our initial benchmark learning the Michealis-Menten interactions demonstrated the strong influence of the data collection duration in the ability of the neural network to recover the originally simulated coupling term. However, the adoption of physiology-informed regularization greatly improved model predictive capabilities, even for extremely short data collection periods.

Furthermore, application on Bergman’s glucose-minimal model showed an improved data fit when using a neural glucose rate of appearance function, while retaining biological plausibility of the simualted results. Finally, the incorporation of a neural network in the EDeS model to describe glucose-dependent pancreatic insulin release also showed an improved model fit to data compared to the original model with the hybrid-model learning the biphasic secretion of insulin in response to a glucose challenge which cannot be simulated using the current insulin secretion terms.

Discussion

The results demonstrate that UDEs maintain the generalisability of conventional mechanistic models and provide accurate predictions of system behavior in line with machine learning performances, while having considerably lower data requirements. While these model can become stable during training due to limitations with the available data, providing additional information in the form of physiology-informed regularisation can greatly improve the resulting models. Furthermore, UDEs provide the potential to allow for rapid extension of current mechanistic models with new biological insights. In particular in medical applications, where low data availability prohibits the use of conventional machine learning approaches.

References