Neural Ordinary Differential Equations (neural ODEs) are a brand new and exciting method to model nonlinear transformations as they combine the two fields of machine learning and differential equations. In this talk we discuss DiffEqFlux.jl, a package for designing and training neural ODEs, and we introduce new methodologies to improve the efficiency and robustness of neural ODEs fitting.