Composites are intrinsically multiscale material systems. Increasing our understanding of how these materials behave at very small scales through multiscale computational mechanics is part of an important shift towards virtual material testing and design that will lead to lighter, more efficient and sustainable composite materials. In particular, concurrent multiscale analysis (FE2) is a powerful tool for unravelling small-scale phenomena in composite materials and deepening our understanding of how they affect macroscale performance. Nevertheless, FE2 still does not see widespread application in industry due to its extreme computational cost: microscale models are embedded at every macroscale material point and must be solved at every iteration of every macroscopic time step. The search for effective ways to accelerate FE2 simulations is a major focus of the emerging field of mechanistic machine learning for solid mechanics. A popular strategy is to train data-driven surrogates that take the place of the original micromodel computations and can provide approximate stress-strain mappings from which predictions can be obtained several orders of magnitude faster than solving the original micromodels. However, training these models to reproduce complex behavior is a highly cumbersome task: purely data-driven surrogates discard well-established physical mechanisms which must be painstakingly learned from data, and strain path dependent material behavior requires training surrogates on data sampled from arbitrarily high-dimensional spaces describing strain paths in time. In this work, we explore a hybrid strategy for building surrogate models for multiscale simulations of composite materials. Instead of relying on fully data-driven models, we start from classical physics-based constitutive models that encapsulate relevant material behavior features (e.g. loading-unloading behavior) we aim to avoid learning directly from data. To increase flexibility in an interpretable way, we let the material parameters of the models evolve in time with latent dynamics learned by a machine learning encoder. We demonstrate the hybrid approach in the context of fiber-reinforced materials with pressure-dependent plasticity behavior. The resulting surrogates inherit numerical robustness and frame invariance directly from their physics-based decoders and provide sensible path-dependent predictions while being trained exclusively on monotonic strain paths.