In this three-part talk, I will present some of our recent efforts that aim to control and adapt neural models to work more effectively in target applications. The first part will focus on how to repurpose a pre-trained neural entailment model for multi-hop QA, and decomposing large QA models to run effectively on mobile devices. In the second part, I will present methods for learning structured latent spaces for better control in modeling and generating event sequences. In the third part, I will talk briefly about modeling target-side syntax for machine translation.