We touched on a lot of these questions in this talk about how we went from a prototype to a production machine learning system: https://vimeo.com/181931334
The main point I haven't seen mentioned that often is to constantly verify your data and your data processing pipeline. We treat these checks as integration tests and run them as part of our continuous integration system. We also use New Relic to monitor model freshness, to be alerted if any part of the pipeline has broken.
years ago i dealt with NN's in automated trading, and little regard was given to source data requirements. it was like NN's were treated as magic boxes rather than black boxes. a magic box will miraculously produce useful output even when you feed it garbage, but a black box has specific requirements for its data source to make it useful. so i like how the author dedicates one entire part of the three-part flow chart to sources and preprocessing. if i deal with NN's again i'll definitely place more emphasis on this aspect.
I would love to hear more from people deploying machine learning techniques at their projects, teams and companies.