Making Sense of ML Metrics
Michael McCourt, Head of Engineering at SigOpt, an Intel company
Machine learning models are only as useful as the metrics for which they are trained and optimized towards. In this talk we’ll discuss common pitfalls when designing and tracking machine learning metrics and how to overcome them. We’ll then focus on intelligent search approaches that explore many metrics at once to better understand the modeling problem and parameter space.
Finally, we’ll discuss techniques for optimizing multiple metrics at once to analyze tradeoffs. This talk will provide useful lessons for developers just getting started in ML, engineers fine-tuning pre-trained models for production, or seasoned researchers developing and training algorithms from scratch.