Speakers:
Tim Obert, Data Scientist at CNN Digital
Ailish Byrne, Engineer at CNN Digital

The User Intelligence team in the Data Intelligence organization at CNN Digital is working on content recommendations. We are still in the experimentation phase, although that involves A/B testing with as much as 20% of cnn.com traffic. Initially experimentation was incredibly slow. We had a single tenant API and struggled with the process for running experiments in Optimizely. We were focused on a small number of relatively complex models. And, we had no offline evaluation process to signal what we should A/B test. We have managed to make a ton of progress on pain points within the past year even though we still have a lot we want to improve on. In this session we will talk about what we have learned and improved and what we want to work on next.