Recommendations at CNN

The User Intelligence team in the Data Intelligence organization at CNN Digital is working on content recommendations. We are still in the experimentation phase, although that involves A/B testing with as much as 20% of cnn.com traffic. Initially experimentation was incredibly slow. We had a single tenant API and struggled with the process for running experiments in Optimizely. We were focused on a small number of relatively complex models. And, we had no offline evaluation process to signal what we should A/B test. We have managed to make a ton of progress on pain points within the past year even though we still have a lot we want to improve on. In this session we will talk about what we have learned and improved and what we want to work on next.