Leading AI-Powered Recruitment with automated MLOps pipelines in production
Industry
Programmatic job advertising
Resources
AWS EKS
Use-case
Recommendation Systems
Reinforcement Learning
TOOLS
There are millions of approaches to build an AI infrastructure today. Intel® Tiber™ AI Studio helped us to adopt the industry’s best tools, with out-of-the-box best practices, saving us endless DevOps hours. Now we are able leverage all the latest data science tools and frameworks like Kafka, Kubernetes, and Spark in one unified user friendly UI, and focus on building the industry leading algorithms that allow us to advance our position as a competitive AI solution.
Eran Brill
Data Scientist at PandoLogic
About PandoLogic
PandoLogic is an AI-based programmatic job advertising platform that intelligently automates and optimizes job advertising spend. Companies faced with complex hiring needs can source quality applicants faster, smarter, and more efficiently with their unique proprietary technology. pandoIQ provides an end-to-end job advertising solution that delivers a significant increase in job ad performance without any wasteful spending.
Overview
PandoIQ’s AI-enabled algorithms are a key competitive advantage in the programmatic job advertising market. These clever algorithms automate processes and take on grunt work to save valuable time for customers. They make better-than-human decisions on the spot using terabytes of data, machine-learning, and AI to make sure their customers get the most from job advertising spend. Their platform is built around AI-enabled algorithms that use over 200+ job attributes and more than 200 billion historical job performance data points to predict the optimal job advertising campaign. The models are continuously being retrained automatically to production.
Challenges
Operationalizing ML on premises with no DevOps or infrastructure
In order to lead the crowded job advertising market, PandoLogic needs to constantly deliver top-of-the-line ML models for their customers like Fedex, Dominos, Postmates and other Fortune 500 organizations. PandoLogic invested in a lean team of data scientists, but their productivity is constantly interrupted with DevOps tasks and infrastructure challenges. Their team lacked the resources and infrastructure to operationalize their impressive models to achieve real business results. PandoLogic knew what solutions needed to be integrated into their workflow, but did not want to invest their time planning, building and maintaining an entire ML infrastructure. To remain competitive, PandoLogic’s data science team needed to focus on building and delivering models rather than building their own infrastructure. They were limited to on premise deployment which caused technical challenges and DevOps overhead. They required dynamic Spark Clusters to handle terabytes of data, which wasted weeks of set up time, and caused major overhead costs to maintain. As data science professionals they preferred to spend their time creating models that added real business value. The PandoLogic team wanted a way to train and deploy on premise and in multiple clouds, without being locked into a single cloud. They needed an easy way to leverage open source tools and the compute resources they already had.
Solution
Out-of-the-box Enterprise-grade ML infrastructure and MLOps
Intel® Tiber™ AI Studio helped PandoLogic quickly adopt an entire enterprise-level ML infrastructure platform from zero to production. What would have taken a team of specialized ML engineers, and months of DevOps time to build was all delivered out-of-the-box with Intel® Tiber™ AI Studio. Intel® Tiber™ AI Studio provides one click integration with AWS data lakes, Kubernetes deployments, Spark Clusters, and cloud compute all with little overhead. It delivered a resource management system that can leverage on-prem resources and automatically bursts to the cloud and terminates with one click. Intel® Tiber™ AI Studio helps PandoLogic insert cloud resources and services into their tech stack seamlessly. PandoLogic leverages Intel® Tiber™ AI Studio’s Flow UI tool to build custom ML pipelines that allows each task to run on a different cloud simultaneously. In a few clicks they are able to take their Jupyter Notebooks to process the data, save metadata and analyze the flow, while using Python to run experiments, and serve their models with Kubernetes, AWS and on-premise servers. Using the AI library, PandoLogic can build custom drag and drop pipelines pulled from their GitHub repo, and use their proprietary algorithms as well as XGBoost, LightGBM, Ensemble Methods and other open source ML and DL frameworks like PyTorch with the Intel® Tiber™ AI Studio platform. Intel® Tiber™ AI Studio’s MLOps solution has accelerated workflows drastically, allowing spinning up a powerful AI environment in one click without manual DevOps. While PandoLogic had a version control system in place, it was not comprehensive. Now with Intel® Tiber™ AI Studio, PandoLogic can version control data, experiments and monitor models in a single
Results
Intel® Tiber™ AI Studio helped PandoLogic quickly adopt an entire enterprise-level ML infrastructure platform from zero to production. What would have taken a team of specialized ML engineers, and months of DevOps time to build was all delivered out-of-the-box with Intel® Tiber™ AI Studio. Intel® Tiber™ AI Studio provides one click integration with AWS data lakes, Kubernetes deployments, Spark Clusters, and cloud compute all with little overhead. It delivered a resource management system that can leverage on-prem resources and automatically bursts to the cloud and terminates with one click. Intel® Tiber™ AI Studio helps PandoLogic insert cloud resources and services into their tech stack seamlessly. PandoLogic leverages Intel® Tiber™ AI Studio’s Flow UI tool to build custom ML pipelines that allows each task to run on a different cloud simultaneously. In a few clicks they are able to take their Jupyter Notebooks to process the data, save metadata and analyze the flow, while using Python to run experiments, and serve their models with Kubernetes, AWS and on-premise servers. Using the AI library, PandoLogic can build custom drag and drop pipelines pulled from their GitHub repo, and use their proprietary algorithms as well as XGBoost, LightGBM, Ensemble Methods and other open source ML and DL frameworks like PyTorch with the Intel® Tiber™ AI Studio platform. Intel® Tiber™ AI Studio’s MLOps solution has accelerated workflows drastically, allowing spinning up a powerful AI environment in one click without manual DevOps. While PandoLogic had a version control system in place, it was not comprehensive. Now with Intel® Tiber™ AI Studio, PandoLogic can version control data, experiments and monitor models in a single