Data Operations: Or How I Learned to Stop Data Wrangling and Love Machine Learning
Film buffs will remember the above scene from Stanley Kubrick’s iconic Dr. Strangelove, in which President Muffley says, “Gentlemen, you can’t fight in here! This is the War Room!” The absurd, satirical line is oddly applicable to what’s happening today in machine learning teams. The machine learning folks shouldn’t be data wrangling- they should be focused on machine learning. But because we don’t often receive our data in a usable format, valuable time is spent transforming, moving, and cleaning data.
And the situation will only get worse as the number of data sources increases. Data streams from data partners, in-house data, and public sources all need to be brought together to extract value from machine learning. This has created an acute need to establish data operations teams that focus on monitoring, moving, transforming, and securing data before any machine learning can take place.