I think the data scientist hiring frenzy may soon collapse as large companies run out of patience with their recently formed data science departments that struggle to deliver ROI (for many reasons, often not the fault of the data scientists). But I think we are not yet at the final iteration of the job market for these sorts of skills. Companies usually don't really want someone who specializes in model tuning and algorithm creation, they want something like a "full-stack data analyst" - someone who acknowledges that the modeling may be 2% of the effort and the rest is business analysis, data wrangling, engineering, stakeholder management, building tools for users/operators, etc., and rolls up their sleeves to deliver an end-to-end solution. There does not yet exist a catchy name for this role, but I bet that in a few years it will be what everyone wants to hire. So skate to where the puck will be...
Great comment. What you say matches my experience, but not entirely. My take:
The 2% is a bit low as an avg. for ML work; I would say modeling/ML work ranges from 0 to 20% across the "data team". Most of the work is still data wrangling/engineering.
I agree that most companies need "full stack data guys". At my current company, there were 2-3 iterations of previous DS teams who had zero impact and left behind nothing in production. Then I came in, and being a full stack guy, I built up a DWH, ETL, dashboards [1] and eventually found opportunities to put ML into production [2].
I don't agree that the hype will end. The reason = companies/CEOs aren't rational and don't know what you said above. They'll just keep hiring DS people/teams and hope that eventually some magic gift pops out. I think this will keep on going for at least 5 years. I get pinged about 1-2 times per week by recruiters for Head of DS type positions in London/Dubai, everybody wants to build data science teams.
Great comment indeed. The number of universities that have created MS in Analytics or Data Science programs and the number of students that have enrolled in them over the last few years is staggering and IMO unsustainable. The quality of these programs is also very uneven - there are a few great ones but many seem to have just cobbled together a few database courses from the CS department, a few classical statistics and probability courses from the Math department and a few marketing courses from the Business department and called it a MS in Analytics program. Likewise, the expectations students have about the type of work they'll be doing and the expectations companies have about the value that will be realized are both completely out of alignment. /rant off
