As AI Reshapes Tech, Microsoft, Others Refocus Dev Structure
The impact of AI on software organizations has been huge, so much so that major companies such as Microsoft are making organizational shifts to keep up with the AI revolution.
Microsoft this week announced a major organizational change with the creation of a new engineering division called “CoreAI — Platform and Tools” in response to the rapid advancement of AI technology.
In a memo to Microsoft employees, Satya Nadella, chairman and CEO of Microsoft said he believes 2025 will be a pivotal year for AI applications, noting that: “More so than any previous platform shift, every layer of the application stack will be impacted. It’s akin to GUI, internet servers, and cloud native databases all being introduced into the app stack simultaneously. Thirty years of change is being compressed into three years!”
Consolidating Teams
The new organization will consolidate several existing teams, including Microsoft’s Developer Division, AI Platform, and some key teams from the Office of the CTO (AI Supercomputer, AI Agentic Runtimes, and Engineering Thrive), Nadella wrote.
Microsoft is responding to a trend that has been gaining momentum over the past year that has shifted from AI as a scientific endeavor to AI an engineering opportunity.
“Spurred on by the incredibly low barrier of entry in terms of both cost and complexity found with AI services like Microsoft Azure OpenAI Services, both data scientists and enterprise app developers alike have been clamoring for API-led access to AI outcomes,” said Brad Shimmin, an analyst at Omdia.
“Given the rapid maturation of supportive frameworks like Microsoft’s own AutoGen and direct hooks into .NET, developers right now can import and call AI API services to perform tasks such as named entity extraction or image recognition without needing to understand anything about the science behind those services,” he added.
New Mission
The new Microsoft Core AI and tools group’s mission is to build an end-to-end Copilot & AI stack for customers to build and run AI apps and agents. This group also will be responsible for building out GitHub Copilot, “thus having a tight feedback loop between the leading AI-first product and the AI platform to motivate the stack and its roadmap,” Nadella said.
Moreover, Microsoft “will build agentic applications with memory, entitlements, and action space that will inherit powerful model capabilities,” Nadella wrote. And “Further, how we build, deploy, and maintain code for these AI applications is also fundamentally changing and becoming agentic,” he wrote.
Nadella said this reorganization reflects Microsoft’s vision of a new “AI-first app stack” that will feature new UI/UX patterns, agent-based runtimes and reimagined management and observability layers with Azure as the fundamental AI infrastructure.
“…Azure must become the infrastructure for AI, while we build our AI platform and developer tools — spanning Azure AI Foundry, GitHub, and VS Code — on top of it. In other words, our AI platform and tools will come together to create agents, and these agents will come together to change every SaaS application category, and building custom applications will be driven by software (i.e., ‘service as software’),” Nadella wrote.
The Power of Agents
David Mytton, CEO of Arcjet told The New Stack that agents stood out to him as the most important aspect of this reorganization.
“GitHub Copilot Workspace has been doing this for a little while now, and it made its way into VS Code with Copilot Edits,” he said. “This is what they’re gearing up for — 2025 is the year of multistep agents. It makes sense to build on Copilot first, then extend it to other Microsoft apps.”
Moreover, Mytton noted that Meta CEO Mark Zuckerberg predicted on a recent Joe Rogan podcast that 2025 would see agents become mid-level engineer coworkers, “so Microsoft changing its org structure sounds like a leading indicator of those types of structural changes,” he said.
AI Is Everywhere
“AI is everywhere. AI is in dev tools, so it’s in GitHub. AI has its own tooling, it is in applications, and in agents that talk to APIs, so it’s in Visual Studio and VS Code. AI is omnipresent, not an island,” Andrew Brust, CEO of Blue Badge Insights, told The New Stack. “So, it only makes sense that AI Platform and Dev Div should come together.”
However, “Ultimately, if Microsoft does AI right, it will be less discrete and more of the element in every part of the stack. I don’t think we’re there yet, and I’m certain there are more reorgs to come, but this one makes sense. It takes AI out of incubatory isolation and makes it part of the developer mainstream, which reflects the way the entire industry will go, eventually.”
Key Players
Jay Parikh will lead this group as executive vice president (EVP) of CoreAI — Platform and Tools. Eric Boyd, corporate vice president (CVP) of AI Platform; Jason Taylor, CVP and deputy CTO of AI Infrastructure; Julia Liuson, president of the Developer Division; and Tim Bozarth, CVP of developer infrastructure, and their respective teams will report to Parikh.
“Over the past few years building GitHub Copilot into the Visual Studio family and Azure, we’ve seen how AI is going to disrupt applications and developer experience,” wrote Amanda Silver in a LinkedIn post. “Bringing it all together into a focused effort will accelerate our energy in this space.”
Moreover, Parikh will work closely with Scott Guthrie, EVP of the Cloud + AI Group, Rajesh Jah, EVP of Experiences + Devices; Charlie Bell, EVP of security; Mustafa Suleyman, CEO of Microsoft AI; and Kevin Scott, Microsoft’s CTO to optimize the entire Microsoft tech stack, Nadella said.
Guthrie will continue leading Cloud + AI, focusing on cloud infrastructure as it grows to become Microsoft’s largest business segment, Nadella added.
“It’s a good sign that Microsoft realizes that AI is changing everything and reorganizing accordingly,” Holger Mueller, an analyst at Constellation Research, told The New Stack. “AI is changing software development, so the merger of the AI platforms and developer tools makes sense. But there will be significant needs for coordination across the divisions, so it’s no surprise that Satya Nadella is referring to doing this as ‘One Microsoft.’ The inflection point from an org structure is that this marks the first time Scott Guthrie gives up product responsibility.”
Skeptical View
Microsoft is no stranger to reorganizing to take advantage of technology innovations, and this move is no exception, Jason Bloomberg, an analyst at Intellyx, told The New Stack.
“It makes sense for any vendor the size of Microsoft to incorporate AI — in particular, generative AI — across product lines. Reorganizing to accomplish this task makes perfect sense,” he said.
However, “Putting on my skeptic’s hat, I wonder if there’s more sizzle than steak here,” Bloomberg said. “After all, Microsoft has been down the ‘agentic AI’ road before with Clippy. Remember Clippy? You could say this move is focused on building ‘Clippy on steroids.’ Does anybody really want that?”
A Whole New Era of Software Development
Omdia’s Shimmin said he believes we’re entering into a new era of enterprise app development, one that reimagines the entire software stack as a much more capable and flexible toolset built increasingly upon agentic systems.
“Instead of building purely deterministic systems through hard-coded imperative methodologies, it seems we’re entering a new phase where developers use a more declarative approach, focusing on defining the outcome rather than writing the linear steps required to reach that outcome.
“So, in all, I think what we’re seeing from Microsoft is a reflection of this suffusion of AI across the enterprise software development stack, something we think is a necessary step already demonstrated to some degree of success by Salesforce with its AgentForce platform enhancements late last year,” he said.
AI Impact on Other Enterprise Vendors
Meta’s Zuckerberg mentioned in the Joe Rogan podcast that he felt he could start using agentic AI systems to replace his mid-level engineers. And Salesforce’s leader Marc Benioff, mentioned famously in late December that he had no intention of hiring “any” engineers in 2025 for the same reason.
“Honestly, I think these large tech firms are jumping the gun a bit here. Relatively advanced autonomous coding assistants like Cline and Aider can certainly build serviceable app ‘starting points’, but I have not seen them actively employed in carrying out the typical workflows that go into maintaining a very large, mature codebase beyond basic documentation, unit testing, and the like,” Shimmin said.
“If these systems do indeed help us generate code at any kind of scale close to what Meta and Salesforce would have us believe, then I am quite certain we’ll be looking at a significant backfill requirement as companies scramble to hire engineers capable of not replacing those displaced by AI but instead maintaining the code built by AI. We’ll see,” he added.
Red Hat’s Approach
As part of its strategy to wrangle new AI advancements, Red Hat this week completed its acquisition of Neural Magic, a pioneer in software and algorithms that accelerate GenAI inference workloads. With Neural Magic, Red Hat adds expertise in inference performance engineering and model optimization.
With Neural Magic’s technology, Red Hat aims to break through the challenges of wide-scale enterprise AI, using open source innovation to further democratize access to AI’s transformative power, the company said.
Neural Magic’s capabilities will be incorporated into Red Hat AI, Red Hat’s portfolio of gen AI platforms. Built with the hybrid cloud in mind, Red Hat AI encompasses:
- Red Hat Enterprise Linux AI (RHEL AI), a foundation model platform to more seamlessly develop, test and run the IBM Granite family of open source-licensed LLMs for enterprise applications on Linux server deployments;
- Red Hat OpenShift AI, an AI platform that provides tools to rapidly develop, train, serve and monitor machine learning models across distributed Kubernetes environments on-site, in the public cloud or at the edge; and
- InstructLab, an approachable open source AI community project created by Red Hat and IBM that enables anyone to shape the future of gen AI via the collaborative improvement of open source-licensed Granite LLMs using InstructLab’s fine-tuning technology.
“Red Hat’s acquisition of Neural Magic is a strategic enhancement to its AI capabilities, facilitating AI deployment across hybrid clouds by leveraging Neural Magic’s expertise in model optimization and inference acceleration,” said Dave McCarthy, an analyst at IDC, in a statement. “This move not only aligns with Red Hat’s commitment to open source innovation but also positions the company to offer more cost-effective, scalable AI solutions that reduce dependency on specialized hardware.”
