How Data Helps Lego Click Developer Experience into Place
LONDON — “We need data to power good developer experiences.”
That’s how Jonas Druedhahl Rask, Lego’s director of developer experience and master brick picker, kicked off his The DevOps Conference talk, along with Waqas Ali, fellow master brick picker and engineering delivery lead for the developer platform.
Not only is Lego a prominent giveaway at most tech events, it’s also an influential tech company known for its innovation. Leveraging a canonical data structure and Backstage, the Lego platform engineering team is now using a lot of inner sourcing and continuous feedback to build an internal developer platform by developers for developers. Here’s how.
The New Developer Journey
The Lego platform team’s work is grounded in the developer persona “Emma.” Druedhahl Rask and Ali walked the audience through Emma’s onboarding.
Like every newbie, Emma’s first task is to work out the complex people, processes and tooling of the product-led organization. This includes API discovery and how to create and update documentation.
“All of this also comes with some challenges: being new and trying to understand how the company operates, what is this structure for the organization?” Ali said. “Then she also needs to find the right tools to do the job. And she would like to identify other teams she can possibly work with.”
The Lego Group is organized into domain-driven, stream-aligned product teams, supported horizontally by a platform engineering team and a series of enablement teams. Each stream-aligned team is in charge of a digital product, and is staffed by a long-standing DevOps team that’s responsible for its development. Emma is onboarding onto one of these product teams.
From Day 1, Emma needs cloud access to deploy to the cloud. At the start of Lego’s platform engineering journey, she would have had to reach out to the cloud enablement team, chat over email and fill out a couple of forms, and it would take almost a week to get a cloud account.
Why so long? Druedhahl Rask grouped the onboarding challenges into:
- Discoverability. As he said, “You need to know someone who knows someone who knows where to go in order to get things done.”
- Automation. Too many humans in the loop and not enough automation, increasing lead time.
- Ownership. When individuals are tied to IT assets, it’s harder to determine ownership.
Add to this, if someone wanted to move to another product team, they would have taken their cloud resources and IT assets with them, which sometimes, he said, would render so much chaos that some developers decided to leave the company altogether.
A Key to Success: Supergraph Data Modeling
Lego’s solution is a canonical data model that spans the whole Lego technology landscape and enables:
- Discoverability.
- Interaction.
- Ownership.
“It’s important in this model that we capture the right entities that will allow us to do this discovery of the different resources and IT assets,” Druedhahl Rask said.
For example, he said, “To power automation efforts, you need to be able to interact with the different entities in this model,” like product teams being able to provision and manage cloud accounts, while the cloud enablement team can adopt FinOps, by measuring the cost across all cloud accounts per product.
This canonical data model — which maps out like an interconnected web — allows for a lot of discoverability, Druedhahl Rask said, while maintaining ownership.
But it wasn’t enough at Lego’s scale.
“If you want to use it for something real, you also need to make it available,” he said. “So we are democratizing access to all of the data within the model by API enabling it.”
The API is the standard that has emerged for how developers prefer to access and extend an internal developer platform.
Next, the Lego platform team adopted the supergraph architecture and operation model, citing inspiration from Netflix and Volvo. The team uses a GraphQL API with a schema federation to bind together one common schema, which is behind an abstracted composition layer — like an internal developer portal — for product and service discoverability. Underneath, a subgraph branches off for each product’s data model.
“It provides a schema that is valid for that entity, and then there’s this guardrail that every entity must match with [a] product either directly or indirectly, so we have this traceability back to a product always,” Druedhahl Rask said. The schema and guardrail are built and managed by the local data owner, which is always a product or enablement team.
This canonical data model gets grammatical with the structure of subject, object and predicate to make searchable relationships across the model. An employee (subject) can have multiple relations to a product (object), like is a developer on or is a lead engineer on (predicate). This makes for a more searchable developer experience.
Instead of tying these data and products to individuals, now the data and the people are tied to products, which are then owned by a team. When you go into a product, you get a holistic view of what the product is about:
- Domain.
- Subdomain.
- Connected cloud accounts.
- Everyone who works on it.
- All applications the product owns.
- All production APIs.
- Documentation.
“It’s all connected with the product, not with people,” Ali said, who are onboarded onto a product and can be transferred from there. “And that’s available to everybody, all the engineers.”
Lego Adopts the Baseplate
The average product developer isn’t aware that all of this is happening — though everything is visible to everyone.
Instead, the platform team has built Baseplate — after the foundation for most Lego builds — an internal developer platform built on top of Backstage, the popular open source platform for building developer portals.
The team takes the data from the subgraph and turns it into a catalog of products within the platform, Ali said.
At this point, Emma, Lego’s developer persona, needs to figure out how to work with other teams:
- What can she do to make her life easier?
- Are there other teams that are doing similar things?
Instead of building an API from scratch, Emma looks at what’s already available. She can search on the Baseplate internal developer platform for all the APIs in production. She can find out information on each API, can connect to it automatically, can find out what product team owns it, and find any of the centralized documentation.
There’s no longer a human in the loop to approve access to in-production assets. Cross-organizational reusability is unlocked.
From Day 1, Emma can contribute to that documentation, which is submitted via pull request to whichever team owns a product. She can also create Backstage software templates, plugins and documentation herself.
Inner Sourcing with Backstage Plugins
Leveraging the Backstage plugin model, Lego embraces a culture of inner sourcing, inviting any developer to contribute to building on Baseplate.
The Lego platform team created a plugin template to allow teams to build and contribute their own Backstage plugins to increase platform self-service.
The team foresaw the common hurdle that Backstage uses TypeScript, which few Lego developers are fluent in. To ameliorate this, they hold two-day internal hackathons, where the platform team embeds with the enablement or product team onboarding to Baseplate. Teams can build whatever they want to help make Baseplate more useful faster, and the platform team helps build the programming language gap.
The platform and cloud enablement teams held a hackathon to build a cloud plugin to create a self-service experience in Baseplate to help devs request and set up a cloud account — cutting this time from about three days to about three minutes. Together, those teams were also able to build FinOps plugins, by removing the ownership of the account from people and attaching it to a product.
“Value gains from this is tremendous,” Ali said, with a 95% decrease in developer hours spent setting up cloud accounts, and 99% of cloud accounts now connected to products, leading to no more orphaned accounts. Ownership is clear to everyone.
Other inner-sourced Backstage plugins include those fostering self-service around documentation, APIs, GitHub, learning and development, and AI products.
How to Measure Developer Experience
“This whole connected, integrated experience that we are providing is also removing this cognitive load from our engineers that they don’t have to ask someone and find the information themselves or find the API or find the cloud account,” Ali said. “Or if something is going down, find the owner of that thing.”
Platform engineering emphasizes constructing frictionless golden paths, so developers want to use the platform — they aren’t forced to. That makes adoption one of the most common ways to measure developer experience and platform success. Lego judges this by asking questions like:
- How many users are on the platform for X number of days?
- How many documentation hits on which documentation?
- How many products are using the platform?
“We are looking at these numbers every day, and we are making decisions based on these numbers,” Ali said. “Baseplate is being used by 92% of the products. That means they’re finding value in it.”
Baseplate currently has:
- About 1,300 users.
- More than 60,000 documentation hits a month.
- More than 400 hits to the cloud plugin a month.
- More than 400 APIs in production, with an Open API specification.
The platform team uses heat maps across domains, products, plugins and more to understand who is using, for example, documentation a lot and comparing that with which team is using the platform the least.
A low adoption rate could be a signal that there’s confusion about what’s available or how to use the platform. That team becomes a priority for outreach and a hackathon to be sure the platform engineers are building for that target audience’s needs.
The platform team is also examining the developer user journey to uncover patterns around user drop-off, such as a product team that begins to onboard, maybe even sets up a cloud account, but doesn’t do the next step of adding new user accounts. That’s when the platform team reaches out to users for feedback, uncovering bugs and filling feature gaps along the way.
“Feedback is a very central part of our platform,” Ali said, including continuous conversations and dedicated Slack channels. “We are hosting community sessions in the company where anybody can join, ask questions, and we show, inspired by Backstage itself.”
The platform team also collects context-based feedback where people can provide feedback on specific features from within Baseplate, which goes directly to the plugin or product owner.
“We will see it,” Ali said of the platform team, “but we are connecting our users with the engineers who are responsible for the platform as well.”
