Apple’s Open Source Roots: The BSD Heritage Behind macOS and iOS
Consider this: Apple’s stylish and user-friendly devices, like the powerful MacBook and the ubiquitous iPhone, owe much of their reliability and performance to an open source operating system. But which open source OS do they use? Although it’s often claimed that Apple’s macOS, iOS, iPadOS, watchOS, visionOS and tvOS are directly descended from FreeBSD, this is actually a misconception.
The true foundation of these operating systems lies in a marriage between Apple’s older OS technologies and NeXTSTEP, which is itself a hybrid of Mach and BSD, predating FreeBSD. While some elements of the FreeBSD userspace were incorporated into these operating systems over the years, the Apple kernel (XNU) does not directly stem from FreeBSD. However, they do share a common BSD heritage.
Understanding BSD Variants
To understand how Apple’s operating systems evolved, it’s crucial to understand the different versions of BSD (Berkeley Software Distribution) and their influence:
The Original BSD at Berkeley
BSD originated at the University of California, Berkeley, in the late 1970s. Developed as an enhancement to AT&T’s original UNIX operating system, BSD introduced many innovations that have become standard in modern operating systems. The first version, 1BSD, was released in 1977, followed by 2BSD in 1978. Significant versions include 4.1BSD (1981) and 4.2BSD (1983), which introduced the Fast File System (FFS), TCP/IP networking and the sockets API, foundational elements still in use today.
FreeBSD
FreeBSD, derived from the original BSD, is celebrated for its performance, advanced networking features and extensive hardware support. It emerged from the 386BSD project in 1993 and has since become widely used in servers, desktops, and embedded systems. FreeBSD’s robust performance and reliability make it a favorite for high-availability applications and internet infrastructure.
NetBSD
NetBSD is famed for its portability across many hardware platforms. Its motto, “Of course it runs NetBSD,” reflects its capability to operate on a vast array of hardware architectures, from servers and desktops to embedded devices and mainframes. Since its inception in 1993, NetBSD has been a leading example of portability and clean design in the BSD family.
OpenBSD
OpenBSD, which forked from NetBSD in 1995, focuses on security, correctness and code simplicity. It is renowned for its rigorous security practices and has pioneered many security technologies such as OpenSSH, PF (packet filter) and secure memory management techniques. OpenBSD’s commitment to security makes it the preferred choice for applications where security is paramount.
Other BSD Variants
Several BSD variants have emerged, each with a unique focus and enhancements. Examples include DragonFly BSD, which focuses on performance and scalability, and Darwin, which forms the core of Apple’s macOS and iOS.
Darwin and XNU: The Core of Apple Operating Systems
At the heart of macOS is the XNU kernel, a hybrid kernel that combines the Mach microkernel, BSD components and the I/O Kit, an object-oriented API for device drivers. This integration ensures that macOS benefits from the Unix-like robustness of BSD while leveraging the flexibility of the Mach microkernel.
Darwin, the open source foundation for Apple’s operating systems (macOS, iOS, watchOS, tvOS and iPadOS), includes the XNU kernel, various BSD components and other open source projects. Darwin’s origins trace back to NeXT, a company founded by Steve Jobs in 1985, after he left Apple. NeXT developed NeXTSTEP, an operating system built on the Mach microkernel and BSD. This evolved into OpenStep and eventually led to Darwin when Steve Jobs returned to Apple, bringing NeXT’s technology with him.
In 2000, Apple released some of the core components of Mac OS X (now macOS) as open source under the Apple Public Source License (APSL), allowing the broader community to benefit from and contribute to its development. This included components like launchd, Grand Central Dispatch and Core Foundation, some of which were later licensed under the more permissive Apache License to encourage wider adoption. However, higher-level components like the Cocoa and Carbon frameworks remained proprietary, preserving Apple’s competitive edge. The GPL and BSD licensed components were not re-licensed, maintaining their original open source licenses.
Detailed Timeline
- 1969: The UNIX operating system is developed by AT&T’s Bell Labs.
- 1977: The first version of BSD (1BSD) is released by the University of California, Berkeley, as an enhancement to AT&T’s original UNIX operating system.
- 1978: 2BSD is released, continuing to build on the foundation of 1BSD with further enhancements and tools.
- 1980: 3BSD is released, introducing more advanced features and improvements.
- 1983: 4.2BSD is released, incorporating major innovations like the Fast File System (FFS) and TCP/IP networking, which become foundational elements for future operating systems.
- 1985: Steve Jobs founds NeXT after leaving Apple.
- 1989: NeXTSTEP is released. It is based on the Mach microkernel and BSD, and combines advanced capabilities with an object-oriented design.
- 1993: FreeBSD and NetBSD are derived from 386BSD, each focusing on performance, security and portability, respectively.
- 1995: OpenBSD is forked from NetBSD, emphasizing security and code correctness.
- 1996: Apple acquires NeXT, bringing NeXTSTEP technology back to Apple and laying the groundwork for Mac OS X.
- 2000: Apple releases the first version of Mac OS X, based on Darwin OS. This OS combines the Mach microkernel with components from BSD, forming the core of Apple’s modern operating systems.
- 2006: OpenDarwin ceases releasing, marking the end of Apple’s efforts to provide a stand-alone, installable version of Darwin OS.
The Evolution and Integration of BSD Components in macOS
|
Component |
Origin |
|
XNU Kernel |
Mach/NeXTStep/OpenStep |
|
Network Stack |
FreeBSD/BSD, with additional NIKE and IOKIT |
|
Virtual File System |
FreeBSD/BSD |
|
Userland Tools |
FreeBSD/BSD |
|
Memory Management |
Mach/NetBSD |
|
Process Model |
Mach IPC, Mach Security Trailers, and Mandatory Access Control (MAC) mechanisms |
Apple’s macOS is a hybridized operating system that integrates various components from different BSD variants, creating a robust and versatile platform. At the heart of macOS lies the XNU kernel, a hybrid kernel that combines elements from Mach, NeXTSTEP and OpenStep, with additional components from BSD. This foundational architecture leverages the strengths of each system to deliver a reliable and high-performance operating system.
Network stack: The network stack in macOS has roots in FreeBSD and other BSD variants, incorporating their reliable and efficient networking capabilities. Early networking elements, such as the TCP/IP stack, were influenced by FreeBSD’s design, renowned for its performance and reliability. Including the kqueue event notification interface, originally developed for FreeBSD, enhances macOS’s ability to handle I/O events efficiently. Apple’s implementation incorporates code from both BSD and FreeBSD, but also includes unique mechanisms like the Network Kernel Extension (NKE), the object-oriented device driver system (IOKit) and disk arbitration layers. These components represent significant divergences from traditional BSD implementations.
Virtual file system: MacOS’s virtual file system components are derived from FreeBSD and other BSD systems, ensuring a stable and secure file management system.
Memory management: MacOS’s memory management system primarily originates from Mach, with influences from NetBSD, particularly the shared/merged buffer cache.
Process model: MacOS’s process model is based on Mach, using Mach system threads at the base layer of the process subsystem.
Mach IPC and security: Mach IPC is extensively used throughout the kernel and userland of macOS and iOS, introducing further differences. For example, Mach security trailers are a fundamental part of trusted IPC. The Mandatory Access Control (MAC) mechanisms used in application sandboxing also diverge significantly from those in *BSD.
Divergence over time: FreeBSD’s reluctance to integrate UNIX03 compatibility changes led to a divergence, with libraries like libc and libm forking despite some cross-pollination efforts. Additionally, initiatives like the NeXTBSD project, which aimed to bring technologies like launchd, Mach IPC and Grand Central Dispatch from Apple’s ecosystem to FreeBSD, were met with limited interest. As a result, the codebases have continued to diverge, especially with Apple’s advancements in custom silicon support and optimizations, further widening the gap between them.
Known integrations and contributions: macOS has benefited significantly from FreeBSD, particularly in networking concepts. While macOS doesn’t fully use FreeBSD’s networking stack, it incorporates several FreeBSD-derived components tailored to meet its unique requirements. Early networking elements in macOS, such as the TCP/IP stack, were influenced by FreeBSD’s design, known for its performance and reliability. Additionally, features like the kqueue event notification interface, originally developed for FreeBSD, have been integrated into macOS, enhancing its ability to handle I/O events efficiently.
Impact on Apple Products
Apple’s adoption of BSD code extends across its entire product line. macOS powers the Mac desktops and laptops, iOS runs on iPhones, iPadOS on iPads, watchOS on Apple Watches, visionOS on AR/VR devices, and tvOS on Apple TVs. Each of these operating systems incorporates BSD components, highlighting the widespread influence of BSD in Apple’s ecosystem.
BSD’s robustness, security and performance features have been instrumental in shaping the stability and efficiency of Apple’s operating systems. Integrating BSD-derived networking concepts and components has enabled macOS to deliver high-performance network capabilities, making it a reliable choice for consumer and enterprise applications. BSD’s advanced memory management and process scheduling also contributed to macOS’s responsiveness and multitasking capabilities.
By blending BSD’s robust features with its proprietary technologies, Apple has created a series of operating systems that are stable, efficient, innovative and highly functional. This symbiotic relationship underscores the importance of open source contributions in the ongoing evolution of Apple’s software ecosystem.
Apple’s Contribution to and Consumption of Open Source
Apple heavily relies on BSD but also contributes to the open source community. The company’s open source website and its GitHub provide access to many components of the Darwin OS, including the XNU kernel, various userland utilities and libraries. These contributions ensure that the open source community can benefit from Apple’s innovations and improvements.
Periodically, Apple updates its open source projects, releasing new versions of Darwin OS components. These updates often coincide with new macOS and iOS releases, showing Apple’s ongoing commitment to the open source community. By sharing its improvements and enhancements, Apple helps advance open source software development, benefiting developers and users.
The State of Darwin OS
Darwin OS, the open source core of Apple’s operating systems, is now released in fragmented pieces rather than as a complete, installable system. Apple distributes components of Darwin separately, such as the XNU kernel repository on GitHub and various userland tools on the Apple Open Source website. This fragmented release strategy makes assembling a complete Darwin OS from Apple’s official sources challenging.
With the OpenDarwin project, Apple once made these pieces able to be installed and stand alone. However, due to limited interest and significant divergences in development, OpenDarwin ceased releasing in 2006. Apple’s intentions with OpenDarwin were commendable, but the forks became too significant to sustain, leading to the current fragmented approach.
Despite the challenges in obtaining a complete Darwin OS from Apple, the open source community has attempted to integrate these components into an installable operating system with varying success. Projects like PureDarwin (and previously OpenDarwin) have aimed to provide a usable version of Darwin OS by integrating the various open source components released by Apple. These community-driven projects highlight the collaborative spirit of open source development and demonstrate the potential of Darwin OS as a stand-alone operating system.
Apple’s Current Relationship With BSD
Apple’s relationship with BSD code is shrouded in mystery. Despite its significant contributions to open source, the company remains somewhat opaque about the exact extent of BSD code in its operating systems. This closed-off development process makes it challenging to pinpoint how much macOS, iOS and other Apple systems still rely on BSD roots. The permissive BSD license allows Apple to use and modify the code without the obligation to disclose where or how it is used, unlike the copyleft licensing used in the GPLv2 for Linux, which requires modifications to be shared.
Apple’s approach to open source software has evolved over time. The company has shifted its focus from CoreOS’s OSS components to other areas, such as programming languages and compiler technologies. Apple has invested significant time and energy into projects like Swift, the clang compiler and the LLVM runtime. These efforts represent some of the most successful examples of Apple’s contributions to the open source community and demonstrate a substantial technology transfer from Apple to BSD, particularly in the compiler toolchain.
Moreover, over the years, Apple’s extensive modifications blur the lines between legacy BSD code and modern adaptations. As Apple has significantly modified and extended the original BSD components, it becomes increasingly unclear which parts are directly inherited from BSD and which are innovations by Apple. This opacity limits our understanding of BSD’s full impact on Apple’s products, highlighting the difference in disclosure requirements between BSD’s permissive license and GPL’s copyleft approach.
There is evidence that Apple continues to use current BSD components in its operating systems. Recent June 2024 commits submitted by Klara, Inc., which upstreams code to the FreeBSD project on behalf of various clients (such as NetApp), indicate ongoing integration and use of FreeBSD code, and demonstrates that despite the closed nature of Apple’s development process, BSD code (particularly FreeBSD) remains a vital component of its operating systems.
Market Impact and Adoption
Apple’s use of BSD in its products has had a substantial impact on the consumer electronics industry and creative content professions. The robustness, security and performance of BSD have made Apple devices highly appealing to consumers and professionals. Industries such as film, music production, graphic design and other creative fields have especially embraced Apple’s products for their reliability and powerful capabilities.
Advanced memory management, process scheduling and high-performance networking capabilities from BSD ensure that Apple’s desktops and laptops deliver the necessary performance for demanding applications. This makes macOS an excellent choice for video editing, music production and graphic design, where stability and performance are crucial.
Other Industries Leveraging BSD
While Apple’s integration of BSD has been instrumental in its success, BSD’s characteristics also make it a preferred choice for other vendors in various industries. Companies like NetApp, Netflix, Juniper (all FreeBSD users) and others use BSD for its advanced features in different applications:
- Security: BSD’s strong networking capabilities and robust security measures benefit the security sector, making it ideal for secure communications and data protection.
- Internet traffic management: BSD’s reliable TCP/IP stack ensures efficient handling of high data volumes, making it suitable for managing internet traffic.
- Embedded systems: BSD’s advanced memory management and process scheduling contribute to the efficiency and reliability of embedded systems.
- Storage solutions: Companies like NetApp leverage BSD’s capabilities to provide robust and efficient storage solutions.
By recognizing BSD’s diverse applications, we can appreciate its influence on Apple’s products and across a wide range of industries, highlighting the versatility and robustness of this open source operating system.
Number of Products and Market Reach
Apple has released a wide range of products that run on macOS, iOS, iPadOS, watchOS, tvOS and visionOS. As a result, many devices worldwide contain BSD code. Thus, the number of shipped and running Apple products, including embedded devices, desktops/laptops and even server computers within the company’s data centers, may be comparable to or even exceed the number of devices and systems running on Linux. This is strong evidence of the widespread adoption and impact of the BSD legacy within Apple’s ecosystem.
Apple’s devices are found in various industries and among different consumer demographics. This widespread market presence demonstrates the significance of BSD’s contributions to the success and reliability of Apple’s operating systems. The seamless user experience, strong performance and security features derived from BSD have made Apple products the preferred choice for millions of users worldwide.
Outlook
Apple has heavily relied on BSD code to develop its operating systems, including macOS, iOS, iPadOS, watchOS, visionOS and tvOS. By integrating BSD components, Apple has established a strong foundation for innovation, stability and performance. Although the exact extent of BSD code in Apple’s systems is not transparent, it is evident that BSD has significantly influenced the design and functionality of Apple’s products.
Looking forward, it is likely that Apple will continue to take advantage of BSD’s strengths while also contributing to the open source community. The ongoing evolution of both BSD and Apple’s operating systems suggests a future where open source collaboration remains essential for technological advancement. Recognizing the importance of BSD in Apple’s success helps us better appreciate the value of open source contributions in shaping the technology we use daily.
References and further reading:
