23rd Oct: It's time for the Second TV Tropes Halloween Avatar Contest! Details here
"It's a UNIX system! I know this!"Quirky, often counterintuitive, but incredibly flexible, UNIX has gone from a little-known research operating system in the 1970s to an entire design philosophy. In its experimental days, UNIX stood in the background, influencing OSes but not making much noise on its own; that changed starting in the mid-1980s, when the first commercial UNIX products appeared, and exploded in the mid-1990s, as OSes either based directly on UNIX or following its principles came to the forefront via the World Wide Web. As of 2014, UNIX-like operating systems have finally managed to outrank Microsoft Windows in terms of usage share, with UNIX-like systems spanning pretty much all the scales and sizes of computing — from uCLinux for microcontrollers and embedded devices, to Apple iOS and Android for smartphones, to Linux and Mac OS X for desktop computers, to Linux, Oracle Solaris, HP-UX and IBM AIX for servers and mainframes. UNIX was once considered unfriendly, terse and somewhat elitist, requiring expensive licensing and large, expensive minicomputers to run; now, almost all of UNIX's source code is freely licensedmuch longer explanation and UNIX derivatives can be found on things as small as a smart phone * , or as large as an IBM mainframe. It's also probably the only OS that came into existence specifically because a bored programmer wanted to play a game. Science-fiction writer Neal Stephenson wrote of Unix, in his essay "In The Beginning Was The Command Line" in January 1999:
— Lex, Jurassic Park
Multics for the rest of usBefore diving into what UNIX is, it'd make sense to mention the OS that inspired it: Multics. Multics was itself a research OS; it was highly ambitious, was highly secure, required expensive IBM-ish mainframes to run, and was also stuck in Development Hell. (Eventually Honeywell-Bull would commercialize it, but that was several years off.) One of the research partners was Bell Labs, then the experimental division of the Bell System, and in 1969, they decided to leave the project. This left one programmer there, Ken Thompson, with not much else to do. Thompson had a pet project going on Bell's Multics machine, a game called Space Travel, but at several dollars (in 1969 money) per run, using it was not economically sound. Looking for a cheaper alternative (as well as a new project to work on), he spotted a spare DEC PDP-7 in his lab, and began porting Space Travel to it. As he did, he found himself recreating various parts of Multics around it; eventually, he had a file system, kernel and utilities ready to go. Since it was kind of like Multics, but "neutered", he decided to call the new system "Unics". Eventually, after getting a more powerful system (a DEC PDP-11) to improve Unics and work on the first real application for it (a typesetting package that eventually became nroff), Ken and his development partner Dennis Ritchie started setting the name in small caps, at which point it mutated into "UNIX," which would have been displayed as "Unix" with Ritchie's formatting. UNIX used to be trademarked by Bell Labs itself, but through several historical accidents it's now controlled by the Open Group, who publishes a "Single UNIX Specification" and grants use of the trademark to OSes that pass a verification suite based on the SUS. Many UNIX-like OSes or even descendants of UNIX are not officially UNIX: for example, BSD, while being descended from UNIX, is not "UNIX"(tm) because the developers did not pay for a trademark certification. Linux, BSD, and the other UNIX-alikes all try to comply with the POSIX standard, however. A lot of tropes associated with UNIX from the start began here; Multics had long, self-explanatory commands with much shorter aliases, and UNIX generally just used the shorter forms (like ls, cp, etc.) directly. Back in those days, your terminal was very likely going to be a clunky old Teletype 33 printing terminal, and when the terminal was capable of about 10 characters per second, brevity counted. Also, the PDP-7 in particular didn't have much memory, and Multics' way of handling communications between running programs would have been massive overkill, so Doug McIlroy came up with "pipes", a feature even non-UNIX OSes like Windows NT use now.
Portability, C and the Berkeley connectionNot long after the first edition of UNIX was published inside Bell Labs, Ken and Dennis started work on making the system portable. In the early 1970s, this was a big deal, since almost all OSes up to that point had been written specifically for the machine they were going to run on. The idea was to write most of the kernel in a higher-level language that could be "compiled" into code for different machines, then add small bits of machine-dependent code where needed to handle things like interrupts and memory. The language Dennis invented for the project was called "C", and after he and Brian Kernighan published a book on it, it became a hit outside Bell Labs and even outside the UNIX community (in fact, it's highly likely the Web browser you're reading this in has C or C++ code in it). Once Unix was ported to C, Bell Labs started allowing researchers at universities to study its insides. Since the Bell System was still a regulated monopoly at the time, and thus couldn't sell computers or OSes, Bell Labs would give university computer science departments access to the UNIX source code for the cost of duplication once they signed a non-disclosure agreement. UNIX became very popular in operating systems classes after this. The lecture notes of one Australian computer science professor, John Lions of the University of New South Wales, were compiled into a book (Lions' Commentary on UNIX 6th Edition, with Source Code, aka the Lions Book) and became a widely-bootlegged underground publication and circulated for years this way due to both the Bell System's licensing strictures and the later UNIX Wars (see below); it was, finally, legally published in book form 20 years later, in 1996. The college with the most influence by far, though, was the University of California at Berkeley, or just "Berkeley" for short. Berkeley's Computer Science Research Group added many new features to the UNIX system, eventually creating their own version of UNIX called the Berkeley Software Distribution, or BSD. Among the many things they did were porting UNIX from the PDP-11 to the VAX (in the process, modernizing the way it handled virtual memory), adding networking support, improving the user input/output functions significantly, and generally cleaning things up. The icing on the cake was the licensing — Berkeley put their changes under a license that allowed free distribution and modification. This meant that anyone with a Bell UNIX license could use BSD, and this also had bigger implications later on. In the meantime, Bell Labs released what would be the last important version of "classic" UNIX, the 7th Edition (otherwise known as "Version 7" or "V7" to fans). V7 was considered by many the last version of UNIX ever, and through ports like Xenix found its way onto many machines not powerful enough to run BSD UNIX. There was also a full rewrite of V7 called "Minix", which aimed to help teach UNIX concepts (and, later, advanced concepts such as microkernels) to students and curious hackers that couldn't get a V7 license from Bell. This was many people's first introduction to UNIX concepts, and like BSD, it would have a huge influence later on.
UNIX goes commercial; GNU is born (with the Free Software Movement)In the early 80s, it looked like things would keep going as they had, with UNIX being primarily an academic research OS, for the foreseeable future. But as usually happens in history, a major change was triggered by something that at first seemed completely unrelated. In the case of UNIX, the seemingly unrelated trigger was American antitrust law. In 1982, AT&T (the Bell System's parent company) lost a long-running lawsuit with the US Department of Justice, and was forced to divest itself of its local telephone service companies (the "baby Bells"). In return, they were finally allowed to enter the computer industry, and commercializing UNIX was at the top of their priorities. They changed the licensing such that various parts of the system were "unbundled" or a la carte, making a usable UNIX system much more expensive. This annoyed many people, but there was little they could do about it at the time. In the meantime, the UNIX group at Bell continued working on the original UNIX tree, which eventually became UNIX System V, the basis of all of AT&T and its partners' commercial UNIX offerings. Most vendors ended up merging it with their own code bases (often based on Version 7) and creating their own UNIX variants such as HP-UX (Hewlett-Packard) and AIX (IBM). Direct ports of System V were available from outside porting houses like Interactive Systems. At around the same time, a researcher at MIT's AI Lab named Richard Stallman decided he was fed up with companies founded by ex-AI-lab-members (such as Symbolics), which often built upon AI lab software and were expected to be used by the AI lab, "hoarding" their innovations and preventing sharing of code. (For more information on this, see "Free as in Freedom") His main concern was that, once the source to various parts of an OS were "nonfree" or made unavailable, fixing or improving on them was impossible (or, at least, extremely difficult), that this could lead to monopoly behavior and rip communities apart (such as Symbolics and the AI lab, which had lost many talented personnel). This was hastened by the death of the AI lab's own operating system, ITS (a sort-of cousin to UNIX—ITS and Multics were both derived from the early operating system CTSS, but seen as more elegant yet less portable), the abandonment of the AI lab by his friends, and the "locking up" of their successors, the Lisp Machines. He envisioned a complete operating system where all of the parts were "free software" and could be modified and changed at will, but with the caveat that the changes had to be shared with others as well. He called his vision "GNU", a recursive acronym for "GNU's Not UNIX!", posted a manifesto describing his intentions to Usenet in 1983, and created the Free Software Foundation to oversee the effort. By 1990, the FSF would have most of the parts of GNU ready, including a compiler, utilities and such, but no kernel to go with it. Thus, Hurd was born. Hurd was to be a completely free and open kernel, a hybrid between a Microkernel and a Monolithic kernel based on the Mach kernel layout of BSD. However, development was slow, and even as of 2013, it is still not considered suitable for everyday use. Development slumped even further when it appeared that the Linux kernel (which we will get to later in the article) had progressed much faster to the point where it is stable enough for production use by the mid-90s, and is also of the right license to integrate with the rest of the GNU parts. On the other side of the US, in Silicon Valley, some of the Berkeley researchers, along with hardware designers from Stanford, got together to make the first computers designed specifically to run UNIX, called workstations. The most famous of these was Sun Microsystems, named after the Stanford University Network, a part of the ARPANET, the ancestor of the Internet. Other companies, like Silicon Graphics, soon followed, and even companies that historically had ignored UNIX before (HP and IBM) got into the fold with the HP 9000 (running HP/UX) and the IBM RT-PC (running either AIX or AOS, a BSD derivative).
The UNIX Wars; Linux and the BSDs appearStarting around 1988, Sun and AT&T entered into an agreement to develop UNIX-based software together, and called the joint venture "UNIX International". As part of the agreement, Sun (who had been using a custom BSD variant called SunOS up to this point) and AT&T agreed to make a "merged" UNIX that would combine the best parts of BSD and System V; this was released as System V Release 4 in 1990, with Sun's version being released under the name Solaris 2. Other vendors, specifically DEC, IBM, and HP, felt snubbed and formed the Open Software Foundation in protest, and began work on a UNIX derivative called OSF/1, which (like NeXTStep and, much later, Mac OS X), was based on the Mach microkernel and parts from 4.3 BSD. In the meantime, the IBM PC and its clones were taking the commercial market by storm, and the Mac was introducing people to a new way of working that many felt no one could touch. UNIX was in a state of disarray; most vendors had proprietary changes to their UNIX builds, none of which were compatible with the others, and porting software was becoming more difficult. This inspired some of the first standards for UNIX, promulgated by the POSIX working groups and by X/Open. Getting everyone to agree would be difficult, but as the PC became more and more powerful and Microsoft began talking about making a version of Windows to compete with UNIX, the impetus to cooperate grew. By the mid-1990s, UNIX International had been disbanded, and OSF stopped development on OSF/1 (leaving DEC to maintain their own branch, renamed "Tru64", for their Alpha machines); OSF merged with X/Open to form the aforementioned Open Group. At Berkeley, a programmer by the name of Keith Bostic, inspired by Stallman's GNU Project (and Berkeley's own separation of their TCP/IP code in "Networking Release 1"), came up with a large project of his own: purge the BSD codebase of proprietary AT&T code to allow Berkeley-based startups to market it without licenses from AT&T. Involving almost all of the UCB Computer Science Research Group, the project was almost completely successful, with the results being released as "Networking Release 2" or "Net/2", and two "forks" soon emerged; "BSD/OS," from startup BSDi, and "386/BSD," a free software product from Berkeley hacker Bill Jolitz. Eventually, AT&T decided to wash their hands of the whole thing, and spun off the UNIX group as UNIX System Laboratories (USL). In one final salvo before concluding the war, USL sued Berkeley and BSDi, claiming copyright infringement and restraint of trade. The possibility of a free UNIX appearing in the near future appeared dim. In 1991, a programmer in Helsinki, Finland named Linus Torvalds, who was inspired by Minix * posted his intent to build a "little" UNIX clone (which he intended to be "just a hobby" and "nothing big like GNU") to Usenet, with some of the work already done. Others agreed to help, and by 1992 Linux, as it was dubbed by one of Linus's helpers, was maturing quickly. Linus made the important decision to put the Linux kernel under the same license as GNU's tools and utilities, making it attractive to developers who appreciated GNU's stance. With commercial UNIX still expensive, and BSD's future unclear, Linux grew quickly, and by 1995 had reached version 1.0; already, there were several vendors offering "distributions" or fully-usable OSes based around Linus's kernel and GNU at that time. Some of them, like Linspire, SLS and AV Linux, came and went, others like Slackware, Fedora (changing names from Red Hat), SuSE and Debian lived on, and new distros appear on the scene periodically. As of 2013, a spinoff of UK-based Ubuntu, Mint, is the most popular distro. Ubuntu was #1 for a few years after it was introduced, however it slipped from the #1 pedestal after making some decisions that were unpopular with the Linux community (the Unity desktop manager and integrated Amazon search being the two most cited offenders) and earning the ire of the Free Software Foundation for "becoming too commercialized". While all this was going on, Jolitz suddenly dropped support for 386BSD, resulting in a set of unofficial patches to the last version appearing. Eventually, these patches were cleaned up, improved, and became the subject of two forks: NetBSD, which focused on porting the Net/2 tapes to as many platforms as possible, and FreeBSD, which sought to continue where 386BSD left off, and make a quality BSD release intended specifically for PC compatibles. The USL lawsuit still loomed large, though, and progress was slow compared to Linux (which wasn't party to the lawsuit and could move at full speed). Finally, USL was bought by Novell, and Novell decided to call a truce. It was decided that, with the exception of small parts of code inside the kernel, that BSD was solely under Berkeley's license, and that the UNIX trademark would be given over to X/Open. The decision meant that NetBSD and FreeBSD could return to full development, and by the late 1990s, development speed was on par with Linux. NetBSD itself would later be forked into OpenBSD, which has an emphasis on security, and FreeBSD 4.0 was forked into DragonFlyBSD, due to a disagreement over an architectural change in FreeBSD 5.0.
The Open Source MovementNot long after this, another development in the software world had a huge impact on UNIX and the Internet. The World Wide Web was finally gaining in popularity after years of development, and the vehicle most people used to access web content was a program called "Netscape Navigator". The company behind it, also called Netscape, had placed the software under a license that allowed free usage for individuals and non-commercial organizations, but required payment for use in corporations; it was hoped that corporations would also invest in their server software, which was top-of-the-line for the time and quite expensive. The problems started when it turned out few people were registering their copies of Netscape; despite what the license said, registration was never mandatory and the program was effectively free to anyone interested. To add to this, Netscape's server software was being severely threatened by the Apache HTTP Server, which was not only free but gaining steadily in market share (and to this day is still the most popular Web server on the Internet by a wide margin, even beating Microsoft's own Internet Information Services). Finally, Microsoft introduced Windows 98 with an "Active Desktop" feature that integrated Web-browser functionality with the Windows desktop. Strangely, Netscape was actually more upset over what Microsoft did (and the loss of affiliate revenue it would have cost them) than the fact that their server business was dying, and decided to sue Microsoft in a landmark antitrust suit. But that wasn't the only strange thing Netscape did in 1998. Inspired by the success of Linux and the BSDs, and no doubt knowing that the company itself was in bad shape, Netscape decided to release the sources to the latest version of Navigator (which had been renamed "Communicator" by then) under a free license. Out of the meetings for this came a term called "open source", which attempted to provide the gist of the free software movement's goals without having to go into licensing details, making it easier for people not up on their legalese to understand what was going on. This was a controversial move, with GNU partisans saying it didn't go far enough, while others argued that the word "free" was too easily misunderstood and indicated that the software was worthless. Either way, the term stuck, and "open source" now indicates any software license that conforms to the Open Source Definition, a set of rules based on the Debian Free Software Guidelines (Debian being a highly influential and somewhat conservative distribution of Linux). As for Netscape, most of the company migrated to Mozilla Foundation (originally known as the Mozilla Organization), a non-profit founded to handle the development of a new browser based on the Communicator code. After a few false starts (including a decision to rewrite most of the client from scratch, a move that rankled some of the Netscape veterans), the new browser was in usable shape by 2003. A slimmed-down version without the built-in mail/news reader or the Web page editor was developed alongside it, and eventually became Mozilla Firefox, and a slimmed-down version consisting of just a mail/news reader eventually became Mozilla Thunderbird (a third-party fork of the HTML editor became Nvu, which was discontinued and forked itself as Kompo Zer). The full-blown browser itself was originally planned to be left for dead, but said group of Netscape veterans saved it by bringing it under their wing, rechristened it Seamonkey, and brought it back in shape. And yes, it has a cult following to this day, again mostly by Netscape veterans who prefer the idea of heaving everything under one roof instead of in separate applications.
UNIX todayNow, in 2013, UNIX and its clones and derivatives are more popular than ever. Apple decided to open-source the kernel and user tools for Darwin, the basis of Mac OS X, and even Solaris, one of the last bastions of old-school commercial UNIX, has since been opened up- just to close again after Sun was bought up by Oracle, which caused much of the community to fork the last open release into Illumos. Various PC-centric distributions of Linux are giving Microsoft a run for its money for the first time in years, especially on low-end PCs that can't run Vista or 7, and a resurgent Mac (running UNIX itself) is competing with Microsoft on the high end. The BSDs are popular in server, networking and security-related applications. Ubuntu, a distribution of Linux, is quite popular with computer-savvy people who appreciate its ease of use and its excellent hardware support, and is even starting to dent the average userdom, despite rankling some users with some recent unpopular moves. On the other hand, those users offended by Ubuntu's "overcommercialization" remained loyal to Linux, most moving on to Linux Mint which is an Ubuntu respin, or on to other distros at worst. There are also several forks of the official Linux kernel being used on high-end electronic devices and smartphones, as well as in networking gear like Ethernet switches and routers, and in many embedded devices. The rise of small, simple, low-cost, low-power "netbooks"—basic laptop computers for $300 or less geared towards Internet functionality at the expense of gaming and multimedia—has also worked in UNIX's favor. Many netbooks have discarded the traditional x86 architecture due to power consumption and cost concerns in favor of the Advanced RISC Machine (ARM) architecture, also used by the vast majority of embedded devices such as cell phones. The only current version of Microsoft Windows for ARM processors is the largely incompatible "Windows RT" (a modified version of Windows 8), and the vast majority of ARM netbooks run some version of UNIX, usually Linux. Chromebooks (netbooks that run Chrome OS, a specialized Gentoo Linux variant aimed at those who spend the majority of their time online) seem to have taken the lion's share of the netbook market as of late due to their low price points and excellent battery life. In addition to netbooks, other portable devices nowadays often contain some code from UNIX or UNIX-like operating systems. Apple's iOS, used on the iPhone, iPod Touch, and iPad, contains code from the Darwin base of Mac OS X; and iOS's main competitor, Google's Android operating system, contains a fork of the Linux kernel at its core. Also, Sony has confirmed that the PlayStation 4's OS is in fact a customized version of FreeBSD. The biggest bombshell of it all, however, is probably when Valve Software decided to create a spin of Debian in 2013, calling it SteamOS and announcing that the OS will be powering its line of Steam Machine consoles, but will also be free to download for anyone interested in building their own custom Steam Machine. This after porting Steam and several games to Linux, as well as encouraging developers who're using the Steam platform to roll out Linux support as well. The move is seen as encouraging, and many fans of Linux believe this to be finally signalling the rise of Linux as a serious competitor of Windows in the gaming arena.