Skip to end of metadata
Go to start of metadata

moved to comments below

While there are many different good OSes and distributions based on common illumos kernel and contributing to its development, the OpenIndiana project remains about the most generic distribution as a most-direct ideological descendant of the OpenSolaris project, and is dubbed the reference distribution. In fact, some OpenIndiana projects are being renamed into illumos namespace these days, while the illumos project will soon use the development, organizational and building processes developed for OpenIndiana. The two projects also share some infrastructure and some of the developers.

This page aims to outline the procedure (a rough outline here with links to detailed pages in the two projects' Wikis) for newcomers to prepare a build environment, get the sources and start building illumos kernel (OS/Net consolidation) and OpenIndiana userland programs, and, hopefully, ultimately fix some bugs, code new features and contribute to the public projects.

When the fixes get upstream from individual projects (including every user's personal tinkering), everybody wins!

Prepare the Build Environment

“If you want to make an apple pie from scratch, you must first create the universe.”
-- Dr. Carl Sagan (American Astronomer, Writer and Scientist, 1934-1996)

illumos is a self-hosting effort. In order to build it, you need to install an illumos-based OS, such as OpenIndiana.

  1. Go to http://openindiana.org/download/ and get the current distribution image of whatever type suits you best.
    The installation media is a LiveCD or a LiveUSB image of an installer (one installer with graphics desktop, another with text-mode and SSH only); there is also an Automated Installer for advanced (usually networked) setups.
    NOTE that you'll likely also need internet access to install further required packages and local zones.
  2. Install the distro to your building machine, see Installing OpenIndiana for a reference procedure if you're new to this.
    NOTE 1: There are many other ways to install (Open)Solaris-based OSes, if you're experienced and brave enough (see for example posts about Automated Installer and Networked Installation, Advanced Manual installation of OpenIndiana from LiveCD media, Experimental installation with split ROOT dataset hierarchy); however, simple standard installations work well enough and are much easier for your peers give advice on – should you need help in troubleshooting something.
    NOTE 2: If you decide to use a VM environment for builds and/or tests, beware that VirtualBox has proven too slow for many developers (as of those versions that they have tested), and illumos builds took days instead of hours. It may work for you – please report if so.
  3. (optional) If you have free internet traffic, and plan to do many experiments, you might want to set up a mirror of OS package repositories, so that your further installations (including installs of local zones) would be local to your LAN networking.
    This optional step is described in Mirroring OpenIndiana (page is being rewritten as of April 2012) and somewhat in Automated Installer and Networked Installation, and would require considerable disk space and download traffic (about 3Gb for "current" OpenIndiana packages, and 55Gb for optional "legacy" packages; few of the latter may be needed – if at all).
    After some successful builds, you'll likely be running repositories of your own compiled software – that's what a successful make ends with.
  4. When your installed host OS is up and running, it is advised to create a "local zone" dedicated for builds, so as not to pollute your host's main operating environment ("global zone"). The procedure to prepare a build zone is outlined here: Building in zones.

    Local zones are a lightweight virtualization technology, one of many goodies introduced by Sun Solaris 10 and OpenSolaris – a separate operating environment which runs within your host OS and has a separate set of installed software, so that the builds and subsequent tests have little impact on your main host OS, or on other local zones. Using ZFS, it is easy to snapshot the zone's filesystems, and roll back in case of major mishaps.
    Being an operating environment, a local zone may have a networking setup of its own – but a dedicated IP address number from your LAN is not required. The local zone may be NATed by its global zone using etherstubs and other virtual networking components, or it may have no networking at all – but repository cloning and other networked operations would have to be run from the global zone then.


Get the Sources and Build Stuff

"make world" or "emerge world"?
-- an innocent question like that can start a holy war elsewhere...

Source codes of illumos and OpenIndiana, as well as some other related projects, are published as "repositories" with Git (git) and Mercurial (hg) source-code management systems. Public copies of these repositories are hosted on a number of servers, including hg.openindiana.org, BitBucket and GitHub; their contents are maintained by corresponding project admins with commit privileges.

Developers working with the projects' source codes clone the projects' public repositories, then they create "workspaces" as instances of their private repository, and compile the code in their workspace.
Also, when the public project moves ahead, the developer's local repository and workspace needs to be updated (or even merged) with the public changes.
People start by compiling a copy of the current public sources, but when they ultimately change some code and fix something, they would report and contribute these changes as described in the next sections.

  1. If you'd like to play with the illumos kernel illumos-gate, see How To Build illumos and How To Build Components. These pages detail the setup of ZFS datasets, compilers and environment scripts needed to build the kernel, as well as running the build procedure and installing the built illumos core packages as a new boot environment on your build host or another test system. The main page also describes how to configure and run the package repository server to install your newly built kernel onto a different machine.
    NOTE: The page as of now documents building of illumos in the global zone (host OS); it may soon be tested for procedural compatibility with local zones and updated accordingly.
  2. If you'd like to play with the OpenIndiana userland programs illumos-userland (mostly – third-party software integrated with illumos/OpenIndiana and not part of the OS core), see Building with illumos-userland for instructions of compiler setup, getting the sources and running the build procedure and installing the results to test. Its several child pages detail the process; in particular, see Contribution Process for more details on forking your own source-code repository. See also the README file and doc directory in the checked-out source code repository.
  3. There is also an OI-SFE ("Spec Files Extra") project for more open-source software to get packaged under Solaris and derived operating systems, including an extra repository of some software known to be patent-encumbered and of limited legal usability in certain countries:
    https://www.illumos.org/projects/oi-sfe
    http://wiki.openindiana.org/oi/Spec+Files+Extra+Repository
    http://wiki.openindiana.org/oi/Building+the+OI-SFE+IPS+Repository
    http://pkgbuild.sourceforge.net/spec-files-extra/
  4. Your building adventures would usually create (or update) your IPS repository of packages with pieces of software. If you want to make a unitary distribution from that, there is an utility called the Distribution Constructor (it takes care of making the ISO and USB images).
    NOTE: the Distro Constructor is known to work only from the global zone.

Report bugs

While using the OS you can stumble upon things that are inconvenient, missing or even outright don't work in your software and hardware environment. Even the small things matter, such as missing entries in man-pages, uninformative error messages returned by programs, and such.

Even if you intend to fix that problem yourself, the proper procedure starts with checking if the problem is not already known, and making sure the problem gets known. You can do that via the bug tracker (needed for contributions down the road) and/or via mailing lists (for open discussion of options, as well as to find out if somebody works on that/related/similar problem already, or can do that quest faster/better/more eagerly than you); probably both paths should be followed.

If you propose a new feature, you can post an RFE (Request For Enhancement or Feature) in the illumos bugtracker; if you think something is broken – you should post a Bug.
Note that some bugs can get rejected by the bugtracker maintainers – either because they duplicate another bug, or because the work described is deemed to be out of scope for the project (near-term or at all), and existing developers won't commit to fixing it. In the latter case, if the feature is still important for you and you would fix it yourself (or find someone who would), you're welcome to revisit the closed bug and assign it to yourself, ultimately integrating the fix as discussed below.

There are many categories (see the list of sub-projects at https://www.illumos.org/projects/ before posting), including these for illumos kernel and userland, OpenIndiana distribution, and integration of third-party software into common repositories or the distribution itself:

It may also be correct to bring attention to your proposals in corresponding Mailing Lists (see above).

There are some recommended procedures to gather information on more serious Bugs which cause program or OS crashes (kernel or program core files and an overview analysis of them) – which can greatly help developers figure out what went wrong:

Contribute

"LGTM" or "+1"
-- a good response to good submissions

When you're comfortable with the building environment and it works for you, you may feel the urge to fix something in the OS, or add to it. When that works, you may want (and are encouraged) to contribute your changes to parent projects.

The process is described in more detail in pages like these:

The general procedure includes:

  1. marking the problem and your involvement in it via the bugtracker for a certain project, so that the problem you are fixing has a name and an issue number (check if a matching issue already exists and is perhaps being worked on by somebody else).
    If you do intend to solve it yourself – assign the bug to yourself; in part, this helps avoid situations where bugs important to you are discarded (closed) by bugtracker admins as being "too vague" or "out of scope" for existing core developers;
  2. cloning the source code repository;
  3. developing a solution;
  4. testing that the solution works and a full compilation of the repository is "clean" (no warnings and complaints from involved compilers and other utilities);
  5. committing the changes to your repository and including the bugtracker issue number and name in the commit comment;
  6. preparing a patch report for public review (for illumos-gate that is automated by the webrev program; for BitBucket projects like illumos-userland there is a button to calculate differences between your and original repositories);
  7. notify the corresponding mailing list about your fix and have some reviewers look at your changes, perhaps repairing the drawbacks they find;
  8. when your fixes are "LGTM"ed (looks good to me) or "+1"ed by several reviewers, request to integrate (RTI) your changes into the common public repositories via the mailing lists.

Thanks, now you are officially one of our heroes, and your name is recorded for posterity! Keep it going!

Labels:
  1. May 10, 2012

    Deirdre: This page needs editing to separate the two.

    Jim: Alternatively, there may (should?) appear pages on building and contributing to illumos OS/Net with other distributions based on it?
    OpenIndiana is grouped with illumos here because by far it is the most common and generic (and well-documented) distro which may be easiest for developers to install. It also has the goal of being the upgrade path for old-time Solaris and OpenSolaris shops looking not to be locked into Oracle vendorship and have a free Solaris-based OS. Other distros are often too specific for a job or packaging method, and may not gain much deserved attention just because of that niche they chose to excel at. They are encouraged to add their description pages to this wiki, as well as clone this page for their development and packaging routines. I think that would produce the most coherent documentation for developers liking one or another distro to get into a working setup from scratch.

    Deirdré: There can and should be such pages for all, either here or on their own sites. But we should not confuse newcomers by playing favorites here. OI is not the only answer to any of the questions, and it is probable that none of us knows how many of our particular distro is being used (or for what). Nor even how to count. On sheer number of CPUs, I'm pretty sure SmartOS wins. (wink) 

    1. May 10, 2012

      Well, to make it clearer - I am not an advocate for any particular distro, except that OpenIndiana seems like the upgrade path I'd take sooner or later in the deployments I maintain (some Solaris 8-10 and mostly OpenSolaris SXCE – no IPS-based ones yet, all SVR4, including some of our in-house packaged software for ourselves and customers).
      It may be true however, that since our current servers are all based on generic distros used for widely different tasks, the overall deployment might benefit from using a common kernel with similar maintenance/admin routines but using different distros optimized for one or another task (i.e. storage, virtualization, interactive GUI and development, etc.)
      Or, maybe, this would increase the maintenance chores beyond reasonable threshold, and using subsets of packages from a sub-optimal generic distro "that has it all" – basically like parts of the huge (Open)Solaris distros are used now on each machine depending on its tasks – would be beneficial instead.

      Still, in terms of gaining contributors (or even just users willing to find and report bugs) the main metric should involve the number of users (admins, developers, maybe companies they work for). It is the sheer amount of brainpower and available hands-on-deck that would move the projects forward (wink)

      The "sheer amount of CPUs" and other environmental aspects does expose the OS to more field-testing with different stresses, hardwares, firmwares and softwares, which also contributes – but more in terms of improving stability and expanding the (common illumos kernel's) HCL.

      ----

      Regarding THIS page, I think we can go both ways: carve out or recreate the pieces pertaining to only an overview of building illumos, and make that a separate page – in particular, other distros can build their similar self-hosted build routines on clones of that page. For example, I am not sure whether all distros have the local zones at all, and if those pieces of advice pertain to them. One such cloned page would be the current text for illumos+OI (like reversing the origin in ZFS clones (wink))...

      I still do think that an overview page like this one should be relatively compact (i.e. in comparison with How To Build illumos alone) but comprehensive as a single-stop resource to turn an enthusiastic newbie with bare hardware (or a VM) and a distro image into a ready-to-go developer/contributor (of a particular distro).

      //Jim