Skip to end of metadata
Go to start of metadata

This is largely based on the work Rich Lowe created as seen at http://hub.opensolaris.org/bin/view/Community+Group+tools/hg_workflow

Please Note: Though this workflow is still valid, the canonical illumos-gate repository is now using git rather than Mercurial, and is hosted as "illumos/illumos-gate" on github.

Populate your workspace

Setup Mercurial

If this is your first time working on the Illumos code, run /opt/onbld/bin/hgsetup to create a suitable ~/.hgrc file in your home directory. This will (among other things) enable the Cadmium extension for Mercurial. Cadmium contains a number of commands that are useful while developing Illumos.

Build

Follow the instructions at How To Build Illumos

Making changes

Make your changes as you normally would. There is no explicit "edit" step required by Mercurial.

Mercurial does not lock files in your workspace for editing, and Cadmium (a Mercurial extension providing wx-like functionality, useful for Illumos development) does not make use of a statically maintained list of changed files (as wx(1) would). You can make changes freely in your workspace without any required bookkeeping.

The hg status command will tell you what you have changed in your repository.

Committing your changes

You may commit changes as and when you choose, a commit is local to a repository until you choose to push it to others.

  • For small fixes you may choose to only commit when you're done with your work, or perhaps prior to a merge before you submit the change for integration.
  • For larger workspaces it's generally best to commit at appropriate intervals in your work.

Cadmium's hg backup command (described below) is not really intended to return you to a point-in-time snapshot of your ongoing work, but to recover a lost (or destroyed) workspace. It is recommended that you move around within your workspace's history using the hg update command to move to prior revisions as necessary.

Check for nits

As with wx(1), Cadmium provides a command that will run a sub-set of the pre-putback checks on your workspace. You may find it useful to run hg nits (cf. wx nits) at intervals to keep your changes generally tidy as you work.

This will check CDDL blocks, copyrights, C style, header format, Java style, permissions, and (absence of) keywords.

Backup your work

You'll want to backup your work at regular intervals, in case of disaster (disk failure, plagues of beetles, possession by the demons of stupidity, and such).

The hg backup command will save the changes (both committed and uncommitted) in the working copy of <your-repo> to a subdirectory named <your-repo> in the ~/cdm.backup directory within your home directory, by default.

As stated above (see "Commit your changes"), these commands do not cross repository boundaries, so you must take separate backups for usr/closed, if appropriate.

hg backup takes a -t flag which, like wx, will only take a backup if changes have been made since the most recent. Backups are ordered by generation number (a monotonically increasing digit).

To restore your workspace, clone the parent and restore from your backup:

hg restore takes a -g flag with which you can specify the generation from which you want to restore.

Merging with the parent

Mercurial merges the lines of history, rather than only file contents. Therefore, you will need to perform merges even if there are no conflicting file changes (though the merges will be simple then, of course).

This will no doubt come as a shock, initially.

By default, if your .hgrc was configured with hgsetup(1), we will default to use TeamWare's filemerge(1) for merges, if TeamWare is in your $PATH, and meld or gpyfm if it is not, and you have one of them.

Merging with committed changes

If your workspace has committed changes, and you pull from your parent workspace, you will likely find that you now have two "heads" (revisions with no children) in your workspace, and that you received output from Mercurial suggesting that you may wish to merge them with hg merge. You almost certainly do. Example:

When you run the hg merge command, Mercurial will merge the two heads, starting your merge application as necessary to resolve content changes in individual files, and asking you questions as necessary to resolve name changes. When complete, your working copy will represent the still uncommitted merge.

You should commit your merge soon after you are finished performing it, perhaps after some verification that you merged correctly.

Merging with uncommitted changes

If you have uncommitted changes in your workspace, you will also need to merge when you use "hg pull -u" or "hg pull && hg update" to update your workspace.

If the pull added two heads, it means you have both committed and uncommitted changes in your repository. In this case, your changes will be as they were, and you may continue working in that state until you see fit to commit, and then hg merge to merge with the extra head.

If you have only uncommitted changes in your repository, then there is no need to merge the history of your workspace and its parent. In this case, hg update (or the -u part of "hg pull -u") will behave similarly to the resolution of a regular merge; your merge application will be started, or you might be asked questions about name conflicts.

Best practices

Don't make other changes along with a merge

It is to your advantage for a merge changeset to only represent a merge, rather than a merge and some other changes you happened to be making at the time. This keeps a clear distinction between each change and makes it easy to backout a specific change if required.

Always commit your work before you merge

As we've seen above, if updating your workspace from its parent would cause their histories to diverge, then you will need to merge your changes with the parent. You should commit your changes to your workspace before doing the merge, such that your merge changeset is entirely separate from any ongoing work in your workspace.

Recommitting your work into a single changeset

You may not integrate your changes into the gate with merged changesets, uninteresting interim changes, etc.

In most cases, you're going to need to make these go away before you push your changes out.

The command to do this is hg recommit (much like wx redelget, etc.)

hg recommit will take your changes to date, and create a single changeset containing those changes.

You may specify a checkin comment on the command line (-m), a file containing your checkin comments (-l), or, if neither are specified, hg recommit will pop up your $EDITOR with a buffer containing all the checkin comments you have used thus far, for you to edit into a form suitable for integration.

The comment format used with Mercurial is a bug ID, followed by its synopsis.

XXX Example (of all 3)

hg recommit will perform certain sanity checks as it runs.

Recommit will fail if you have uncommitted changes in your workspace.
You must commit them, when appropriate, then recommit.

Recommit will fail if your workspace has more than one head, or branch.
You must merge your changes such that your workspace contains only a single head (hg heads), and then recommit. Recommit will also offer to take a backup if there have been changes since your last backup, before actually recommitting your changes. For the sake of all that is good and sane, say "yes."

Best Practices

In general, you should recommit your work as late as possible (though prior to requesting an integration), this way your full local history is available to you, and backed up, should you need to refer to it at a later point.

Even though Cadmium will ask whether you wish to backup before recommitting a workspace, you should always answer Yes.

Run the pre-putback checks on your workspace

Cadmium's hg pbchk command will run a full set of pre-putback checks on your workspace (cf. wx pbchk).

This includes checking CDDL blocks, checkin comments, copyrights, C, Header and Java style checks, checks for executable files which you intend to integrate, that you have not added any tags, that you have not added any branches, nor changed the name of the main branch, that your work does not include SCCS ID keywords, and that your RTI is approved.

In most cases, each and every one of these checks should pass. You should already know if you will be exempt from one or more of these rules.

If passed the Hg global option -qpbchk and nits will be silent about passing checks, only providing output for checks that fail.

Integrating your changes (putback, push)

XXX: fill me in with current Illumos process

Running a project gate

Most of how you choose to run your project gates is entirely your own decision, these are merely recommendations, though it is probably worth your time to read them anyway.

Of course, when your project integrates you will have to follow most of the steps outlined in the previous section too.

Syncing up with your parent gate

At intervals, you're going to want to sync your project gate with its parent (illumos-gate, for instance). You probably want to do this on a regular schedule and at suitable points, what this schedule or these points are is entirely up to you, though we would advice you not merge constantly, both for the sake of providing you a stable basis for work, and to avoid filling your project gate history with minor merges. Merging as builds close may work fine in many cases, others may have different preferences.

Syncing with the parent is different in the case of the project gate, as you wish to preserve history of the merge, among other things.

Clone a fresh child of your project gate

You should never do real work directly within your project gate, do the work in a child and push the changes to your gate when you are finished, as normal.

Pull from your project gate's parent at the point you wish to sync up with:

Merge the head from your project with the head from the parent

The above did not specify the -u flag, so the working copy remained at the head of your project, this way your project is the first side of the merge, and the parent the second.

Commit your merge

Commit the merge, using some sensible message of your choosing, maybe something like "Sync with onnv_97".

Building the merge

If you intend to run the merge through a full build to verify it, you'd be well served to build in children of the merge workspace, leaving the workspace in which you performed the merge free of detritus.

Make sure everything is as you expect

The first few times through this, you may wish to use Cadmium's hg list and hg pdiffs with the -p option to specify which parent to use, to make sure that the changes relative to your project gate and/or its parent are as you expect.

Push the merge to your project gate

Push the changes out to your project gate.

DO NOT recommit the merges used to sync your project gate with its parent. There are multiple reasons for this, discussed below.

 

  • They represent valuable history to you and the rest of your project,… so you don't want them to go away.
  • There is no good way to achieve this. If you recommit relative to your project gate, you will not only remove the merge changeset you created, but also the changesets from your project gate's parent since, relative to your project gate, those changes are local to your merge workspace.

So when you do this, you're throwing out a lot of history from the parent of your project gate.
This is not fatal, in that when you next sync up, you'll pull all that history again, then merge it again, but you're making work for yourself as well as throwing away information.
Eventually, when you plan to integrate, you would have to pull all this history (throughout the life of your project), and recommit again against the gate to which you intend your project to integrate.

Don't do it.

You may notice that the above may take some time, during which others may be changing your project gate, and causing you to possibly have to merge with them.

In our experience this has been fairly rare, and how it should be dealt with is a project specific choice.

Never recommit within your project gate

You should never, ever, recommit the contents of your project within the project gate.

This will considerably disrupt each and every person who has a child of your project gate (your co-workers), throw away valuable project history, and serve no practical purpose.

Don't do it, ever.

Backing out undesirable changes

Mercurial's hg backout command is the best way to achieve this.

It will create an inverse changeset on top of the change to be backed out, which you may then need to merge - if the changeset to be backed out is not the tip (using the -m flag to backout will start the merge for you).

This will want two commit messages, once for the backout changeset itself, and once for the merge (if necessary).

As with syncing up with your gate's parent, you should perform backouts in a child of your project gate, and then push them to the project gate itself.

If your backout required a merge, you should probably recommit it, as its function is plain enough, and the extra merge in the history is not useful to you.

Cherry-picking changes from your project gate's parent

From time to time, you may wish to have changes from the parent in your project gate where a merge would be inconvenient, or outside of the schedule you're using for your merges.

You have several options, which of them is correct is largely up to you:

  • You could merge out of schedule, which is probably better if what you want is large, or depends on other changes.
  • You can hg export that revision from the parent, and import that diff into a child of your project gate, test, and the push to your project gate.
    The latter can be somewhat automated using Mercurial's transplant extension, which is not documented here.
  • If you choose to do a full merge, it should be performed as all other merges, and as documented above.

Integration

Integrating your project is much like integrating any other change you have made, however there are a few other things which you should do.

Do the integration from a child of your project gate:

  • Take a fresh child of your project gate, run hg pbchk within that child. If file changes are necessary, integrate those changes into your project gate, and update the child you intend to integrate from.
  • Once pbchk is clean, beyond checks that depend upon your checkin comments, recommit within the child of your project gate.
  • Perform any test builds within children of this child, keep the child you intend to integrate from as pristine as possible.
  • Integrate from this child.

This leaves your actual project gate untouched, so its history can be kept around as long as you think may be valuable.

Where to get help

  • Mercurial's built-in help is available via hg help <command> or hg help <extension> (eg. hg help loghg help cdm).
  • Bryan O'Sullivan (a Mercurial contributor), has written a book "Distributed revision control with Mercurial" which is freely available from: http://hgbook.red-bean.com/
    You should bookmark it, and perhaps print out the pdf and keep it on your desk for a while.
Labels: