Tales from the Gryphon

Or theming PS1

I am not fanatical about my command prompt. No sirree, not me. It is just that I spend half my life either staring at an Emacs window, or at an xterm command prompt; so even a marginal boost in productivity goes a long way. And I am often logged in over ssh to machines half a continent away, and am still comfortable enough on the Linux VT to spend time there, and I often do not have the GUI gew-gaws feeding me data. Hmm. Data. inpuuut. No. Must focus.

With all these open xterms and Emacs terminal mode frames floating around, it is easy to lose track of where I am on each terminal, and what the working directory is. So I want my command prompt to help me keep track of where I am. If the terminal is an xterm, the title can be setup like

 "user@host:../shotened/path/to/current/working/dir"

I want to know what machine, id (am I root?), and directory I am in. If I am deep down in the labyrinths of some work related directory tree, I want the path to be pruned, from the left, one component at a time. However, this does not help me on the console; so I also want the path to be in the command prompt; but it should not take up too much of the command line; and ideally, should just go away as I type a longish command. Gawd, I love zsh.

I once did a analysis of the command history. The most often used command sequences were

 cd some-place, ls

So. pwd and ls. I really really want to see the directory listing when I change directory to a new one. As one grows old, memorizing the directory contents for dozens of machines all the time taxes the gray cells a bit. So, I figure, why not let the command prompt handle all that? Having the current working dir always visible cuts down a heck of a lot on the pwd commands, and so all that’s left is to insist that the command prompt thingy always run ls after a change directory. Simple enough. Saves on typing. And time. And this is not just some crazy talk.

I want help with noticing whether the previous command exited with an error status (useful for commands that normally do not create an output).

If I am logged in to a machine on battery power, I want to know that. I also like visual cues to the amount of power remaining (good for my laptop on the long flights). I don’t want to have to know f the machine uses APM or ACPI, I just want my prompt color to change as the power fades.

I want my command prompt to let me know if I am in a directory which is under version control, and if so, what branch I am on. (I occasionally have to come in contact with arch, bzr, git, subversion, svk, and mercurial). If I am in a version controlled project, where I am relative to the root of the checked out tree is often more important than the absolute path, so I want to see relative paths, not absolute paths.

I want to know if there are uncommitted files in the working directory. Visually. I want to be reminded if I am in the middle of an ongoing

 rebase -i .

This is not asking for too much, is it?

So, here is a screen-shot of this in action: I start from my home directory, go to a directory not under version control, go to a project under git, then a different project with uncommitted files, and then finally to a subversion checkout. All with an angry fruit salad of colors warrantied to make Martin Krafft want to claw his eyes out . If you use zsh, then just grab hold of this, and then do:

 autoload -U promptinit
 promptinit
 prompt manoj

This might be a bit of a hit on slow machines, but even my laptop is a core 2 duo, so I do not find it noticeable.

Oh, and if you liked this article, you might also like Theming Emacs, and Theming XTerms.

Or configuring LS COLORS

After successfully Theming Emacs, I decoded to turn my efforts to providing more copacetic colors in other aspects of my working environment. I’ll write more about my effort to write my own color theme for fvwm when it is closer to being done. For now, I’ll concentrate on “ANSI Colors” in XTerms, and how one may theme those. The old Tektronics (or ANSI) terminals had 8 colors, namely, black, red, green, yellow, blue, magenta, cyan, and white. These colors had a normal, and a bright mode. The default set of colors is OK, but is not optimal, since they have to work on both dark and light backgrounds; however, I use mostly dark background colors.

Given that, I set to design my own colors, using fully laturated colors, with a base luminosity of 70, and equally spaced in chroma — and used the same saturation and chroma, but a luminosity of 80, for the bright set. This results in a pleasant pastely set of colors, which are e4asy on the eyes, when used on very dark backgrounds. The configuration can be seen in XTerm. I set the colors for rxvt as well as xterm. Now, the most noticeable effect of these changes is if using the color option for ls, therefore the next thing to do was to configure the colors used by ls using a utility dedicated for that task, namely, dircolors. By trial and error, I modified the default configuration provided by dircolors and ended up with dir colors, with the results as you see.

Using org mode to write ikiwiki posts

I have been much taken by org-mode in Emacs recently. Firstly, this has allowed me to actually effectively implement the tenets of “Getting things done”, which is amazing by itself. I always used to start with great enthusiasm, which rarely lasted longer than a week. Until org-mode. Then this is so amazingly flexible. It has better handling of tables and even a spreadsheet mode than anything I have used — and it has a pretty decent HTML export function. I practically live in org-mode on work days now. I strongly recommend org-mode, from things like agendas, TODO lists, calendars, appointments, articles, and now Ikiwiki posts.

I rarely ever use markdown, apart from blogging using Ikiwiki – and thus, since I blog so infrequently, I rarely ever remember all the syntax. On the other hand, org-mode syntax is now at my finger tips — so I decided it was time to write a Ikiwiki plugin so I can now write my blog posts using syntax I can actually remember. It is still rough around the edges – firstly, this requires emacs23 (or, on Debian, there is the unofficial emacs-snapshot package), secondly, it uses hueristic hacks to prune away the bits of the stand alone HTML page that org-mode creates; all I can say is that it currently works for me, as you can see from this post. How git this is for mainlne inclusion is another matter.

The code is here: org.pm. It can be dropped in to the Ikiwiki plug-in directory, and that’s all you need. With this, Ikiwiki pre-process things in the input file, and then passes it to emacs to htmlize it.

---

Update Added screenshots

---

I spend a lot of time working in front of a screen (many hours in a dimly lit room) and eye fatigue is an issue. A consistent color scheme, especially one which is easy on the eyes, is important — and it also helps to have way of doing that where the directives are not all scattered all over my Emacs-lisp setup. Enter Emacs Color Themes, available as the Debian package emacs-goodies. Aided with a HSL color picker, I set around trying to create a dark color theme for Emacs.

It does not help that I am blue-green color blind, so subtle variations are often lost on me. I do want to use color contrast to increase productivity, but I also want to avoid the jarring angry fruit salad look, and so I am in the process of crafting a logical color scheme that is high contrast enough for me, without being too unpleasing.

In circumstances where there a lot of related faces that can be viewed, for example, the Gnus group buffer, consistent and logical color choices are the only sane option. Gnus groups can be news (bluish) or mail (greenish), have states (large number of unread messages, normal, and empty). The large number unread groups have highest luminosity (appear brighter), and the empty one have lower luminosity (appear greyer), but have the same hue and saturation. Sub states and group priorities are rendered using a color series which has constant luminosity and saturation, and vary in hue by a constant separation — so all the related groups have the same brightness ({mail,news}/{unread,normal,empty}), and a graded selection of foreground colors. It sounds more complicated that it looks. The eye is drawn naturally to the unread groups, and first to the mail, then USENET groups (which is my preference).

Similar color variations occur for individual messages in a group; high scoring messages bubble to the top, and have a higher luminosity. This color schema has made me slightly faster at reading mail/USENET. In the message itself, quoted mail messages from different people are color coordinated, with high contrast between citations that are close to each other in the hierarchy, so it is less likely that one misunderstands who said what in a long conversation.

The resulting scheme covers programming languages, Gnus, Erc, mail, org-mode, CUA-mode, apt-utils, bbdb, compilation buffers, changelog mode, diff and ediff, eshell, and more. This has allowed me to consolidate all my color directives into a single file, and is in a format that might be usable by others. See the wiki page for details about how to use and switch color themes in Emacs.

Enjoy.

“Are you rebasing or merging?” seems to be the 64 thousand dollar question over in vcs-pkg discussions. Various people have offered their preferences, and indeed, several case studies of work flows have been presented, what is lacking is an analysis of the work-flow; an exploration of which methodology has advantages, and whether there are scenarios in which the other work flow would have been better.

Oh, what are all these work flows about, you ask? Most of the issues with packaging software for distributions have a few things in common: there is a mainline or upstream source of development. There are zero or more independent lines of development or ongoing bug fixes that are to be managed. And then there is the tree from which the distribution package is to be built. All this talk about packaging software work flows is how to best manage asynchronous development upstream and in the independent lines of development, and how to create a coherent, debuggable, integrated tree from which to build the distributions package.

The rebasing question goes to the heart of how to handle the independent lines of development using git; since these lines of development are based off the main line of development, and must be periodically synchronized. Here is a first look at a couple of important factors that will have bearing on that question, and packaging software for a distribution using Git in general. This is heavily geared towards git (nothing else does rebases so easily, I think), but some of the concepts should be generic. I am not considering the stacked set of quilt patches source controlled with Git in this article (I don’t understand that model well enough to do an analysis)

As a teaser, there is a third answer: neither. You can just add an independent line of development, and just let it sit: don’t rebase, and don’t merge; and in some circumstances that is a winning strategy.

I have been using Arch to package my Debian packages since 2003; which means that Arch has had a good long run as my SCM of choice. I have been using CVS for a few years before I moved to arch, and the migration took me about six months, since it involved a while new philosophy of packaging; I am hoping that migrating to git would not involve such a major paradigm shift, and thus be less disruptive and time consuming. What follows is a narrative of my efforts to get educated about Git.

This article is meant to be an annotated, selective, organized set of links to information about Git. How does it differ from the myriad of other link collections about Git proliferating on the web? Well, the value add is in the annotations and the organization: while not quite a narrative of my exploration, this is an idealized version of what I think my discovery process should have been, to be most effective. Staging the information is important; google finds one lots of information that is incomprehensible to someone just coming to Git. This selection of links is actually selective; I have included only pointers to resources that fed me information at the level that I could handle at that stage, and I have eliminated links to information that was not new at that point. I have tried to select the best (in terms of information and clarity) of breed for each kind of information source I have come across so far.

There is a caveat: while still a beginner, though I am able to better judge now what is confusing to a beginner than I shall be when I have become more familiar with the system, I am still enough of a novice not to trust my judgement on what really is best practice. I can fix the latter as I gain experience, but then I’ll need to be careful not to overload on complexity too early in the learning curve.

On the down side, this selection is subjective, and probably shall be even in the long term: I include what appealed to me, and will probably miss loads of pointers to information that I have not yet come across. However, I hope this will make it easier for other people to reach the same goal: use git for their version control needs.

Have fun.

One of the things I have been tasked to do in my current assignment is to create a dashboard of the status of various software components created by different contractors (participating companies) in the program. These software components are built by different development groups, utilizing unlike toolsets, languages and tools — though I was able to get an agreement on the VCS (subversion — yuck). Specifically, one should be able to tell which components pass pre-build checks, compile, can be installed, and pass unit and functional tests. There should be nightly builds, as well as builds whenever someone checks in code on the “release” branches. And, of course, the dashboard should be HTTP accessible, and be bright and, of course, shiny.

My requirements were that since the whole project is not Java, there should be no dependencies on maven or ant or eclipse projects (or make, for that matter); that it should be able to do builds on multiple machines (license constraints restrict some software to Solaris or Windows), not suck up too much time from my real job (this is a service, if it is working well, you get no credit, if it fails, you are on the hot seat). And it should be something I can easily debug, so no esoteric languages (APL, haskell — and Python :P)

So, using continuous integration as a google search term, I found the comparison matrix at Damage Control

I looked at anthill, and cruisecontrol, and the major drawback people seemed to think it had was that configuration was done by editing an XML file, as opposed to a (by some accounts, buggy) UI is not much of a factor for me. (See this, and also this ). I also like the fact that it seems easier to plug in other components. I am uncomfortable with free software that has a “commercial” sibling; we have been burned once by UML software with those characteristics.

Cruisecontrol, Damagecontrol, Tinderbox1 & tinderbox2, Continuum, and Sin match. I tried to see the demo versions; Sin’s link led me to a site selling Myrtle Beach condo’s, never a good sign. Continuum and Damagecontrol were currently down, so I could not do an evaluation. So, here are the ones I could get to with working demo pages: http://cclive.thoughtworks.com/ and http://tinderbox.mozilla.org/showbuilds.cgi?tree=SeaMonkey

Cruisecontrol takes full control, checking things out of source control; and running the tests; which implies that all the software does build and run on the same machine — this is not the case for me. Also, CC needs to publish the results/logs in XML; which seems to be a good fit for the java world; but might be a constraint for my use case.

I like the tinderbox dashboard better, based on the information presented; but that is not a major issue. It also might be better suited for a distributed, open source development model; cruisecontrol seems slightly more centralized — more on this below. cruisecontrol is certainly more mature; and we have some experience with it. Tinderbox has a client/server model, and communicates via EMAIL to a number of machines where the actual build/testing is done. This seems good.

Then there is flamebox — nice dashboard, derivative of tinderbox2; and seems pretty simple (perhaps too simple); and easily modifiable.

However, none of these seemed right. There was too much of an assumption of a build and test model — and few of them seemed to be a good fit for a distributed, Grid-based software development; so I continued looking.

.

I finally decided CABIE

Continuous Automated Build and Integration Environment. Cabie is a multi-platform, multi-cm client/server based application providing both command line and web-based access to real time build monitoring and execution information. Cabie builds jobs based upon configuration information stored in MySQL and will support virtually any build that can be called from the command line. Cabie provides a centralized collection point for all builds providing web based dynamic access, the collector is SQL based and provides information for all projects under Cabie’s control. Cabie can be integrated with bug tracking systems and test systems with some effort depending on the complexity of those systems. With the idea in mind that most companies create build systems from the ground up, Cabie was designed to not have to re-write scripted builds but instead to integrate existing build scripts into a smart collector. Cabie provides rapid email notification and RSS integration to quickly handle build issues. Cabie provides the ability to run builds in parallel, series, to poll jobs or to allow the use of scripted nightly builds. Cabie is perfect for agile development in an environment that requires multiple languages and tools. Cabie supports Perforce, Subversion and CVS. The use of a backend broker allows anyone with perl skills to write support for additional CM systems.

The nice people at Yo Linux have provided a Tutorial for the process. I did have to make some changes to get things working (mostly in line with the changes recommended in the tutorial, but not exactly the same. I have sent the patches upstream, but upstream is not sure how much of it they can use, since there has been major progress since the last release.

The upstream is nice and responsive, and have added support in unreleased versions for using virtual machines to run the builds in (they use that to do the solaris/windows builds), improved the web interface using (shudder) PHP, and and all kinds of neat stuff.

One of the reasons I have only blogged 21 times in thirty months is because of the very kludgey work flow I had for blogging; I had to manually create the file, and then scp by hand, and ensure that any ancillary files were in place on the remote machine that serves up my blogs.

After moving to ikiwiki, and thus arch, there would be even more overhead, were it not so amenable to scripting. Since this is arch, and therefore creating branches and merging is both easy and natural, I have two sets of branches — one set related to the templates and actual blog content I server on my local, development box, and a parallel set of branches that I publish. The devel branches are used by ikiwiki on my local box; the remote ikiwiki uses the publish branch. So I can make changes to my hearts content on the devel branch, and the merge into my publish branch. When I commit the publish branches, the hook function ensure that there is a fresh checkout of the publish branch on the remote server, and that ikiwiki is run to regenerate web pages to reflect the new commit.

The hook functions are nice, but not quite enough to make blogging as effortless as it could be. With the movge to ikiwiki, and dissociation of classification and tagging from the file system layout, I have followed the lead of Roland Mas and organized my blog layout by date; posts are put in blog/$year/$month/$escaped_title. The directory hierarchy might not exist for a new year or month. A blog posting may also show in in two different archive indices: the annual archive index for the year, and a monthly index page created for every month I blog in. However, at the time of writing, there is no annual index for the next year (2007), or the next month (January 2007). These have to be created as required.

All this would get quite tedious, and indeed, would frequently remain undone — were it not for automation. To make my life easier, I have blogit!, which takes care of the niggling details. When called with the title of the prospective post; this script figures out the date, ensures that the blog directory structure exists, creating path components and adding them to the repository as required, creates a blog entry template, adds the blog entry to the repository, creates the annual or the monthly archive index and adds those to the repository as needed, and finally, calls emacs on the blog posting file. whew.

I have long been interested in moving the debian-policy package away from CVS, but had never quite managed to gather enough motivation to do the switch. Debian policy has long had an Alioth project, but I finally managed to file a support ticket, and nag Wichert into creating the arch project for policy.

The first step was to convert the CVS version into an arch repository, and this is where cscvs comes in. cscvs is nice, but I am told that Canonical has a better, private version, and helpful folks there offered to do a managed conversion to arch for me using these cutting edge tools. I declined, being persnickety enough to want to convert Debian technical policy using tools in Debian itself. And, apart from two change-sets (numbers 117 and 125), cscvs managed to do the conversion to arch (the bazaar flavour) nicely on its own (well, after a few false starts as I climbed the learning curve). Came to 283 change-sets. Have a look at my Debian policy branch. It can be registered at http://arch.debian.org/arch/private/srivasta/archive-etch/.

The next step was to create a baz archive on arch.debian.org, and use Clint’s ACL recipe to allow people in the dbnpolicy group to have write access. And then, since I wanted this branch to also have the full set of distinct patch logs, I cycled through all 286 patches in my local branch, replayed and committed them into the remote branch one by one. You may browse the public, or release, branch as well. This version can be registered at http://arch.debian.org/arch/dbnpolicy/etch/.

I finally got around to updating the SELinux UML recipe page to the latest 2.6.11.2 kernel. Of course, the new linux UML kernel expects a newer version of SELinux policy (19) than I had in my root_fs, so I’ll have to rebuild a new root file system. I wanted to get the newest version of the SELinux tools into Sid, but I seem to be coming down with a flu like thing. Since I would like to see Etch be SELinux capable, it is important that the barrier of entry be low for people wanting to play around with it.