by date
Plan for Joel Baxter
Plan for Joel Baxter
| Name: | Joel Baxter |  |
|---|---|---|
| Date Posted: | Nov 23, 2002 | |
| Rating: |  Not Rated | |
| Public: | YES | |
| Comments: | YES | |
| RSS Feed: |  or Subscribe with  . | |
| Profile Page: | View profile page for Joel Baxter |
Blog post
The Daily: More on build automation.
The Daily for Nov 21 2002:
TGE bug report 79 resolved, I hope. An example of a simple oversight having weird consequences.
I'm going to split this update into pieces. First, now, some more rambling about build automation. Then at the usual time (i.e. the wee hours) maybe some more about other topics.
I did a Web search (Google, freshmeat, SourceForge, CPAN) and trawled out the following links for further examination:
Automatic CVS Update
Automize
BuildRobot
cvs-get
gbuild
CruiseControl
fuel
MidWatch
Software Testing Automation Framework
tinderbox
VCS perlmod
The necessary question at this point is what the general plan is going to be, to help me choose the right assistance.
I could keep one copy of the source up-to-date on my Linux box and samba-export it for use by Windows boxes, so it's not absolutely necessary that the solution for automatically snagging stuff from CVS be crossplatform. It's also not necessary that it include much in the way of scheduling smarts, as I can use cron for that. On the other hand, it is necessary that it at least work on Linux... and crossplatform would be nice, at least, for the possibility of including Mac builds in the future.
As for the automated builds, my preference is that the solution be crossplatform, to minimize the variety of software I have to maintain. This disposes of BuildRobot, fuel, and MidWatch (unless I can't find a crossplatform solution).
So, taking the remaining links in order:
- ACU is a collection of sh and awk scripts. This looks pretty good, as it constantly keeps the files up to date (rather than updating on a schedule), and it apparently understands how to talk to the CVS server through an ssh tunnel, which is what our setup requires.
- Automize isn't freeware, but it's fairly cheap (50 bucks). Looks like it does many things that I'm not interested in, though. And from the online documentation it's not clear how helpful it would be.
- cvs-get is a Perl script that invokes the cvs client. For this level of functionality I'd probably rather just write something myself.
- gbuild has some cvs-get-like functionality, and then some additional Linux-only build functionality.
- CruiseControl might be a useful solution for automating the builds; and it's Java, so it's at least allegedly crossplatform.
- STAF looks like it could be very powerful/flexible, but it would take a bit of study. Also not Mac compatible. Maybe come back to this later.
- tinderbox is a bunch of Perl scripts. Some of the tinderbox client scripts could be useful for build automation; the server scripts could also be used in conjunction with whatever build automation solution I use, to display the results. Not really a turnkey thing though, would take some noodling to set it up.
- The VCS Perl module presents a nice interface for dealing with CVS. This would probably be a good thing to use if I end up having to roll my own updater.
My plan of attack, then, is to try out ACU first of all. If this works then we can set up any number of systems to keep a directory automatically-updated with the latest code for the purposes of regular builds. This is the sort of thing we did the last time I used CVS on a big project, although we had a hackier approach to it.
Assuming that works, we then need something to build the code. We don't really need to run continuous builds like some "serious" software projects do, but daily would be nice. So I'll take a look at the capabilities of CruiseControl and tinderbox. One issue I need to keep in mind is testing; obviously at a minimum I need to know if the build failed, but other tests would be good too. As a first cut I'd like the build process to start up a dedicated server (killing the previously running server) and notify me if that fails.
Dunno when I'll get to this, but I'll summarize the results when I have them.
TGE bug report 79 resolved, I hope. An example of a simple oversight having weird consequences.
I'm going to split this update into pieces. First, now, some more rambling about build automation. Then at the usual time (i.e. the wee hours) maybe some more about other topics.
I did a Web search (Google, freshmeat, SourceForge, CPAN) and trawled out the following links for further examination:
Automatic CVS Update
Automize
BuildRobot
cvs-get
gbuild
CruiseControl
fuel
MidWatch
Software Testing Automation Framework
tinderbox
VCS perlmod
The necessary question at this point is what the general plan is going to be, to help me choose the right assistance.
I could keep one copy of the source up-to-date on my Linux box and samba-export it for use by Windows boxes, so it's not absolutely necessary that the solution for automatically snagging stuff from CVS be crossplatform. It's also not necessary that it include much in the way of scheduling smarts, as I can use cron for that. On the other hand, it is necessary that it at least work on Linux... and crossplatform would be nice, at least, for the possibility of including Mac builds in the future.
As for the automated builds, my preference is that the solution be crossplatform, to minimize the variety of software I have to maintain. This disposes of BuildRobot, fuel, and MidWatch (unless I can't find a crossplatform solution).
So, taking the remaining links in order:
- ACU is a collection of sh and awk scripts. This looks pretty good, as it constantly keeps the files up to date (rather than updating on a schedule), and it apparently understands how to talk to the CVS server through an ssh tunnel, which is what our setup requires.
- Automize isn't freeware, but it's fairly cheap (50 bucks). Looks like it does many things that I'm not interested in, though. And from the online documentation it's not clear how helpful it would be.
- cvs-get is a Perl script that invokes the cvs client. For this level of functionality I'd probably rather just write something myself.
- gbuild has some cvs-get-like functionality, and then some additional Linux-only build functionality.
- CruiseControl might be a useful solution for automating the builds; and it's Java, so it's at least allegedly crossplatform.
- STAF looks like it could be very powerful/flexible, but it would take a bit of study. Also not Mac compatible. Maybe come back to this later.
- tinderbox is a bunch of Perl scripts. Some of the tinderbox client scripts could be useful for build automation; the server scripts could also be used in conjunction with whatever build automation solution I use, to display the results. Not really a turnkey thing though, would take some noodling to set it up.
- The VCS Perl module presents a nice interface for dealing with CVS. This would probably be a good thing to use if I end up having to roll my own updater.
My plan of attack, then, is to try out ACU first of all. If this works then we can set up any number of systems to keep a directory automatically-updated with the latest code for the purposes of regular builds. This is the sort of thing we did the last time I used CVS on a big project, although we had a hackier approach to it.
Assuming that works, we then need something to build the code. We don't really need to run continuous builds like some "serious" software projects do, but daily would be nice. So I'll take a look at the capabilities of CruiseControl and tinderbox. One issue I need to keep in mind is testing; obviously at a minimum I need to know if the build failed, but other tests would be good too. As a first cut I'd like the build process to start up a dedicated server (killing the previously running server) and notify me if that fails.
Dunno when I'll get to this, but I'll summarize the results when I have them.
Recent Blog Posts
| List: | 06/16/04 - Plan for Joel Baxter 11/26/02 - Plan for Joel Baxter 11/23/02 - Plan for Joel Baxter 11/21/02 - Plan for Joel Baxter 11/20/02 - Plan for Joel Baxter 11/19/02 - Plan for Joel Baxter 11/15/02 - Plan for Joel Baxter 11/14/02 - Plan for Joel Baxter |
|---|
Submit your own resources!| Joel Baxter (Nov 23, 2002 at 10:03 GMT) |
I decided to go ahead and get ACU running tonight, and I have, and now I want to go to bed rather than writing a .plan. :-)
Short notes about ACU:
It's not a particularly easy thing to set up. You need to be (or become) versed in user account creation/managament, CVS administration, ssh keys, sudo configuration, and probably a few other little things. The instructions also aren't quite correct, and the advertised ability to use an ssh tunnel was missing.
However, once all the rather extensive setup was finished, I didn't have to do much fixing or adding to the scripts to get them working and get the ssh-tunnelling going. So, overall a win, compared to doing it from scratch.
The CVS manual, BTW, has an example of using a command in the CVSROOT/loginfo file to keep an up-to-date version of a module on the same system as the CVS repository itself. The ACU approach just extends that idea to work on remote systems as well... basically, the command in the loginfo file runs a script that ssh-s into the remote machine and does the necessary CVS client command from there. Conceptually straightforward, tho there's a lot of cruft that needs to be set up both on the CVS server host and the system hosting the checkout. Ah well, it's done now.
| Joel Baxter (Nov 23, 2002 at 11:23 GMT) |
The script that was doing ssh started printing the error message "You have no controlling tty. Cannot read passphrase." Guess what the problem was?
Need another clue? OK, I eventually got around to trying ssh manually from the shell prompt. It asked me for a password for the remote account, rather than using the account's DSA key to automatically authenticate. Hmmm?
Solution: the home directory of the account on the remote system must NOT be group-writeable, or sshd rejects key-based authentication attempts.
A little while ago I had changed that account's home directory permissions from 700 to 770 because I wanted to be able to access it from my normal user account, which was in the same user group. Changing the permissions to 750 instead made sshd happy again, and is equally workable for my purposes, so huzzah or something. Bedtime now for sure.
--
BTW one flaw in ACU as it currently stands, is that to update a single file in the mirror it does a CVS update on the entire module. This isn't terribly slow with the number of files in TGE, but it creates a noticeable pause... probably wouldn't be too hard to improve this so that it only updates the affected files.
| Phil Carlisle (Nov 23, 2002 at 12:09 GMT) |
Just had a thought, why not run Wine on the linux box and compile the windows executable on the same box? I know its not a perfect solution, but it does save having another box AND it cuts down a potential point of failure (windows box getting access to the linux box).
One thing Ive seen which probably is pretty useful and might be funky to have is a way to kick of an autobuild from a webserver. This would build the executable, package it up and drop the final installer into a remote directory with a build report going to the person who asked for it, and anyone else interested.
Phil.
| Joel Baxter (Nov 23, 2002 at 19:15 GMT) |
However, since I already do have this other Windows box available, it doesn't hurt to use it. :-)
You must be a member and be logged in to either append comments or rate this resource.
