Main

Code Notes Archives

January 25, 2003

Update all your operating systems (again, still) - Jan 25, 2003

There's lots of bleary eyed network admins out here this morning. Looks like another* MS SQL server vunerability being exploited out there. We know of at least one server that was driving 80Mb/s to the Internet.

All this can be avoided by patching your operating systems correctly. Most users can go to their Internet Explorer browser (the big blue "e" on your start menu) and on the Tools menu, choose Windows Update. Follow the instructions. For whatever other beefs one might have with Microsoft, they do their critical update service very well. Its free, and it could save you thousands of dollars.

http://update.microsoft.com

If you run if you run Linux or some other operating system please consult your manuafaturer's update pages, or install some program to automatically download the required software.

RedHat Linux
Apple
SGI (Irix)

* Symantec reports that this activity is related to vunerabilities discovered in 2002, and at the time this is posted the incident has not yet been published at NIPC, CERT or similar sites.

February 4, 2003

How often should I be doing this? - Feb 4, 2003

Recently a client who had just had a server compromised by the SQL slammer worm told us he had "applied the patch this summer." By some accounts, this should have protected him. But it got us to thinking about lies, damned lies and statistics again. And how often should we tell our clients that thier systems should be updated? And what's it going to cost?

bar graph of number of updates, RedHat Vs Microsoft

The graph above represents (in a crude fashion) the number of security advisories released by our two primary Operating Systems vendors over the past few years. That could provide a point of departure, and don't worry this isn't an OS crusade. We think the manufacturers do a fine job of supplying the patches. Problems occur when they aren't applied correctly or more frequently when they aren't applied in time.

Last year RedHat released 293 security or bug advisories. These are not all security related, and certainly a lot of the advisories were for bugs not related to security issues. But that's around one per work day. The potential cost impact is between almost none to review an advisory for software you don't use, to the hour or so that a kernel upgrade will typically involve; these occur about three times a year. It works out to about 51 hours a year.

Or, you can look at it from the other side of the coin. Some Windows boxes were disabled by Code Red and NIMDA in 2001 and more recently we noticed large disruptions on the Intenernet because of the SLAMMER worm which affected (and was spread by) unpatched Microsoft SQL Servers. These disruptions to business cost the global economy heavily. The cost can be measured in price of recovering from the intrusions and to get running again, the business lost while operations are disrupted, and sometimes actully loosing customer confidence. On the web, if they don't have confidence in you, they have a choice.

Some figures place the costs of Code Red at 2.6 billion dollars world wide. While 2.6 billion is a rather abstract number, the more recent SLAMMER worm caused outages in Bank America Corp's ATM network, a direct effect that most Americans can probably relate to. In addition to banking networks, other networks one would expect to be isolated from Internet attack were affected. In an extream instance showing potential global nature of this issue, China Telecom a nationwide phone company shut down all Internetional calls for the weekend, with only limted service being restored

JSW4.NET offers a service for a nominal monthly fee to keep your business servers up to date if you have a dedicated Internet connection with secure remote access.

March 3, 2003

Attaching External Style Sheets - Mar 3, 2003

CSS2 is the current standard for allowing authors to specify style information in web documents. CSS Contains information about the size, placement, color and other visual aspects of HTML entities. The current CSS specifcation is CSS2 which is a superset of CSS1 and allows designers to specify style for various media, such as print.

One of the first things you will find about CSS styles is that they can be defined in a number of places. By specifing styles via an external file designers can use one or several sheets across a whole site. Minor sitewide design changes are then greatly simplified by making the changes in a single file, or stylesheet. The following are the two methods of "attaching" a stylesheet file named mystyle.css

or

I provide some more examples of multpile style sheets being imported, but before you flip over to that, I should give you the official links to HTML, XHTML and CSS because those are the standards I write about and (try to) design to. The official standards body for these languages is The W3 Consortium - w3c.org.

In either case, a link or import via style element, the attribute type specifies the language of the style sheet. It is always "text/css" although it could be some other language in the future without breaking the modular nature of web documents. (Of course CSS works for pure XML too, so its unlikely to be unseated.)

When using the "rel" attribuite in a link tag, and author can also specify an alternative style sheet by setting the value of that attribute to "alternate" should force a choice on users. I tested this for the article you're reading and IE6.0 does not seem to support this part of the standard. So, for cross browser compatibility we usually use the @import method to specify more than one style sheet. (I always wondered why we did it that way. )

The idea of defining styles in an external sheet could be further modularized by grouping certain kinds of styles, such as table formating, into individual files. Using the import method, this is accomplished as follows:

Whether you choose to write your pages utilizing multiple sheets or only a single style sheet, sites that are written to standards will survive the test of time bettter than those that are written to the "anything that looks right is OK" standard. Those sites seem to break almost every time the powers that be release a new browser. With the exception noted above, it's my recent experience that the browsers become more and more standards compliant with every successive release.


March 4, 2003

Sendmail Vulnerability - Mar 4, 2003

I really hope to get some more articles about web development posted soon, but so much time is spent keeping up to date on systems and servers sometimes the articles get put on the back burner. Anyway, all systems administrators should be aware by now that as of yestrday morning their Sendmail based mail servers were vulnerable. This is a pretty serious exploit which would give total control over the server to the attacker. Sendmail is software that is bundled with almost all Unix and Linux operating systems. Some estimates place Sendmail on 72% of all Internet mail servers.

Internet Security Systems Inc. is the company who discovered the vunerability and has helped coordinating the response.

This information was released yesterday:

http://www.sans.org/webcasts/030303.php (A free archive of the webcast is available, sign up is required. It's pretty cool if you want to hear a the guys who discovered and developed fixes for these security issues. Bonus information about hardening your mail server is included.)

http://www.nipc.gov specifically http://www.nipc.gov/warnings/advisories/2003.htmSendmail has both an Open Source, free product and a commercial product.

What's up with that?

Interestingly enough, as of this posting the Department of Homeland Security has not recognized this as a threat. This is presumably more about getting the information posted than an actual shortcoming in thier threat assesment process. We searched the site for 'sendmail' and 'Sendmail'. All the Operating Systems vendor's we checked had updates or patches available on their web sites.

March 12, 2003

Sweet Trick - Mar 12, 2003

I was checking out the Hiveware site today, and they have an interesting solution to the problem of putting email links on web pages. In the early days of the web, this was one of the first ways in which we used to explain the inherent difference of the medium to prospective clients: It allowed for simple and direct feedback from user (potential clients from anywhere in the world) 24 hours a day, 365 days a year. GREAT!

Then someone wrote program to parse the email address out of any web pages, and set that spider loose on the net harvesting unsuspecting addresses off of the new entruprentuers' sites. SPAM is a real problem, but that feedback mechanism is critical to providing an additional channel for customer service.

Hiveware uses JavaScript in their free product "Enkoder" to scramble email adresses in order to confuse the address harvesters. It makes addresses hard for a script to harvest.

We came accross this while we were reading an article about how to use Apache's mod_rewrite to block access to the web servers. As Mark Pilgrim says on his site:

You will never stop all abusive behavior from all automated robots and rude programs, but you can minimize their effects and reduce the abuse to acceptable levels.

If you get over to his site, be sure to see "100" a work in progress consitsting of photopraphs and creative writing. I guess eventually there will be 100 items here?

March 17, 2003

Not quite blank - Mar 17, 2003

(Note on 7/26/03: Below you will find a bad reference to Apendix A of the HTML 4.0 specification. This is incorrect. The reference is actually in Appendix A of the CSS 2.0 specification.)

A lot of artists fear a blank canvas. It's a almost sterotype that writers get blocked when faced with a blank page. I'm a little this way about style sheets. I use pretty plain HTML typically without class or id attributes if possible. How many different classes of h1 can we reasonably expect users to be able to apply?

I remembered that when I was learnings CSS I had found "the default" style sheet. At the end of the HTML4.0 specification, there is a typical style sheet, based on "extensive research into current UA practice." Which means that all the default CSS selectors are specified here in such a way that if we don't override the values with our own style declarations, this is what is rendered. (Sort of.)

See the sample and a more complete explanation of the file in Appendix A.

April 8, 2003

Entry-level MT hacks - Apr 8, 2003

If anyone's been looking (and I know of three... a vast number relatively speaking) over the past week or so, some changes have been happing to the style and to a lesser degree the layout of AdvisorBits. This is not a totally original design. I am still using Movable Type's default templates (mostly) and the stylesheet is taken more directly than you might imagine from the "Stormy" style that is supplied with Movable Type.

The big changes are how posts are selected for display on the main page, and changing the <div> with the id "links" on the right side of the page.

I write about things as I am inspired. Inspiration often comes to me when my clients ask questions or raise issues. Although I am really commmitted to keeping this site fresh, I do not have a firm "I will post something every day, or every week" kind of schedule.

That is one of the reasons I removed the calendar. I have also changed the "<MTEntries>" from the default of showing the last few days as configured through the MT interface; to showing the last three entries without regard for the age of the posts. The code for this is in the "index template" and it looks like:

<MTEntries lastn="3">

I've also used a couple of CSS tricks to make the "links" <DIV> appear more like a navigation area. Previously I though it looked like, well, the right hand side of the page. I have put a border on all four sides of the area, instead of just one dotted line visually separating the primary content from the navigation. I have also used the frequently misunderstood position:relative in order to tie the navigation box visually into the whole page. It still breaks if the user agent viewing window is too narrow. (Your browser doesn't have enough pixels, its not my problem.) Here's the changes I made to the MT Stormy style sheet with regard to <div id="links">:

#links { position:relative; left:-80px; background:#ccc; margin:5em 0 0 0; width:160px; padding:5px; border: 1px solid #630; }

It should be noted that we're still validating.

And one day I know I'll get rid of all the warnings too. As always, your comments are welcome. Next on the list of improvements is an about page, a privacy page, and some new stuff on the navigation area.

April 9, 2003

I know, I know... - Apr 9, 2003

So, although yesterday's AdvisorBits design work looked beautiful in Explorer 6, it broke in Netscape 7 and Opera 6. Problem was that when the content div is floated left, you can't set the width on the links div. I imagine the content div is floated so that the links div comes after the content div in the code. This will cause it to break gracefully with links at the bottom of the page in browsers that don't understand css. (Like Netscape on my SGI-O2)

For those of you that don't use a modern VISUAL browser, I apologize in advance. Maybe I could put in a link that only renders in aural browser that says "Go right to the text". I read the latest AlertBox, and I think I know wht he's talkin about. I'm just not sure what to do about this. (And it's more than enough of a discussion for its own post.)

By swaping the order of the divs, and floating the links div right I get much closer to what I want, in fact it was erie, because Opera rendered more or less like I ultimately wanted it to, but Netscape and IE both gave me what I expected. I am aware that if you use a broswer that doesn't understand this that you'll probably get lost.

Or if you resize the browser in such a way that the fixed width of the links div... anyway, the 65% width had to go. By playing with the right margin and padding of the content I was able to get about what I expected in all three browsers. Here's how the CSS ended up:

#content {
margin-right:95px;
padding-right:95px;
padding-bottom:10px;
background:#ccc;
border-right:1px solid #333;

}

#links {
float:right;
width:150px;
background:#ccc;
margin:5em 10px 10px 10px;
padding:10px;
border: 1px solid #630;
}

Remember that I switched the order of the divs in the index.html template, and I really have to figure out a better way around that. It may validate for CSS but its really ugly for accessible. But now it looks like I want it to. I'm pretty sure it will validate again with only the same warnings about inherited colors.


April 15, 2003

Linux Cheat Sheet - Apr 15, 2003

O'Reilly Network: [April 15, 2003] has a complete listing of Linux commands and example usage from agetty to znew.

If you don't use agetty I'm not suprized its usually called by init to set terminal parameters. And znew converts ".Z" zip files to ".gz" zip files of course.

If I start posting more of these straightforward links with only the briefest of comments, it's because I installed "bookmarklets". Which are only kinda working, but that's another story.

April 23, 2003

Built-in Index Goodies - Apr 23, 2003

The Apache web server is argueably the coolest thing since sliced bread. On a site I maintain, we have archives of a few public mailing lists. I originally set these up to get mail out of my own in box, and found that it was convenient to have the resources close to me on the web. Subsequently, I have added a search feature and opened it up for public access.

We keep one year's worth of messages to each mailing list. If for instance you wanted to see our MySQL list archives, you would use this URL:

http://www.jsw4.net/info/list-archives/mysql/

Apache has a default module called autoindex_module, and in general, if the server cannot find a document named index.html in a directory that is publically accessible, it will construct a listing of the files in that directory. (Or sometimes, it will tell the viewer they don't have permission to view the directory. It depends on the site specific configuration.)

I've frequently wanted to change the order that the listing is shown in. For instance, the names in the directory above sort from oldest to newest at the bottom of the list. But users will most frequently want to see the recent entries first.

In noticed the listing is rendered with hyperlinks for column headings. To my pleasure, the interface is as you would expect it (if you know that at least one OS I use allows file lists to be sorted by clicking on the heading of such lists) and by clicking on the name heading I was able to change the default sort order from ascending order of names, to decending order. Notice the URL change that has occured:

http://www.jsw4.net/info/list-archives/mysql/?N=D

By reading the docs, I learned that one can also change the default sort order in the Apache configuration file, by use of the IndexOrderDefault Ascending|Descending Name|Date|Size|Description configuration directive.

Hats off the the folks at Apache, again.

May 9, 2003

Swiss army search tool - May 9, 2003

Dave's Quick Search Deskbar is a likely to replace my Google Toolbar as most useful Internet Information tool. The idea with both is simple, to put a search box on every screen. Google does this with its Internet Explorer Google toolbar. The downside to this is that the broswer has to be open in order for the search tool to be accessible, and although its ability to search parts of Google is extreamly strong, it searches only Google.

Dave's Toolbar sits in my Windows task bar. (So I guess it won't work for Mac or Linux.) Dave's Toolbar searches Yahoo, and CPAN, both other places I look. It searches SlashDot, the Bible, Walmart On-Line and the Yellow Pages to give you an idea of how many places it works with. It even tracks packages with UPS and FedEx I guess.

The interface is a little tough to get used to, there is only and edit box and on menu button. That may be a matter of of my taste or experience, but I wonder if the average user will understand and use all the extra fetures. There was a glitch unlocking my XP toolbar, but other than that it installed very easily. As an added bonus it shows the time so I was able to recapture some screen real estate by turning off the Windows clock.

Both tools are worth having. (And to disprove the free lunch theory, they're both free.)

May 12, 2003

Reinstalling is no fun - May 12, 2003

This article describes a good basic strategy to take when you are forced by a security compromise to reinstall your servers. It hasn't happened to me in a couple of years, but Mathew Tanase struck a cord with me in "Starting from Scratch: Formatting and Reinstalling after a Security Incident" when he says:

There is a point you reach in the recovery process, after you have done a little digging, put a finger on what might have gone wrong, where you come to the proverbial "fork in the road". Every security professional or systems administrator has faced the decision at some point in his or her career: is it better to try to repair the damage, or just reinstall the system and start from scratch?

May 15, 2003

Stupid Pet Tricks #1 - May 15, 2003

Here are some useful (you figure out how useful) things one can do with the amazing Dave's Toolbar I mentioned last week:

Convert temperatures right in the edit box:

temp 70 /f
results in
70°F = 21°C 294°K 530°R

Web designers will find it particularly useful to open a browser window at a certain resolution. For instance I am writing this in the window that opened when I entered this in my toolbar:
winres 800 600 http://www.advisorbits.com

(Or a URL like that anyway...)

Here's one to gererate dummy text for web page mock-ups:

lipsum 12

(If you don't want to get Dave's tool bar just to see what that one does, check this out: http://www.lipsum.com/ for a useful web designer's tool.)

Track Packages :
UPS 1Z23495743892095 (*example tracking number only)
Brings up the summary package tracking information for this tracking number in a web browser window. Haven't tested all shippers, but the menu has UPS, FedEx, DHL, AirborneExpress, and the US Postal Service. (Does this mean Dave's toolbar can "Go Postal?" I'll have to do a security audit. )

While this is not the first time I have seen any of these tools, it is the first time I have seen them neatly gathered together in one place on my Windows desktop. Look for future editions of Stupid (useful?) Pet Tricks (with Dave's Toolbar) as I find other cool things it can do.

May 23, 2003

Picture Upload Program - May 23, 2003

As most technical web professionals know, read any CGI programming news group, mailing list, or web based forum long enough and someone will ask a question about uploading files to the server via a form in the user's browser. Its something we all have to do for clients from time to time, and as an administrator, I can tell you it does represent a security exposure.

One should be careful how they accomplish this, pay attention to authentication and authorization, and be careful what is done with the uploaded files until it is known what they are.

Over the years, I have used a variety of CGI programs that have accomplished this for me, and those of you who use perl and use CGI(.pm) will be glad to know that this is supported.

This article is not about programming or even the perl way to accomplish this specific task. This post is to point out a php program for posting photo's to the web in a photo album format. There are tons of other programs that do this, in a variety of languages presumably larger than the two I have mentioned.

But I used Gallery, and you can see the example here. What is useful to some of my clients is the ability to upload binary image files right in a browser, without needing to use FTP. It is limited in the kinds of files that can be uploaded, only pictures are allowed for saftey.

June 11, 2003

Configuring CPAN - Jun 11, 2003

The Comprehensive Perl Archive Network (CPAN) is a collection of reusable perl code building blocks that is freely available. What this means to webmasters is that a number of prewritten scripts are available to use to build scipts you need without having to write every last line of code yourselves. (If you don't even know what perl is, stop reading this and go find out.)

CPAN is also the name of a perl module you can download from the archive. Usually when you first setup perl, you will also setup the CPAN module and then subsequently you can use that module to download other modules from the archive.

To start CPAN in interactive mode from the command line you type:


perl -MCPAN -e shell


The first time you enter the CPAN shell you will be asked a series of configutation questions; the location of various programs on your computer and the you are presneted with a list of mirror sites which you can choose the one(s) nearest to you.

Once you have configured CPAN you can see your configuration by typing :


o conf


This will show you all the current setting for CPAN, and if you ever need to reconfigure the initial values you set, you can do:


o conf init


There are a whole series of configuration varibles you can set, and the ones that got me thinking about htis article are the history file. For some time the CPAN sheel has had readline capabilities, which means that you can get a histoory of the commands you have used in a session. (Amoung other things that his means.)

However, the last time I upgraded CPAN, when I exited it told me that I had notspecified a historyfile so none was written. This made me think about reading the man pages ('man CPAN' not 'man cpan') to see what other variables are in the configuration.


o conf histfile ~/.cpan/histfilename
o conf histsize 200
commit


Specifies that I want a file name histfilename to be created and I want it to be 200 lines long. Now when I exit from CPAN and come back in, I can use command line history to see what commands I executed last time I used the interface.

July 15, 2003

More on Shrike - Jul 15, 2003

So I finally got around to an install of RH9. There was a glitch at the end of the first disk and the install died. When I rebooted, I had enough of an install complete that it booted and most services started. I had half a mind to see what would happen if I launched KDE.

Fortunately it was the half that doesn't stay focused on one thing for long, and I rebooted to the installation.

The install started again, but once I made all the basic selections (keyboard, mouse, time zone, etc) and the package selection (um, just upgrade whatever you recognize and leave the rest alone) I was able to upgrade the only redhat installation left on my hard disk, the one that just failed.

And it worked. The install skipped to where it died, installed the rest of the RPMS and everything came back fine.

Except Apache. Sometime between 7.2 and 9 RedHat updated the world's most popular web server from 1.3.27 to 2.0.40 and that played hell with the mod_perl application I am working on here. I got Apache upgraded and running and then it turns out my handler needs upgrading too.

For most users this shouldn't present as much of and issue as I think relatively few people write thier own mod_perl handlers. And those that do are probably more aware of these kinds of changes than I was.

July 27, 2003

In case you were on Mars... - Jul 27, 2003

This week both Cisco and Microsoft announced what amount to network crippling vunerabilities, and the availability of fixes for them. You should have reviewed all your Cisco devices running IOS to apply a workaround or to update the IOS image on your routers, switches and other Cisco gear. If you have an outside vendor who is responsible for performing this work, you should contact them to make sure your network has been protected.

Most Windows machines have a feature called "Automatic Update", which shows a little globe icon by the clock on your desktop when your machine needs updates. This week the globe was there a lot.

If you're still running impending demise of 98.)

The vendors have released the fix, and both of them have home page links to articles and information about the issues. I have written about this before, and I will again.

You've been warned.

The Cisco advisory, can be found at thier web site, and has been updated several times.

The Microsoft Advisory has been updated at the security site, and the original advisory can be found from the update page.

I noticed that Microsoft thanks LSD, the security professionals who informed them of the issue and helped to develop the fix.

(Note on Aug 4: Since I wrote this Microsoft has sent two different mailings to me urging me to apply this update. I think they are trying really hard to get the word out. Maybe there's a message in this fact.)

August 29, 2003

Tarpits - Aug 29, 2003

As of version 1.2.7a the netfilter firewall tool, iptables has had a built in target called TARPIT, but the first I had heard of it was the other day on Security Focus, in an article by Tony Bautts. The TARPIT target is an offshoot of the LeBrea project. The LeBrea project is the brainchild of Tom Liston, and you can read about Lebrea at HackBusters.net and at its new offical site, at Sourceforge.net. What the TARPIT target does is slow the spread of infection down, presumably so we have more time to react to outbreaks.

The idea is that when an offending scan comes in we accept the connection and then we set it up ion such a way that the attacker can't close the connection for 12 to 24 minutes.

We know that the msblaster.exe attack was targeting port 135 to spread. So on interfaces and addresses where we don't expect port 135 traffic we accept the traffic and hang the attacking computer up.

The iptables command would look something like:

iptables -A input -p tcp --dport 135 -j TARPIT

If you run RedHat you'll need to pick up an updated iptables rpm over at gnumonks (www.gnumonks.org), and while you're there, you may want to check out the matching ulogd rpm for packet logging features.

Please don't deploy this in a production environment until you have tested it out first. This could cause some big problems for you if you accidentally misconfigure it. You've been warned, now have fun in the tarpits.


October 15, 2003

Better write this down - Oct 15, 2003

Every once in a while for some reason, CPAN fails to work. I get these insane directory names and the whole install fails. I was installing Crypt::OpenPGP which is written by Ben Trott, the same guy who writes MoveableType (the software used to publish Advisor Bits) and this happened to me.

Hey Ben, that's a lot of prerequisites!

I had to install Digest-MD2 the old way in order to get the OpenPGP to work. It took me long enough to remember that I thought it was worth writing here so I can look it up in the future. (Good use #107 for Advisor Bits!)


Here's the generic instructions for installing Perl mods when CPAN is fussed up, for those of you lucky enough to have always used CPAN:

1) Download the file:

wget ftp://ftp.someserver.com/some/long/path/To/YourModule-1.01.tar.gz

2) unpack the stuff

tar -xvzf YourModule-1.01.tar.gz

3) change into the directory

cd YourModule-1.01/

4) Configure the module for compilation and installation

perl ./Makefile.pl

If you need to install anything first, you will find out now.

5) Make the module and test it before installation

make
make test

6) Install the module if everything worked to this point

make install

7) You're done.

November 2, 2003

CSS only rollover - Nov 2, 2003

In the continuing quest for standards compliance and clean code, a "rollover" that doesn't require client side JavaScript is not only cool as cats pajamas, but it does somethings a lot better too.

For instance, Dreamweaver has been known to miss the "OnMouseOver" images when moving files and adjusting links. (Forget about using wget, it does not interpret JavaScript so it never gets all the images.)

This idea is lifted straight from the master, Eric Meyer. He has a complete explanation of the technique at the O'Reilly Network.

Internet Explorer (version 6 point whatever is current today) kind of flashes the images when changing them a bit, but the effect is not too bad. Opera 7 handles the hover event smoothly. You can see my sketch of this idea at http://www.chathamselfstorage.com/rates.html

January 3, 2004

Sneak Preview this Weekend - Jan 3, 2004

I am putting the finishing touches on the JSW4.NET main company site. This is only the third redesign of the JSW4.NET site since it was founded in 1999. There are a few things to polish up and the form script still has to be written. Undoubtedly, I need to run the spelling checker on several pages that I missed.

The CSS on this site is designed very carefully so that the main body of the text is always at the top of the page (code wise) and special care has been taken to make the CSS nice for print media. I changed the visibility attribute of the

that has all the navigation stuff to "hidden" and although that caused it to not print, the space that would have been taken by the element usually caused an extra page to print.


The solution was to change the display attribute to "none". I was more familiar with the use of the display attribute to control whether an element was inline or a block; it can also be set to several other values.

January 9, 2004

Watching the master at work - Jan 9, 2004

There are a lot of Linux Engineers out there with more experience than me. I always read their posts when I get in trouble, and some times it helps me out. I was really close last night when one of the few guys I actually know who certifiably "Knows his sh*t waaay better than me" showed up.

My problem was I had half of a mirror set (Linux software RAID1) and the other half had gone south during a reboot. The half I had wouldn't boot. Murphy's law explicitly states that when I have part of a mirror set fail, the bootstrap will never be on the drive that survives. Murphy is one of the reasons I was suprised to see Dan L.

Given that I now have a reconstructed and booted system, I thought I would record the steps I took, so that if someone else encounters the same situation it may be of use to them.

The box all this was setup on is an entry level Intel server board, with integrated IDE controller and dual 80GB IDE drives. The RAID was setup as part of the installation process of RedHat 7.3, and I use GRUB as the bootloader.

To repair the RAID, I was using a RedHat install disk and booting to Linux Rescue. You can get a tiny Linux booted up to mount partitions and try to fix systems using this option. Debian has something like this too on their install CD.

I managed to partition the new drive exactly the same as the old drive. I should also mention that when the drive failed I initially tried some totally desparate switching of slave to master and cable swaping to get the other good drive to boot. No dice. Somehow at the end of this whole story when I finally booted and got everything running, it was OK that the drive on the second IDE channel had migrated to the primary IDE channel, and that the jumper setting had been changed. I really have to like an OS that can stand up to my hardware skills.

Once Dan became involved things started to come together. The first thing he noted was that none of the partitions were marked as bootable. Using fdisk, he made the partition which contained the kernel images (i.e. /boot) bootable. I am going to have to do more reading about this bootable partition thing; I have not used this in the past. On the other hand one of my most stable boxes has the swap partition on each drive marked as bootable. Making the /boot partition bootable makes a lot more sense to me. YMMV, this is a whole other topic.

In RH73 fdisk the option to mark a partition as bootable is "a". I am told I should look into cfdisk, in fact the main page for fdisk says to choose cfdisk over fdisk.

Once the disk was setup, he mounted the root partition from the drive that was good, and performed a chroot to the temporarily mounted root partion. This enabled him to edit the raidtab to mark the new disk as "failed" so that once the operating system saw it the kernel wouldn't accidentally reconstruct the wrong (i.e., blank) half of the mirror. That looks something like this for each md device:

raiddev /dev/md0
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda2
raid-disk 0
device /dev/hdd2
raid-disk 1
failed-disk 1

Dan also suggested we disconnect the new drive to be doubly sure there were no accidental reconstructions. Good plan.

Since I use GRUB, it is already installed. One of the very cool things about GRUB is that it has a tiny shell which allows you to setup the boot environment and also to install the bootloader. Once you're in the shell you can tell grub to install a bootstrap using a command something like this where "hd0" is the master drive on the primary IDE channel:

grub> install (hd0)And sure enough, the darn thing booted to grub and from there the system came up. The RAID barely complained about running in degraded mode because of the drive that we had disconnected and marked failed in the raidtab. It came up again when we reconnected the device too. Dan was hungry and I was tired, so we decided to call it a night after the box came up and the services were all running again.

Besides which, I couldn't very well ask him for the help I needed to figure out how to reconstruct the arrays after everything else he had done. Fortunately the NEW documentation for NEW raidtools (2.4 kernel and above) is very good and it worked.

To reconstruct the array use the raidhotadd command:

raidhotadd /dev/md0 /dev/hdd2It doesn't look like anything is happening, but you can check your /proc/mdstat to see whats going on. array reconsruction took a few minutes for small partitions and more than an hour for a 60GB partition. Reconstruction occurs in the background, so load on the server may effect this time. The drives can remain mounted and active while reconstruction occurs.

[root@cp root]# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hdd9[0] hda9[1]
104320 blocks [2/2] [UU]
md6 : active raid1 hdd8[0] hda8[1]
66565184 blocks [2/2] [UU]
md3 : active raid1 hdd7[0] hda7[1]
3076352 blocks [2/2] [UU]
md1 : active raid1 hdd6[0] hda6[1]
513984 blocks [2/2] [UU]
md5 : active raid1 hdd5[0] hda5[1]
6144704 blocks [2/2] [UU]
md4 : active raid1 hdd3[0] hda3[1]
104320 blocks [2/2] [UU]
md0 : active raid1 hdd2[0] hda2[1]
722816 blocks [2/2] [UU]
unused devices:

I think since I manually marked the disk failed I am going to have to manually remove the failed entry from the raidtab, but I am going to deal with that when I am on-site.

I have been in this same situation three times in 5 years, and this is the first time I actually got reconstructed arrays working cleanly. I don't really want to confess to the lengths I have gone to in order to fix this issue, but I guess I already have.

January 16, 2004

Common Gateway Interface (CGI) - Jan 16, 2004

What is CGI-BIN?

It's a term that gets thrown about all the time in "web circles". CGI, CGI-BIN, Common Gateway Interface all generally refer to an environment where users "run" a program on a server, using a web browser on a client computer to submit input. The program may store the data, or lookup information based on the data, the possibilities are being expanded all the time. Whatever the program does with the input data, it typically also returns some information or confirmation (output) to the user's browser.

CGI-BIN is usually a directory on a server that is setup with special permissions. These permissions include allowing Internet users to run, or execute, the programs. On some web servers, any file with the extension ".cgi" can run without regard to what directory the script file is located in.

The University of Kansas Academic Computing Center has a fairly detailed description of CGI and HTML forms.

There is a CGI standard defined here. NSCA was the first widely available web server, and so in the early days of the Internet, other web server software such as Netscape conformed more or less to these standards.

That document is also mentioned on the W3C site's own comprehensive listing of information related to the various technical implementations of CGI, the Common Gateway Interface.

March 30, 2004

GIMP turns 2.0 - Mar 30, 2004

The GNU Image Manipulation Program (GIMP) version 2.0 was released on the 23rd of this month. The GIMP is a freely distrubuted OpenSource program that does exactly what its title says. It is most frequently compared to the commercial program, PhotoShop, which is made by Adobe.

The GIMP is available in a number of languages and for a variety of platforms, including Windows, OSX and Linux. If you want to get a copy for yourself, follow the linkson the GIMP for Windows page.

April 8, 2004

Apparently hell has frozen over - Apr 8, 2004

A friend of mine who is an editor by profession called me this morning to ask what a DS1 is. She also mentioned an article on eWeek that tipped us to this:
http://sourceforge.net/projects/wix/ is Microsoft's first open source project. (No kidding.)

The author, Steven Vaughan-Nichols, sees it as a page from the Evil Empire's playbook described in the Halloween documents.

April 10, 2004

The envelope please - Apr 10, 2004

One of the issues that will have to be overcome before Linux can become the desktop Operating System of choice is the real and percieved lack of applications. There are simply more applications out there for Windows. That is why it is a "real" lack.

Advocates such as the Open Source Directory, would like to overcome this somewhat unfounded perception. They publish a comprehensive listing of Open Source software that is available, and not just for Linux. As mentioned here in Advisorbits, there is Open source software in the Windows world too. This "Editors Choice Awards of 2003" article is a bit old, but when I found it it made me look at a few applications I hadn't known about. (I am also proud to be a user of many of the applications mentioned, including Moveable Type. )

I was confused at first, because I am also familiar with the http://dmoz.org/ Open Directory Project. These sites do not seem to be directly related.

One software projects that caught my eye was the Open Office project. I am going to start evaluating Open Office as an alternative to many components of MS Office. Its time to upgrade my business PCs, and cost is a factor. The evaluation will center on interoperability with MS-Office 2000 files, and usability. How much training will be required to make the transistion? We'll find out and let you know.

June 18, 2004

There ought to be a law - Jun 18, 2004

Today's Internet Storm Center Handler's Diary tells a story that would be funny if I didn't know it to be sadly true in many cases. See "Mailbag: How to tell your consultant is a fraud."

August 5, 2004

Thanks, Mark - Aug 5, 2004

I confess, my first PC was a MacPlus. (The original MacPlus if you're old enough to know there was more than one model year.) So I have always been in essence a drag and drop guy. Over the years I picked up a little DOS, and now I use a lot of Linux. I work on a command prompt more than almost everyone I know.

The real truth is that the GUI background has softened me and made me pretty much a poser compared to the guys who really know how to use a *NIX shell. I learn what I need because I look things up and for some unknown reason I understand man pages. (And computer software manuals, and I know how to Google.)

I was looking for some syntax examples so I could reboot rk Sobels' new book, which also just happens to have the answer to all the even number questions in the book. It even conveniently incuded an example that had everthing but the exact time I wanted.

So here's how it looks:


[root@colin root]# at 0600 tomorrow
warning: commands will be executed using (in order) a) $SHELL b) login shell c) /bin/sh
at> shutdown -r now
at>
job 1 at 2004-08-06 06:00
[root@colin root]#


If you want to see what that will setup (not for the weak of heart) you could do:

[root@colin root]# at -c 1


A note: the is caused by typing a Ctrl-d and the number "1" in the second command is whatever job number you wish to see listed.

August 15, 2004

Wireless intrusion tools - Aug 15, 2004

I've used Ethereal for some years now. This software was updated the other day, if you have an older copy you might want to update for new features. If you use Ethereal, you know that it is an Open Source network protocol analizer that runs under Linux, Unix and Windows.

Developers and security professionals may use a protocol analizer to capture packets on the the network and look inside the traffic to troubleshoot netowrk problems or even to look for signs of intrusions. Network Chemistry has a similar product that uses parts of Ethereal, and prvides additional functionality. Their "Packetizer" software, which makes use of the Open Source philosphy of cooperation is licensed under the GNU license, and the source code is freely available. Packetizer is a protocol anaylizer that is specifically designed to look at wireless LAN packets. The Packetizer is part of a larger suite of security products that Network Chemistry offers.

I thought the site and the product merit mention as an example of how a commercial organization can contribute to and benefit from the Open Source community.

If you're running wireless, for heavens sake, please run some kind of IDS too. If you don't know about these 2 tools you should look into them and also do penetration testing on the wireless segments of your LAN.

August 27, 2004

Fun with XP SP2 and NMAP - Aug 27, 2004

Nmap Hackers: Windows XP SP2: Nmap Fix and Further Information contains a useful explanation from the author and a clearly marked rant.

Fyodor has released a version of NMAP that provides a workaround for the fact that Microsoft has removed access to RAW tcp sockets in XP Service Pack 2. He's also given a critique of Microsoft's policy which has effectively broken the useful systems and network administration software he wrote, NMAP. One of the footnotes led me to a detailed explaination of the changes posted on the Microsoft site. As I read over the list of changes at Microsoft it occurs to me that the same people who have problems with their computing networks because of misconfigured host based firewalls will probably be more than a little affected by the changes to Microsoft's strategy here.

I think that well configured and maintained XP machines in small business networks will benefit from some of the features in SP2. If you support XP hosts, be prepared for problems, and plan to spend time dealing with these issues over the next few months.

August 30, 2004

Splitting nerd hairs - Aug 30, 2004

Sometimes I write these posts and I am wonder if my clients understand the stuff I write about, and sometime I wonder if it hasn't just become one of those ranting (or rambling) nerd blogs even though I really try hard not to rant. And then, as if to cheer me up, Google points me to a whole other realm of nerdly wonder, namely Eric Smith's blog about all things SPAM.

spamblogging - A discussion of all things spam. Stopping, tracking, responding, and more.

I was looking for info on how to use SPF on Microsoft Exchange, and more general info about the feelings in the mail administrator community about whether the latest anti SPAM trend is going to fix things or not. I got a lot of good information and two great quotes from spamblogging.

Who's to say what's wrong and what is right? All I know is that SPF/Sender ID sure isn't going to stop spam, and it sure isn't going to make my life all that different. But if it can help cut back on spoofing and even reduce spamming only a bit - then I'm down with it.

And even though he over generalizes and calls RMS an Open Source talking head, I think this line by Eric Smith in a post titled "Big Surprise: RMS favors OSS over that of Microsoft", is rather amusing and almost certainly on target:

All in all, it is a bunch of nerd egos making cracking sounds as they butt into each other, trying to split hairs over things that don't necessarily even apply.

August 31, 2004

Nice looking Groupware - Aug 31, 2004

Just got back from a test drive of Dicole MimerDesk. Its a web based groupware software project that has a nice list of useful features. It is an OSI project that I heard about on the CGI:Application list.

Dicole Groupwork and Collaborative Learning Environment

September 2, 2004

Gimp Lomo Effect - Sep 2, 2004

My friend Jesse is a genius with Photoshop. He can take just about any photo and make it look good; I oughta know because hes helped me out of a design jam a couple of times. But me, I can't afford those professional kinds of tools when my primary use will be to shrink my photos of flowers from my garden. When I mentioned the GIMP to Jesse, he kind of mumbled some things I dont quite remember as if to imply theres stuff that Photoshop can do that GIMP can't.

One of the features he likes about his tool was some ability to store procedures and repeat them. I found out there's a thing in GIMP called script-fu which is an implimentation of the scheme programing language. I don't think this ability to write our own scheme scripts to manipulate images is much use to me or Jesse, but the ability to plug in other people's scripts sure is. I have been working through some of the tutorials at the GIMP web site and the various plug-ins are mentioned in some of the tutorials.

LOMOed shot of dog in mist

And I found out there's a script-fu to apply lomo effects. (Although the script didn't work because it was for an older version of GIMP, I found a refernce to this set of instructions that allows me to create my own lomo effect. Thijs van der Vossen · Faking Lomo) Do I like it?

Which brings me back to Jesse, because I had never heard of this until he mentioned it on his own blog at http://plasticmind.com/weblog/. Lomo is a Russian optics company that makes camera which takes pictures that have their own unique quality of color and light.

September 9, 2004

Firewalls Up Mr. Scott! - Sep 9, 2004

James Doohan, aka Chief Engineer Montgomery Scott got his star on the Hollywood walk recently in case you missed it. I haven't watched StarTrek in some time, but if I tuned in StarGenX or whatever the most recent version is, I would expect the latest Captain Whatshisname to be saying things like "Firewalls Up, Mr. Scott".

Microsoft's Number 1 of three steps to protecting your PC is Use an Internet Firewall. I don't always agree with Microsoft, but these days you really should have some kind of firewall between you and the Internet.

Firewalls come in hardware and software varieties. I tend to favor a hardware firewall for several reasons, although host based software firewalls have their advantages too.

A hardware firewall router for broadband will generally also provide several switch ports so you can also network several computers with the same piece of equipment. Its been my experience that once these are setup correctly there's less chance the average end-user will try to reconfigure it, and correspondingly less risk of leaving holes open.

Personal, or software based firewalls are nice because they are so easy to update. If an exploit is released, users these days seem to feel comfortable downloading applications and following instructions, so a software based firewall may be easier to maintain.

Ideally, end users would understand the intricacies of packet filtering and IP routing, and they could maintain a mixed environment of hardware AND software based firewalls for the extra protection this kind of layered approach offers. However in my experience, a single firewall at the perimeter of these stub networks is likely to be the most uniform and fairly fool proof way to offer an acceptable trade between usability and security.

I've just seen too many turned off host based firewalls to really trust my clients' network security to them. I've also seen installations of host based firewall software that were so restrictive that they couldn't update themselves; and VPN connections dead as a doornail.

If you have an office LAN you should talk to your consultant or integrator. They will be able to tell you what kind of firewall you have now, and if it has been reviewed and updated recently. (If you have a consultant or an integrator, and you don't have any firewall protecting your LAN from the Internet, you may have a larger problem.)

SOHO hardware firewalls start at around $50, and they're pretty small so you can even take them with you on the road. If you can get someone to configure it for you once, chances are good you'll never have to mess with it again.

In the end, whatever firewall that users will actually put up and maintain between them and the Internet is the firewall that's best.

A couple of postscript notes here:

1) Microsoft's three steps leave out a very important one too, use strong passwords.

2) Remember that wireless thingies can be hacked from a distance of 2000 feet; perimeter firewalls don't even begin to address that hole. (Why would someone break into your firewall, if they can just hack away at your wireless LAN on the other side until they get in?)

3) Firewalls actually come in more than two flavors, there also are three technical categories: packet filtering; proxy based; stateful packet inspection.

September 12, 2004

XP Service Pack 2 - Sep 12, 2004

In case there was any doubt about the advice in the the aritcle about service pack 2, "Fun with XP SP2 and NMAP", I think users should apply the patch if they can. (Unless of course someone else is responsible for the machine, such as a network administrator.)

Users may have heard security experts claim that XP SP2 has vunerabilities. This may be true, but so are these facts:

1) Some of the vunerabilities existed in previous versions of Windows XP and simply remain unaddressed.

2) Security is about layers, and this service pack impliments some new layers, the net effect is a good thing.

3) A lot of the stuff the experts are speaking about is pretty esoteric stuff, requiring certain conditions to pre-exist and also requiring the user to be more or less a zombie. Larry Seltzer had a good quote about this on the BugTraq list.

W

here do we draw the line on this social engineering stuff? If I send an e-mail to someone telling them to flush their iPod down the crapper does that mean the iPod is vulnerable to a toilet attack?

So, to summarize: YES, apply serivce pack 2 on Windows XP machines that you are responsible for. NO, do not flush your iPod.

Related reading: "A Feast of Egos", by Tim Mulen of Security Focus

October 13, 2004

First Post from Firefox - Oct 13, 2004

Firefox is the newest browser from Netscape's Mozilla project; get yours here. I am posting from this new browser today because all of my links and bookmarks imported fine from IE. In fact, I barely notice the difference in the way this new free browser renders my MT interface.

Mozilla is an Open Source project, so support of it at AdvisorBits should come as not suprise. Another thing I like is that Firefox claims to be more secure because it doesn't run ActiveX controls. Instinctively this appeals to me as a user, although I am also a fan of some applications where I think I will need ActiveX controls.

If Firefox is the default broswer it forces the user to choose to use IE in those instances where the ActiveX controls are trusted, and so it makes the user think about security.

Users who are conscious of security are good things in my book.