EOF newline and consistant EOL goodness (utility)

Well, I got tired of fixing stuff by hand when I imported 1.5.2 into my local subversion repository, so I made a shell script to do this. I figured a few other linux devs might find it handy, so I'm releasing it to the wild after using and refining it over the last week or so.

What problems are solved by this shell script ?

Many files due to various editor and user weirdnesses don't have a newline at the end of the file. gcc pedantically complains about this and there is no way to turn off the warning. This shell script automates fixing those files.

Also, some files have mixed EOLs in them. Some lines have DOS type CR/LF combinations, other lines end in the unix LF convention and I think I even found a few that end in CR only. This shell script automates fixing those files as well. Since I have autoprops setup and enabled with subversion, it doesn't like to import these files because of the mixed EOL conventions in them.

One caveat though, is that all files processed by the shell script (*.[ch], *.c[cs], *.html, *.gui, *.txt and *.mk) will have their modification date and time changed to the current time causing your next build to compile everything. Since you'll generally only be using this script when you get a new TGE release or add a resource or two to the project, I don't find this too inconvenient in my development process.

You can download the script from here fixtextfiles or copy it from the code block below (missing a few comments to get under maximum post length):

#!/bin/bash
#
#       fixtextfiles - cleans up EOL and missing newline at EOF.
#
#       Requires dos2unix (from tofrodos package in ubuntu feisty, other
#       distros or unix variants might find it in a different package).
#
#       Requires printf (from coreutils package in ubuntu feisty, other
#       distros or unix variants might find it in a different package).
#
#       Requires ed (from ed package in ubuntu feisty, other
#       distros or unix variants might find it in a different package).
#
#       CAVEATS: Touches all your text/source files, changing the last
#       modification date/time to current.  So, your next build will
#       likely compile all your files.
#
#       License: Use however you want to, just don't claim it as your
#       own.

# Turn off GLOB processing so our find conditions don't
# get molested.
#
set -f

# Files matching these patterns will be processed by both ed and dos2unix.
# Add or subtract "find" expressions as you see fit.
#
FIND_CONDITIONS="( -iname *.[ch] -o -iname *.c[cs] -o -iname *.html -o -iname *.gui -o -iname *.txt -o -iname *.mk )"

# Use dos2unix to clean up EOLs.  This will force all EOLs in a file
# to have a NEWLINE (\n) at the end of a line, even if the file has
# mixed EOL as many of the files do.  This makes it possible to
# import into CMS repositories like svn and cvs without complaints.
#
echo "Forcing unix EOL on text files."
find . -type f $FIND_CONDITIONS -print0 | xargs -0 dos2unix

# Use ed to insure there is a newline at the end of the file so
# gcc doesn't complain.
#
echo "Sending all text files through ed to add missing newlines."
find . -type f $FIND_CONDITIONS -printf '%P\n' | while read i
do
        # Using printf for some pretty printing, comment it out
        # if your don't have the package installed or don't like
        # what it does.
        printf '\t%-70s\r' "$i"

        # ed, vi and a few other unix editors are adamant that
        # the files they write end with a newline.  Take advantage
        # of that quirk by forcing ed to read and write the file.
        # This will insure that all files written are terminated
        # by a newline, but otherwise the file contents remain
        # unchanged.  The file's modification date/time will be
        # updated for all files this touches.
        #
        echo "wq" | ed "$i"  > /dev/null 2>&1
done

echo