Tag Archives: Bzip2

LFS 6.8 (Part 10): The Second Part of the Rest of the Temporary Toolchain

The LFS temporary toolchain is going to take a while to cover completely, so I’m breaking it up into several parts for readability.

I’m about halfway through building the LFS temporary toolchain, and encountering mostly smooth sailing.

Grep

According to Wikipedia, “The name [Grep] comes from the ed command g/re/p (global / regular expression / print).” What Grep does is search files for the patterns you specify.

Grep installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Grep was supposed to build in 5 minutes, but with the test suites took a total of 19 minutes.

Gzip

Gzip is an older and only fairly efficient compression tool, but it works well enough that it is still a near-universal compression format.

Gzip installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

I calculated Gzip to build in less than 5 minutes, but took 8 minutes. Those test suites! Well, at least I’m getting practice in what to look for.

M4

M4 is (as the LFS book puts it) a macro processor typically used to pre-process source code for a compiler. What does it do that’s so different from any other scripting/macro language? I don’t know, but it probably is very well suited to pre-processing source code for compilers, since that’s what its used for 😛 .

M4 installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

M4 should have taken 10 minutes to build, but test suites inflated that number to 29 minutes.

Make

Ah, Make is our friend! Make automates the compiling of programs by reading that Makefile you keep seeing in the source directory and compiling the program accordingly. I can’t imagine what it would be like doing all of the steps by hand. The LFS book would be huge!

Anyway, Make is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

For all its versatility, Make was only supposed to need 5 minutes to build, but with its test suite, 7 minutes.

Patch

Patch modifies source code by applying the changes specified in a patch file. This makes it easy to apply small fixes with accuracy without having to wait for the developers to update the whole program.

Patch is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Patch was projected for a less-than 5 minute build, and finished in a satisfying 4 minutes. Including the test suite!

The test suite threw up two errors, though. One error seemed to be dependent on the ed text editor, and the LFS live CD apparently didn’t have it; however, it shouldn’t have run anyway, so no biggie. The other error I couldn’t even track down, and seems to be a failure of cat or something. Since I couldn’t find a way to fix it (or even figure out what it was), I decided to forge ahead regardless.

Perl

Perl is a general-purpose scripting language, often represented with a camel mascot. There is much that can be said about Perl, but you have Google for that 🙂 .

First I had to apply a patch “to adapt some hard-wired paths to the C library” and run Perl’s configure script to build a minimal Perl for testing purposes (which is all it will be used for in the temporary toolchain). When I installed Perl, I only needed the utilities and one library, so it built in a lot less time than it will later. Perl installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

I did skip the test suite, since the LFS book informed me that all of the rest of Perl would be built as well for the test; because Perl was only partially built, “installing” the temporary toolchain Perl was really just copying directories. Again, because this is a minimal version of Perl for the toolchain, it should only take 39 minutes to build, but it took 2 hours 9 minutes. I have no idea why so slow; I guess Pressie is getting tired.

Sed

Sed is a commonly used *nix tool for editing text, and is especially handy for shell scripts.

Installing Sed was straightforward, and Sed was installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

5 minutes is all Sed should have taken to install, but the optional test suite inflated that to a full 7 minutes.

Tar

Tar stands for “tape archiver”, so you can guess how far back its roots go. Tar made backup to tapes easy by concatenating files into a single .tar file. This made backups to tape much easier, as the tape didn’t have to rewind and fast forward back and forth between a single file and the master file table.

Anyway, Tar was installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Tar should have taken 15 minutes to build, but with the optional test suite, Tar took a full 57 minutes to build (all okay, though).

Texinfo

Texinfo is a suite of utilities for manipulating info pages (a longer, more detailed version of the man pages).

Texinfo was installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Simple to build, Texinfo was scheduled for a 10 minute build, but took 12 minutes with the optional test suite.

Xz

Xz is an up-and-coming compression format that hopes to mostly replace less efficient formats like Gzip and Bzip2. It is great for source code, as Xz compresses text even better than either Gzip or Bzip2.

Xz installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

15 minutes expected, 17 minutes taken (test suite included).

Now What?

All the packages for the temporary toolchain are now installed to /tools (in the temporary toolchain, /mnt/lfs/tools), and I am ready for the final steps in preparing the temporary toolchain for action!

LFS 6.8 (Part 9): The First Part of the Rest of the Temporary Toolchain

The LFS temporary toolchain is going to take a while to cover completely, so I’m breaking it up into several parts for readability.

After building the compiler and C libraries, I need to build numerous small, but not insignificant, utilities for the temporary toolchain.

Tcl

Tcl is the Tool Command Language, a flexible scripting language that is commonly used on *nix systems.

Tcl, Expect, and DejaGNU are used in the test suites for the versions of Binutils and GCC I will build for the final LFS system. According to LFS, “[i]nstalling three packages for testing purposes may seem excessive, but it is very reassuring, if not essential, to know that the most important tools are working properly.” After the test suites, though, these programs are unnecessary, so I won’t be building them again for the final LFS system. To build Tcl, I first cd‘ed into the unix/ subdirectory of the Tcl source and ran configure and make there. I also ran the optional test suite, just because. Tcl installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Tcl was supposed to take about 25 minutes to build, but with the test suites it took a total of 38 minutes to build.

I made the Tcl libraries writable so I can strip debugging symbols at the end of the temporary toolchain build process (to recover extra space, and presumably speed things up a bit). Next, I installed Tcl’s private headers, since Expect needs them, and made a symlink to tclsh.

Expect

Expect is a popular extension to Tcl that makes it easy to script interactions with *nix programs (in fact, Expect is one of the main reasons people use Tcl in the first place).

Before compiling Expect, I ran a script to make it use the known-good /bin/stty; this prevents Expect from trying to use a (non-existent) /usr/local/bin/stty that can be present in the live CD build environment. I configured Expect to look in /tools/lib (in the temporary toolchain, /mnt/lfs/tools/lib) for the freshly compiled Tcl libraries, and to build without the included Expect scripts, since they are unneeded. I also ran the optional test suite, which experienced eleven failures, but the LFS book says not to worry about Expect test suite failures, since they apparently fail at random with no consequences (“under certain host conditions that are not within our control” to be exact). Expect is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

I expected Expect to build in 5 minutes, but instead Pressie expectorated Expect in 6 minutes.

DejaGNU

DejaGNU is a program testing framework written with Expect.

LFS uses DejaGNU 1.4.4, which was released in 2004. Naturally, GNU released DejaGNU 1.5 six days after the LFS 6.8 book was released 😛 .

However, I’m using version 1.4.4, so I applied a patch containing several fixes to the old version before compiling. DejaGNU is built and installed in a single step, so had to run the optional test suite after the installation rather than before. DejaGNU installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

DejaGNU was supposed to build in less than 5 minutes, and Pressie delivered a shining build time of 2 minutes.

Ncurses

Ncurses is a handy set of libraries that lets programs define a text interface without having to rewrite the interface for every different terminal program. Think of it as a text counterpart to GTK+ or Qt.

Ncurses was fairly easy to compile, with a few options enabled that ensure it will be usable once I chroot into the final LFS system. I did not run the test suites for Ncurses since they can only be run once Ncurses is installed, and the LFS book did not provide explicit instructions for not screwing things up. Ncurses is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Estimated build time was 34 minutes, but Ncurses built in a brisk 24 minutes. Way to go, Pressie!

Bash

Bash is the Bourne-Again SHell, a Swiss Army tool for interacting with your computer in a super-efficient way on the command line. There are seriously way too many features and capabilities in Bash for me to do them justice: just know that there are a lot.

Bash has a built-in malloc (memory allocation) function, but it apparently breaks a lot, so I compiled Bash to use Glibc’s more stable malloc. Temporary toolchain Bash is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Bash was supposed to build in 25 minutes, and it only took 30 minutes.

Finally, I created a sh symlink to bash. Similar to the cc symlink for gcc, this allows for scripts to be generic, and for system administrators to use a program of their choice.

Bzip2

Bzip2 is a compression/decompression utility that is newer and better than Gzip, but older and not quite as good as Xz. It is one of the most commonly used compression methods for distributing source code (though Gzip is still pretty popular).

Bzip2 is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

I expected Bzip2 to compile in less than 5 minutes, and Pressie breezed through in only 3 minutes.

Coreutils

Coreutils is a giant wad of basic system utilities. I think the only reason they’re all together is that it was easier than having dozens of individual packages that would all have to be installed anyway.

I needed to tweak Coreutils to build the hostname program, which the Perl test suite will need later on. Coreutils installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Coreutils was supposed to build in 34 minutes, but took 1 hour 35 minutes. (I ran the optional test suite, which is probably why Pressie took so long. No errors yet!)

One of the tests failed in the (optional) Coreutils test suite, but I didn’t record which one. I hope this doesn’t come back to bite me….

Lastly, I manually installed this temporary toolchain version of su as su-tools according the the LFS book instructions. su is not truly installable yet, as I’m building and installing the temporary toolchain as the user lfs, and only root can install a su that can setuid root. The manually installed su-tools will be used in tests in the final LFS system, and I’ll be able to keep a fully functional su accessible in the LFS live CD build environment.

Diffutils

Diffutils is a collection of tools for comparing files and directories.

Diffutils installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Diffutils was supposed to build in less than 5 minutes, but took 18 minutes. (Again, running the optional test suite likely inflated the numbers.)

File

File is a very old, very solid *nix program for determining filetypes and reading files. According to its authors, File is used in every known BSD and every known Linux distribution — which is pretty cool!

File is also installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Since File is a fairly small program, it was projected to compile in only 10 minutes, though it actually took 7 minutes, even with the test suite!

Findutils

Findutils has several GNU programs to find files, as well as to generate and maintain a file database (to find files faster).

Findutils is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Findutils was supposed to compile in 15 minutes, though it actually took 16 minutes with the test suite.

Gawk

Gawk is the GNU implementation of the AWK language. AWK lets you script the manipulation of text files.

Gawk is installed to /tools (in the temporary toolchain, /mnt/lfs/tools).

Gawk was supposed to take 10 minutes to build, and actually took 10 minutes. Hooray! Even the test suite didn’t slow Pressie down!

More to come!

So far, no major problems have stopped my progress in building LFS. Let’s hope my luck holds!