Build an RPM package from source on Linux...

Sometimes, we develop some helper scripts or tools, to help us in our day-to-day work - from automation tools, to reporting tools, to development tool, etc. As a Database Administrator/Engineer, I do spend some time writing helper scripts (mostly Bash and C++) and Automation tools (using Ansible). distributing these tools across systems sometimes, becomes a hassle, so what easy can we have this managed centrally and easily distributed - RPMs (Redhat Package Manager).

In this article, I'll give a walk-through of how we can achieve this. For the purpose of this article, we'll be packaging a simple bash script that checks if a server has a running installation of MariaDB or MySQL server. (N/B: This may not be the standard way or best practise, of building an RPM, but the idea is to give an idea of how and feel).

First, let us understand the basic build environment and some key concepts.

The root of an rpm build environment tree is the?"~/rpmbuild"?directory, which would contain 6 sub-directories. we will create the following sub-directories in the ~/rpmbuild directory:

• BUILD - this directory is used during the build process of our RPM package. basically, it is where the source code /script of what we want to package are built, temporary files are stored, moved, etc.
• BUILDROOT - this directory contains the files resulting from the build/compilation of the package inside the BUILD directory, reflecting the structure of the target system inside a sub-directory with the package name.
• SOURCES - this directory holds the sources (source code, scripts, etc) that needs to be compiled/built into the RPM package. it is usually a tarball file (.tar.gz or .tgz)
• SPEC - this directory contains the .spec file - which defines how the package is to be built. This file is quite important in the process (as we'll come to see) and good attention should be paid, when writing up this file.
• RPMS - this directory contains the RPM package built, for different architectures (noarch, x86-64, etc) based on what is specified in the .spec file
• SRPMS - this directory holds the .src.rpm packages. A source rpm package does not belong to any architecture or distribution, rather the actual .rpm packages are based on the .src.rpm package. this usually consists of the original compressed tar file, spec file and the patches which are required to create the binary package file. (A Binary package file contains all the source files along with platform specific additional information to install/uninstall the package, as well as application libraries of functions compiled for a particular architecture)

Spec File: this file contains the instructions and information needed to build our rpm package. it has an extension of .spec. the .spec file is usually logically divided into two (2) sections:

1. Preamble - this section contains information about the package being built, as well as defining any dependencies (both "Build" and "runtime" dependencies). it has several tags like: Name (basename of package), Version (upstream version of package), Release (release number), License, Url, Source0 (location of the source files), BuildArch (architecture target of the package), BuildRequires (build dependencies), Group, Requires (runtime dependencies), etc.
2. Body - this section would contain the following, that are relevant to the build process:

• %prep - this tag prepares the source for the building process. it does some background processes like - removing previous build, expand the source (.tar.gz) file, etc
• %build - this is responsible for performing the actual build process.
• %description - this contains the description about the package
• %install - contains instruction for the installation of the build package. can be executed as a bash script.
• %files - contains a list of files that are part of the package. complete paths are required here. note: if a file is not defined here, it will not be part of the package.
• %clean - this section would instruct the RPM to clean up /remove any files that are not part of the package's normal build area.

To expand the definition of a macro, you can use the rpm --eval "<macro>" command. e.g. rpm --eval "%{_binddir}"

• %setup - this is used in the %config section of the spec file, and basically has the following functions: extracts the source code/script into the BUILDDIR directory > switches into the extracted directory > sets the appropriate file permissions inside of it.
• %{make_build} - this is used in the %build section of the spec file, and basically it just runs the make command with a predefined set of options, to compile the source code/script.
• %{make_install} - this is used in the %install section of the spec file, and basically runs make install with the DESTDIR parameter, used to instruct the command to install the compiled package files to given directory instead of the real system.

We'll see more of these in play, when we write the .spec file for our package.

Macros: are an important and useful aspect when building RPM packages, as well as writing up the spec file. they let us perform some actions and reference things automatically. We'll explore just a bit of them here (a full list can be found at here)

First, let us look at some RPM directory macros:

• %{_topdir} - this references the ~/rpmbuild directory.
• %{_builddir} - this references the BUILD directory
• %{_buildrootdir} - this references the BUILDROOT directory
• %{_rpmdir} - this references the RPMS directory
• %{_sourcedir} - this references the SOURCES directory
• %{_specdir} - this references the SPEC directory
• %{_srcrpmdir} - this references the SRPMS directory
• %{_sysconfigdir} - this references the /etc directory
• %{_prefix} - this references the /usr directory
• %{_bindir} - this references the /usr/bin directory

Okay, now we have a good understanding of some basic concepts, let us proceed to actually Building the (RPM) Package. (N/B: unless absolutely necessary, do not build a package as root user)

• Install rpmbuild and gcc - in my case, I already have it installed, but depending on your distro, this package should be installed on your Linux server.
• create the rpm root directory and other needed sub-directories.

[michael@ansibleserver ~]$ mkdir -p ~/rpmbuild/{BUILD,BUILDROOT,SOURCES,SPECS,RPMS,SRPMS
[michael@ansibleserver ~]$ ll rpmbuild/
total 0
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 BUILD
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 BUILDROOT
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 RPMS
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 SOURCES
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 SPECS
drwxrwxr-x. 2 michael michael 6 Oct 29 08:04 SRPMS}        

• Create the script to be packaged in a directory, create a tarball file from the folder and move the tar file into the rpmbuild/SOURCES directory.

[michael@ansibleserver ~]$ mkdir -p testpkg-1.0.0
[michael@ansibleserver ~]$ cd testpkg-1.0.0/
[michael@ansibleserver testpkg-1.0.0]$ vi testpkg
[michael@ansibleserver testpkg-1.0.0]$ chmod +x testpkg?
[michael@ansibleserver ~]$ tar --create --file testpkg-1.0.0.tar.gz testpkg-1.0.0/ -vv
drwxrwxr-x michael/michael? ?0 2022-10-29 08:16 testpkg-1.0.0/
-rwxrwxr-x michael/michael 702 2022-10-29 08:16 testpkg-1.0.0/testpkg
[michael@ansibleserver ~]$ ll
drwxrwxr-x. 8 michael michael? ? 89 Oct 29 08:04 rpmbuild
drwxrwxr-x. 2 michael michael? ? 21 Oct 29 08:16 testpkg-1.0.0
-rw-rw-r--. 1 michael michael 10240 Oct 29 08:18 testpkg-1.0.0.tar.gz
[michael@ansibleserver ~]$ mv testpkg-1.0.0.tar.gz ~/rpmbuild/SOURCES/
[michael@ansibleserver ~]$ ll ~/rpmbuild/SOURCES/
total 12
-rw-rw-r--. 1 michael michael 10240 Oct 29 08:18 testpkg-1.0.0.tar.gz        

• Create a <package_name>.spec file in the ~/rpmbuild/SPEC directory. pay attention to the content, and see how I've matched things like name, version, prefix and also used the macros.

[michael@ansibleserver SPECS]$ vi testpkg.spec
[michael@ansibleserver SPECS]$ cat testpkg.spec?
Name:	? ?     testpkg	
Version:	    1.0.0
Release:	    1%{?dist}
Summary:	    A simple rpm package to check for existence of a Mariadb Database Service
BuildArch:? ? ? noarch


Group:		    Unspecified
License:	    GPL
URL:		    https://github.com/Nyele93/rpm_tool
Source0:	    %{name}-%{version}.tar.gz
  

BuildRequires:	bash
BuildRequires:? gcc
Requires:	    bash


%description
A simple rpm package to check for existence of a Mariadb Database Service


%prep
%setup -q


%install
rm -rf $RPM_BUILD_ROOT
mkdir -p $RPM_BUILD_ROOT/%{_bindir}
cp %{name} $RPM_BUILD_ROOT/%{_bindir}


%clean
rm -rf $RPM_BUILD_ROOT


%files
%doc
%{_bindir}/%{name}


%changelog
* Sat Oct? 29 2022 Michael Amadi <[email protected]> - 1.0.0
- First version being packaged        

If you wish, you can use the rpmlint command, to check for errors in the spec file - syntax: rpmlint </path/to/specfile/specfile>

Now that we have the source file and the spec file ready, next step is to run the rpmbuild command, to build the package. rpmbuild can take a number of parameters, but for the purpose of this article, we'll give a short description of few: rpmbuild -bb | -bs| ba :

- bb : build binary

- bs : build source

- ba: build both (both source and binary)

[michael@ansibleserver ~]$ tree rpmbuild
rpmbuild
├── BUILD
├── BUILDROOT
├── RPMS
├── SOURCES
│? ?└── testpkg-1.0.0.tar.gz
├── SPECS
│? ?└── testpkg.spec
└── SRPMS


6 directories, 2 files
[michael@ansibleserver ~]$ rpmbuild -ba rpmbuild/SPECS/testpkg.spec
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.xwkACA
+ umask 022
+ cd /home/michael/rpmbuild/BUILD
+ cd /home/michael/rpmbuild/BUILD
+ rm -rf testpkg-1.0.0
+ /usr/bin/tar -xof /home/michael/rpmbuild/SOURCES/testpkg-1.0.0.tar.gz
+ cd testpkg-1.0.0
+ /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
+ exit 0
Executing(%install): /bin/sh -e /var/tmp/rpm-tmp.VQIH7A
+ umask 022
+ cd /home/michael/rpmbuild/BUILD
+ '[' /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64 '!=' / ']'
+ rm -rf /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
++ dirname /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
+ mkdir -p /home/michael/rpmbuild/BUILDROOT
+ mkdir /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
+ cd testpkg-1.0.0
+ rm -rf /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
+ mkdir -p /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64//usr/bin
+ cp testpkg /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64//usr/bin
+ /usr/lib/rpm/check-buildroot
+ /usr/lib/rpm/redhat/brp-ldconfig
/sbin/ldconfig: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf: No such file or directory
+ /usr/lib/rpm/brp-compress
+ /usr/lib/rpm/brp-strip /usr/bin/strip
+ /usr/lib/rpm/brp-strip-comment-note /usr/bin/strip /usr/bin/objdump
+ /usr/lib/rpm/brp-strip-static-archive /usr/bin/strip
+ /usr/lib/rpm/brp-python-bytecompile '' 1
+ /usr/lib/rpm/brp-python-hardlink
+ PYTHON3=/usr/libexec/platform-python
+ /usr/lib/rpm/redhat/brp-mangle-shebangs
Processing files: testpkg-1.0.0-1.el8.noarch
Provides: testpkg = 1.0.0-1.el8
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires: /bin/bash
Checking for unpackaged file(s): /usr/lib/rpm/check-files /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
Wrote: /home/michael/rpmbuild/SRPMS/testpkg-1.0.0-1.el8.src.rpm
Wrote: /home/michael/rpmbuild/RPMS/noarch/testpkg-1.0.0-1.el8.noarch.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.EsN4Ry
+ umask 022
+ cd /home/michael/rpmbuild/BUILD
+ cd testpkg-1.0.0
+ rm -rf /home/michael/rpmbuild/BUILDROOT/testpkg-1.0.0-1.el8.x86_64
+ exit 0        

from the above output, our build succeeded (exit status of 0). we can check the tree structure of our rpmbuild directory again, to see the generated files. (also notice the cleanup when we ran the build command).

[michael@ansibleserver ~]$ tree rpmbuild
rpmbuild
├── BUILD
│? ?└── testpkg-1.0.0
│? ? ? ?└── testpkg
├── BUILDROOT
├── RPMS
│? ?└── noarch
│? ? ? ?└── testpkg-1.0.0-1.el8.noarch.rpm
├── SOURCES
│? ?└── testpkg-1.0.0.tar.gz
├── SPECS
│? ?└── testpkg.spec
└── SRPMS
? ? └── testpkg-1.0.0-1.el8.src.rpm


8 directories, 5 files        

as expected, our .src.rpm and .rpm files exist. now we'll proceed to install the package, test out its functionality and also uninstall.

[michael@ansibleserver ~]$ sudo rpm -ivh rpmbuild/RPMS/noarch/testpkg-1.0.0-1.el8.noarch.rpm
Verifying...? ? ? ? ? ? ? ? ? ? ? ? ? ################################# [100%]
Preparing...? ? ? ? ? ? ? ? ? ? ? ? ? ################################# [100%]
Updating / installing...
? ?1:testpkg-1.0.0-1.el8? ? ? ? ? ? ? ################################# [100%]
[michael@ansibleserver ~]$ ll
total 256
drwxr-xr-x. 2 michael michael? ? ?23 Oct 26 20:30 Desktop
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Documents
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Downloads
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Music
drwxr-xr-x. 2 michael michael? ? ?53 Sep 26 17:41 Pictures
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Public
drwxrwxr-x. 8 michael michael? ? ?89 Oct 29 08:04 rpmbuild
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Templates
drwxr-xr-x. 2 michael michael? ? ? 6 Dec? 2? 2021 Videos
[michael@ansibleserver ~]$ pwd
/home/michael
[michael@ansibleserver ~]$ testpkg
==========================================
This is an RPM package for Michael's demo.
==========================================


result: No Database previously installed
----------------------------------------
[michael@ansibleserver ~]$ sudo yum remove testpkg.noarch
Dependencies resolved.
============================================================================================================================================
?Package? ? ? ? ? ? ? ? ? ? ? ? ?Architecture? ? ? ? ? ? ? ? ? ?Version? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?Repository? ? ? ? ? ? ? ? ? ? ? ?Size
============================================================================================================================================
Removing:
?testpkg? ? ? ? ? ? ? ? ? ? ? ? ?noarch? ? ? ? ? ? ? ? ? ? ? ? ?1.0.0-1.el8? ? ? ? ? ? ? ? ? ? ? ? ? ?@System? ? ? ? ? ? ? ? ? ? ? ? ?702??

Transaction Summary
============================================================================================================================================
Remove? 1 Package

Freed space: 702??
Is this ok [y/N]: y
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
? Preparing? ? ? ? :? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? 1/1?
? Erasing? ? ? ? ? : testpkg-1.0.0-1.el8.noarch? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1?
? Verifying? ? ? ? : testpkg-1.0.0-1.el8.noarch? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?1/1?
Removed:
? testpkg-1.0.0-1.el8.noarch? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ??
Complete!


Now we'll run the same command, and should get the error of command not found, as package has been uninstalled.
[michael@ansibleserver ~]$ testpkg
bash: testpkg: command not found...
[michael@ansibleserver ~]$?        

And there you have it! our built RPM package works as expected, and adheres to all RPM behaviours. Now we can package our script/source code into an RPM package to be distributed on your Linux servers. go ahead and play with the SPEC file (Arch, cleanup, %install, etc). happy building!!!