Packaging protobuf-c for EL5

How boring a title that is!  Okay, we're RPMing up Protobuf-c - a c binding for the protobuf kit from Google - for EL5.  We need it to do something far more interesting later -- so these are the giants whose shoulders we stand upon.

I cheated;  let's be honest about that, and just how much I cheat to make my life easier.  It's a lot!  In this case, I stole the protobuf-c RPM as built in RHbz593559 for EL6, and which is supposed to be started for EL5 but hasn't dropped yet.  So we'll just finish that port.

1. Go get it:
   wget http://download.fedoraproject.org/pub/epel/6/SRPMS/protobuf-c-0.15-2.el6.src.rpm
2. Unpacking it is harder because of a strange change to the package format, where they seem to ignore compatibility.  So we need to do an rpm2cpio|cpio instead of a trivial rpm-i .  It's no big deal, but watch for it:
   

   rpm2cpio protobuf-c-0.15-2.el6.src.rpm | cpio -idumv

3. Because the EL6 one has problems, we need to patch around those, but they're trivial;  it's surprising that we see this in what should be a quality product.  I worry about just how many people have this kind of blinders on.
   

   @@ -10,6 +10,7 @@

    Source1:        http://protobuf-c.googlecode.com/svn/tags/%{version}/LICENSE
   
    BuildRequires:  protobuf-devel
   +BuildRoot:     %{_tmppath}/%{name}-root
   
    %description
    Protocol Buffers are a way of encoding structured data in an efficient yet@@ -41,9 +41,13 @@

    make check
   
    %install
   +[ "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT
    make install DESTDIR=$RPM_BUILD_ROOT
    rm -f $RPM_BUILD_ROOT/%{_libdir}/libprotobuf-c.la
   
   +%clean
   +[ "$RPM_BUILD_ROOT" != "/" ] && rm -rf $RPM_BUILD_ROOT
   +
    %post -p /sbin/ldconfig
    %postun -p /sbin/ldconfig

4. now build:
   

   rpmbuild -ba protobuf-c.spec

And that's kinda it.  Can you see how simple a package-port this is?

Can you see how simple code-compatibility things we learned in first-year CompSci are being ignored here to our detriment?  This is so disappointing, and it wouldn't be a problem if I haven't had this very same conversation with someone in a position to fix it.

Upgrading Cobbler from 2.0 to 2.2 -- and overcoming WSGI woes

If you've used Cobbler for a while, you'll want to upgrade it.  Naturally!

There's a problem with the upgrade, though:  the 2.0 version seems to use mod_python, and the new one uses mod_wsgi.  No problem, right?  So you install mod_wsgi as part of the process:

# service httpd start
Starting httpd: Syntax error on line 10 of /etc/httpd/conf.d/cobbler.conf:
Invalid command 'WSGIScriptAliasMatch', perhaps misspelled or defined by a
module not included in the server configuration
                                                           [FAILED]

So your server won't start.  Yay!

What's really going on is

1. mod_python and mod_wsgi don't play well together
2. mod_wsgi is impotent on install and needs activation
3. mod_python is still the go-to for rendering the configs, which now use syntax it can't handle

The remedy is simple:

1. remove mod_python.  It can't be used, so let's get it out to avoid dep- and other issues.
   
   

   rpm -e mod_python

2. create a mod_wsgi config
   
   

   cat > /etc/httpd/conf.d/05-load-wsgi.conf
   LoadModule wsgi_module modules/mod_wsgi.so
   

3. restart httpd
   
   

   service httpd restart

And that's it:

Stopping httpd:                                            [FAILED]
Starting httpd:                                            [  OK  ]

And you're back up and running.

Is it disappointing that it doesn't Just Work?  For sure.  Could you figure it out if you were a mod_python user or an expert, and knew the hell WSGi was?  Maybe.  But I'm not, and I think that as an app user it's not really on me to be an expert.  You may argue how proficient one needs to be to use any device, but I'm thinking it's not ready for prime-time yet.  Boo!

Diffing Configs After an RPM install

Some daemon not starting?  Think you may have mucked with the config and you don't know how?  Let's talk about stashing your configs in an SCM, you lamer, or maybe doing a backup every month or so?  (ha, gotcha) For now, let's find out what you've done.

[root@gator ~]# mkdir /tmp/bz75819
[root@gator ~]# yumdownloader --destdir !$ `rpm -qf --qf "%{name}\n" /etc/httpd/conf/httpd.conf`
yumdownloader --destdir /tmp/bz75819 `rpm -qf --qf "%{name}\n" /etc/httpd/conf/httpd.conf`
[root@gator ~]# rpm2cpio /tmp/bz75819/*.rpm | cpio -tv | grep /etc/httpd/conf/httpd.conf
6573 blocks
-rw-r--r--   1 root     root        33726 Oct 20 14:05 ./etc/httpd/conf/httpd.conf
[root@gator ~]# rpm2cpio /tmp/bz75819/*.rpm | cpio -i --to-stdout */etc/httpd/conf/httpd.conf | diff -u - /etc/httpd/conf/httpd.conf
6573 blocks
--- -   2012-03-03 16:05:07.411845000 -0800
+++ /etc/httpd/conf/httpd.conf  2012-03-03 16:05:01.000000000 -0800
@@ -131,7 +131,7 @@
 # prevent Apache from glomming onto all bound IP addresses (0.0.0.0)
 #
 #Listen 12.34.56.78:80
-Listen 80
+#Listen 80

 #
 # Dynamic Shared Object (DSO) Support

So that's what you've done.  Yeah, it looks contrived, but I saw that this week.  So when I see it again this week, I know what to do.

Have you ever needed to find out just what you've changed in a config since a package has been installed?

Installing and Using MSSQL from PHP on Centos5/RHEL5

The boss wants to use an MSSQL (a Sybase derivative) to power a PHP page on Linux.  You're thinking to yourself that the boss has finally lost it as you nod and smile politely.

Thing is, it's possible. And it's actually not so hard.  Installing it is tricky;  it's like this:

1. yum install php-mssql

That's it.  It'll pull in both freetds (from EPEL) and unixodbc if it's not already installed.  How easy was that?  Okay, you may need to install the EPEL repository as well, but that's cake.

Now to use it.  Here's a sample.php:

sed 's:#:<:' <<-EOF >/tmp/sample.php
#?php
try {
  if (function_exists("mssql_connect")) {
    if (($m = mssql_connect("db.host.com:1433", "user", "passwd")) !== FALSE) {
      if (function_exists("mssql_query")) {
        $res = mssql_query("SELECT @@VERSION", $m);
        $row = mssql_fetch_array($res);
        print_r($row);
      } else
        throw new Exception("mssql_query function doesn't exist");
    } else
      throw new Exception("connection failed");
  } else
    throw new Exception("mssql_connect function doesn't exist");
  mssql_close($m);
} catch(Exception $e) {
  echo "#PRE>";
  print_r($e);
  echo "#/PRE>";
}
?>

EOF

And that should easily return a valid query and response.  And that's nice, because while the boss may well be crazy, his crazy grin looks better over free coffees.

ReCollecting the FileProvides

I use APT for RPMs, as it's the most versatile tool out there.  Some folks don't seem to understand that APT was ported for RPMs about a decade ago, and those using Yum for their RPM management are encouraged to do so, lest their horizons grow too broad.  The truly genius folks at Conectiva used APT to great benefit, and allowed massive flexibility;  upgrading in-place over a major release was a possible, easy and well-tested procedure, as I recall.  But this isn't for the Apt-RPM genius so recently localized at Conectiva and now lost to the rest of the world.

Lately there's been another change to the repository format and layout as used by YUM;  this is nothing new, as many open-source de-facto standards are developed and grow in the same kind of seeming vacuum, and the gaijin must adapt.  If you're suffering as I was, your apt-get update invocations will look like this:

E: Error occured while processing sec (CollectFileProvides)
E: Problem with MergeFileProvides /var/lib/apt/lists/archive_cobbler_repo%5fmirror_epel5-i386_repodata_277d21a2341fe766d0daff22b2846905517bcd71-primary.sqlite

I'm using Cobbler to mirror many, many repos locally; almost anywhere I have more than one machine.  The update to the createrepo invocation will either come from /etc/cobbler/settings or the createrepo flags in a particular repository item.

Here's my change:

--- /etc/cobbler/settings~      2011-09-23 07:03:38.000000000 -0700
+++ /etc/cobbler/settings       2011-09-26 14:24:32.000000000 -0700
@@ -58,7 +58,7 @@
 # enables working with Fedora repos from F11/F12 from EL-4 or
 # EL-5 without python-hashlib installed (which is not available
 # on EL-4)
-createrepo_flags: "-c cache -s sha -C --update"
+createrepo_flags: "-q -c cache -s sha --update -d"

 # if no kickstart is specified to profile add, use this template
 default_kickstart: /var/lib/cobbler/kickstarts/default.ks

You will need to adjust to suit, as the patch will not apply cleanly unless you've been mucking about there already, but take the patch and make yours match the result.  Restart cobbler when you're done and your repos will be updating madly as expected.

As always, sorry if the Blogspot editor butchers my blockquotes.

Fixing the Dependency Graphs in Bugzilla 3 EPEL RPM

Got the Epel bugzilla RPM?  I did, and it's pretty good, considering how challenging BZ is to package in a way that even begins to satisfy FHS and FSSTND.  Some collections of app-like data files are difficult to package correctly.

But the dependency graphs were pooched.  They seemed to want to send me off to http://host/var/lib/bugzilla/webdot/foo.png.  See why?  Yeah, but the problem is deeper than that, and it's not just a minor packaging bug:  bugzilla code actually uses a static CGI-bin dir to dictate the location of a dynamic spool dir containing the product of the dot calls -- the dependency graphs.  This is muy bad in terms of FHS.

But, after wading through a dozen PL files to get to where I could either mod the template, the code or the PM file containing the constants which dictate the locations and how one relates to the other ...

I just dumped in a cheap apache conf.d hack to provide a proper location for the png files without changing their real or interpreted location and without rendering in a cleaner fashion at all:

cat <<-EOF> /etc/httpd/conf.d/bugzilla-fix.conf
Alias /var/lib/bugzilla/data/webdot /var/lib/bugzilla/data/webdot
EOF

I mean, why mod the code if the next install will just hose the changes anyway (talk to me after to do the Dependency Graph Arrow Fix I've done for the last 8 years or so) and why change the templates, its generator or the PM file to suit?  Toss in a symlink so they just go to the right place while the code thinks it is generating the PNGs in the wrong place anyway?

The thing was bound to still be a bit of a mess either way, when I was done, and while I expect the package made a herculean effort to make the excellent package that it is, the source code uses so many wrong literal strings for everything that it's a big hairy ball of squidbait I didn't want to start to get into.  So I chased the code, did some soul-searching and optest for the ugliest but most reliable and maintainable fix I could.

Sorry.

Maintaining Repos in Kickstarted Machines After Install

After you've installed a machine, its install-time repository config in /etc/yum.repos.d is pretty much set.

Bah, I say! Bah! Just keep it updated.

Kickstart (cobbler):

#set yumconfcronfilename = "/etc/cron.daily/50-yum-config-stanza"
cat << EOECYCS > $yumconfcronfilename
#!/bin/sh
$yum_config_stanza

sed -ne '
        /^baseurl=/{
                s/baseurl=/repomd /
                s://:__:
                s:/: :
                s:__://:
                p
        }
        ' /etc/yum.repos.d/cobbler-config.repo \
          > /etc/apt/sources.list.d/cobbler-config.list
EOECYCS
chmod a+x $yumconfcronfilename

If you're not running cobbler, set it into place by hand:

cat << EOECYCS > /etc/cron.daily/50-yum-config-stanza
#!/bin/sh
wget "http://archive/cblr/svc/op/yum/profile/centos5-i386-minimal" --output-document=/etc/yum.repos.d/cobbler-config.repo

sed -ne '
 /^baseurl=/{
  s/baseurl=/repomd /
  s://:__:
  s:/: :
  s:__://:
  p
 }
 ' /etc/yum.repos.d/cobbler-config.repo \
   > /etc/apt/sources.list.d/cobbler-config.list
EOECYCS

chmod a+x /etc/cron.daily/50-yum-config-stanza

That's dereferenced for you. The actual profile's going to be way off, though, so don't use that one verbatim. Find your own:

awk -F/ '/^url/{print $NF}' anaconda-ks.cfg

As usual, watch carefully for the way in which the 'new', 'better' blogspot editor makes an artistic puree of the quoted stuff;  grain of salt, kids.

Installing VMware Tools in Linux after ESX 4.1

Tried to install VMware-Tools lately?  Not sure how it goes on Windows, but on Linux it's a bit more challenging than it used to be.  Used to be there was an RPM on the ISO that popped into the Linux machine's vCDROM drive when the option to install VMware Tools was chosen.

No more.  Like ESXi4.0 before it, ESX4.1 seems to ship with an ISO containing, for Linux, a little tarball with the stuff.

Those who've worked for a distro or packaged anything will know that shipping and installing bare tarballs is really not the best way to go.  Sure, it's fine for the cowboys who maintain 1-2 machines at the home or home-office, but for anyone who has to maintain more than one, has to roll back or verify a package level, or who just wants a clean system, a good package is the only way to go.

Note that I said good package.  If you hear anyone mention "dependency hell" or anything like that, they weren't paying attention.  Stop using bad packages!  Stop mixing repositories.  The pathetic blunt object that pass for packages is the topic of many blogwails past and many blogwails to come.

The great news?  VMware seem to have a few packages for us.  In fact, they come in 4 flavours of RedHat and some even for the you-bunt-you crowd.  Seems all you have to do is mark up the apt config ...

cat <<EOF> /etc/apt/sources.list.d/esx.list

repomd  http://packages.vmware.com tools/esx/4.1/rhel$(VERSION)/$(ARCH)
EOF
apt-get update

... or yum config if you don't build your own packages and know where apt excels ...

cat <<EOF> /etc/yum.repos.d/esx.repo

[esx]
name=VMware-tools for RHEL\$releasever - \$basearch

baseurl=http://packages.vmware.com/tools/esx/4.1/rhel\$releasever/\$basearch
enabled=1
gpgcheck=1
gpgkey=http://packages.vmware.com/tools/VMWARE-PACKAGING-GPG-KEY.pub

EOF

yum update

and after refreshing, just install the package:

apt-get install vmware-tools
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
   vmware-open-vm-tools (8.3.2-257589.el5)
   vmware-open-vm-tools-common (8.3.2-257589.el5)
   vmware-open-vm-tools-kmod (8.3.2-257589.el5)
   vmware-open-vm-tools-nox (8.3.2-257589.el5)
   vmware-open-vm-tools-xorg-drv-display (10.16.7.0-0.257589.el5)
   vmware-open-vm-tools-xorg-drv-mouse (12.6.4.0-0.257589.el5)
   vmware-open-vm-tools-xorg-utilities (8.3.2-257589.el5)
   vmware-tools-common (8.3.2-257589.el5)
   vmware-tools-nox (8.3.2-257589.el5)
The following packages will be REPLACED:
   xorg-x11-drv-vmmouse (12.4.0-2.1)
   (by (10.13.0-2.1)
   vmware-open-vm-tools-xorg-drv-mouse) ()
    (12.4.0-2.1)
   xorg-x11-drv-vmware (10.13.0-2.1)
   (by ()
   vmware-open-vm-tools-xorg-drv-display) (12.4.0-2.1)
The following NEW packages will be installed:
   vmware-open-vm-tools (8.3.2-257589.el5)
   vmware-open-vm-tools-common (8.3.2-257589.el5)
   vmware-open-vm-tools-kmod (8.3.2-257589.el5)
   vmware-open-vm-tools-nox (8.3.2-257589.el5)
   vmware-open-vm-tools-xorg-drv-display (10.16.7.0-0.257589.el5)
   vmware-open-vm-tools-xorg-drv-mouse (12.6.4.0-0.257589.el5)
   vmware-open-vm-tools-xorg-utilities (8.3.2-257589.el5)
   vmware-tools (8.3.2-257589.el5)
   vmware-tools-common (8.3.2-257589.el5)
   vmware-tools-nox (8.3.2-257589.el5)
0 upgraded, 10 newly installed, 2 replaced, 0 removed and 0 not upgraded.
Need to get 44.8kB/13.7MB of archives.
After unpacking 41.1MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-kmod 8.3.2-257589.el5 [524kB]
Get:2 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-common 8.3.2-257589.el5 [5283kB]
Get:3 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-nox 8.3.2-257589.el5 [2631B]
Get:4 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-xorg-drv-mouse 12.6.4.0-0.257589.el5 [17.5kB]
Get:5 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-xorg-drv-display 10.16.7.0-0.257589.el5 [33.4kB]
Get:6 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools-xorg-utilities 8.3.2-257589.el5 [7828kB]
Get:7 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-open-vm-tools 8.3.2-257589.el5 [2811B]
Get:8 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-tools-common 8.3.2-257589.el5 [39.4kB]
Get:9 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-tools-nox 8.3.2-257589.el5 [2635B]
Get:10 http://packages.vmware.com tools/esx/4.1/rhel5/x86_64/ vmware-tools 8.3.2-257589.el5 [2763B]
Fetched 44.8kB in 0s (46.5kB/s)
Committing changes...
Preparing                                ############################## [100%]
Updating / installing
  vmware-open-vm-tools-xorg-drv-display- ############################## [100%]
  vmware-open-vm-tools-xorg-drv-mouse-12 ############################## [100%]
  vmware-open-vm-tools-kmod-8.3.2-257589 ############################## [100%]
  vmware-open-vm-tools-common-8.3.2-2575 ############################## [100%]
  vmware-open-vm-tools-nox-8.3.2-257589. ############################## [100%]
  vmware-open-vm-tools-xorg-utilities-8. ############################## [100%]
  vmware-open-vm-tools-8.3.2-257589.el5. ############################## [100%]
  vmware-tools-common-8.3.2-257589.el5.x ############################## [100%]
  vmware-tools-nox-8.3.2-257589.el5.x86_ ############################## [100%]
  vmware-tools-8.3.2-257589.el5.x86_64   ############################## [100%]
Cleaning up / removing
  xorg-x11-drv-vmmouse-12.4.0-2.1.x86_64 ############################## [100%]
  xorg-x11-drv-vmware-10.13.0-2.1.x86_64 ############################## [100%]
Done.

While I avoid Yum, it should look like this:

[root@0-50-56-bc-0-7 ~]# yum install vmware-tools
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * addons: mirror.raystedman.net
 * base: mirror.skiplink.com
 * extras: mirror.skiplink.com
 * updates: styx.biochem.wfubmc.edu
Setting up Install Process
Resolving Dependencies
:
[blah blah]
:
Dependencies Resolved

==================================================================================================================================
 Package                                           Arch               Version                               Repository       Size
==================================================================================================================================
Installing:
 vmware-open-vm-tools-xorg-drv-display             x86_64             10.16.7.0-0.257589.el5                esx              33 k
     replacing  xorg-x11-drv-vmware.x86_64 10.13.0-2.1

 vmware-open-vm-tools-xorg-drv-mouse               x86_64             12.6.4.0-0.257589.el5                 esx              17 k
     replacing  xorg-x11-drv-vmmouse.x86_64 12.4.0-2.1

 vmware-tools                                      x86_64             8.3.2-257589.el5                      esx             2.7 k
Installing for dependencies:
 vmware-open-vm-tools                              x86_64             8.3.2-257589.el5                      esx             2.7 k
 vmware-open-vm-tools-common                       x86_64             8.3.2-257589.el5                      esx             5.0 M
 vmware-open-vm-tools-kmod                         x86_64             8.3.2-257589.el5                      esx             512 k
 vmware-open-vm-tools-nox                          x86_64             8.3.2-257589.el5                      esx             2.6 k
 vmware-open-vm-tools-xorg-utilities               x86_64             8.3.2-257589.el5                      esx             7.5 M
 vmware-tools-common                               x86_64             8.3.2-257589.el5                      esx              39 k
 vmware-tools-nox                                  x86_64             8.3.2-257589.el5                      esx             2.6 k

Transaction Summary
==================================================================================================================================
Install      10 Package(s)
Upgrade       0 Package(s)

Total download size: 13 M
Is this ok [y/N]: y
Downloading Packages:
:
[ more blah blah ]
:
Running Transaction
  Installing     : vmware-open-vm-tools-xorg-drv-display                                                                     1/12
  Installing     : vmware-open-vm-tools-kmod                                                                                 2/12
  Installing     : vmware-open-vm-tools-common                                                                               3/12
  Installing     : vmware-open-vm-tools-nox                                                                                  4/12
  Installing     : vmware-tools-common                                                                                       5/12
  Installing     : vmware-tools-nox                                                                                          6/12
  Installing     : vmware-open-vm-tools-xorg-drv-mouse                                                                       7/12
  Installing     : vmware-open-vm-tools-xorg-utilities                                                                       8/12
  Installing     : vmware-open-vm-tools                                                                                      9/12
  Installing     : vmware-tools                                                                                             10/12
  Erasing        : xorg-x11-drv-vmmouse                                                                                     11/12
  Erasing        : xorg-x11-drv-vmware                                                                                      12/12

Installed:
  vmware-open-vm-tools-xorg-drv-display.x86_64 0:10.16.7.0-0.257589.el5
  vmware-open-vm-tools-xorg-drv-mouse.x86_64 0:12.6.4.0-0.257589.el5
  vmware-tools.x86_64 0:8.3.2-257589.el5

Dependency Installed:
  vmware-open-vm-tools.x86_64 0:8.3.2-257589.el5                      vmware-open-vm-tools-common.x86_64 0:8.3.2-257589.el5
  vmware-open-vm-tools-kmod.x86_64 0:8.3.2-257589.el5                 vmware-open-vm-tools-nox.x86_64 0:8.3.2-257589.el5
  vmware-open-vm-tools-xorg-utilities.x86_64 0:8.3.2-257589.el5       vmware-tools-common.x86_64 0:8.3.2-257589.el5
  vmware-tools-nox.x86_64 0:8.3.2-257589.el5

Replaced:
  xorg-x11-drv-vmmouse.x86_64 0:12.4.0-2.1                        xorg-x11-drv-vmware.x86_64 0:10.13.0-2.1

Complete!

 The experience will be very similar -- if a bit more verbose.


That, my friends, may be it.  It seems convoluted but, really, once you ship a metapackage out with a proper apt+yum config, and your Linux VMs upgrade themselves overnight (well, maybe the test ones get the new metaRPMs first) then you'll be so much more pleased at the ability to do all that automatically.  Add in Spacewalk and/or cobbler and stir for best results.