Red Hat Network Satellite tips
I manage a Red Hat Network (RHN) Satellite Server in our lab for our Red Had Enterprise Linux (RHEL) hosts, and throughout the years I’ve gathered a few tips and hints that have made my life easier. Some of these tips can probably be found in the official Red Hat documentation already, others are not as obvious as you would think. Feel free to add your own; and as always, comments and feedback are welcome.
Remote commands
Problem:
Remote commands are default executed from /tmp
In some organisations is filesystem /tmp mounted with the “noexec” option. Therefor scripts can’t be executed remotely.
This is due to the environment variable TMPDIR
Solution:
change the environment variable TMPDIR to a directory where the filesystem is mounted
at least exec options ( default options is good enough )
Removing stale system entries
Problem: The RHN satellite server has a limited number of server entitlements. Every time a server is registered it takes up one entitlement. When a server is re-imaged or re-purposed, its entry in the RHN database (wasting one entitlement) until it is manually removed using the RHN web interface, which can quickly become a maintenance nightmare.
Solution: The script delete-old-systems-interactive (which is part of the ‘spacewalk-utils’ package) can be used to automatically remove stale server entries from the RHN database. I run this daily in my lab as a cron job.
#!/bin/bash
#
# Removes stale entries from RHN database after x days of server not checking in
#
# Admin password can be provided in --password argument in cleartext (not a good
# idea) or saved in /etc/rhn/$username-password in cleartext (also not a good
# idea but at least file can be protected with appropriate access permissions).
#
# Without the --force argument, it only tells you what servers would have been
# removed but does not actually do it.
LOG=last-systems-removed.log
days=21
/usr/bin/delete-old-systems-interactive --idle=${days}d \
--username=$username --force > $LOG
# email log - optional
mail <email> -s "Systems removed from RHN (if any)" < $LOG
Managing VM entitlements
If you manage a lot of RHEL VMs with RHN, you will be better off purchasing “Virtualization Platform” entitlements from Red Hat, which allow you to entitle an unlimited amount of VMs running on the same physical host, instead of each one consuming regular entitlements. Once you get “Virtualization Platform” entitlements, do the following:
1. Register the hypervisor (running KVM or Xen) to RHN and add the “Virtualization Platform” entitlement to it. You should then see a “Virtualization” tab when you click on the system.
2. Be sure you set your hypervisor for configuration management. Add the child channel “RHN Tools for RHEL” and install the following RPMs, which will be needed to allow remote actions on it.
# yum –y install osad rhncfg rhncfg-actions rhncfg-client rhncfg-managemen
3. Allow remote actions:
# rhn-actions-control --enable-all # rhn-actions-control --report deploy is enabled diff is enabled upload is enabled mtime_upload is enabled run is enabled
4. Start osad service so that remote actions are executed quickly rather than waiting for the systems to check in:
# service osad start # chkconfig osad on
You can now register your VMs to RHN, and even though the VMs will be listed in the Systems tab, they will not consume a regular entitlement. You can compare the number of systems registered with the number of entitlements consumed in Admin —> Organization —> Subscriptions.
Managing rhn and yum caches
Once day I noticed that the directory /var was unusually large, over 5 GB. I looked at the obvious choice first (/var/log), but my log files were ok. I then realized that I had two very large directories:
/var/cache/rhn 2.1 GB
/var/cache/yum 3.1 GB
/var/cache/rhn contains cached metadata used for the channel synchronization process between your RHN server and rhn.redhat.com. This cache can be refreshed at any time (which will also reduce its size) but will also slow down the synchronization process. It’s probably best to leave this as-is, the default refresh level provided by Red Hat is more than adequate.
/var/cache/yum contains cached metadata and files downloaded via yum, and is common to all systems that use yum to manage RPMs (not just RHN servers). Caching information here can speed up operations like “yum list”, but it can also take up significant amounts of space. You can control whether you want caching enabled or not in /etc/yum.conf. In my case, I had caching enabled (which I have now turned off) and had also recently upgraded a ton of RPMs, which was the reason why my cache was so large. I proceeded to wipe out the cache with “yum clean all” and now the directory is down to a more manageable 51 MB.
Oracle database tables’ size
Below is a list of the database tables in my RHN server:
# su -l oracle -c 'db-control report' Tablespace Size Used Avail Use% DATA_TBS 12.9G 11.5G 1.3G 89% SYSAUX 500M 199.3M 300.6M 40% SYSTEM 400M 255.9M 144M 64% TEMP_TBS 1.2G 0B 1.2G 0% UNDO_TBS 1.4G 662.5M 861.4M 43% USERS 128M 64K 127.9M 0%
The DATA_TBS table holds all system and package information, and can grow in size pretty quickly as you add channels and systems to your RHN server. When this table fills up to 100%, the RHN server becomes inoperable.
To increase the size of the DATA_TBS table, use the following:
# su -l oracle -c ‘db-control extend DATA_TBS’
This command will extend the DATA_TBS table by 512MB. Run it again if you want to extend it by another 512 MB.
However, there is an alternate method to increase the size of your tables that gives you better control over increasing the database tables. The first thing to know is that the files that hold these tables are located in the directory /rhnsat/data/rhnsat, so let’s take a look:
# ls -l /rhnsat/data/rhnsat -rw-r--r-- 1 oracle oracle 10141696 Nov 18 13:56 control_01.ctl -rw-r--r-- 1 oracle oracle 10141696 Nov 18 13:56 control_02.ctl -rw-r--r-- 1 oracle oracle 524296192 Nov 18 11:25 data_01.dbf -rw-r--r-- 1 oracle oracle 524296192 Nov 18 13:55 data_02.dbf -rw-r--r-- 1 oracle oracle 524296192 Nov 18 13:56 data_03.dbf -rw-r--r-- 1 oracle oracle 524296192 Nov 18 11:07 data_04.dbf -rw-r--r-- 1 oracle oracle 524296192 Nov 18 11:28 data_05.dbf ..... -rw-r--r-- 1 oracle oracle 536879104 Nov 18 11:28 data_23.dbf -rw-r--r-- 1 oracle oracle 536879104 Nov 18 11:28 data_24.dbf -rw-r--r-- 1 oracle oracle 536879104 Nov 18 04:47 data_25.dbf -rw-r--r-- 1 oracle oracle 536879104 Nov 18 04:47 data_26.dbf -rw-r--r-- 1 oracle oracle 104858112 Nov 18 13:56 redo_1001.log -rw-r--r-- 1 oracle oracle 104858112 Nov 18 13:56 redo_1002.log -rw-r--r-- 1 oracle oracle 104858112 Nov 17 16:30 redo_1101.log -rw-r--r-- 1 oracle oracle 104858112 Nov 17 16:30 redo_1102.log -rw-r--r-- 1 oracle oracle 524296192 Nov 18 13:05 sysaux.dbf -rw-r--r-- 1 oracle oracle 419438592 Nov 18 13:56 system.dbf -rw-r--r-- 1 oracle oracle 262152192 Nov 17 22:00 temp_01.dbf -rw-r--r-- 1 oracle oracle 536879104 Nov 14 22:01 temp_02.dbf -rw-r--r-- 1 oracle oracle 524296192 Nov 18 13:56 undo_01.dbf -rw-r--r-- 1 oracle oracle 536879104 Nov 18 13:53 undo_02.dbf -rw-r--r-- 1 oracle oracle 134225920 Nov 18 04:47 users_01.dbf
So let’s say that instead of increasing the DATA_TBS table by 512 MB, we’d like to increase it my 1024 MB. As you can see above (highlighted in red above) the last file used for the DATA_TBS table is data_26.dbf, so for consistency purposes, I will use data_27.dbf for the next file:
# su - oracle # ORACLE_SID=rhnsat sqlplus /nolog SQL> CONNECT / AS SYSDBA SQL> ALTER TABLESPACE data_tbs ADD DATAFILE 2 '/rhnsat/data/rhnsat/data_27.dbf' SIZE 1024M; SQL> quit;The size of the DATA_TBS table is now:
# db-control report Tablespace Size Used Avail Use% DATA_TBS 13.9G 11.5G 2.3G 82% SYSAUX 500M 199.3M 300.6M 40% SYSTEM 400M 255.9M 144M 64% TEMP_TBS 1.2G 0B 1.2G 0% UNDO_TBS 1.4G 662.5M 861.4M 43% USERS 128M 64K 127.9M 0%