System daemons (xinetd, inetd, rsyslogd and cron)

Certify and Increase Opportunity.
Be
Govt. Certified Linux Administrator

Back to Tutorial

A daemon is a type of program on Unix-like operating systems that runs unobtrusively in the background, rather than under the direct control of a user, waiting to be activated by the occurance of a specific event or condition.

Unix-like systems typically run numerous daemons, mainly to accommodate requests for services from other computers on a network, but also to respond to other programs and to hardware activity. Examples of actions or conditions that can trigger daemons into activity are a specific time or date, passage of a specified time interval, a file landing in a particular directory, receipt of an e-mail or a Web request made through a particular communication line. It is not necessary that the perpetrator of the action or condition be aware that a daemon is listening, although programs frequently will perform an action only because they are aware that they will implicitly arouse a daemon.

Daemons are usually instantiated as processes. A process is an executing (i.e., running) instance of a program. Processes are managed by the kernel (i.e., the core of the operating system), which assigns each a unique process identification number (PID).

There are three basic types of processes in Linux: interactive, batch and daemon. Interactive processes are run interactively by a user at the command line (i.e., all-text mode). Batch processes are submitted from a queue of processes and are not associated with the command line; they are well suited for performing recurring tasks when system usage is otherwise low.

Daemons are recognized by the system as any processes whose parent process has a PID of one, which always represents the process init. init is always the first process that is started when a Linux computer is booted up (i.e., started), and it remains on the system until the computer is turned off. init adopts any process whose parent process dies (i.e., terminates) without waiting for the child process’s status. Thus, the common method for launching a daemon involves forking (i.e., dividing) once or twice, and making the parent (and grandparent) processes die while the child (or grandchild) process begins performing its normal function.

Some daemons are launched via System V init scripts, which are scripts (i.e., short programs) that are run automatically when the system is booting up. They may either survive for the duration of the session or be regenerated at intervals.

Many daemons are now started only as required and by a single daemon, xinetd (which has replaced inetd in newer systems), rather than running continuously. xinetd, which is referred to as a TCP/IP super server, itself is started at boot time, and it listens to the ports assigned to the processes listed in the /etc/inetd.conf or in /etc/xinetd.conf configuration file. Examples of daemons that it starts include crond (which runs scheduled tasks), ftpd (file transfer), lpd (laser printing), rlogind (remote login), rshd (remote command execution) and telnetd (telnet).

The daemons referenced in /etc/init.d are configured to be run as Linux services. Services are programs that are started and stopped through the init scripts in the /etc/init.d directory. Many of these services are launched when the system is booted. The /sbin/service utility provides a consistent interface to executing the init scripts. The init scripts provide a consistent interface to managing a service by providing options that start, stop, restart, query status, and perform other actions on services.

inetd

inetd (internet service daemon) is a super-server daemon on many Unix systems that provides Internet services. For each configured service, it listens for requests from connecting clients. Requests are served by spawning a process which runs the appropriate executable, but simple services such as echo are served by inetd itself. External executables, which are run on request, can be single- or multi-threaded. First appearing in 4.3BSD, it is generally located at /usr/sbin/inetd.

Often called a super-server, inetd listens on designated ports used by Internet services such as FTP, POP3, and telnet. When a TCP packet or UDP packet arrives with a particular destination port number, inetd launches the appropriate server program to handle the connection. For services that are not expected to run with high loads, this method uses memory more efficiently, since the specific servers run only when needed. Furthermore, no network code is required in the service-specific programs, as inetd hooks the sockets directly to stdin, stdout and stderr of the spawned process. For protocols that have frequent traffic, such as HTTP and POP3, a dedicated server that intercepts the traffic directly may be preferable.

The /usr/sbin/inetd daemon provides Internet service management for a network. This daemon reduces system load by invoking other daemons only when they are needed and by providing several simple Internet services internally without invoking other daemons.

The inetd daemon starts by default each time you start your system. When the daemon starts, it reads its configuration information from the file specified in the ConfigurationFile parameter. If the parameter is not specified, the inetd daemon reads its configuration information from the /etc/inetd.conf file.

Once started, the inetd daemon listens for connections on certain Internet sockets in the /etc/inetd.conf. The /etc/inetd.conf file describes to the inetd daemon how Internet service requests on Internet sockets should be handled. When the inetd daemon receives a request on one of these sockets, it determines which service corresponds to that socket and then either handles the service request itself or invokes the appropriate server.

The ftpd, rlogind, rexecd, rshd, talkd, telnetd, and uucpd daemons are started by default by inetd. The tftpd, fingerd, and comsat daemons are not started by default unless they are uncommented in the /etc/inetd.conf file. The /etc/inetd.conf file can be updated by using the System Management Interface Tool (SMIT), the System Resource Controller (SRC), or by editing the /etc/inetd.conf.

The list of services that will be serviced is given in a configuration file, usually /etc/inetd.conf. A GUI for managing the configuration file is an optional accessory. The daemon may need a signal in order to re-read its configuration. For an example, telnet can be configured as follows (line taken from a machine running AIX version 5.1):

telnet stream tcp6   nowait root   /usr/sbin/telnetd     telnetd -a

The first word, telnet, is the official name of the service. It is resolved using the system database to map port numbers and protocols to service names. In this case, /etc/services should contain:

telnet         23/tcp

The second and third words describe the type of socket and underlying protocol respectively. The /etc/protocols database is consulted. The fourth word is the wait/nowait switch. A single-threaded server expects inetd to wait until it finishes reading all the data. Otherwise inetd lets the server run and spawns new, concurrent processes for new requests. The fifth word is the user name, from the /etc/passwd database, that the service program should run as.

Finally, the path and the arguments of an external program are given. As usual, the first argument is the program name. In the example, inetd is told to launch the program /usr/sbin/telnetd with the command line arguments telnetd -a. inetd automatically hooks the socket to stdin, stdout, and stderr of the server program.

Generally TCP sockets are handled by spawning a separate server to handle each connection concurrently. UDP sockets are generally handled by a single server instance that handles all packets on that port.

Some simple services, such as echo, are handled directly by inetd, without spawning an external server. In recent years, because of the security limitations in the original design of inetd, it has been replaced by xinetd, rlinetd, ucspi-tcp, and others in many systems. Distributions of Linux especially have many options and Mac OS X (beginning with Mac OS X v10.2) uses xinetd. As of version Mac OS X v10.4, Apple has merged the functionality of inetd into launchd.

The services provided by inetd can be omitted entirely. This is becoming more common where machines are dedicated to a single function. For example, an HTTP server could be configured to just run httpd and have no other ports open. A dedicated firewall could have no services started.

xinetd

It is an open-source super-server daemon which runs on many Unix-like systems and manages Internet-based connectivity. It offers a more secure alternative to the older inetd (“the Internet daemon”), so most modern Linux distributions use it instead of that.

The xinetd daemon conserves system resources, provides access control and logging, and can be used to start special-purpose servers. xinetd can also be used to grant or deny access to particular hosts, provide service access at specific times, limit the rate of incoming connections, limit the load created by connections, and more.

xinetd runs constantly and listens on all ports for the services it manages. When a connection request arrives for one of its managed services, xinetd starts up the appropriate server for that service.

The configuration file for xinetd is /etc/xinetd.conf, but the file only contains a few defaults and an instruction to include the /etc/xinetd.d directory. To enable or disable an xinetd service, edit its configuration file in the /etc/xinetd.d directory. If the disable attribute is set to yes, the service is disabled. If the disable attribute is set to no, the service is enabled. You can edit any of the xinetd configuration files or change its enabled status using the Services Configuration Tool, ntsysv, or chkconfig. For a list of network services controlled by xinetd, review the contents of the /etc/xinetd.d directory with the command ls /etc/xinetd.d.

Configuration of xinetd resides in the default configuration file /etc/xinetd.conf and configuration of the services it supports reside in configuration files stored in the /etc/xinetd.d directory. The configuration for each service usually includes a switch to control whether xinetd should enable or disable the service.

An example configuration file for the RFC 868 time server:

# default: off

# description: An RFC 868 time server. This protocol provides a

# site-independent, machine readable date and time. The Time service sends back

# to the originating source the time in seconds since midnight on January first

# 1900.

# This is the tcp version.

service time

{

disable         = yes

type           = INTERNAL

id             = time-stream

socket_type     = stream

protocol       = tcp

user           = root

wait           = no

}

# This is the udp version.

service time

{

disable         = yes

type          = INTERNAL

id             = time-dgram

socket_type     = dgram

protocol       = udp

user           = root

wait           = yes

}

The lines with the “#” character at the beginning are comments without any effect on the service. There are two service versions the first one is based on the Transmission Control Protocol (TCP), the second one is based on the User Datagram Protocol (UDP). The type and planned usage of a service determines the necessary core protocol. In a simple way, the UDP can not handle huge data transmissions, because it lacks the abilities to rearrange packages in a specified order or guarantee their integrity, but it is faster than TCP. TCP has these functions, but it is slower. There are two column in each versions inside the Braces. The first is the type of option, the second is the applied variable.

The disable option is a switch to run a service or not. In most cases the default state is yes. To activate the service change it to no. There are three types of services. The type is INTERNAL if the service is provided by xinetd, RPC when it based on Remote procedure call, they are commonly listed in the /etc/rpc file, or it can be UNLISTED when the service is neither in the /etc/services nor in the /etc/rpc files. The id is the unique identifier of the service. The socket_type determines the way of data transmission through the service. There are three types: stream, dgram and raw. This last one is useful, when we want to establish a service based on a non-standard protocol.

With the user option it is possible to choose a user to be the owner of the running service. It is highly recommended to choose a non-root user for security reasons. When the wait is on yes the xinetd will not receive request for the service if it has a connection. So the number of connections is limited to one. It provides very good protection when we want to establish only one connection per time.

There are many more options available for xinetd. In most Linux distributions the full list of possible options and their description is accessible with a “man xinetd.conf” command. To apply the new configuration a SIGHUP signal must be sent to the xinetd process to make it re-read the configuration files. This can be achieved with the following command: kill -SIGHUP “PID”. PID is the actual process identifier number of the xinetd, which can be obtained with the command pgrep xinetd.

rsyslogd

Rsyslogd is a system utility providing support for message logging. Support of both internet and unix domain sockets enables this utility to support both local and remote logging.

The /etc/rsyslog.conf file contains the configuration information that rsyslogd needs to run. The default configuration file that ships with most systems is sufficient for most standard needs. But you may find that you have to tweak the file a little if you want to do any additional fancy things with your logs—like sending local log messages to remote logging machines that can accept them, or logging to a database, or reformatting logs, etc.

rsyslog is an “advanced” version of sysklogd where the config file remains the same (you can copy a syslog.conf file directly into rsyslog.conf and it works) ; but you have a lot of new cool stuff coming with it :

• You can listen to TCP/UDP/… connections, with restrictions (ports, Source IPs)
• You can load a lot of modules
• You can discriminate the log filtering by program, source, message, pid etc. (for instance, each message tagged with the message “connexion closed” to the file closed.log)
• You can discard message after one or more rules Visit http://www.rsyslog.com which is very good indeed

Syslog-ng is “Next-Gen”. I think it’s the best way to manage logs : everything is object (source, destination, filter, and the very forwarding rule) and the syntax is clear. I doubt in terms of functionality that rsyslog and syslog-ng are different.

Rsyslog is a rocket-fast system for log processing. It offers high-performance, great security features and a modular design. While it started as a regular syslogd, rsyslog has evolved into a kind of swiss army knife of logging, being able to

• accept inputs from a wide variety of sources,
• transform them,
• and output the results to diverse destinations.

Rsyslog has a strong enterprise focus but also scales down to small systems. It supports, among others, MySQL, PostgreSQL, failover log destinations, ElasticSearch, syslog/tcp transport, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part.

Messages enter rsyslog with the help of input modules. Then, they are passed to ruleset, where rules are conditionally applied. When a rule matches, the message is transferred to an action, which then does something to the message, e.g. writes it to a file, database or forwards it to a remote host.

Upon startup, rsyslog reads its configuration from the rsyslog.conf file by default. This file may contain references to include other config files.

A different “root” configuration file can be specified via the -f <file> rsyslogd command line option. This is usually done within some init script or similar facility.

Statement Types – Rsyslog supports three different types of configuration statements concurrently:

• sysklogd – this is the plain old format, taught everywhere and still pretty useful for simple use cases. Note that some very few constructs are no longer supported because they are incompatible with newer features. These are mentioned in the compatibility docs.
• legacy rsyslog – these are statements that begin with a dollar sign. They set some configuration parameters and modify e.g. the way actions operate. This is the only format supported in pre-v6 versions of rsyslog. It is still fully supported in v6 and above. Note that some plugins and features may still only be available through legacy format (because plugins need to be explicitly upgraded to use the new style format, and this hasn’t happened to all plugins).
• RainerScript – the new style format. This is the best and most precise format to be used for more complex cases. The rest of this page assumes RainerScript based rsyslog.conf.

The rsyslog.conf files consists of statements. For old style (sysklogd & legacy rsyslog), lines do matter. For new style (RainerScript) line spacing is irrelevant. Most importantly, this means with new style actions and all other objects can split across lines as users want to.

In general it is recommended to use RainerScript type statements, as these provide clean and easy to read control-of-flow as well as no doubt about which parameters are active. They also have no side-effects with include files, which can be a major obstacle with legacy rsyslog statements.

For very simple things sysklogd statement types are still suggested, especially if the full config consists of such simple things. The classical sample is writing to files (or forwarding) via priority. In sysklogd, this looks like: mail.info /var/log/mail.log or mail.err @server.example.net

This is hard to beat in simplicity, still being taught in courses and a lot of people know this syntax. It is perfectly fine to use these constructs even in newly written config files. As a rule of thumb, RainerScript config statements should be used when

• configuration parameters are required (e.g. the Action… type of legacy statements)
• more elaborate control-of-flow is required (e.g. when multiple actions must be nested under the same condition)

It is usually not recommended to use rsyslog legacy config format (those directives starting with a dollar sign). However, a few settings and modules have not yet been converted to RainerScript. In those cases, the legacy syntax must be used.

There are two types of comments

• #-Comments – start with a hash sign (#) and run to the end of the line
• C-style Comments – start with /* and end with */, just like in the C programming language. They can be used to comment out multiple lines at once. Comment nesting is not supported, but #-Comments can be contained inside a C-style comment.

Directives are processed from the top of rsyslog.conf to the bottom. Order matters. For example, if you stop processing of a message, obviously all statements after the stop statement are never evaluated. Flow control is provided by control structures and filter conditions.

Data manipulation is achieved by set, unset and reset statements. Every input requires an input module to be loaded and a listener defined for it. Full details can be found inside the rsyslog modules documentation. Once loaded, inputs are defined via the input() object.

Outputs are also called “actions”. A small set of actions is pre-loaded (like the output file writer, which is used in almost every rsyslog.conf), others must be loaded just like inputs. An action is invoked via the action(type=”type” …) object. Type is mandatory and must contain the name of the plugin to be called (e.g. “omfile” or “ommongodb”). Other parameters may be present. Their type and use depends on the output plugin in question.

Rulesets and rules form the basis of rsyslog processing. In short, a rule is a way how rsyslog shall process a specific message. Usually, there is a type of filter (if-statement) in front of the rule. Complex nesting of rules is possible, much like in a programming language.

Rulesets are containers for rules. A single ruleset can contain many rules. In the programming language analogy, one may think of a ruleset like being a program. A ruleset can be “bound” (assigned) to a specific input. In the analogy, this means that when a message comes in via that input, the “program” (ruleset) bound to it will be executed (but not any other!).

Options of rsyslogd

• -D     Runs the Bison config parser in debug mode. This may help when hard to find syntax errors are reported. Please note that the output generated is deeply technical and orignally targeted towards developers.
• -d     Turns on debug mode.
• -f config file Specify an alternative configuration file instead of /etc/rsyslog.conf, which is the default.
• -i pid file Specify an alternative pid file instead of the default one. This option must be used if multiple instances of rsyslogd should run on a single machine.
• -n     Avoid auto-backgrounding. This is needed especially if the rsyslogd is started and controlled by init(8).
• -N level Do a coNfig check. Do NOT run in regular mode, just check configuration file correctness. This option is meant to verify a config file. To do so, run rsyslogd interactively in foreground, specifying -f <config-file> and -N level. The level argument modifies behaviour. Currently, 0 is the same as not specifying the -N option at all (so this makes limited sense) and 1 actually activates the code. Later, higher levels will mean more verbosity (this is a forward-compatibility option). rsyslogd is started and controlled by init(8).
• -C     This prevents rsyslogd from changing to the root directory.
• This is almost never a good idea in production use. This option was introduced in support of the internal testbed.
• -v     Print version and exit.

Each message has a facility and a priority. The facility tells you from which subsystem the message originated, and the priority tells you how important the message is. These two values are separated by a period. Both values have string equivalents, making them easier to remember. The combination of the facility and priority makes up the “selector” part of a rule in the configuration file.

cron

Cron is a system daemon used to execute desired tasks (in the background) at designated times. A crontab is a simple text file with a list of commands meant to be run at specified times. It is edited with a command-line utility. These commands (and their run times) are then controlled by the cron daemon, which executes them in the system background. Each user has a crontab file which specifies the actions and times at which they should be executed, these jobs will run regardless of whether the user is actually logged into the system. There is also a root crontab for tasks requiring administrative privileges. This system crontab allows scheduling of systemwide tasks (such as log rotations and system database updates).

Crontab section

Each of the sections is separated by a space, with the final section having one or more spaces in it. No spaces are allowed within Sections 1-5, only between them. Sections 1-5 are used to indicate when and how often you want the task to be executed. This is how a cron job is laid out:

minute (0-59), hour (0-23, 0 = midnight), day (1-31), month (1-12), weekday (0-6, 0 = Sunday), command

01 04 1 1 1 /usr/bin/somedirectory/somecommand

The above example will run /usr/bin/somedirectory/somecommand at 4:01am on January 1st plus every Monday in January. An asterisk (*) can be used so that every instance (every hour, every weekday, every month, etc.) of a time period is used. Code:

01 04 * * * /usr/bin/somedirectory/somecommand

The above example will run /usr/bin/somedirectory/somecommand at 4:01am on every day of every month.

Comma-separated values can be used to run more than one instance of a particular command within a time period. Dash-separated values can be used to run a command continuously. Code:

01,31 04,05 1-15 1,6 * /usr/bin/somedirectory/somecommand

The above example will run /usr/bin/somedirectory/somecommand at 01 and 31 past the hours of 4:00am and 5:00am on the 1st through the 15th of every January and June.

The “/usr/bin/somedirectory/somecommand” text in the above examples indicates the task which will be run at the specified times. It is recommended that you use the full path to the desired commands as shown in the above examples. Enter which somecommand in the terminal to find the full path to somecommand. The crontab will begin running as soon as it is properly edited and saved.

You may want to run a script some number of times per time unit. For example if you want to run it every 10 minutes use the following crontab entry (runs on minutes divisible by 10: 0, 10, 20, 30, etc.)

*/10 * * * * /usr/bin/somedirectory/somecommand

which is also equivalent to the more cumbersome

0,10,20,30,40,50 * * * * /usr/bin/somedirectory/somecommand

• The -l option causes the current crontab to be displayed on standard output.
• The -r option causes the current crontab to be removed.
• The -e option is used to edit the current crontab using the editor specified by the EDITOR environment variable.

After you exit from the editor, the modified crontab will be checked for accuracy and, if there are no errors, installed automatically. The file is stored in /var/spool/cron/crontabs but should only be edited via the crontab command.

Crontab commands are generally stored in the crontab file belonging to your user account (and executed with your user’s level of permissions). If you want to regularly run a command requiring administrative permissions, edit the root crontab file:

sudo crontab -e

Depending on the commands being run, you may need to expand the root users PATH variable by putting the following line at the top of their crontab file:

PATH=/usr/sbin:/usr/bin:/sbin:/bin

It is sensible to test that your cron jobs work as intended. One method for doing this is to set up the job to run a couple of minutes in the future and then check the results before finalising the timing. You may also find it useful to put the commands into script files that log their success or failure, for example:

echo “Nightly Backup Successful: $(date)” >> /tmp/mybackup.log