Book Excerpt

Book Excerpt: DevOps Troubleshooting: Linux Server Best Practices

Kyle Rankin — Wed, 13 Mar 2013 14:00:00 +0000

This excerpt is from the book, 'DevOps Troubleshooting: Linux Server Best Practices' by Kyle Rankin, published by Pearson/Addison-Wesley Professional, ISBN 0321832043, Nov 2012, Copyright 2013 Pearson Education, Inc. For more info please visit: http://www.informit.com/store/devops-troubleshooting-linux-server-best-practices-9780321832047

Chapter 8: Is the Website Down? Tracking Down Web Server Problems

Get Web Server Statistics

Although there’s a fair amount of web server troubleshooting you can perform outside of the server itself, ultimately you will get in a situation where you want to know this type of information: How many web server processes are currently serving requests? How many web server processes are idle? What are the busy processes doing right now? To pull data like this, you can enable a special server status page that gives you all sorts of useful server statistics.

Both Apache and Nginx provide a server status page. In the case of Apache, it requires that you enable a built-in module named status. How modules are enabled varies depending on your distribution; for example, on an Ubuntu server, you would type a2enmod status. On other distributions you may need to browse through the Apache configuration files and look for a commented-out section that loads the status module; it may look something like this:

LoadModule status_module /usr/lib/apache2/modules/mod_status.so

After the module is loaded on Ubuntu systems, the server-status page is already configured for use by localhost. On other systems you may need to add configuration like the following to your Apache configuration:

ExtendedStatus On

#
# Allow server status reports generated by mod_status,
# with the URL of http://servername/server-status
# Uncomment and change the ".example.com" to allow
# access from other hosts.
#

    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from localhost ip6-localhost
#    Allow from .example.com

Note that in this configuration example, we have really locked down who can access the page by saying deny from all hosts and only allow from localhost. This is a safe default because you generally don’t want the world to be able to view this kind of debugging information. As you can see in the commented-out example, you can add additional Allow from statements to add IPs or hostnames that are allowed to view the page.

For Nginx, you would add a configuration like the following to your existing Nginx config. In this example, Nginx will only listen on localhost, but you could change this to allow other machines on your local network:

Go to Full Article

Book Excerpt: A Practical Guide to Fedora and Red Hat Enterprise Linux

Mark Sobell — Thu, 10 Nov 2011 17:09:53 +0000

by Mark Sobell

The init Daemon

The init daemon is the system and service manager for Linux. It is the first true process Linux starts when it boots and as such, has a PID of 1 and is the ancestor of all processes. The init daemon has been around since the early days of UNIX, and many people have worked to improve it. The first Linux init daemon was based on the UNIX System V init daemon and is referred to as SysVinit (System V init daemon).

Because SysVinit does not deal well with modern hardware, including hotplug devices, USB hard and flash drives, and network-mounted filesystems, Fedora/RHEL recently replaced it with the Upstart init daemon (http://upstart.ubuntu.com/ and http://upstart.ubuntu.com/wiki). Fedora 15 has moved past Upstart to systemd init daemon, which is described next. Several other replacements for SysVinit are also available. One of the most prominent is initng (http://initng.sourceforge.net/trac). In addition, Solaris uses SMF (Service Management Facility), and MacOS uses launchd.

The systemd init Daemon (Fedora)

The name systemd comprises system, which systemd manages, followed by d. Under UNIX/Linux, daemon names frequently end in d: systemd is the system daemon. At boot time, systemd renames itself init, so you will not see a process named systemd. However, init is simply a link to systemd:

$ ls -l /sbin/init
lrwxrwxrwx. 1 root root 14 04-22 08:47 /sbin/init -> ../bin/systemd

The name is also a play on words with System D, a reference to the French dérouillard (to untangle) or démerder. System D is a manner of responding to challenges that requires fast thinking, adapting, and improvising.

The systemd init daemon is a drop-in replacement for SysVinit; most of the administration tools that worked with SysVinit and Upstart work with systemd. Although systemd is new, most of the user interfaces pertinent to administrators will remain stable (http://www.freedesktop.org/wiki/Software/systemd/InterfaceStabilityPromise). A GUI to systemd is under development.

More Information

Local

Use apropos to list man pages that pertain to systemd (apropos systemd). Some of the most interesting of these are systemd, systemctl, systemd.unit, and systemd.special.

Web

Go to Full Article

Book Excerpt: The Python Standard Library by Example

Doug Hellmann — Thu, 20 Oct 2011 15:20:31 +0000

by Doug Hellmann

Chapter 3: Algorithms

Python includes several modules for implementing algorithms elegantly and concisely using whatever style is most appropriate for the task. It supports purely procedural, object-oriented, and functional styles. All three styles are frequently mixed within different parts of the same program.

functools includes functions for creating function decorators, enabling aspect-oriented programming and code reuse beyond what a traditional object-oriented approach supports. It also provides a class decorator for implementing all rich comparison APIs using a shortcut and partial objects for creating references to functions with their arguments included.

The itertools module includes functions for creating and working with iterators and generators used in functional programming. The operator module eliminates the need for many trivial lambda functions when using a functional programming style by providing function-based interfaces to built-in operations, such as arithmetic or item lookup.

contextlib makes resource management easier, more reliable, and more concise for all programming styles. Combining context managers and the with statement reduces the number of try:finally blocks and indentation levels needed, while ensuring that files, sockets, database transactions, and other resources are closed and released at the right time.

3.1 functools—Tools for Manipulating Functions

Purpose Functions that operate on other functions.

Python Version 2.5 and later

The functools module provides tools for adapting or extending functions and other callable objects, without completely rewriting them.

3.1.1 Decorators

The primary tool supplied by the functools module is the class partial, which can be used to “wrap” a callable object with default arguments. The resulting object is itself callable and can be treated as though it is the original function. It takes all the same arguments as the original, and it can be invoked with extra positional or named arguments as well. A partial can be used instead of a lambda to provide default arguments to a function, while leaving some arguments unspecified.

Partial Objects

This example shows two simple partial objects for the function myfunc(). The output of show_details() includes the func, args, and keywords attributes of the partial object.

import functools 

def myfunc(a, b=2): 
  """Docstring for myfunc().""" 
  print ’ called myfunc with:’, (a, b) 
  return 

def show_details(name, f, is_partial=False): 
    """Show details of a callable object.""" 
    print ’%s:’ % name 
    print ’ object:’,f

Go to Full Article

Book Excerpt: Practical Guide to Ubuntu Linux, A, 3rd Edition

LJ Staff — Thu, 11 Nov 2010 15:00:00 +0000

by LJ Staff

Excerpt from Practical Guide to Ubuntu Linux, A, 3rd Edition.

By Mark G. Sobell
Published by Prentice Hall
ISBN-10: 0-13-254248-X
ISBN-13: 978-0-13-254248-7

Securing a Server

Two ways you can secure a server are by using TCP wrappers and by setting up a chroot jail. This section describes both techniques.

TCP Wrappers: Secure a Server (hosts.allow and hosts.deny)

Follow these guidelines when you open a local system to access from remote systems:

Open the local system only to systems you want to allow to access it.
Allow each remote system to access only the data you want it to access.
Allow each remote system to access data only in the appropriate manner (readonly, read/write, write only).

libwrap

As part of the client/server model, TCP wrappers, which can be used for any daemon that is linked against libwrap, rely on the /etc/hosts.allow and /etc/hosts.deny files as the basis of a simple access control language (ACL). This access control language defines rules that selectively allow clients to access server daemons on a local system based on the client’s address and the daemon the client tries to access. The output of ldd shows that one of the shared library dependencies of sshd is libwrap:

$ ldd /usr/sbin/sshd | grep libwrap
		libwrap.so.0 => /lib/libwrap.so.0 (0xb7ec7000)

hosts.allow and hosts.deny

Each line in the hosts.allow and hosts.deny files has the following format:

daemon_list : client_list [: command]

where daemon_list is a comma-separated list of one or more server daemons (such as portmap, vsftpd, and sshd), client_list is a comma-separated list of one or more clients, and the optional command is the command that is executed when a client from client_list tries to access a server daemon from daemon_list.

When a client requests a connection to a server, the hosts.allow and hosts.deny files on the server system are consulted in the following order until a match is found:

If the daemon/client pair matches a line in hosts.allow, access is granted.
If the daemon/client pair matches a line in hosts.deny, access is denied.

Go to Full Article

Book Excerpt: Linux Kernel Development, 3rd Edition.

LJ Staff — Fri, 15 Oct 2010 13:00:00 +0000

by LJ Staff

Excerpt from Linux Kernel Development, 3rd Edition.

By Robert Love
Published by Addison-Wesley Professional
ISBN-10: 0-672-32946-8
ISBN-13: 978-0-672-32946-3

Chapter 2: Getting Started with the Kernel

In this chapter, we introduce some of the basics of the Linux kernel: where to get its source, how to compile it, and how to install the new kernel. We then go over the differences between the kernel and user-space programs and common programming constructs used in the kernel. Although the kernel certainly is unique in many ways, at the end of the day it is little different from any other large software project.

Obtaining the Kernel Source

The current Linux source code is always available in both a complete tarball (an archive created with the tar command) and an incremental patch from the official home of the Linux kernel, http://www.kernel.org.

Unless you have a specific reason to work with an older version of the Linux source, you always want the latest code. The repository at kernel.org is the place to get it, along with additional patches from a number of leading kernel developers.

Using Git

Over the last couple of years, the kernel hackers, led by Linus himself, have begun using a new version control system to manage the Linux kernel source. Linus created this system, called Git, with speed in mind. Unlike traditional systems such as CVS, Git is distributed, and its usage and workflow is consequently unfamiliar to many developers. I strongly recommend using Git to download and manage the Linux kernel source.

You can use Git to obtain a copy of the latest “pushed” version of Linus’s tree:

$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

When checked out, you can update your tree to Linus’s latest:

$ git pull

With these two commands, you can obtain and subsequently keep up to date with the official kernel tree. To commit and manage your own changes, see Chapter 20, “Patches, Hacking, and the Community.” A complete discussion of Git is outside the scope of this book; many online resources provide excellent guides.

Installing the Kernel Source

The kernel tarball is distributed in both GNU zip (gzip) and bzip2 format. Bzip2 is the default and preferred format because it generally compresses quite a bit better than gzip. The Linux kernel tarball in bzip2 format is named linux-x.y.z.tar.bz2, where x.y.z is the version of that particular release of the kernel source. After downloading the source, uncompressing and untarring it is simple. If your tarball is compressed with bzip2, run

$ tar xvjf linux-x.y.z.tar.bz2

If it is compressed with GNU zip, run

Go to Full Article

Book Excerpt: A Practical Guide to Linux Commands, Editors, and Shell Programming

LJ Staff — Fri, 05 Mar 2010 20:53:47 +0000

by LJ Staff

This article is an excerpt from the new 2nd Ed. of Mark Sobell's book, A Practical Guide to Linux Commands, Editors, and Shell Programming, published Nov. 2009 by Prentice Hall Professional, ISBN 0131367366, Copyright 2010 Mark G. Sobell. For additional sample content from a selection of chapters, please visit the publisher site: www.informit.com/title/0131367366

Go to Full Article

Book Excerpt: Chapter 4 of Citizen Engineer

LJ Staff — Fri, 19 Feb 2010 15:26:39 +0000

by LJ Staff

This chapter excerpt is from Citizen Engineer: A Handbook for Socially Responsible Engineering by David Douglas and Greg Papadopoulos, with John Boutelle, published by Prentice Hall Professional, ISBN 0137143923, Copyright 2010 David Douglas and Greg Papadopoulos. For more info, please visit the publisher page: www.informit.com/title/0137143923

Go to Full Article