Apache

Breaking Up Apache Log Files for Analysis

Dave Taylor — Mon, 27 May 2019 11:00:00 +0000

Dave tackles analysis of the ugly Apache web server log.

I know, in my last article I promised I'd jump back into the mail merge program I started building a while back. Since I'm having some hiccups with my AskDaveTaylor.com web server, however, I'm going to claim editorial privilege and bump that yet again.

What I need to do is be able to process Apache log files and isolate specific problems and glitches that are being encountered—a perfect use for a shell script. In fact, I have a script of this nature that offers basic analytics in my book Wicked Cool Shell Scripts from O'Reilly, but this is a bit more specific.

Oh Those Ugly Log Files

To start, let's take a glance at a few lines out of the latest log file for the site:


$ head sslaccesslog_askdavetaylor.com_3_8_2019
18.144.59.52 - - [08/Mar/2019:06:10:09 -0600] "GET /wp-content/
↪themes/jumpstart/framework/assets/js/nivo.min.js?ver=3.2
 ↪HTTP/1.1" 200 3074
"https://www.askdavetaylor.com/how-to-play-dvd-free-windows-
↪10-win10/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
 ↪AppleWebKit/537.36 (KHTML, like Gecko) Chrome/
 ↪64.0.3282.140 Safari/537.36 Edge/18.17763 X-Middleton/1"
 ↪52.53.151.37 - - [08/Mar/2019:06:10:09 -0600] "GET
 ↪/wp-includes/js/jquery/jquery.js?ver=1.12.4 HTTP/1.1"
 ↪200 33766 "https://www.askdavetaylor.com/how-to-play
↪-dvd-free-windows-10-win10/" "Mozilla/5.0 (Windows NT
 ↪10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)
 ↪Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763
 ↪X-Middleton/1" 18.144.59.52 - - [08/Mar/2019:06:10:09
 ↪-0600] "GET /wp-content/plugins/google-analytics-for-
↪wordpress/assets/js/frontend.min.js?ver=7.4.2 HTTP/1.1"
 ↪200 2544 "https://www.askdavetaylor.com/how-to-play
↪-dvd-free-windows-10-win10/"
 ↪"Mozilla/5.0 (Windows NT 10.0; Win64; x64)
 ↪AppleWebKit/537.36 (KHTML, like Gecko)
 ↪Chrome/64.0.3282.140 Safari/537.36 Edge/18.17763
 ↪X-Middleton/1"

It's big and ugly, right? Okay, then let's just isolate a single entry to see how it's structured:


18.144.59.52 - - [08/Mar/2019:06:10:09 -0600] "GET
 ↪/wp-content/themes/jumpstart/framework/assets/js/
↪nivo.min.js?ver=3.2 HTTP/1.1" 200 3074
"https://www.askdavetaylor.com/how-to-play-dvd-free-windows-
↪10-win10/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140
 ↪Safari/537.36 Edge/18.17763 X-Middleton/1"

That's still obfuscated enough to kick off a migraine!

Fortunately, the Apache website has a somewhat clearer explanation of what's known as the custom log file format that's in use on my server. Of course, it's described in a way that only a programmer could love:

Go to Full Article

Integrating Web Applications with Apache

Andy Carlson — Tue, 22 Aug 2017 12:36:27 +0000

by Andy Carlson

When you deploy a web application, how do end users access it? Often web applications are set behind a gateway device through which end users can access it. One of the popular products to act as an application gateway on Linux is the Apache Web Server. Although it can function as a normal web server, it also has the ability to connect through it to other web servers.

In this article, I discuss what it takes to integrate a web application into Apache. This includes integrating the HTTP protocol functionality, customizing content to render properly and reusing pieces of configuration. Once you understand those basic bits of functionality, you'll have the tools you need to maximize your web applications' usability. So, let's get started!

Crash Course in RegEx

A mechanism that I use throughout this article that might need a brief introduction is Regular Expressions (or regex). Regex is used to define a text pattern to search for within a URL or to find and replace text within content, such as HTML or JavaScript. The text processing command sed uses regex to do searches and substitutions.

For each example below there will be three parts: input, regex pattern and output. The pattern will be applied to the input text and determine the value of the output text.

Example 1:


Input:
  Name: Frank Sinatra
  Genre: Jazz
  Name: 2Pac
  Genre: Rap
  Name: Reel Big Fish
  Genre: Ska

Regex pattern: "^Name: "

Output:
  Name: Frank Sinatra
  Name: 2Pac
  Name: Reel Big Fish

This example searches the input text for text that matches the pattern "^Name: ". This pattern says, "Look for the text 'Name: ' at the beginning of each line." Since there are two lines that begin with that text, only those two lines are returned. While "^" represents the beginning of a line, "$" represents the end of a line. So if you were to apply the pattern "a$", two lines would be returned (Frank Sinatra and Ska). Let's expand on that example and use the input from Example 1 with a new pattern.

Example 2:


Regex pattern: "^Name: [0-9]"

Output:
  Name: 2Pac

As you can see, I've taken the original regex pattern and added [0-9] to the end. This will search for a single character that can be any number from 0 to 9, which is why "2Pac" was the only line returned. You also can specify a range with alphabetic characters ([a-z] or [A-Z]).

Along with pattern selection, you also can do substitution with regex. There are two formats for regex substitutions: s|pattern|replace|modifier or s/pattern/replace/modifier. In Apache, I find it easier to use the pipe-style substitution. Example 3 uses the same input with a new pattern.

Example 3:

Go to Full Article

Apache Web Servers and SSL Encryption

Carlie Fairchild — Tue, 12 May 2015 14:47:58 +0000

by Carlie Fairchild

Congratulations! You’ve decided to set up a Web site. The site might be for your personal use, for sharing family pictures, for a blog, for an SaaS application, or any number of other possibilities. In all of those cases, people will access your site using the Hypertext Transfer Protocol (HTTP). HTTP has evolved and improved through the years, but one thing about it hasn’t changed—the fact that all of the traffic sent on an HTTP connection is unencrypted.

The bottom line is that whether you want or need to do so, adding HTTPS to a site you’re running isn’t very hard to do. In this Geek Guide, I walk through what SSL/TLS is (and isn’t), how you can create or buy a certificate, how to install that certificate into an Apache server and then how to configure Apache such that a subset of URLs on your system are covered by SSL.

Go to Full Article

Consolidate: Put Your Servers into a VirtualBox VM

Michael Reed — Thu, 30 Sep 2010 13:00:00 +0000

by Michael Reed

Rather than installing a server, such as a web server, directly onto your main computer, why not install it in a VM? This sort of setup has a few advantages of security and convenience. These days, spreading resources out into the cloud is the in-thing, but consolidation is often underexploited. Hosting a server in a virtualizer such as VirtualBox is often a good approach for casual or occasional server needs on a home network.

Go to Full Article