nginx

Words, Words, Words--Introducing OpenSearchServer

Marcel Gagné — Wed, 07 Aug 2019 13:46:43 +0000

How to create your own search engine combined with a crawler that will index all sorts of documents.

In William Shakespeare's Hamlet, one of my favorite plays, Prince Hamlet is approached by Polonius, chief counselor to Claudius, King of Denmark, who happens to be Hamlet's stepfather, and uncle, and the new husband of his mother, Queen Gertrude, whose recently deceased last husband was the previous King of Denmark. That would be Hamlet's biological father for those who might be having trouble following along. He was King Hamlet. Polonius, I probably should mention, is also the father of Hamlet's sweetheart, Ophelia. Despite this hilarious sounding setup, Hamlet is most definitely not a comedy. (Note: if you need a refresher, you can read Hamlet here.)

For reasons I won't go into here, Hamlet is doing a great job of trying to convince people that he's completely lost it and is pretending to be reading a book when Polonius approaches and asks, "What do you read, my lord?"

Hamlet replies by saying, "'Words, words, words." In other words, ahem, nothing of any importance, you annoying little man.

Shakespeare wrote a lot of words. In fact, writers, businesses and organizations of any size tend to amass a lot of words in the form of countless documents, many of which seem to contain a great deal of importance at the time they are written and subsequently stored on some lonely corporate server. There, locked in their digital prisons, these many texts await the day when somebody will seek out their wisdom. Trouble is, there are so many of them, in many different formats, often with titles that tell you nothing about the content inside. What you need is a search engine.

Google is a pretty awesome search engine, but it's not for everybody, especially if the documents in question aren't meant for consumption by the public at large. For those times, you need your own search engine, combined with a crawler that will index all sorts of documents, from OpenDocument format, to old Microsoft Docs, to PDFs and even plain text. That's where OpenSearchServer comes into play. OpenSearchServer is, as the name implies, an open-source project designed to perform the function of crawling through and indexing large collections of documents, such as you would find on a website.

I'm going to show you how to go about getting this documentation site set up from scratch so that you can see all the steps. You may, of course, already have a web server up and running, and that's fine. I've gone ahead and spun up a Linode server running Ubuntu 18.04 LTS. This is a great way to get a server up and running quickly without spending a lot of money if you don't want to, and if you've never done this, it's also kind of fun.

Go to Full Article

ModSecurity and nginx

Elliot Cooper — Mon, 24 Sep 2018 11:30:00 +0000

by Elliot Cooper

nginx is the web server that's replacing Apache in more and more of the world's websites. Until now, nginx has not been able to benefit from the security ModSecurity provides. Here's how to install ModSecurity and get it working with nginx.

Earlier this year the popular open-source web application firewall, ModSecurity, released version 3 of its software. Version 3 is a significant departure from the earlier versions, because it's now modularized. Before version 3, ModSecurity worked only with the Apache web server as a dependent module, so there was no way for other HTTP applications to utilize ModSecurity. Now the core functionality of ModSecurity, the HTTP filtering engine, exists as a standalone library, libModSecurity, and it can be integrated into any other application via a "connector". A connector is a small piece of code that allows any application to access libModSecurity.

A Web Application Firewall (WAF) is a type of firewall for HTTP requests. A standard firewall inspects data packets as they arrive and leave a network interface and compares the properties of the packets against a list of rules. The rules dictate whether the firewall will allow the packet to pass or get blocked.

ModSecurity performs the same task as a standard firewall, but instead of looking at data packets, it inspects HTTP traffic as it arrives at the server. When an HTTP request arrives at the server, it's first routed through ModSecurity before it's routed on to the destination application, such as Apache2 or nginx. ModSecurity compares the inbound HTTP request against a list of rules. These rules define the form of a malicious or harmful request, so if the incoming request matches a rule, ModSecurity blocks the request from reaching the destination application where it may cause harm.

The following example demonstrates how ModSecurity protects a WordPress site. The following HTTP request is a non-malicious request for the index.php file as it appears in Apache2's log files:


GET /index.php HTTP/1.1

This request does not match any rules, so ModSecurity allows it onto the web server.

WordPress keeps much of its secret information, such as the database password, in a file called wp-config.php, which is located in the same directory as the index.php file. A careless system administrator may leave this important file unprotected, which means a web server like Apache or nginx happily will serve it. This is because they will serve any file that is not protected by specific configuration. This means that the following malicious request:


GET /wp-config.php HTTP/1.1

will be served by Apache to whomever requests it.

Go to Full Article

nginx and WordPress

Reuven M. Lerner — Tue, 08 Nov 2016 12:13:35 +0000

by Reuven M. Lerner

In my last article, I took an initial look at nginx, the high-performance open-source HTTP that uses a single process and a single thread to service a large number of requests. nginx was designed for speed and scalability, as opposed to Apache, which was designed to maximize flexibility and configuration. But through the years, nginx has become increasingly flexible as well, with a growing number of plugins and modules that can be used to customize its configuration. Between the performance, increasingly good documentation and convenience, it's no wonder nginx has been increasingly popular.

It's also no surprise that WordPress, the open-source blogging and CMS platform, has become hugely popular. I've heard people say that 10% of websites are now run using WordPress. Even if that's not precisely true, there's no doubt that a huge number of sites are powered by WordPress. I'm a mostly satisfied WordPress user, having converted my main site and my two ebook sites to it in the past year after years of using it to power my blog.

So, I thought it would be interesting to demonstrate how easy it is to set up WordPress with nginx, given the popularity of each of these systems alone as well as together. In my last article, I described how you can set up a plain-vanilla PHP system with nginx; WordPress is a bit more complex, but less than you might think. Starting with a bare-bones Linux installation, let's walk through the configuration needed to get WordPress up and running.

The Basics

In order to install WordPress and nginx together, you're going to need three basic software systems installed: WordPress, nginx and MySQL. The first two are pretty obvious, given this article's goal; the third is a byproduct of using WordPress, which works exclusively with MySQL.

So, on my Ubuntu Linux machine, I would run the following:


$ sudo apt-get install mysql-server mysql-client nginx-core
 ↪php5-cli php5-fpm php5-mysql

This installs a very large number of packages, but it will give you the core of what you need to get your system up and running. Notice that you're not installing WordPress here, so that you can install it manually, using the source code. Indeed, installing WordPress via apt-get also means installing Apache; although it's certainly possible to undo this choice, the benefits of installing WordPress on your own outweigh those of doing it via a package manager.

You will, as part of this installation, need to choose a password for your MySQL root user. This is an important part of security on your system, so do try to use a strong password.

Go to Full Article

Linaro Ltd.'s OpenDataPlane

James Gray — Wed, 02 Nov 2016 15:14:35 +0000

by James Gray

The OpenDataPlane (ODP) project is a founding initiative by the Linaro Networking Group to produce an open-source, cross-platform application programming interface (API) for the networking Software Defined Data Plane. Linaro Ltd. recently announced the availability of the first Long Term Support (LTS) Monarch release of OpenDataPlane, which will enable other projects to leverage the acceleration provided by the ODP APIs now that the code base will be fully supported for the foreseeable future.

Linaro adds that work already has begun on network protocol stacks, such as OpenFastPath (OFP), products like the nginx web server accelerated with ODP and OFP and libraries like OpenSSL that provide crypto acceleration via ODP. In addition, ODP and ODP-based products, such as OFP, nginx and OpenSSL, now can be made available as packages in popular Linux distributions like Debian, CentOS and OpenEmbedded. To accompany the release, Linaro launched a validation test suite that permits users and vendors to verify API compatibility between different ODP implementations.

Go to Full Article

Let's Automate Let's Encrypt

Andrei Lukovenko — Tue, 01 Nov 2016 10:08:36 +0000

by Andrei Lukovenko

HTTPS is a small island of security in this insecure world, and in this day and age, there is absolutely no reason not to have it on every Web site you host. Up until last year, there was just a single last excuse: purchasing certificates was kind of pricey. That probably was not a big deal for enterprises; however, if you routinely host a dozen Web sites, each with multiple subdomains, and have to pay for each certificate out of your own dear pocket—well, that quickly could become a burden.

Now you have no more excuses. Enter Let's Encrypt a free Certificate Authority that officially left Beta status in April 2016.

Aside from being totally free, there is another special thing about Let's Encrypt certificates: they don't last long. Currently all certificates issued by Let's Encrypt are valid for only 90 days, and you should expect that someday this term will become even shorter. Although this short lifespan definitely creates a much higher level of security, many people consider it as an inconvenience, and I've seen people going back from using Let's Encrypt to buying certificates from commercial certificate authorities for this very reason.

Of course, if you are running multiple Web sites, having to renew several certificates manually every three months quickly could become annoying to say the least. Some day you even may forget (and you will regret that forgetfulness). Let's leave routines to computers, right?

If you are using Apache under a Debian-based distribution, Let's Encrypt already has you covered with the libaugeas0 package, and it is capable of both issuing and renewing certificates. If, like me, you prefer nginx and want to have zero-downtime automatic certificate updates with industrial-grade encryption, keep reading. I'm going to show you how to get there.

First things first—some assumptions and requirements:

You are running the nginx Web server/load balancer, and you are going to use it for TLS termination (that's a fancy, but technically correct way of saying "nginx will handle all this HTTPS stuff").
nginx serves several Web sites, and you want HTTPS on all of them, and you are not going to pay a single dime.
You also want to get the highest grade on the industry standard for SSL tests—SSL Lab's SSL server test.
You do not enjoy the idea of running some not-so-well-sandboxed third-party code on your server, and you would rather have this code in a Docker container.
Naturally, you are lazy (or experienced) enough, so you want to write some scripts that will re-issue all certificates way before they expire.

Go to Full Article

nginx

Reuven M. Lerner — Tue, 27 Sep 2016 13:30:00 +0000

by Reuven M. Lerner

Engineers love to think that they make decisions based on pure logic and merit. But of course, everyone has biases in terms of programming languages, editors and other technologies—biases that probably can be defended in technical terms, but that often come down to an emotional argument as much as a technical one. (Except in the case of Emacs, of course, which is clearly the best editor by all objective standards.) The problem with such biases is that they can cause people to make choices and decisions that feel comfortable, but aren't necessarily right.

Case in point: I've been using the Apache HTTP server for many years now. Indeed, you could say that I've been using Apache since before it was even called "Apache"—what started as the original NCSA HTTP server, and then the patched server that some enterprising open-source developers distributed, and finally the Apache Foundation-backed open-source colossus that everyone recognizes, and even relies on, today—doing much more than just producing HTTP servers.

Apache's genius was its modularity. You could, with minimal effort, configure Apache to use a custom configuration of modules. If you wanted to have a full-featured server with tons of debugging and diagnostics, you could do that. If you wanted to have high-level languages, such as Perl and Tcl, embedded inside your server for high-speed Web applications, you could do that. If you needed the ability to match, analyze and rewrite every part of an HTTP transaction, you could do that, with mod_rewrite. And of course, there were third-party modules as well.

Things got even better through the years as the Web got larger, and Web sites were expected to do more and more. Scalability became an important issue, and Apache handled it with (not surprisingly) a variety of modules that implemented different back-end schemes. You could have the traditional mix of processes, or use threads, or combinations of the two.

Beyond the flexibility, it was clear that Apache httpd was well maintained, well documented and stable. Installation was easy, upgrades were easy—really, everything was easy.

So, it's no surprise that Apache always has been my first choice when it comes to HTTP servers. And yet, I always knew in the back of my mind that I really should spend more time checking out other options. In particular, one alternative stood out—nginx.

Go to Full Article