Nginx. From Beginner to Pro. Rahul Soni

Transcription

1 Nginx From Beginner to Pro Rahul Soni

2 Nginx: From Beginner to Pro Rahul Soni Kolkata, West Bengal India ISBN-13 (pbk): ISBN-13 (electronic): DOI / Library of Congress Control Number: Copyright 2016 by Rahul Soni This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director: Welmoed Spahr Acquisitions Editor: Louise Corrigan Development Editor: Corbin Collins Technical Reviewer: Eduardo Balsa Editorial Board: Steve Anglin, Pramila Balen, Laura Berendson, Aaron Black, Louise Corrigan, James DeWolf, Jonathan Gennick, Todd Green, Robert Hutchinson, Celestin Suresh John, Nikhil Karkal, James Markham, Susan McDermott, Matthew Moodie, Natalie Pao, Gwenan Spearing Coordinating Editor: Nancy Chen Copy Editor: Karen Jameson Compositor: SPi Global Indexer: SPi Global Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY Phone SPRINGER, fax (201) , orders-ny@springer-sbm.com, or visit Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please rights@apress.com, or visit Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. ebook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales ebook Licensing web page at Any source code or other supplementary materials referenced by the author in this text is available to readers at For detailed information about how to locate your book s source code, go to source-code/. Printed on acid-free paper

3 This book is dedicated to the open source community at large who keep pushing the limits of what software can achieve!

4

5 Contents at a Glance About the Author...xvii About the Technical Reviewer...xix Acknowledgments...xxi Chapter 1: Introduction to Nginx Web Server... 1 Chapter 2: Installing Nginx Chapter 3: Nginx Core Directives Chapter 4: Nginx Modules Chapter 5: Nginx Core Architecture Chapter 6: Hosting Web Sites on Nginx Chapter 7: Nginx and Dynamic Content Chapter 8: Load Balancing with Nginx Chapter 9: Log Analysis, Monitoring, and Automation Chapter 10: SSL, Security, and Authentication Chapter 11: Upgrading and Migrating Chapter 12: Troubleshooting Tips and FAQ Index v

6

7 Contents About the Author...xvii About the Technical Reviewer...xix Acknowledgments...xxi Chapter 1: Introduction to Nginx Web Server... 1 HTTP Basics... 1 What Is a Web Server?... 3 Seven Reasons Why You Should Be Using Nginx... 4 It s Fast... 4 It Can Accelerate Your Application... 4 It Has a Straightforward Load Balancer... 4 It Scales Well... 4 You Can Upgrade It On the Fly... 5 It s Affordable to Install and Maintain... 5 It s Easy to Use... 5 Main Features of Nginx... 5 More Than Just a Web Server... 5 Modular Design... 6 Asynchronous Web Server... 6 Reverse Proxy and Load Balancing Capability... 6 Low Resource Requirement and Consumption...6 Unparalleled Performance... 7 Multiple Protocol Support: HTTP(S), WebSocket, IMAP, POP3, SMTP... 7 SSL Termination... 7 HTTP Video Streaming Using MP4/FLV/HDS/HLS... 7 vii

8 CONTENTS Extended Monitoring and Logging... 8 Graceful Restarting... 8 Upgrades without Downtime Using Live Binaries... 8 Enterprise Features of Nginx Plus... 8 Support Available with Nginx Plus... 9 Advantages of Nginx Plus... 9 Advanced HTTP and TCP Load Balancing... 9 Session Persistence... 9 Content Caching Enhanced Capabilities... 9 Application Health Checks HTTP Live Streaming (HLS) and Video on Demand (VOD) HTTP Dynamic Streaming (HDS/VOD) Bandwidth Management for MP4 Media Live Activity Monitoring Nginx Commercial Support Differences between Apache and Nginx History Performance Resource Requirements Availability Proxy and Load Balancing Server Static vs. Dynamic Content Configuration Modules (or Plug-Ins) Documentation Support Summary...15 viii

9 CONTENTS Chapter 2: Installing Nginx Preparing Your Environment Installing Nginx Using Pre-Built Packages Install Nginx Pre-Built Package Nginx Folder Structure Uninstall Nginx Downloading Nginx from Source Downloading Nginx Extracting the Nginx Archive Understanding the Nginx Source Directory Installing Nginx Binaries Build Tools for Compilation Install Dependent Packages Compiling Nginx Understanding the./configure Script Compile-Time Options Compiling and Installing Nginx Enable Nginx Service on Reboot Troubleshooting Service Start Verifying Web Server Installation Firewall Configuration Nginx in Amazon Elastic Compute Cloud (EC2) Summary...47 Chapter 3: Nginx Core Directives Location of Configuration Files What Are Directives? Context Types Understanding the Default Configuration Simple Directives Events Context ix

10 CONTENTS HTTP Context The conf.d Folder Server Context Location Context Location Context Special Cases Verify the Correctness of Configuration Allow Directory Listing Deny Access to Any Specific Location Proxy the Requests to Apache Proxy the Requests to FastCGI Nginx Variables A Quick Note about Nginx Official Documentation Summary...75 Chapter 4: Nginx Modules What Are Modules? Module Installation Module Categories How Does a Module Work? Module Structure How Modules Fit in Nginx Configuring Default Modules for Optimal Performance Core Module Events Module HTTP Module Enabling Optional Modules Gzip Module FastCGI Module Basic Authentication x

11 CONTENTS Enabling Third-Party Modules PAM Authentication Summary...95 Chapter 5: Nginx Core Architecture A Quick Analogy The Master Process Processes vs. Threads The Worker Process State Machines Update Configuration Upgrade HTTP Request Processing in Nginx Summary Chapter 6: Hosting Web Sites on Nginx Web Server Setup Connecting Host and Guest Servers User Creation Sample Applications Uploading Content Hosting Websites Websites Using Different Names Websites Using Domain Name Internal Redirects Sites Using Different Ports Wildcard Mapping Blocking Access Domain Name Mapping IP-Based Hosting Mixed Name-Based and IP-Based Servers xi

12 CONTENTS Common Configuration Mistakes Let s Use Root Inside Location Block Monolithic Configuration Files Unnecessary Complications Listening on Hostname Summary Chapter 7: Nginx and Dynamic Content Sudo Scare Installing MySQL Installing PHP Configure Nginx for WordPress The MEAN Stack Installing MongoDB Installing Node.JS Installing Express.JS Installing Angular.JS Creating a MEAN Application Configure Nginx for MEAN Stack Summary Chapter 8: Load Balancing with Nginx Defining High Availability Load Balancing for High Availability Hardware Load Balancer Software Load Balancer Load Balancing in Nginx Clean Up the Servers Create Web Content Configure WFE1 and WFE Set Up NLB Server xii

13 CONTENTS Load Balancing Algorithms Most Suitable Algorithm Load Balancing Scenarios Nginx Routing Request to Express/Node.js Passing the HOST Header Forwarding IP Information Buffering Nginx Caching Server Directive Additional Parameters Configure Nginx (PLUS) for Heath Checks Activity Monitoring in Nginx (PLUS) Summary Chapter 9: Log Analysis, Monitoring, and Automation Error Log Access Log What to Log? Log Buffers Conditional Log Log Compression Syslog Analyze Logs tail ngxtop GoAccess Custom Error pages Benchmark Apache Benchmark JMeter Cloud-Based Benchmarking xiii

14 CONTENTS Baseline Monitoring Nginx PLUS Automation Summary Chapter 10: SSL, Security, and Authentication Tools to Protect Network Traffic Capturing Network Traffic with Wireshark Using Secure HTTP SSL Encryption Enabling SSL Nginx Optimizing HTTPS Servers Creating a Certificate Request Web Server Security Creating the Password File Configuring Nginx Password Authentication Summary Chapter 11: Upgrading and Migrating Controlling Nginx Command-Line Parameters Migrating from Apache to Nginx Feature Comparison Configuration Comparison Summary Chapter 12: Troubleshooting Tips and FAQ First, What You Should Not Do First Commandment of Troubleshooting: Isolate the Issue Scenario 1: Page Cannot Be Displayed in the Browser Scenario 2: Conflicting Ports xiv

15 CONTENTS Scenario 3: Bad Permissions Scenario 4: Bad Configuration Scenario 5: Rewrite Rules Scenario 6: Log Only Your Requests Important Tools for Web Administrators ping traceroute top htop atop uptime free ifconfig or ip addr ulimit nslookup powertop iotop iptraf tcpdump WireShark Nagios zabbix w lsof Common Pitfalls to Avoid Chmod Having Root Inside Location Block Using if Blocks Passing Uncontrolled Requests to PHP Rewrite Issues Using Hostname in Configuration xv

16 CONTENTS Frequently Asked Questions Is there an option to compare Nginx and Nginx Plus? Is there a location for sample configurations? How can I redirect from www to no-www and vice versa? How can I write all http requests to https while maintaining a sub-domain? How can I find which flags Nginx was compiled with? Is there any mechanism for detailed debugging? How many third-party modules does Nginx have? What happens if I have Nginx Plus and the license expires? Is there design or consulting help available? Summary Index xvi

17 About the Author Rahul Soni worked at Microsoft for a decade ( ) before becoming completely fascinated and then pulled into open source technologies. He consulted on IIS,.NET, and SharePoint during his Microsoft tenure. There was no looking back when he realized how much could be achieved using the open source stack. Nginx and the MEAN stack floored him completely, and he founded Attosol Technologies ( ) to pursue his passion. This book is an honest attempt from a guy who has always been into a proprietary stack, but ultimately found that the open source community is exciting enough to leave the corporate world. Rahul is a man of simple taste and loves reading, listening to music, and traveling. When not working, he can be found hanging around with his wife Neha and their two kids Anika and Aarav. xvii

18

19 About the Technical Reviewer Eduardo Balsa has worked in system administration for the last 13 years, focusing on server scaling and security. His work on the Fintech and web marketing sector has led him to push web server software to its limits and to gather an extensive knowledge of the ins and outs of their deployment, configuration, and day-to-day maintenance. Eduardo is adept at system-wide automation and of the K.I.S.S. principle and passionate about tinkering and photography. You can find him online at xix

20

21 Acknowledgments There are many people who I would like to acknowledge since they have directly or indirectly played a huge role in getting this book released. Igor Sysoev - The creator of Nginx who solved a problem that pushed web delivery to the next level. The open source community at large, which has helped Nginx along its way. Apress, for their support in releasing this book. All the editors at Apress, who reviewed the book and helped me express myself better. Nancy Chen, the coordinating editor, for being flexible and supportive throughout! Microsoft, where I learned a lot about IIS and other web technologies. And my family, who understood and supported me more than I ever imagined possible when I took up this project! xxi