Web server monitoring

Hello, und welcome to this video about monitoring Web Servers with PRTG

My name is Kimberley Trommler and I'm a systems engineer at Paessler.
In this video, I'm going to discuss the different aspects of web server monitoring, and how you can perform these using PRTG. Web servers are a critical part of every business: as online shops, for customer communication, partner portals, or as part of your corporate marketing strategy. As such, it is essential to ensure that your webservers are performing correctly, to avoid failures and to avoid business losses. The most important question to start with, is: What, exactly, should I monitor? What components and services are most important? Your webservers depend not only on the underlying hardware - they're also dependent on applications and network services such as DNS or AD. Let's look at the different areas of web server monitoring that need to be considered. We can differentiate between two main categories of web server monitoring: the user experience level, and the system or IT level.

In the user experience level, we should monitor the quality of user experience, which includes both the content the user receives, and the response times for loading the content. Since content can be static or dynamic, we should be able monitor both. In addition to serving up the content of the web site, web servers also need to perform additional supporting functions such as performing user authentication, providing security features such as certificates and encryption, forwarding to other URLs, reporting HTTP status codes, and logging web server activity.

This set of functions is partially at the user experience level, because the user sees the results, and partially at the system level, because they're part of fundamental business processes.

Underneath this services level come the hardware devices such as web, application and database servers, and network devices such as switches and firewalls.
I've divided these into two sections: the health of the server, and the basic network-level services. The server health includes not only the hardware, but also the operating system and application-level services that are running on that server. The basic network services include having enough bandwidth, that DNS is working, that certain ports are reachable, and that devices such as proxies or load balancers are functioning correctly.

I'm now going to show you how you can monitor each of these areas using PRTG, starting with how to monitor content and user experience.

Content & User Experience

PRTG offers the following sensors for monitoring content and user experience: Let's look at each of these inside PRTG. Start at "Web Server" Group

- The simplest sensor, the HTTP Sensor, loads a web page and monitors the response time. If the sensor doesn't receive an answer, or if it receives a 404 status code, then the sensor goes into Down state.

- The HTTP advanced sensor shows not only the response time, but also the bytes received, download bandwidth speed, and time to first byte.

o For each channel in the sensor, you can set individual thresholds and alarms. o Show settings for "Time to first Byte" o If we look at the Settings for this sensor, we see that it also supports authentication and you can use it to check for specific content on the page. So you can simulate a login into the website, and check that specific content is or is not appearing on the resulting page. o Other configuration options include special HTTP settings, or simulating specific user agents. - With the HTTP content sensor, you can check up to 50 results that are returned by an HTTP request, with each result in square brackets. You can use this to monitor services, performance counters or any other numerical value from your webserver that you make available to PRTG.

- The HTTP Full Web Page sensor uses a real browser, running in the background, to download an entire web page and to measure the full download time of all elements on the page. You can select whether PRTG should use Chromium, PhantomJS or Internet Explorer as the browser for this sensor.

- The most powerful sensor is the HTTP Transaction sensor, which lets you simulate an entire transaction in your web application. You can test logging into the web application, and then simulate clicking through a series of up to 10 URLs, and PRTG checks reach ability and specific content on each of those pages This sensor is an excellent tool for testing entire business processes that run on your web servers - you can test the user login, the ordering process, payment processing, and the shipping process. We use this sensor ourselves, to monitor our own web shop.

- If you need help recording the list of URLs for a transaction sensor, you can use our free URL recorder tool, which is on our website, under Products/ Networking Freeware/ URL Recorder

- The HTTP XML/REST sensor retrieves an XML file from the URL you specify, and parses the file, to return either an entire node, or values for specific nodes. In this example, we're counting the number of times the node is found. - And, to finish up the HTTP sensors for today, the HTTP Push sensors let you receive HTTP Push messages. This is also known as a "webhook". You can use this to let other systems trigger actions inside PRTG. For example, you could use this to get an alert when someone comments on a blog post.

There are HTTP Push sensors that count the number of push messages received, and others that use the values in the push messages as channel values inside PRTG.

In addition to these HTTP sensors, Paessler offers special sensors for monitoring specific applications and services. For example, the Passive Application Performance Sensor, which we can see here, monitors TCP connection timing using a packet sniffer. It can measure the performance of many web applications without needing direct access to the client or the server. It measures the time needed for each step of the TCP handshake, the number of connections to the service observed during the last polling period, and packet level statistics such as the total number of packets per second and the number of dropped packets.

PRTG also offers sensors specific to a few common web servers and web services. If you're using Apache, there are two Apache sensors that use mod_status to collect data about the apache server. The Performance Status sensor shows the CPU load, the uptime, requests per second, bytes per request, and the number of current busy and idle worker threads. The Totals sensor shows the accesses and the amount of data transferred by the apache server.

The Windows IIS Application sensor uses windows performance counters or WMI to monitor the IIS server or applications running on the IIS server. It can return the number of bytes sent and received, as well as the number of post, get, and CGI requests, not found errors, anonymous and known users, and received and sent files per second.

And, if you're using pingdom to monitor your website, you can include the pingdom results into PRTG using the pingdom sensor. Now that we've seen the PRTG sensors for content and user experience, let's now turn to the supporting services.

Supporting Tasks

As we saw at the beginning of this video, your web server needs to do more than just serve up content. It also needs to perform encryption, authentication, and redirection, to return HTTP error codes, and to perform logging and backups. PRTG offers sensors that can be used to monitor each of these tasks.

For the server security, there's the SSL Security Check Sensor, which checks connectivity to the server using various SSL and TLS protocols . The HTTP SSL Certificate Expiry will check when the server certificate expires.

To test authentication, redirection and HTTP status codes, you can use the HTTP transaction sensor or the HTTP content sensor, both of which we've already seen.

For logging and backup, you can use File sensors or Email sensors to test that the logs are being written correctly and that the backup process has run correctly.

If you're using Google Analytics on your website, you can include the analytics data into PRTG using the google analytics sensor.

Server Health

Moving on to the next area of monitoring, we're now going to look at the server health. Here, PRTG offers quite a number of sensors. The ones you choose will depend on what kind of server you're using. The sensors shown here are the most common ones, but if you'd like more detail than these sensors, the list of all 200 sensor types is available on our website. The SNMP-based sensors will work for most servers, and there are Windows-specific and Linux-Specific sensors for each of those operating systems.

If you're running VMWare, Hyper-V or Xen, there are sensors for both host and virtual machines for each of those environments. Moving up from the hardware and operating system level, we'll need sensors for the critical services running on each server. Most web applications, for example, use a database as part of the back end. PRTG offers SQL sensors for each of the major SQL databases. As an example of an SQL sensor, let's look at at a MySQL sensor. This sensor shows

Then, if you'd like to monitor that specific processes are running, you can monitor individual processes in Windows or Linux using the Windows Process Sensor or the SSH Script Sensor. As: an example, let's look at a Windows process sensor that monitors the apache process on that machine: It shows the absolute working set, private bytes, the number of threads, handles, and instances, as well as the average CPU usage and the total CPU usage of the process.

Basic Network Services

However, knowing that our server is running well isn't the entire story. We also need to know that the underlying network and network services are running.

You can monitor network devices and bandwidth using various methods, including SNMP, netflow, or packet sniffing. Which sensors are the best for your environment will depend on the vendors you're using and the level of detail you would like to see. Since each of your applications and services run on a certain TCP or UDP port, you can use the Port Sensor or Port Range sensor to ensure that all relevant ports are open and responding.

To ensure that basic network services such as DNS or active directory are running, PRTG offers sensors specific to those services, including DNS, DHCP and LDAP sensors.

These sensors will connect to each of those services as if they were a normal client, to monitor that the service is responding and that the response is correct.

Best practices: what should I monitor?

Now that we've seen the different sensors that you could use to monitor your web servers, I'd like to come back to the question we started with: what, exactly, should I monitor? There are lots of options and you need to decide which ones make the most sense in your environment:

• Keep it simple - avoid information overload.

The most important point is to keep it simple. We've just seen lots and lots of different sensors, with lots of configuration options. You don't want to use all of these sensors at the same time. Please carefully consider which ones will give you the most appropriate information for your infrastructure.

• Pinpoint the most critical spots in your infrastructure Next, consider what parts of your infrastructure are most critical. Your web shop is probably more important to your business, than, say, an employee information portal. You also already have a good idea of some parts of your infrastructure that are always causing problems. Put more focus on areas that are known issues or bottlenecks. Set thresholds according to your baselines

Warning and alarm thresholds need to be set individually for every environment. You need to determine what is normal behaviour for your infrastructure, and set thresholds accordingly.

• Define notifications and escalation steps for critical issues Set up notifications so you get alerts immediately when there's a problem, and use escalation steps to ensure that critical problems are dealt with promptly. You can also use notifications to alert other administrators about planned maintenance.
• Do multi-site monitoring to ensure broad reachability You can use PRTG's remote probes to monitor your website from multiple locations at the same time, to ensure that it's reachable from within your entire company or from around the world.
• Use the auto-discovery function to easily find changes in your IT environment Use the built-in auto-discovery to recognize changes in your environment, so that you can add new devices to the active monitoring immediately. If you run auto-discovery on a schedule, you'll always have the most up-to-date information about your intrastructure available.
• Visualise your Results to detect issues quickly To visualise your critical monitoring data easily, or to share monitoring results with colleagues or customers, use PRTG Maps to create dashboards for each interest group. For example, you can create overview maps for management, or detailed maps for administrators. As a concrete example, here's a map that visualizes each area of web site monitoring, with a quick traffic-light status showing the health of that area, so you can pinpoint the root of a problem quickly.

KPIs and SLAs

To help decide what to monitor and to decide what to put on a dashboard, you'll need to determine which KPIs are most relevant to your business. As an example, you might want to include the following monitoring values, but these are specific to each environment, so these are only examples:

• Server = CPU and Memory load Traffic = Throughput Web =

Page load time • SQL =

SQL query • AD =

Port response • DNS =

Check a record • DHCP =

IP request

Once you've decided on the best KPIs for you, you can determine baseline values for each of these KPIs, and then set up thresholds and alerting for each one. When you know what KPIs are relevant, then you can define SLAs for each one, and then PRTG can monitor and report on whether you're meeting the SLAs or not.

So, that brings me to the end of this video about web server monitoring with PRTG. If you have any questions about web server monitoring, or about anything else in PRTG, please feel free to contact us at any time, at [email protected]. Thank you, and good bye.