Have A Big Website - Worried About Server Load

Have we ever thought how big websites could be configured to handle massive amount of traffic? How thousands of businesses worldwide are facing challenges everyday to make sure there are no connection failures due to heavy load or congestion of traffic at the website.

The answer is straight, every organization will have to monitor and baseline amount of traffic by going for stress test to find out if the current hardware could be supportive enough to handle a specific amount of load. One such great software is the ?Web Application Testing? or WAPT, which is a great utility for any website?s sustainability. This software is load, stress and performance testing tool for web sites and intranet applications with web interface. WAPT is designed for MS Windows 2000/XP/2003 and Windows 98/Me operating systems. The question that should be asked is why should we consider WAPT or why should the organization consider Load or stress testing? We certainly believe, major performance issues would arise if stress test has not been performed.

This could very well mean that after the server reaches a certain level of concurrent connection made by visitors, it would consume its hardware resources to process every visitor?s request. If there is a lot to process, the server might give priority to web application than its local resource ending up in a system crash.

Thus, the goal of WAPT is very simple; it helps the organization?s web servers to become active at every stage of stress, whether it?s a web development or website traffic processing power. WAPT can perform a stress test by simulating several thousands of concurrent connections to check the web server/website performance and if it finds any bottlenecks it would warn and suggest fixing this issue by recommending the very next thing.
There are lots of new features added in WAPT 4.0
- You can use multiple virtual profiles to test a single scenario
- There is support for IP spoofing where each virtual user can run with individual and unique IP address
- Each Virtual can be configured with different username and password and is compatible with Windows Integrated Authentication or Basic Configuration.
- You can see and configure the type of reports even a test run summary report is generated
- There is command line interface available
- There is a support for persistent cookies
- All the test scenario and profiles are stored in XML format The Main features of WAPT 4.0
- You can have the application run and understand multiple languages
- There is cool recording feature which you can use to record any configuration and stress test simulations for later review or demonstrations as it supports playback of HTTPS/SSL 2.0 & 3.0 pages. You can also record the user connection speed using the keep-alive connections to a HTTP 1.0 or HTTP 1.1 pages or even HTTPS/SSL pages.
- You can generate data for run-times on specific scheduled intervals by randomizing the delay between the page hits.
- It also has a support for proxy servers for HTTP, HTTPS, SOCKS4 & 5 and supports cookies as well.
- All the authentication request can be handled using NTLM (windows Integrated Authentication) or using the basic authentication.
- If you would like to call the website by a different name, then you also have the facility to add custom host headers
- All this the scenarios can be configured using the user friendly Wizard
- You can choose the reports to be either graphical or in plain text along with full virtual server logs
A must tool for administrator or other user who would to maintain their website efficiently.

Ted Peterson writes for CoreDownload, a shareware archive where you can find utilities like WAPT to test your server

Running Your Own Web Server

This article will cover the basics about running your own web server out of your home. However, before I provide some direction on how to go about doing this I am going to give you the "pain points". I am also going to give you some reasons why running your own web server might not be for you.

Why you don't want to run your own web server?

If you are looking to sell something online you should look at a business web hosting solution. Two reasons. One, you are not going to be able to harden your server enough to make it secure for conducting business transactions. Second, you need to ensure that your site is available to your customers 24/7/365 or as close to 100% uptime as you can get. With web hosting you can typically get 99% uptime. You are very unlikely to come close to this trying to run your own server. Your uptime will be based on the uptime of your internet service provider. Plus every time you update Windows you will likely need to reboot your machine. And, are you familiar with...

- How to setup and configure a router?
- The difference between a physical and a software firewall?
- Do you know how to setup, configure, and secure a web server?
- Do you know enough about networking to run and maintain a server?

It is certainly within the realm of a fairly technically savvy person to setup and administer their own web server on the internet, but it takes some work, and any claims that it is easy to do should be met with guarded skepticism. This advice is coming to you from someone that ran their own web server on the internet for almost two years.

On the other hand it is pretty easy to (relatively speaking) setup and run a web server against your localhost (127.0.0.1) address. This web server will only be available to you local PC, but can serve as the test platform for your web site.

Some direction on running your own web server

Well, if you have read the last section and are still reading then you are looking for some direction. I am going to be offering information from the perspective that you are somewhat familiar already with the terms I am throwing about. The links I have provided here will go into more detail. Again, setting up and maintaining your own web server is not easy and requires a broad range of technical knowledge.

I will be showing you how to setup the Apache web server on your PC running under Windows. I am running Windows XP - Professional. You can run a web server using Windows XP - Home or Windows 2000. I mention this as any specific guidance I am giving you might vary if you are using something other than XP Professional. Also, I have not tried doing this on any other version of Windows.

Getting the software

You can go directly to Apache and get the webserver. But, what if you want to add blog software, forum software, Php, and other packages that do not come standard with the web server itself? Like I said this stuff is not easy. But wait, I am about to turn you on to one of the most comprehensive web packages available. This web package could probably be installed by your Grandmother without difficulty! All you need to do is download the file (it is a huge 186mb file). Once you download it installation is so simple it is ridiculous. The folks that did this are just brilliant. Most software shops could take a lesson from them in installation and configuration management.

This software comes bundled with blog software, forum software, MySQL database, apache security module and much more. The price is right as well. This site is not really easy to find in the search engines which goes to show that sometimes some of the best stuff on the web is buried quite deep. The url is http://www.devside.net/

Figuring out if IIS (Internet Information Server) is enabled?

I am assuming that you are going to install and use Apache as your web server. As such you need to see if IIS is running on your machine as there might be a port conflict as port 80 is the default port.

Some misc. advice (You may likely need to do a bit more digging to deal with the advice given here.)

Setting up your router

- DO NOT open up your entire DMZ to the internet are you will be toast!!!
- Only open up port 80 and port 23 (if you plan on letting people download stuff using FTP. Note, they can download files using http which runs on port 80).

Port Forwarding
- Find a service like ZoneEdit. This is what I use. It is free for low to medium traffic sites.

Your ISP

- Some ISP's block port 80. You can still run your webserver, but it will have to be done on another port (81, 82, etc...)
- Some ISP's have policies that do not allow their customers to run websites. You should read your internet service agreement for details.

For another decent link to a site with good information on setting up a web server go to http://www.diywebserver.com/

Eric Matthews is the owner of http://www.anglesanddangles.com which provides information on web development covering technologies like PHP, Html, Perl CGI, as well as other resources to assist you with your website.

Web-Based SMTP Server Monitoring

Web-based monitoring of SMTP email servers is normally conducted using one of two different approaches. The first uses a socket level connection to verify whether or not the target server is operational. The second more comprehensive method involves sending round-robin messages to your email system and its subcomponents. Depending on your needs, hardware, and company size both have validity and a rightful place. To determine what is best for your company, let's examine both in more detail and the advantages and disadvantages of each.

Socket connection is the most prevalent method used by web-based monitoring companies due to its simplicity. A DNS check is normally conducted, then a socket level connection to port 25 of your email server is attempted. If DNS is ok and SMTP port 25 handshaking occurs, the server is viewed as operational. If not, the appropriate notification methods are triggered. The socket connection method advantages are: speed, low bandwidth, and no agent or server changes are required. Because of its simplistic design, socket connection monitoring can be used to monitor SMTP servers your company does not own or manage. The disadvantage to this approach is accuracy. Just because DNS is operational and handshaking occurs, it does not mean the server or subcomponents are actually processing messages.

The second, less used, method involves a round-robin process that actually sends test messages to your email systems and waits for the message to be returned. As with the socket connection method, a DNS check is normally performed first. A test message is then sent by the monitoring company to your email system. The message travels through your gateways, any subcomponents such as antivirus and content/spam filters, through the firewall to the designated internal server. The internal server, based on a pre-established server-based rule, returns the message back to the monitoring company. If the message is received by the monitoring company within the preset time, 15 or 30 minutes for example, the email system and subcomponents are viewed as operational. If not, the appropriate notification methods are triggered.

If you are using Microsoft Outlook & Exchange, a server-based rule would look something like this: Apply this rule after the message arrives from EmailVerify@SiteRecon.com and with "Email Verification for" in the subject forward it to "Webmaster@SiteRecon.com

The advantage of this approach includes the ability to check the complete message process for failures. For example, a gateway checked using the socket connection method could be fully operational, yet a connected spam filtering appliance or antivirus process could be hung halting message processing. Depending on the monitoring company, the ability to include a specific type of attachment to the round-robin process is also available. Including an attachment such as the Eicar.Txt text pattern for example, permits subcomponents such as antivirus to be fully tested. The disadvantages to this approach are: increased processing by your servers as this testing method sends 96 messages per day using a 15 minute monitoring interval; creation of an email account to receive the test message; and establishment of a server based rule.

The diagram outlines the process for both round-robin and socket connect monitoring methods.

Clearly, the round-robin method offers a more comprehensive testing process than the socket connection method. However, depending upon your needs both process have a rightful place in verifying your email systems are up and operational.

If you do not have the ability to establish server based rules, your only choice is to use monitoring based on the socket connection method. If you have the ability to establish rules, the round robin process offers a much more comprehensive testing method.

Socket connection type monitoring is offered by a wide range of companies and prices vary widely. Round-robin processing is offered by SiteRecon and Websitepulse and range from $11 to $29 per month depending on the monitoring frequency.

Lew Newlin?is CTO of Information Solutions, Inc. that operates SiteRecon.com. SiteRecon specializes in email monitoring?and web site monitoring?for Internet service providers and businesses

Monitor the Uptime of Your Websites and Servers

Websites are nowadays used to do business and earn profit. If you belong to those who own such a website or just planning to start on-line business, you will have to monitor the uptime of your websites or servers. Do not let your clients to let you know about your downtime.

There are companies, which websites are essential for their business success, as they are generating the most or the whole of the company's revenue. Maybe also you belong to them. When this kind of website (server) is down, your business is affected. If your website is hosted by another company, you probably have some sort of "uptime" guarantee. Nevertheless, what exactly is uptime and how do you measure it. The basic definition of uptime is the period of time when your site is up and running, being accessible and able to satisfy customers.

The opposite, when your site is not accessible, it is down. And that is how you measure downtime. Faulty routers, malfunctioning LANs, or a loss of electrical power typically causes most of the downtimes; only rarely are caused by a natural disaster. Most often, it is caused by failures from the telecommunications company or an application failure, not a fire, flood or other catastrophe.

The cost of downtime calculation of course depends on many factors, like the products and services you provide, the size of your company, the number of online sales, etc. Even if you do not sell any products or services online, there is still a cost of downtime, but then in terms of damage of reputation etc. Calculate the cost of downtime is difficult and varies from one company to another. There is no "average business", so the statistics that quantify the cost of downtime for average businesses are not helpful.

The cost of downtime include direct costs such as the labor charges for a team of technologists who had to resolve the outages. The indirect costs are much more difficult to calculate and include potential lost revenue, reductions in worker productivity, damaged reputation with customers and in the marketplace, lost future sales and the cost of storing unsold goods. Financial analysts and accountants at your company can help you come up with the factors for your particular business.

If your website is generating profit for your business, you will for sure monitor it by an external monitoring service. The on-line market offers you to choose from different services to get uptime reports, statistics and get notifications as soon as your website is down. You can not let your customers to inform you about your own downtime.

To understand the importance of being aware of downtime, and to be alerted as soon as possible when a problem occurs, have a look at these numbers:
- According to a report by Cumulus Research Partners website downtime, caused by problems such as network failures, costs European businesses more than ? 5 billion a year.
- In the automotive industry downtime is often worth some $1.000 a minute.
- According to a recent study conducted by ARC Advisory Group, accounting for nearly five percent of total North American production, more than $20 billion is lost each year due to unscheduled downtime. Clearly, the traditional "fail and fix" approach to maintenance is no longer a viable MRO strategy.

Many hosting providers guarantee 99 % uptime. 99 % uptime sounds good, but means that your site could be down more than 3.5 days a year, and in today's Internet that is unacceptable. Comparing to that, 99.9 % uptime is much better, but with 8 hours and 45 minutes of possible downtime do not have to be always enough. The table below shows the percentage of uptime and the related downtime per year.

99 % ........ 87 hours, 36 minutes (more than 3.5 days)
99.9 % ...... 8 hours, 45 minutes, 36 seconds
99.99 % ..... 52 minutes, 33.6 seconds
99.999 % .... 5 minutes, 15.36 seconds
99.9999 % ... 31.68 seconds

As you can see, a 99 % uptime may not sound too bad, but it can cost you quite a lot of loss in revenues.

There are more than 30 uptime monitoring websites worldwide, while some of those have also affiliate partners. Among those are: Alertra, AlertSite, Dotcom monitor, InternetSeer, Jaguard, RedAlert, SiteUptime, WatchMouse, WebsitePulse and others. Almost all of them offer some prepaid packages based on monthly fees, varying by number of possible monitored sites (servers), additional services and by the complexity of the service. The price of their basic prepaid package is $5 - $40 per month. For this price you can monitor only 1, or maximum of 3 devices, choose from quite a lot of protocols, allow to send notifications to multiple contacts via e-mail, IM, pager or SMS, allow to choose between daily, weekly, or monthly reports by email and of course uptime performance and statistics available on-line. The Professional packages can go up to $180 per month or even higher.

There are just two completely free 24/7 monitoring services: Montastic and mon.itor.us.

Montastic's (http://www.montastic.com) biggest advantage is simplicity of the service, while it offers just basic service including real time monitoring and alerting by e-mail one contact person - the registred user, or by RSS feed, when the website is down and when it is back again. It allows to monitor only http websites, limited to the number of 100 (what is not a limit at all), checking it every 10 minutes from two different locations.

Meanwhile mon.itor.us (http://mon.itor.us) provides network, website and server monitoring service supporting 11 protocols with possibility to monitor unlimited number of devices and alert unlimited number of contact persons by e-mail, RSS feed, IM message, pager or SMS. There are also other remarkable features as personalized interactive interface, where you can add server performance and availability tests, set daily, weekly or monthly reports sent by e-mail. Tests are performed from 3 geographically distributed servers, and they are always adding more interesting features as they are still in beta version.

Peter Cernak works at Sourcio (http://www.sourcio.com). The company develops mon.itor.us and offers a wide range of IT services and Open Source solutions for small and medium-size businesses

Hard Disks for Web Servers Explained

A web server's hard disk drives are vital because they are where your website data is actually stored. There are several important technical terms to describe hard disk performance. They sound complicated at first, but they are actually very simple. Knowing what these terms mean can help you to choose a hard disk drive, or understand whether a web hosting company is giving you a good deal on a server. Here are the main terms used to describe hard disk drives.

Capacity-- How much data can be stored on the drive. This is measured in Gigabytes (GB). Note that hard disk manufacturers use what is known as a decimal Gigabyte, which is 10 to the power 9 or 1,000,000,000 bytes. Computers use what is known as a binary Gigabyte, which is 2 to the power 30 or 1,073,741,824 bytes. Therefore the hard disk capacity in GB claimed by the manufacturer is smaller than the number your computer will report. This has been standard practise for many years, but it can catch out beginners. You will need to choose a disk with several times more capacity than your website takes up, in order to leave space for future growth.

Spindle Speed-- This is simply how fast the platters (the hard metal disks that store the data) rotate. It is usually between 5,400 and 15,000 rpm. In a low to mid range web server you should be looking for a 7,200 rpm drive. A high end server would have 10,000 or 15,000 rpm drives. The faster the spindle speed, the faster the drive can read and write data, so faster is better.

Seek time-- A measure of how long it takes for a hard disk drive to move its heads to the correct part of the platters to find the data that it is looking for. This is measured in milliseconds (ms). This will be somewhere around 10 ms, give or take 5 ms. The lower the figure, the faster the drive finds the data.

Cache-- The drive stores some data in Random Access Memory known as the cache buffer. It is much faster to access the buffer than it is to find data on the drive. The drive controller keeps data that you have recently used in the cache in case you need it again, and it also reads data from the disk that it thinks you might ask for next. This makes it faster to access data, so long as the data you asked for is in the cache. Otherwise the drive will have to find it on disk. Cache memory is especially important for accessing small, frequently used files. Cache sizes start from around 8 Megabytes, the more the better.

Sustained Transfer Rate-- How fast the drive can transfer large files that do not fit into the cache buffer, or small files that were not in the cache. In other words, this is the maximum rate that the drive can read files from the disk itself. The greater the rate, the better, especially if most of your data is stored in large files.

Interface-- This is the connection between the drive and the computer's processor. Older servers use PATA, also known as Parallel ATA or IDE. Newer servers use SATA, also known as Serial ATA. High-end, expensive servers use SCSI, and very high-end servers use Fibre Channel. If you are just starting out with your first dedicated web server, you will probably want a SATA hard drive.

RAID-- This is where two or more hard disks are connected together, which gives improved speed and/or reliability compared to a single hard disk drive. You should consider RAID hard disks for your web server to improve reliability.

Now you know the vocabulary, you will be better able to choose hard disk drives when you rent or buy a web server.

Jimmy Martin is a writer and webmaster. You can read more of his guides at www.eejitsguides.com

Virtual Dedicated Server Hosting - The Best of Both Worlds

Virtual Dedicated Server hosting, or VDS, is the term for a physical server which, by means of software, has been partitioned into a number of "virtual" machines operating independently of each other as if they were standalone dedicated servers.

VDS - Advantages

The big advantage of VDS over a fully dedicated server is cost. Typically priced at less than one fourth the cost of full dedicated hosting, VDS hosting provides nearly all the same advantages. And compared to a shared web hosting plan, VDS is only slightly more expensive providing an excellent balance of economy and control.

With a VDS all physical resources such as memory, disk space, and the CPU continue to be shared, but each virtual server operates in a fully independent manner. For example, each VDS can run its own unique operating system and can be set up in any configuration imaginable.

The main advantage of a VDS is that each administrator maintains control at the root directory level of his or her own virtual server. Access at this level permits the web site owners to do a lot of neat things:

• Administer their own file directories
• Assign domain names that resolve to a primary domain without having to go through the web host
• Create and add e-mail accounts
• Assign email addresses
• Manage their own log files
• Maintain passwords
• Set permissions
• Delete and install software

VDS administrators effectively have the ability to do whatever an administrator can do on a real, physical server.

Security

Besides providing significantly greater control over the web hosting environment, the virtual dedicated server hosting is inherently more secure than any shared hosting server can be. Because web sites hosted on a shared server have to use the same operating system, a hacker who gets access to the server's root directory could damage all the websites residing on that server.

On the other hand, A VDS is compartmentalized such that a hacker gaining entry via one account could in no way access any of the others. Each VDS is totally invisible to each other and it's impossible to transfer root-level access capability from one account to another.

Virtual Dedicated Servers can be configured by the hosting company in a variety of ways so it's important that you understand how your web host has allocated the server's resources.

Typically the host would distribute the physical server resources evenly based on how many accounts and/or web sites are being hosted. Thus, with ten virtual servers on one physical server, each would be allocated ten percent of the storage, bandwidth, memory, and CPU time.

VDS - Disadvantages

Interestingly, the disadvantages of hosting on a VDS are pretty much the same as what the advantages are. If you're not sure what you are doing, having the full control that virtual dedicated hosting allows can be very risky.

For example, you can accidentally delete files, or set permissions inappropriately, or allow viruses and other malware to enter the system and, overall, just make a real mess of things. So if you lack the technical skills to administer a web server, and are unable or unwilling to learn them, virtual dedicated hosting may not be a good choice for you.

However, if your site has reached a size where shared web hosting no longer accommodate the demands of your business, virtual dedicated hosting can provide a highly cost-effective alternative to "real" dedicated server hosting.

The Bottom Line

When selecting a virtual dedicated hosting plan, make it your business to understand how all the server resources are allocated, what the available options are for operating systems, how many VDS accounts will reside on each server, as well as the terms and conditions for upgrading and downgrading to a different plan.

Virtual dedicated hosting is the real deal when you need a robust yet economical home for your web site.

Anthony Hamill is the founder of Web-Host-Watch.Comwhich provides webmasters with all the information they need to select a web hosting plan that's right for them.