Eight Ways to Blacklist with Apache’s mod_rewrite
mod_rewrite. In addition to facilitating
site security, the techniques presented in this article will improve your
understanding of the different rewrite methods available with mod_rewrite. Blacklist via Request Method
This first blacklisting method evaluates the client’s request method. Every time a client attempts to connect to your server, it sends a message indicating the type of connection it wishes to make. There are many different types of request methods recognized by Apache. The two most common methods areGET and POST requests, which are required for “getting” and
“posting” data to and from the server. In most cases, these are the only
request methods required to operate a dynamic website. Allowing more request
methods than are necessary increases your site’s vulnerability. Thus, to
restrict the types of request methods available to clients, we use this block
of Apache directives:<IfModule mod_rewrite.c> RewriteEngine On ServerSignature Off Options +FollowSymLinks RewriteCond %{REQUEST_METHOD} ^(delete|head|trace|track) [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>REQUEST_METHOD
in the rewrite condition. First we invoke some precautionary security measures,
and then we evaluate the request method against our list of prohibited types.
Apache will then compare each client request method against the blacklisted expressions
and subsequently deny access to any forbidden requests. Here we are blocking delete and head because they are unecessary, and also blocking trace and track because they violate the same-origin rules for
clients. Of course, I encourage you to do your own research and establish your
own request-method security policy.Blacklist via the Request
The next blacklisting method is based on the client’s request. When a client attempts to connect to the server, it sends a full HTTP request string that specifies the request method, request URI, and transfer-protocol version. Note that additional headers sent by the browser are not included in the request string. Here is a typical example:GET blog/index.html HTTP/1.1This long request string may be checked against a list of prohibited characters to protect against malicious requests and other exploitative behavior. Here is an example of sanitizing client requests by way of Apache’s
THE_REQUEST variable:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{THE_REQUEST} ^.*(\\r|\\n|%0A|%0D).* [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>Blacklist via the Referrer
Blacklisting via the HTTP referrer is an excellent way to block referrer spam, defend against penetration tests, and protect against other malicious activity. The HTTP referrer is identified as the source of an incoming link to a web page. For example, if a visitor arrives at your site through a link they found in the Google search results, the referrer would be the Google page from whence the visitor came. Sounds straightforward, and it is.Unfortantely, one of the biggest spam problems on the Web involves the abuse of HTTP referrer data. In order to improve search-engine rank, spambots will repeatedly visit your site using their spam domain as the referrer. The referrer is generally faked, and the bots frequently visit via
HEAD requests for the sake of
efficiency. If the target site publicizes their access logs, the spam sites
will receive a search-engine boost from links in the referrer statistics.Fortunately, by taking advantage of
mod_rewrite’s
HTTP_REFERER variable, we
can forge a powerful, customized referrer blacklist. Here’s our example:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{HTTP_REFERER} ^(.*)(<|>|'|%0A|%0D|%27|%3C|%3E|).* [NC,OR] RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?adult(-|.).*$ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?poker(-|.).*$ [NC,OR] RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-|.)?drugs(-|.).*$ [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>HTTP_REFERER
variable and as many rewrite conditions as necessary. In this case, we are
blocking a series of potentially malicious characters in the first condition,
and then blacklisting any referrer containing the terms “adult”, “poker”, or
“drugs”. Of course, we may blacklist as many referrer strings as needed by
simply emulating the exisiting rewrite conditions. Just don’t get carried away
— I have seen some referrer blacklists that are over 4000 lines long!Blacklist via Cookies
Protecting your site against malicious cookie exploits is greatly facilitated by using Apache’sHTTP_COOKIE variable. HTTP
cookies are chunks of data sent by the server to the web client upon
initialization. The browser then sends the cookie information back to the
server for each subsequent visit. This enables the server to authenticate
users, track sessions, and store preferences. A common example of the type of
functionality enabled by cookies is the shopping cart. Information about the
items placed in a user’s shopping cart may be stored in a cookie, thereby
enabling server scripts to respond accordingly. Generally, a cookie consists of a unique string of alphanumeric text and persists for the duration of a user’s session. Apache’s
mod_cookie module generates cookie
values randomly and upon request. Once a cookie has been set, it may be used as
a database key for further processing, behavior logging, session tracking, and
much more. Unfortunately, this useful technology may be abused by attackers to
penetrate and infiltrate your server’s defenses. Cookie-based protocols are
vulnerable to a variety of exploits, including cookie poisoning, cross-site
scripting, and cross-site cooking. By adding malicious characters, scripts, and
other content to cookies, attackers may find and exploit sensitive
vulnerabilities.The good news is that we may defend against most of this nonsense by using Apache’s
HTTP_COOKIE
variable to blacklist characters known to be associated with malicious cookie
exploits. Here is an example that does the job:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{HTTP_COOKIE} ^.*(<|>|'|%0A|%0D|%27|%3C|%3E|).* [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>HTTP_COOKIEs containing the specified
list of prohibited characters. In this list you will see characters generally
required to execute any sort of scripted attack: opening and closing angle
brackets, single quotation marks, and a variety of hexadecimal equivalents.
Feel free to expand this list with additional characters as you see fit. As
always, recommendations are most welcome.Blacklist via Request URI
Use of Apache’sREQUEST_URI variable is frequently seen
in conjunction with URL canonicalization. The REQUEST_URI variable targets the
requested resource specified in the full HTTP request
string. Thus, we may use Apache’s THE_REQUEST
variable to target the entire request string (as discussed above),
while using the REQUEST_URI
variable to target the actual request URI. For example, the REQUEST_URI variable refers to the “blog/index.html” portion of the
following, full HTTP request line:GET blog/index.html HTTP/1.1For canonicalization purposes, this is exactly the type of information that must be focused on and manipulated in order to achieve precise, uniform URLs. Likewise, for blacklisting malicious request activity such as the kind of nonsense usually exposed in your server’s access and error logs, targeting, evaluating, and denying malicious URL requests is easily accomplished by taking advantage of Apache’s
REQUEST_URI variable.As you can imagine, blacklisting via
REQUEST_URI
is an excellent way to eliminate scores of malicious behavior. Here is an
example that includes some of the same characters and strings that are blocked
in the upcoming 4G Blacklist:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REQUEST_URI} ^.*(,|;|:|<|>|">|"<|/|\\\.\.\\).* [NC,OR] RewriteCond %{REQUEST_URI} ^.*(\=|\@|\[|\]|\^|\`|\{|\}|\~).* [NC,OR] RewriteCond %{REQUEST_URI} ^.*(\'|%0A|%0D|%27|%3C|%3E|).* [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>REQUEST_URI
variable. Here we are denying any URL requests containing
invalid characters, including different types of brackets, various
punctuational characters, and some key hexadecimal equivalents. Of course, the
possibilities are endless, and the blacklist should be customized according to
your specific security strategy and unfolding blacklisting needs.Blacklist via the User Agent
Blacklisting via user-agent is a commonly seen strategy that yields questionable results. The concept of blacklisting user-agents revolves around the idea that every browser, bot, and spider that visits your server identifies itself with a specific user-agent character string. Thus, user-agents associated with malicious, unfriendly, or otherwise unwanted behavior may be identified and blacklisted in order to prevent against future access. This is a well-known blacklisting strategy that has resulted in some extensive and effective user-agent blacklists.Of course, the downside to this method involves the fact that user-agent information is easily forged, making it difficult to know for certain the true identity of blacklisted clients. By simply changing their user-agent to an unknown identity, malicious bots may bypass every blacklist on the Internet. Many evil “scumbots” indeed do this very thing, which explains the incredibly vast number of blacklisted user-agents. Even so, there are certain limits to the extent to which certain user-agent strings may be changed. For example, GNU’s Wget and the cURL command-line tool are difficult to forge, and many other clients have hard-coded user-agent strings that are difficult to change.
On Apache servers, user-agents are easily identified and blacklisted via the
HTTP_USER_AGENT variable.
Here is an example:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{HTTP_USER_AGENT} ^$ [OR] RewriteCond %{HTTP_USER_AGENT} ^.*(<|>|'|%0A|%0D|%27|%3C|%3E|).* [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^.*(HTTrack|clshttp|archiver|loader|email|nikto|miner|python).* [NC,OR] RewriteCond %{HTTP_USER_AGENT} ^.*(winhttp|libwww\-perl|curl|wget|harvest|scan|grab|extract).* [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>mod_rewrite module, enable the rewrite
engine, and proceed to deny access to any user-agent that includes any of the
blacklisted character strings in its name. As with our previous blacklisting
techniques, here we are prohibiting angle brackets, single quotation marks, and
various hexadecimal equivalents. Additionally, we include a handful of
user-agent strings commonly associated with server attacks and other malicious
behavior. We certainly don’t need anything associated with libwww-perl hitting our server, and many
of the others are included in just about every user-agent blacklist that you
can find. There are tons of other nasty user-agent scumbots out there, so feel
free to beef things up with a few of your own.Blacklist via the Query String
Protecting your server against malicious query-string activity is extremely important. Whereas static URLs summon pages, their appended query strings transmit data and pass variables throughout the domain. Query-string information interacts with scripts and databases, influencing behavior and determining results. This relatively open channel of communication is easily accessible and prone to external manipulation. By altering data and inserting malicious code, attackers may penetrate and exploit your sever directly through the query string.Fortunately, we can protect our server against malicious query-string exploits with the help of Apache’s invaluable
QUERY_STRING variable. By taking advantage of this
variable, we can ensure the legitimacy and quality of query-string input by
screening out and denying access to a known collection of potentially harmful
character strings. Here is an example that will keep our query strings squeaky
clean:<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{QUERY_STRING} ^.*(localhost|loopback|127\.0\.0\.1).* [NC,OR] RewriteCond %{QUERY_STRING} ^.*(\.|\*|;|<|>|'|"|\)|%0A|%0D|%22|%27|%3C|%3E|).* [NC,OR] RewriteCond %{QUERY_STRING} ^.*(md5|benchmark|union|select|insert|cast|set|declare|drop|update).* [NC] RewriteRule ^(.*)$ - [F,L]</IfModule>QUERY_STRING
variable to check all query-string input against a list of prohibited
alphanumeric characters strings. This strategy will deny access to any URL-request
that includes a query-string containing localhost references, invalid
punctuation, hexadecimal equivalents, and various SQL
commands. Blacklisting these enitities protects us from common cross-site
scripting (XSS), remote shell attacks, and SQL
injection. And, while this a good start, it pales in comparison to the new
query-string directives of the upcoming 4G Blacklist. ;)Blacklist via IP Address
Last but certainly not least, we can blacklist according to IP address. Blacklisting sites based on IP is probably the oldest method in the book and works great for denying site access to stalkers, scrapers, spammers, trolls, and many other types of troublesome morons. The catch is that the method only works when the perpetrators are coming from the same location. An easy way to bypass any IP blacklist is to simply use a different ISP or visit via proxy server. Even so, there is no lack of mindless creeps out there roaming the Internet, who sit there, using the same machine, day after day, relentlessly harassing innocent websites. For these types of lazy, no-life losers, blacklisting via IP address is the perfect solution. Here is a hypothetical example demonstrating several ways to blacklist IPs:# block individual IPs<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REMOTE_ADDR} ^123\.456\.789\.1 [OR] RewriteCond %{REMOTE_ADDR} ^456\.789\.123\.2 [OR] RewriteCond %{REMOTE_ADDR} ^789\.123\.456\.3 [OR] RewriteRule ^(.*)$ - [F,L]</IfModule> # block ranges of IPs<IfModule mod_rewrite.c> RewriteEngine On RewriteCond %{REMOTE_ADDR} ^123\. [OR] RewriteCond %{REMOTE_ADDR} ^456\.789\. [OR] RewriteCond %{REMOTE_ADDR} ^789\.123\.456\. [OR] RewriteRule ^(.*)$ - [F,L]</IfModule> # alt block IP method<Limit GET POST PUT> order allow,deny allow from all deny from 123. deny from 123.456. deny from 123.456.789.0</Limit>mod_rewrite
and its associated REMOTE_ADDR
variable. Each of the hypothetical IPs listed represent a
specific, individual address. Then, in the next code block, we are blocking
three different ranges of IPs by omitting numerical data
from the targeted IP string. In the first line we target any
IP beginning with “123.”,
which is an enormous number of addresses. In the second line, we block
a different, more restrictive range by including the second portion of the
address. Finally, in the third line, we block a different, much smaller range
of IPs by including a third portion of the address.Then, just for kicks, I threw in an alternate method of blocking IPs. This is an equally effective method that enables you to block IP addresses and ranges as specifically as necessary. Each
deny line pattern-matches according to
the specified IP string.Dealing with Blacklisted Visitors
In each of these eight blacklisting techniques, we respond to all blacklisted visitors with the server’s default “403 Forbidden” error. This page serves its purpose and requires very little to deliver in terms of system resources, however there is much more that you can do with blacklisted traffic. Here are a few ideas:Redirect to home page
More subtle than the 403 error, this redirect strategy routes blocked traffic directly to the home page. To use, replace the
RewriteRule directive (i.e., the entire line) with the
following code:RewriteRule ^(.*)$
http://your-domain.tld/ [F,L]Redirect to external site
The possibilities here are endless. Just make sure you think twice about the destination, as any scum that you redirect to another site will be seen as coming from your own. Even so, here is the code that you would use to replace the
RewriteRule directive in
any of the examples above:RewriteRule ^(.*)$
http://external-domain.tld/some-target/page.html [F,L]Redirect them back to their own site
This is one of my favorites. It’s like having a magic shield that reflects attacks back at the attacker. Send a clear message by using this code as the
RewriteRule directive in any of our
blacklisting methods:RewriteRule ^(.*)$
http://%{REMOTE_ADDR}/ [F,L]Custom processing
For those of you with a little skill, it is possible to redirect your unwelcome guests to a fail-safe page that explains the situation to the client while logging all of the information behind the scenes. This is perhaps the most useful approach for understanding your traffic and developing an optimal security strategy. The code would look something like this, depending on your file name and its location:
RewriteRule ^(.*)$ /home/path/blacklisting-script.php
[F,L]
Không có nhận xét nào:
Đăng nhận xét