Why Fail2Ban Can Fail With VoIP

Wednesday, January 26, 2011

J. Oquendo


Plenty of engineers and administrators have been raving about using Fail2Ban in Asterisk, Trixbox and similar deployments and while there is some value to using it, sometimes the value collides with the risks of using Fail2Ban.

Because many have swore by Fail2Ban, I will explain why I choose not to rely on it and before someone comes around to state: "you're biased, you don't know what you're talking about, your data is skewed, you have a narrow point of view..." or other similar comments, I will explain my views as concisely (READ ACCURATE NOT ARROGANT) as possible.

Firstly, VoIP does not revolve around Asterisk, derivatives of Asterisk, Linux variants, BSD variants and so on. So for those touting Fail2Ban as the next best thing for counter VoIP attacks, you are the ones seeing protection and security from a narrow scope. For I have clients who use Allworx as well as pbxnsip which run on Windows. Fail2Ban? Right.

Secondly, there is a big difference in an individual instance of Asterisk versus working at the carrier level. Fail2Ban will never see the light of day on any of my Session Border Controllers. It would not make sense to load up my processor which is supposed to pass packets as quickly as possible for the sake of filling up any iptables rules on addresses that will constantly change.

Lastly... 'Blacklist this why don't ya!!!'

Let's get into the butchery of using Fail2Ban on a Managed Service Provider slash carrier scale. For starters, Fail2Ban's documentation concerning VoIP is limited to Asterisk [1] but never fear as we're engineers this means that technically, we should have the capacity to modify Fail2Ban as we see fit.

With that side, pick your poison, how many iterations of Fail2Ban implementation do we want to mix and mingle to get it working. For the "high end customers" using Avaya IP Office and say Cisco Call Manager on Windows, you're SOL [2], but again, never fear, there are alternative solutions out there. They're called "good design and administration."

So for the first counterpoints: "The world does not revolve around Linux, BSD, Asterisk, etc.," let's assume it did - the world did revolve around Asterisk and or Linux. We'll assume every PBX installation is an Asterisk installation on Linux, pick your own distro. We fire up Fail2Ban and begin blacklisting away. As a managed service provider, I see plenty of SIP devices lose registration on a constant basis.

Could be that they reset themselves, could be an update that broke authentication, could be someone simply reset the device. In any event, the device is now registering incorrectly. Fail2Ban to the rescue. On the one hand, it could alert me to my client not being able to register, on the other hand, it will also block my client.

As a carrier, sometimes my role is to validate the claims of a client having an issue. This means, I have to at times, register devices as one of my clients in order to replicate the problem. Oops, human error, I entered the wrong password, did I remember to *whitelist* myself? Was I working from home today? Humans make mistakes, irrelevant? Sort of.

Moving along though, from the carrier perspective where a PBX can have thousands of registrations, do I want to add the additional task of administrating bans caused by Fail2Ban? The answer is no.

  • Customer: "I can't connect"
  • Engineer: "I don't see why not..." (thinks, let me check iptables (remember I did want to side with you on Asterisk and Linux))
  • Engineer: thinks to himself "aha! Darnit, they were blocked with Fail2Ban"

How much time do you think you could spend diagnosing this issue? Let's say 1 minute per day. One minute per day in this instance multiplied by how many clients? See how this can be counterproductive? Logging into ONE of my current managed PBXs. This one PBX currently has 6720 clients on it. Overall I manage somewhere to the tune of 75 PBXs and trunk with over 1500 companies.

Total call records : 155703
(min): 854960:51
ASR: 69.50%
Successful: 148214
Shortest: 0.01
Average: 7:54
Median: 2.24

If I average say 1,000 clients per PBX I manage, this could mean 75,000 accounts spread across who knows how many networks. If 1% of my managed clients complained, I'd waste 750 minutes - if I spent 1 minute on each client - to diagnose an issue. 12 1/2 hours per day wasted. Let me half that number and spread it across the week. 6 1/4 hours per week in wasted time. Do I want to give myself another job by deploying Fail2Ban? Not really.

Moving to the second counterpoint - the carrier level. As an ITSP VoIP slash security engineer, I have zero knowledge of who should be registering to a SIP trunk I provide. Once I provide a SIP trunk to my peer, they're on their own. What's suggested here by the Fail2Ban fanclub? Should I step in the mix and block registrations on a PBX I don't manage? That probably would not only break SLAs but could lead to a potential lawsuit.

Furthermore, the goal of my SBC is to process a call as soon as it gets in, not go through the millisecond motions of determining "who is on first, what is on second" it is not the sort of role I want my SBC playing. Let me be fair about this though, so I decide to side with those who believe in Fail2Ban and somehow deploy it on my SBC. At what point in time does my blacklist become a nightmare.

Finally, because I don't want to continue down this explanation road... "Blacklist this why don't ya!!!"

samplepbx:~/sipvicious# cat rogueaccounts.txt
yourethe weakest link goodbye
yourethe weakest link goodbye
yourethe weakest link goodbye
samplepbx:~/sipvicious# ./svwar.py -d rogueaccounts.txt samplepbxserver.com & echo sayanora

Would you be willing to place your bet I can force Fail2Ban to accept a higher prefix? I don't want to throw out ideas for attackers, but in order for you to properly blacklist, your whitelist game better be better than the malice an attacker can think of. A simple seq in a loop can force Fail2Ban to block off multiple networks in seconds. What is you're fix then? Whitelist all, blacklist after. Counterproductive don't you think.

Remember, security is a cat and mouse game. You fix it, someone will break it. So for those who misinterpret my writings and confuse "arrogance" with experience, perhaps you haven't been exposed to the amount of VoIP I come across.

Now for those who want to go a step further and state: "wait you hypocrite... You run a blacklist!!!" [3] I suggest trying to think outside of the box about the information I post with my blacklist and how it differs from Fail2Ban. To do this, you need to understand what goes on with a VoIP attack. My inference based on SOLID data from a running honeypot points to the following:

Attacker (let's say at --> launches sipvicious scan --> mypbx

So an attacker launched a scan, who cares, it happens and even if I outright blacklist him, odds are, he is on a throwaway address or compromised host. Nevertheless he scans. At some point in time, he WILL come across the honeypot I left in plain sight for him. Attacker now gets an account and registers:

Attacker (NOW COMING FROM --> registers --> mypbx

See the dilemma here? Data shows that an attack USING a compromised account is almost NEVER the same as the one doing the scans. Blacklist? Failed2Ban So why post the data on the VoIP Abuse Project? There is a method to the madness in the sense that if you DO see a bruteforcer who is constantly on the list, odds are, that machine is heavily compromise and in constant use.

So if you're say a law enforcement official, you DEFINITELY want to keep an eye on that machine. Further, my blacklist consists of more than just bruteforcers, in fact, they don't even bug me to be honest with you. What keeps me interested are the accounts actually registering and making calls:

$ wget -qO - www.infiltrated.net/vabl.txt|awk '/ADN/' | tail -n 1 | ADN | VABL | 4436 | | AS-NLAYER | US | ANCHORFREE.COM | ANCHORFREE INC | 01120107935053

So I have someone coming from Anchorfree who registered to my PBX and placed a call to: 01120107935053 Guess what? 1) This attacker is nowhere to be found with terms like "wrong password", "Username/Auth mismatch", "registration for peer" or any other term Asterisk uses for errors. Fail2Ban? WILL ALWAYS fail to see this. "But why post the data" again, if you don't see the benefit of it, I don't know what to tell you.

From my perspective, its a study of VoIP based attacks with a nudge to others: "Hey, this rogue host CONNECTED AND MADE CALLS through my system you DEFINITELY want to block him and be on the lookout for the number he called!" Or... "Hey these are the rogue hosts connecting to me. See that one netblock right there? They're definitely worth blacklisting"

wget -qO - www.infiltrated.net/vabl.txt|awk '/ADN/ && /TEDATA/{print $9|"sort -u"}'

This differs from outright blocking everyone who knocks on your door. Also, aside from all of this, I ALWAYS suggest blocking ALL allowing in trusted period. Why bother with the headaches? So when I receive messages like:

"I deploy on-site PBXes based on a souped-up and super-customized version of trixbox, with an added layer of security using Fail2ban, a generic Intrusion Protection script that I improved and customized for Asterisk-based systems.

I'm sorry, I erroneously thought you were interested in serious cooperation, and not a self-righteous git.

Your blurb about Fail2ban is full of wrong assumptions, and your Mr.-Know-It-All attitude defines pretty much the picture."

Please take the time to understand that while your point of view may be correct for your current deployment/situation where you have a single PBX running on your "souped up, super customized, uberduber version of Trixbox" the reality is the world doesn't revolve around Trixbox, Asterisk, Linux, Fail2Ban, etc., there are always two sides of the equation here. I sincerely doubt you're going to push 9,000,000 minutes of traffic daily over your Trixbox install successfully with Fail2Ban.

[1] http://www.fail2ban.org/wiki/index.php/Category:VOIP

[2] http://www.urbandictionary.com/define.php?defid=676823&term=SOL

[3] http://infiltrated.net/index.php?option=com_content&view=article&id=17&Itemid=23


J. Oquendo


Possibly Related Articles:
VoIP SLAs HoneyPot Fail2Ban PBX
Post Rating I Like this!
The views expressed in this post are the opinions of the Infosec Island member that posted this content. Infosec Island is not responsible for the content or messaging of this post.

Unauthorized reproduction of this article (in part or in whole) is prohibited without the express written permission of Infosec Island and the Infosec Island member that posted this content--this includes using our RSS feed for any purpose other than personal use.