For the previous year and a half, I have worked for ITS Security at The University of Rhode Island doing such things as managing Snort sensors, being a code monkey and such. When i started here, we were using BASE to view our snort alerts. At the time, our snort signature management was of the “a file is either enabled or disabled” variety; there was no further discretion. As such, filtering through all of the noise was a challenge. More challenging, perhaps, than at other sites running snort: our residential network plays host to a variety of traffic, the likes of which was surely not in the thoughts of people who wrote the rules. (Someone fetches a server list for an online game, and it triggers the storm worm rules? Oh boy.) My first project here was to write software to aide us in filtering through the hoise. The result of that project was SMAC: Snort Management and Assessment Center. (We come up with an acronym, then figure out what it means; the M really has no meaning.) SMAC was made up of profiles, and each profile contained a list of ips and signatures which were considered high fidelity. As an example, one could have a SMAC profile containing ip addresses of known botnet servers, and a selection of high-fidelity rules for detecting command and control channels. Upon going to SMAC, one could select a profile, and then begin viewing alerts in that profile. It is important to note that SMAC was not an alert viewer entire of itself, but rather, an enhanced searching interface to BASE; alerts were still viewed in BASE, and there were many links to BASE. At the time, SMAC seemed like a step in the right direction. And so we continued to use SMAC for about a year. Over that year, features were added to SMAC. My favorite added feature was the addition of a “GROUP BY” to SMAC’s main query. One could select fields to group on: source ip,destination ip, signature. The GROUP by allowed us to “ignore” repeat offenses and greatly aided in log analysis. Using the groups, SMAC would query as normal, but note the number of alerts returned in a GROUP. Clicking on this number brought the user to a page where he could view the alerts in that group. As a final level of drilling down, one could still view the alert in BASE, search for an IP in BASE, or search for a Signature from BASE.
Although SMAC was great, we had a huge problem: BASE was too slow. BASE is designed to work in your environemnt; it is not designed for maximum speed. When our database grew upwards of 1 million alerts, BASE would crumble under the pressure. The solution we finally arrived at was aanval, which we are using to date. Aanval’s lack of advanced searching features prompted me to overhaul my rule management strategies: instead of “ignoring” the noise, I had to STOP the noise. I accomplished this by using Aanval’s sensor management tools and the built-in signature management. The signature management seemed pretty awesome for the first few seconds, but I quickly realized that it was not. To begin with, aanval does not know how to count: it displays an incorrect rule count for many of our rules files. I deduced that this was partially due to the fact that the aanval people don’t know how to parse rules: they failed to account for the fact that there can be whitespace between sid: and the sid. (this was never the case with the community rules, and was inconsistent in the Emerting Threats rules.) They also failed to account for newlnes (slash-escaped) in rules. Rules that aanval could not parse (if that’s what you call it) could not be enabled. I submitted a bug report with the specifics on this, and was told that it would be fixed. I also suspsect that aanval does not delete an SID from the enabled list when it is no longer in the file. Over time, this would skew the enabled count. Meanwhile, I continued using the signature management in aanval. This lead to other gripes: Aanval (stupidly) keeps track of an enabled list of SIDs, as opposed to a disabled list. When the signatures were updated, I had to go through them and enable the new ones. This might sound simple, but it required remembering which ones I had intentionally disabled.
And then Aanval 4 was released. Oh boy was I happy! The aanval team spent months hyping this up; it was rumored to be the best thing since silced bread! I installed Aanval 4 and even tried it for about 5 minutes. I was not impressed. It was clear to me that the aanval people are just random peopole making a security-related application. One might assume that they have experience doing the task that their application is supposed to accomplish, but it is clear looking at aanval 4 that they haven’t a clue as to what security is. The new features included the ability to change my background, and ohhhhhhh a flash-based interface? So now my IDS looked pretty, but it appeared to have 1/2 the functionality as before. The signature management system which I had loved so much was GONE. Let’s recap: Aanval is now UGLY and USELESS. A few weeks later, the signautre management system was re-released. I guess they jumped the gun on the inital release. I was happy about this, as I figured those pesky bugs were finally fixed. Nope. The code was exactly the same, and had exactly the same flaws as the previous version. In fact, most of Aanval 4 seemed exactly the same as aanval 3…. well, excluding the clunky, revammped, shit-pile GUI. I resubmitted the bug report, this time, including a *patch*. They’d have to be pathetically lazy not to fix it then. Update: It is partially fixed. The rules spanning multiple lines is still not fixed, but the (rev|sid):\s*[0-9]+; is *sort* of fixed. They used [ ]* where they should have used \s*, so tabs are not accounted for. *sigh*.
Still not fixed, and my snort rules are only getting more out-of-date. The time came, last week, to take matters into my own hands. I wrote my own signature parser class in PHP, using the Snort source code as an example (actually, that was second rewrite. First time, I did it however the hell I wanted. Second time, I parsed it character-by-charcter, used states, et cetera. That version is far more “intelligent” than the snort signature parser. The third time, I finally decided to read the snort code and figure out how they did it.). As it turns out, snort rule parsing is very odd. It essentially breaks the rule options into an array of tokens by splitting at semicolons — even if the semicolon is inside a quoted value. As it turns out, quotes don’t really have any syntactical meaning in snort rules. I discovered this while trying to figure out how a rule was still being parsed with a regex that contained a doulbe quote (example — pcre:”/”/”; is parsed “correctly” by snort.) When i was coding a stateful parser, I expected these nested quotes to have syntactical meaning, but upon learning that they did not, I figured that snort kept track of an in_pcre state or something, so I added this so quotes had no syntactical meaning inside regex. I did this by using the / as a delimiter for the regex, but later learned that it’s also okay to use an unescaped / in the regex. It is not, however, okay to use an unescaped semi-colon in the regex. A semi-colon would cause snort’s mSplit(…) function to split the regex. ODD.
Armed with my signature parser and KISS, I set about writing my own signature management system. That was about four hours ago; I’m “done” now. About as done as I’ll be for today, anyway. It’s prettier than aanval’s. It works. (also different from aanval!). It’s simple for now, though I plan on unsimplifying it in the future. The code could use some neatening as well. But for now, it works. See scripts for the signature parsing code (I’ll continue to update that, most likely). As for the signature management, it needs work. Maybe I’ll release it when i’m ready, or if there’s demand, which there won’t be because no one will read this anyway :). proofread… later.