It's the weekend, and I'm lazy so I haven't read these yet. No commentary.
2WoO Day 7: Supporting New Applications the Right Way by Michael Starks
WoO Day 7 : Tidbits by Jason Frisvold
My contribution: OSSEC Rules 101
The mailing list discussion: Day 7: Making it happen: who, what, when and how?
So what did everyone think of this year's Week of OSSEC? A big thanks to Michael Starks for making this happen!
Saturday, October 23, 2010
OSSEC Rules 101
As I've already written about decoders, it's time to cover rules. Rules are phase 3 in our 3 phase plan (pre-decoding, decoding, and rules). The syntax guide for writing rules can be found in the OSSEC documentation: here.
I will still be using the ossec-logtest progam to help illustrate the rules. I'll also be starting with the same log message as before:
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
Just to refresh your memory, the beginning of the message gets chopped off during pre-decoding and decoding, so we will be left with "Failed password for ddp from 172.16.51.1 port 56588 ssh2" as the material for creating a rule. Here is the output of feeding the above message through ossec-logtest:
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
dstuser: 'ddp'
srcip: '172.16.51.1'
This will be out reference for writing the rules.
There are a number of ways for OSSEC to evaluate a log message to determine the rule that will be triggered. Much like decoders the first rule that matches will be triggered, so the order of the rules is important. Since OpenSSH is a reasonably complicated group of applications, there will probably be a few rules associated with it (much like there are a number of decoders).
All OSSEC rules start the same way:
<rule id="NUMBER" level="NUMBER">
The id is just an identification number to reference the rule. The level determines the severity of the event, and can be any number between 0 and 15. Here's a handy wiki entry detailing the various severity levels.
Since I am using examples that are already in place I can just use the id and level of the real rule:
<rule id="5700" level="0" noalert="1">
The option 'noalert="1"' means that this rule will never trigger an alert. We are going to use this rule to group all ssh rules together, so we don't want any sshd event we do not yet have a rule for to trigger Rule 5700.
It can be helpful to write a rule to help identify sshd log messages. We can do this a number of ways, but since we have an sshd decoder already, we'll match on the decoder.
<decoded_as>sshd</decoded_as>
With this option, the rule will catch all log events that are decoded (in Phase 2) as sshd. Now we will add a description to this rule to help us understand it better later, and close the rule out:
<description>SSHD messages grouped.</description>
<./rule>
Here's the rule all together:
<rule id="5700" level="0" noalert="1">
<decoded_as>sshd</decoded_as>
<description>SSHD messages grouped.</description>
</rule>
This is the first rule in the sshd_rules.xml file. Above it is the line:
<group name="syslog,sshd,">
This creates two groups called "sshd" and "syslogd." These groups can be used in rules and reports. More on this later.
With rule 5700 in place my log message would match it, but not report itself as matching it (because of the noalert="1" option). So I need another rule to match the specific message I'm seeing. We'll start it the same way as the last message, and add another option:
<rule id="5716" level="5">
<if_sid>5700</if_sid>
As you may be able to guess, this is the 17th rule in sshd_rules.xml (5700-5716). It's classified as a level 5 rule because one mistyped password may not be a big deal.
The <if_sid> option is similar to the <parent> option in decoders, this rule will only be checked if rule 5700 matched the log message. This should help optimize the ruleset, and not force OSSEC to check too many unrelated rules when a log message comes in.
The next thing we want to do is somehow match the log message. We'll add the following to do just that:
<match>^Failed|^error: PAM: Authentication</match>
What this line does is tries to match its value to the log message. Remember the "^" symbol from the decoder post? If not, the symbol means that the character immediately following it should be the first character in the log message. The pipe symbol, "|", is a logical "or." In this case the value "^Failed" or "^error: PAM: Authentication" can match in the rule. Without the pipe we'd have to create two rules, one for each string. Using the pipe means we can be lazy and only write one rule to handle both possible messages.
So with this <match> option we're looking for the first part of the log message to be either "Failed" or "error." With the above log message we do indeed see "Failed." Remember that we are only dealing with the log section, as given to us in the decoding phase (Failed password for ddp from 172.16.51.1 port 56588). So this <match> will match our log message.
We want to be able to identify what is happening, and the description is what will be sent in the alert. Here is the description:
<description>SSHD authentication failed.</description>
We also want to be able to search for these types of events (mis-typed passwords) for reports, so we'll also add a group to identify it as an authentication failure and close out the rule:
<group>authentication_failed,</group>
</rule>
This group will be in addition to the main groups posted above. The complete group listing for this rule will be "syslog,sshd,authentication_failed."
Here is the complete rule:
<rule id="5716" level="5">
<if_sid>5700</if_sid>
<match>^Failed|^error: PAM: Authentication</match>
<description>SSHD authentication failed.</description>
<group>authentication_failed,</group>
</rule>
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
dstuser: 'ddp'
srcip: '172.16.51.1'
**Phase 3: Completed filtering (rules).
Rule id: '5716'
Level: '5'
Description: 'SSHD authentication failed.'
**Alert to be generated.
As I mentioned above one failed login may not be a big deal. Bob mis-types his password at least once a day. But what happens if a bot out there is trying to brute force a password? That's a big deal, and something that should be reported. Here's a full rule to help group rule 5716 events:
<rule id="5720" level="10" frequency="6">
<if_matched_sid>5716</if_matched_sid>
<same_source_ip />
<description>Multiple SSHD authentication failures.</description>
<group>authentication_failures,</group>
</rule>
Since this is multiple attempts using the wrong password the level is now set to 10. Frequency is an option I haven't introduced yet. Frequency specifies how many times a rule must match before this rule will fire. In this case a failed login has to happen 6 times before rule 5720 will match.
<if_matched_sid> is similar to <if_sid>. There may be a difference, but I don't really know it. When creating rules with the frequency option, use <if_matched_sid>.
<same_source_ip /> just tells OSSEC that the 6 failed attempts must come from the same source IP address. If Bob, Lisa, Angela, Todd, Jason, and Herbert all mistype their passwords at the same time from different systems we don't want this alert to fire.
The rest of the rule is pretty standard so I'm not going to explain it.
Frequency can be very useful, and even combined with a couple of other options. Here is another brute force rule dealing with bad ssh logins:
<rule id="5712" level="10" frequency="6" timeframe="120" ignore="60">
<if_matched_sid>5710</if_matched_sid>
<description>SSHD brute force trying to get access to </description>
<description>the system.</description>
<same_source_ip />
<group>authentication_failures,</group>
</rule>
In addition to frequency this rule also uses timeframe and ignore. These options are pretty simple. In this rule a user has to match rule 5710 6 times in a timeframe of 120 seconds. If that 6th event is 121 seconds after the first, this rule will not match. After rule 5712 has fired it will not fire again for 60 seconds thanks to the ignore option. This is to help keep you from getting a flood of alerts.
You'll also note that the description is broken into two lines. This is purely for readability reasons, the lines will be combined in the alert.
I'm not sure why these two rules have different options. I'd like to think that if one of them has the timeframe and ignore options they both should, or neither of them should. I'll have to look into that.
Since ossec-logtest doesn't keep state between the messages fed into it I can't really show a 5720 or 5712 alert using it.
There are a lot more options available in rules. For instance you can use <regex> in a rule. You won't be able to pull information out with parentheses in a regex, that's only available in decoders. Options like <category>, <srcip>, <user>, and <category> can be used to match specific elements. <category>syscheck</category> can be used to modify a syscheck alert. Here's an example:
<rule id="10999" level="15">
<category>syscheck</category>
<match>/var/ossec/etc/ossec.conf</match>
<description>ossec.conf has been modified!</description>
</rule>
This rule would set off an alert of level 15 if syscheck has noticed that /var/ossec/etc/ossec.conf has been modified.
Since we know Bob mis-types his password at least once per day we can do the following to not have alerts fire when he mistypes his password:
<rule id="110000" level="0">
<if_sid>5716</if_sid>
<user>bob</user>
<description>Ignore bob</description>
</rule>
If we want to limit it even further we can limit this to his source IP:
<rule id="110000" level="0">
<if_sid>5716</if_sid>
<user>bob</user>
<srcip>192.168.1.23</srcip>
<description>Ignore bob</description>
</rule>
There are two more options I want to mention: alert_by_email and no_email_alert. The first always sends an email when that event is triggered, no matter what level is set. This can be useful if you have determined that an event isn't a good candidate for a high level, but you always want to see an email on it. The second means the alert never sends an email. I'm not sure how useful this one is, but I'm guessing someone wanted it.
There's still a lot of information to cover, but it'll have to wait for future blog posts. This one is long enough. If I didn't explain something here well enough please let me know!
I will still be using the ossec-logtest progam to help illustrate the rules. I'll also be starting with the same log message as before:
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
Just to refresh your memory, the beginning of the message gets chopped off during pre-decoding and decoding, so we will be left with "Failed password for ddp from 172.16.51.1 port 56588 ssh2" as the material for creating a rule. Here is the output of feeding the above message through ossec-logtest:
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
dstuser: 'ddp'
srcip: '172.16.51.1'
This will be out reference for writing the rules.
There are a number of ways for OSSEC to evaluate a log message to determine the rule that will be triggered. Much like decoders the first rule that matches will be triggered, so the order of the rules is important. Since OpenSSH is a reasonably complicated group of applications, there will probably be a few rules associated with it (much like there are a number of decoders).
All OSSEC rules start the same way:
<rule id="NUMBER" level="NUMBER">
The id is just an identification number to reference the rule. The level determines the severity of the event, and can be any number between 0 and 15. Here's a handy wiki entry detailing the various severity levels.
Since I am using examples that are already in place I can just use the id and level of the real rule:
<rule id="5700" level="0" noalert="1">
The option 'noalert="1"' means that this rule will never trigger an alert. We are going to use this rule to group all ssh rules together, so we don't want any sshd event we do not yet have a rule for to trigger Rule 5700.
It can be helpful to write a rule to help identify sshd log messages. We can do this a number of ways, but since we have an sshd decoder already, we'll match on the decoder.
<decoded_as>sshd</decoded_as>
With this option, the rule will catch all log events that are decoded (in Phase 2) as sshd. Now we will add a description to this rule to help us understand it better later, and close the rule out:
<description>SSHD messages grouped.</description>
<./rule>
Here's the rule all together:
<rule id="5700" level="0" noalert="1">
<decoded_as>sshd</decoded_as>
<description>SSHD messages grouped.</description>
</rule>
This is the first rule in the sshd_rules.xml file. Above it is the line:
<group name="syslog,sshd,">
This creates two groups called "sshd" and "syslogd." These groups can be used in rules and reports. More on this later.
With rule 5700 in place my log message would match it, but not report itself as matching it (because of the noalert="1" option). So I need another rule to match the specific message I'm seeing. We'll start it the same way as the last message, and add another option:
<rule id="5716" level="5">
<if_sid>5700</if_sid>
As you may be able to guess, this is the 17th rule in sshd_rules.xml (5700-5716). It's classified as a level 5 rule because one mistyped password may not be a big deal.
The <if_sid> option is similar to the <parent> option in decoders, this rule will only be checked if rule 5700 matched the log message. This should help optimize the ruleset, and not force OSSEC to check too many unrelated rules when a log message comes in.
The next thing we want to do is somehow match the log message. We'll add the following to do just that:
<match>^Failed|^error: PAM: Authentication</match>
What this line does is tries to match its value to the log message. Remember the "^" symbol from the decoder post? If not, the symbol means that the character immediately following it should be the first character in the log message. The pipe symbol, "|", is a logical "or." In this case the value "^Failed" or "^error: PAM: Authentication" can match in the rule. Without the pipe we'd have to create two rules, one for each string. Using the pipe means we can be lazy and only write one rule to handle both possible messages.
So with this <match> option we're looking for the first part of the log message to be either "Failed" or "error." With the above log message we do indeed see "Failed." Remember that we are only dealing with the log section, as given to us in the decoding phase (Failed password for ddp from 172.16.51.1 port 56588). So this <match> will match our log message.
We want to be able to identify what is happening, and the description is what will be sent in the alert. Here is the description:
<description>SSHD authentication failed.</description>
We also want to be able to search for these types of events (mis-typed passwords) for reports, so we'll also add a group to identify it as an authentication failure and close out the rule:
<group>authentication_failed,</group>
</rule>
This group will be in addition to the main groups posted above. The complete group listing for this rule will be "syslog,sshd,authentication_failed."
Here is the complete rule:
<rule id="5716" level="5">
<if_sid>5700</if_sid>
<match>^Failed|^error: PAM: Authentication</match>
<description>SSHD authentication failed.</description>
<group>authentication_failed,</group>
</rule>
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
dstuser: 'ddp'
srcip: '172.16.51.1'
**Phase 3: Completed filtering (rules).
Rule id: '5716'
Level: '5'
Description: 'SSHD authentication failed.'
**Alert to be generated.
As I mentioned above one failed login may not be a big deal. Bob mis-types his password at least once a day. But what happens if a bot out there is trying to brute force a password? That's a big deal, and something that should be reported. Here's a full rule to help group rule 5716 events:
<rule id="5720" level="10" frequency="6">
<if_matched_sid>5716</if_matched_sid>
<same_source_ip />
<description>Multiple SSHD authentication failures.</description>
<group>authentication_failures,</group>
</rule>
Since this is multiple attempts using the wrong password the level is now set to 10. Frequency is an option I haven't introduced yet. Frequency specifies how many times a rule must match before this rule will fire. In this case a failed login has to happen 6 times before rule 5720 will match.
<if_matched_sid> is similar to <if_sid>. There may be a difference, but I don't really know it. When creating rules with the frequency option, use <if_matched_sid>.
<same_source_ip /> just tells OSSEC that the 6 failed attempts must come from the same source IP address. If Bob, Lisa, Angela, Todd, Jason, and Herbert all mistype their passwords at the same time from different systems we don't want this alert to fire.
The rest of the rule is pretty standard so I'm not going to explain it.
Frequency can be very useful, and even combined with a couple of other options. Here is another brute force rule dealing with bad ssh logins:
<rule id="5712" level="10" frequency="6" timeframe="120" ignore="60">
<if_matched_sid>5710</if_matched_sid>
<description>SSHD brute force trying to get access to </description>
<description>the system.</description>
<same_source_ip />
<group>authentication_failures,</group>
</rule>
In addition to frequency this rule also uses timeframe and ignore. These options are pretty simple. In this rule a user has to match rule 5710 6 times in a timeframe of 120 seconds. If that 6th event is 121 seconds after the first, this rule will not match. After rule 5712 has fired it will not fire again for 60 seconds thanks to the ignore option. This is to help keep you from getting a flood of alerts.
You'll also note that the description is broken into two lines. This is purely for readability reasons, the lines will be combined in the alert.
I'm not sure why these two rules have different options. I'd like to think that if one of them has the timeframe and ignore options they both should, or neither of them should. I'll have to look into that.
Since ossec-logtest doesn't keep state between the messages fed into it I can't really show a 5720 or 5712 alert using it.
There are a lot more options available in rules. For instance you can use <regex> in a rule. You won't be able to pull information out with parentheses in a regex, that's only available in decoders. Options like <category>, <srcip>, <user>, and <category> can be used to match specific elements. <category>syscheck</category> can be used to modify a syscheck alert. Here's an example:
<rule id="10999" level="15">
<category>syscheck</category>
<match>/var/ossec/etc/ossec.conf</match>
<description>ossec.conf has been modified!</description>
</rule>
This rule would set off an alert of level 15 if syscheck has noticed that /var/ossec/etc/ossec.conf has been modified.
Since we know Bob mis-types his password at least once per day we can do the following to not have alerts fire when he mistypes his password:
<rule id="110000" level="0">
<if_sid>5716</if_sid>
<user>bob</user>
<description>Ignore bob</description>
</rule>
If we want to limit it even further we can limit this to his source IP:
<rule id="110000" level="0">
<if_sid>5716</if_sid>
<user>bob</user>
<srcip>192.168.1.23</srcip>
<description>Ignore bob</description>
</rule>
There are two more options I want to mention: alert_by_email and no_email_alert. The first always sends an email when that event is triggered, no matter what level is set. This can be useful if you have determined that an event isn't a good candidate for a high level, but you always want to see an email on it. The second means the alert never sends an email. I'm not sure how useful this one is, but I'm guessing someone wanted it.
There's still a lot of information to cover, but it'll have to wait for future blog posts. This one is long enough. If I didn't explain something here well enough please let me know!
Friday, October 22, 2010
Second Annual Week of OSSEC Roundup: Day 6
2WoO Day 6: Running Multiple Instances on One Box by Michael Starks
WoO Day 6 : Layin' Down The Law by Jason Frisvold
Video of a web tool to view OSSEC alerts by @tatehansen. It looks pretty neat to me. Here is his email to the ossec list explaining that it's ruby on rails and mongodb.
WoO Day 6 : Layin' Down The Law by Jason Frisvold
Video of a web tool to view OSSEC alerts by @tatehansen. It looks pretty neat to me. Here is his email to the ossec list explaining that it's ruby on rails and mongodb.
Thursday, October 21, 2010
Second Annual Week of OSSEC Roundup: Day 5
A few weeks ago Mischael Starks msg'd me on IRC (#ossec on freenode!) about a crazy idea he had. He wanted to present Daniel Cid (creator of OSSEC, but you knew that) with a plaque to thank him for all of the work he's done. Here's dcid's blog post with information about the plaque: OSSEC Award daemon
Showing appreciation for the developers of your favorite projects is important. They work hard, often using their precious free time to make their creation the best that it can be. Most of the time they see the reported problems, critical blog posts, or angry tweets. They probably don't see a lot of the good stories, and I bet it can be a bit of a drag. So send them a thank you email, or submit a patch, buy them something from their Amazon wish list, or get them a beverage of their choice the next time you see them at a conference. I hear buying them pizza is a good idea.
I know I owe the OSSEC developers a big big thank you to a lot of developers for a lot of projects, but this post is about OSSEC. So a big thank you to dcid, jrossi, mstarks, atomicturtle, and others I can't think of at the moment! OSSEC's great because of you guys. OSSEC Team CONTRIBUTORS
2WoO Day 5: Taming File Integrity Alerts by Michael Starks has some great information on syscheck alerts. The syscheck_control -u kind of helps you create a new baseline. Although this will leave a window of time when syscheck won't be able to help you.
WoO Day 5 : Decoders Unite! by Jason Frisvold is a nice basic introduction to OSSEC decoders. I just want to remind everyone that we love user contributions! Feel free to send changes and additions to the mailing list, that's how OSSEC gets support for more logs.
OSSEC got a mention in the Internet Storm Center's Tools updates - Oct 2010 post! Thanks to Jim Clausing and the rest of the ISC handlers for the great resource!
And last but not least, here's the mailing list discussion for the day. The topic is '2WoO Day 5: Shared intelligence: what does an attack.' I'd love to see the OSSEC logs from a real penetration test (or even a real attack), but I doubt anyone would release that kind of information. Maybe in the future one of the capture the flag (CTF) competitions would do us the favor of installing OSSEC on a target?
If I see anything else posted, I'll update.
Showing appreciation for the developers of your favorite projects is important. They work hard, often using their precious free time to make their creation the best that it can be. Most of the time they see the reported problems, critical blog posts, or angry tweets. They probably don't see a lot of the good stories, and I bet it can be a bit of a drag. So send them a thank you email, or submit a patch, buy them something from their Amazon wish list, or get them a beverage of their choice the next time you see them at a conference. I hear buying them pizza is a good idea.
I know I owe the OSSEC developers a big big thank you to a lot of developers for a lot of projects, but this post is about OSSEC. So a big thank you to dcid, jrossi, mstarks, atomicturtle, and others I can't think of at the moment! OSSEC's great because of you guys. OSSEC Team CONTRIBUTORS
2WoO Day 5: Taming File Integrity Alerts by Michael Starks has some great information on syscheck alerts. The syscheck_control -u kind of helps you create a new baseline. Although this will leave a window of time when syscheck won't be able to help you.
WoO Day 5 : Decoders Unite! by Jason Frisvold is a nice basic introduction to OSSEC decoders. I just want to remind everyone that we love user contributions! Feel free to send changes and additions to the mailing list, that's how OSSEC gets support for more logs.
OSSEC got a mention in the Internet Storm Center's Tools updates - Oct 2010 post! Thanks to Jim Clausing and the rest of the ISC handlers for the great resource!
And last but not least, here's the mailing list discussion for the day. The topic is '2WoO Day 5: Shared intelligence: what does an attack.' I'd love to see the OSSEC logs from a real penetration test (or even a real attack), but I doubt anyone would release that kind of information. Maybe in the future one of the capture the flag (CTF) competitions would do us the favor of installing OSSEC on a target?
If I see anything else posted, I'll update.
Wednesday, October 20, 2010
Second Annual Week of OSSEC Roundup: Day 4
2WoO Day 4: Five Tips & Tricks for OSSEC Ninjas! by Michael Starks
Spot the Difference by Jason Frisvold
Today's discussion topic: Day 4: What bugs you: problems, challenges and room for improvement.
My contribution: WIP OSSEC Rules
Just saw this one: http://www.ossec.net/dcid/?p=208
Spot the Difference by Jason Frisvold
Today's discussion topic: Day 4: What bugs you: problems, challenges and room for improvement.
My contribution: WIP OSSEC Rules
Just saw this one: http://www.ossec.net/dcid/?p=208
Work in Progress OSSEC Rules
A number of the services and daemons I run on my OSSEC network don't have rules and decoders in the default OSSEC install. It's not uncommon for me to fire up mutt and see a number of 1002's in my mailbox from unsupported or less supported daemons. To deal with this I write rules and decoders for most of the daemons I use. I don't like to see log messages that don't have a corresponding OSSEC rule, even if they are benign.
Due to my sometimes odd choices in software I've had to write quite a few rules. They started out bad, really bad. I think I've gotten a bit better at writing rules though, mostly with the help of the OSSEC community. I even contribute decoders and rules back to the project. Of course Daniel Cid is a busy man, and doesn't always have time to look over my work. So I created an unofficial OSSEC rules repository. This repo contains rules and decoders that I'm working on.
The rules and decoders I've written are not part of the OSSEC project, this is something I've been doing on my own. You can't blame OSSEC for any of the (many) mistakes I've made with them. My plan is to make regular(-ish) releases of these rulesets, available for anyone to download and use. I'm slowly integrating my rule changes into the default OSSEC ruleset, and occassionally bugging dcid to pull my changes into the main tree.
I currently have 4 files ready for download. wip-ossec-rules-2.5.1.2.tar.gz is a tarball of the rule files and decoder (rules/*_rules.xml and etc/decoder.xml). It also contains a file (etc/rules.config) that contains the information you will need to put in ossec.conf to use the new rules. Untarring the tarball should be done in a temporary directory and the files copied over. You can use log-test to test the rules and decoders before deploying them.
wip-ossec-rules-2.5.1.2.tar.gz.sig will be the signature file to check with GNUPG or PGP. pubkey.txt is the gpg key, and wip-ossec-rules-2.5.1.2.cksum is the sha1/md5 checksum file.
I plan on numbering the releases for these WIP rules based on the version of OSSEC that's out at the time. For instance the number above, 2.5.1.2, is release 2 for version 2.5.1. The next release I do will be 2.5.1.3.
The first release included the local_rules.xml, which I thought was unnecessary. The second release also puts rule 5719 into the invalid_login group.
If you use these rules and decoders, please let me know if you have any issues. Open up a trouble ticket, send me an email, hit me up on twitter, I'm ddpbsd on freenode (in #ossec, of course!), whatever. Just let me know. I'll fix any issues as fast as I possibly can.
Contributions will also be accepted. If you look through the rulesets you'll see names other than my own. I currently take rules, decoders, and log messages off of various mailing lists or google searches (with some obfuscation) and use them for this little project. So don't be surprised if you see a log message you submitted to a mailing list at some point. I do try to credit the original source when possible.
I know I've got a lot of work ahead of me with this "little" project, but I'm looking forward to it. It's taught me a lot so far, and I know I still have a lot to learn.
Public key:
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.10 (OpenBSD)
mQENBEy9yNYBCAC/A6b0YiXXsBaWso3+mCYRSAAAAbMegueyWNvTfentn4Sy5Sm2
M1T/mcYK9eUHySgdZqyidKpvx63aZg1CrqoZqFN8c9VtoDjoCWa0Iv23zeC2W1Ce
897zp6jN0vp30ZzDw48698tnj7mAL4OH+1YjQeA64X+qh9GpLXHzrZ3ulJmVhjR3
2u0P7+pz3b7nVBG1IR2rsSLyyKpHPsVhmLO0668useMH1bQlTqFceaZrKi2KONip
VCb0BqWdP8MOnJ87g0FlRSsXPo9/F5/bTMPNONAKIervk6/ind9mF3yEcj2DXUMA
VODDJ5L7piG98oj8TB118o0RqLJ/L5nYh74XABEBAAG0GWRkcGJzZCA8ZGRwYnNk
QGdtYWlsLmNvbT6JAT4EEwECACgFAky9yNYCGwMFCQHhM4AGCwkIBwMCBhUIAgkK
CwQWAgMBAh4BAheAAAoJEGZNz8PPZP7i20EIAIQKuNPemYxdHc8EmoG4ACz+RoJ3
/B49Jedd1ASueuGqP+xFmLYriBacY/OmR36LeqlxGjCjO/Ln29oWTpMV/E+RM8ag
WW2P9C1DNRBQl5b0IGKt8NyH9OQ74x2QRvFVewg0laqmEjAjSNyi6suTHv+PRATq
DeNG7AVHAHOaQbv+fLPYNDv4aOl7URtaTfyQiHNchA97YQjSTBEKp+kE94NQrMgd
92YBlDr3SSNojMgqlHHhKuAGXLhuUxqg3O0bvZYDkHbbrKhp1Vm0sJM0B7CUOiQ+
49K/rsaESCeAKNx9DunZWg0oxqd1M7h2r34j3y3lm5EdIHpzBdD/kqm/YQW5AQ0E
TL3I1gEIAKpSu1kFhYTIhoaFyVddnIZyua4Xud0Q5V2vAAnWQq0rtkS3H/nh5dPt
NtFjT5hMKe2slDipnDbnaC48jfDjCFmUYsOutRpo8eDs/wtkI6C/atTqZXwxntIG
10i/c36Atrg7iV0fMYEgq105N7GivMNDFdi4oE8k2+59ICW/VG4W4zJa5DRXjbdL
K8lA1zd/f/i+pOSc+we9oAvPJh5AnPHv0fOxV3Dfx6qCDDdO05kWyqR91TxbwOe+
irSZkPL/VVkR+4H+guWmy/Vcj2TrOqii0Yv/D7o+GALbrt+5AwH16DhlYXIkGB+3
mz4wAbkB5gHU9RZjcgbtmwMRF1zLHNUAEQEAAYkBJQQYAQIADwUCTL3I1gIbDAUJ
AeEzgAAKCRBmTc/Dz2T+4kzZB/9YmRwgRNOr8UQdPIR0O70GLQMT4ahEWh9IeDlR
ATSqb9WpEjU4UDZTz7Y/+ZTERV+gHV3dFqHSn2W+j9zZwnNnmulYb1F2iB3sLtrg
dFO+N8X9+7H4EbLYrupr3gBMja8fuBT6zkI1s+lyCuHEUGtAj6mPjCRkhF4JULY5
lFyn71j5gASXqzVVHYFyYd8SPj6AY+umvUoDr9zSD/TDxJXXO5Vm+W3ergn1Ktbo
U9yhQAc3XNxTL8L4REb16+g5cNU5DuuHfBhLwhvgH7+41Nj5nEcO0S7k6aXzbYVP
nbKxg0qaaFYSwVVFjlQ+23z2How0rF/AAZjJnMBI07EV4wZO
=oclq
-----END PGP PUBLIC KEY BLOCK-----
Due to my sometimes odd choices in software I've had to write quite a few rules. They started out bad, really bad. I think I've gotten a bit better at writing rules though, mostly with the help of the OSSEC community. I even contribute decoders and rules back to the project. Of course Daniel Cid is a busy man, and doesn't always have time to look over my work. So I created an unofficial OSSEC rules repository. This repo contains rules and decoders that I'm working on.
The rules and decoders I've written are not part of the OSSEC project, this is something I've been doing on my own. You can't blame OSSEC for any of the (many) mistakes I've made with them. My plan is to make regular(-ish) releases of these rulesets, available for anyone to download and use. I'm slowly integrating my rule changes into the default OSSEC ruleset, and occassionally bugging dcid to pull my changes into the main tree.
I currently have 4 files ready for download. wip-ossec-rules-2.5.1.2.tar.gz is a tarball of the rule files and decoder (rules/*_rules.xml and etc/decoder.xml). It also contains a file (etc/rules.config) that contains the information you will need to put in ossec.conf to use the new rules. Untarring the tarball should be done in a temporary directory and the files copied over. You can use log-test to test the rules and decoders before deploying them.
wip-ossec-rules-2.5.1.2.tar.gz.sig will be the signature file to check with GNUPG or PGP. pubkey.txt is the gpg key, and wip-ossec-rules-2.5.1.2.cksum is the sha1/md5 checksum file.
I plan on numbering the releases for these WIP rules based on the version of OSSEC that's out at the time. For instance the number above, 2.5.1.2, is release 2 for version 2.5.1. The next release I do will be 2.5.1.3.
The first release included the local_rules.xml, which I thought was unnecessary. The second release also puts rule 5719 into the invalid_login group.
If you use these rules and decoders, please let me know if you have any issues. Open up a trouble ticket, send me an email, hit me up on twitter, I'm ddpbsd on freenode (in #ossec, of course!), whatever. Just let me know. I'll fix any issues as fast as I possibly can.
Contributions will also be accepted. If you look through the rulesets you'll see names other than my own. I currently take rules, decoders, and log messages off of various mailing lists or google searches (with some obfuscation) and use them for this little project. So don't be surprised if you see a log message you submitted to a mailing list at some point. I do try to credit the original source when possible.
I know I've got a lot of work ahead of me with this "little" project, but I'm looking forward to it. It's taught me a lot so far, and I know I still have a lot to learn.
Public key:
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.4.10 (OpenBSD)
mQENBEy9yNYBCAC/A6b0YiXXsBaWso3+mCYRSAAAAbMegueyWNvTfentn4Sy5Sm2
M1T/mcYK9eUHySgdZqyidKpvx63aZg1CrqoZqFN8c9VtoDjoCWa0Iv23zeC2W1Ce
897zp6jN0vp30ZzDw48698tnj7mAL4OH+1YjQeA64X+qh9GpLXHzrZ3ulJmVhjR3
2u0P7+pz3b7nVBG1IR2rsSLyyKpHPsVhmLO0668useMH1bQlTqFceaZrKi2KONip
VCb0BqWdP8MOnJ87g0FlRSsXPo9/F5/bTMPNONAKIervk6/ind9mF3yEcj2DXUMA
VODDJ5L7piG98oj8TB118o0RqLJ/L5nYh74XABEBAAG0GWRkcGJzZCA8ZGRwYnNk
QGdtYWlsLmNvbT6JAT4EEwECACgFAky9yNYCGwMFCQHhM4AGCwkIBwMCBhUIAgkK
CwQWAgMBAh4BAheAAAoJEGZNz8PPZP7i20EIAIQKuNPemYxdHc8EmoG4ACz+RoJ3
/B49Jedd1ASueuGqP+xFmLYriBacY/OmR36LeqlxGjCjO/Ln29oWTpMV/E+RM8ag
WW2P9C1DNRBQl5b0IGKt8NyH9OQ74x2QRvFVewg0laqmEjAjSNyi6suTHv+PRATq
DeNG7AVHAHOaQbv+fLPYNDv4aOl7URtaTfyQiHNchA97YQjSTBEKp+kE94NQrMgd
92YBlDr3SSNojMgqlHHhKuAGXLhuUxqg3O0bvZYDkHbbrKhp1Vm0sJM0B7CUOiQ+
49K/rsaESCeAKNx9DunZWg0oxqd1M7h2r34j3y3lm5EdIHpzBdD/kqm/YQW5AQ0E
TL3I1gEIAKpSu1kFhYTIhoaFyVddnIZyua4Xud0Q5V2vAAnWQq0rtkS3H/nh5dPt
NtFjT5hMKe2slDipnDbnaC48jfDjCFmUYsOutRpo8eDs/wtkI6C/atTqZXwxntIG
10i/c36Atrg7iV0fMYEgq105N7GivMNDFdi4oE8k2+59ICW/VG4W4zJa5DRXjbdL
K8lA1zd/f/i+pOSc+we9oAvPJh5AnPHv0fOxV3Dfx6qCDDdO05kWyqR91TxbwOe+
irSZkPL/VVkR+4H+guWmy/Vcj2TrOqii0Yv/D7o+GALbrt+5AwH16DhlYXIkGB+3
mz4wAbkB5gHU9RZjcgbtmwMRF1zLHNUAEQEAAYkBJQQYAQIADwUCTL3I1gIbDAUJ
AeEzgAAKCRBmTc/Dz2T+4kzZB/9YmRwgRNOr8UQdPIR0O70GLQMT4ahEWh9IeDlR
ATSqb9WpEjU4UDZTz7Y/+ZTERV+gHV3dFqHSn2W+j9zZwnNnmulYb1F2iB3sLtrg
dFO+N8X9+7H4EbLYrupr3gBMja8fuBT6zkI1s+lyCuHEUGtAj6mPjCRkhF4JULY5
lFyn71j5gASXqzVVHYFyYd8SPj6AY+umvUoDr9zSD/TDxJXXO5Vm+W3ergn1Ktbo
U9yhQAc3XNxTL8L4REb16+g5cNU5DuuHfBhLwhvgH7+41Nj5nEcO0S7k6aXzbYVP
nbKxg0qaaFYSwVVFjlQ+23z2How0rF/AAZjJnMBI07EV4wZO
=oclq
-----END PGP PUBLIC KEY BLOCK-----
Tuesday, October 19, 2010
Second Annual Week of OSSEC Roundup: Day 3
Abusing OSSEC the Countermeasures - Michael Starks showed us how to break OSSEC yesterday, today he's showing us how to protect against these attacks.
Contributing to OSSEC - A post by Daniel Cid on giving back to the OSSEC community specifically, but the principles are probably the same for all Open Source/Free Software projects.
Meet the Agent by Jason Frisvold gives a great introduction to configuring agents through ossec.conf and shared/agent.conf
These next 2 posts seem to be the "go to" source for Bigfix and OSSEC integration:
Using Bigfix for Mass Deployments of OSSEC agents for Windows by Shawn Jefferson
Monitoring your OSSEC installation with Bigfix also by by Shawn Jefferson
My contribution: OSSEC Decoders 101
EDIT:
And here's another just sent to the list: OSSEC to the rescue
The discussion topic of the day: 2WoO Day 3: Time to share: rules, configs, tips and tricks.
Contributing to OSSEC - A post by Daniel Cid on giving back to the OSSEC community specifically, but the principles are probably the same for all Open Source/Free Software projects.
Meet the Agent by Jason Frisvold gives a great introduction to configuring agents through ossec.conf and shared/agent.conf
These next 2 posts seem to be the "go to" source for Bigfix and OSSEC integration:
Using Bigfix for Mass Deployments of OSSEC agents for Windows by Shawn Jefferson
Monitoring your OSSEC installation with Bigfix also by by Shawn Jefferson
My contribution: OSSEC Decoders 101
EDIT:
And here's another just sent to the list: OSSEC to the rescue
The discussion topic of the day: 2WoO Day 3: Time to share: rules, configs, tips and tricks.
OSSEC decoders 101
In a previous post I mentioned that writing rules and decoders for OSSEC can be easy. Now I'm going to attempt to backup that claim bydetailing the process of writing a decoder. A similar post for writing rules will come later. To do this I'll use decoders for OpenSSH that are currently included in OSSEC, and an old log message from a Fedora 11 system.
The log message:
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
To help write the decoders I'll be using the ossec-logtest application. ossec-logtest takes input from stdin and traverses the decoders and rules looking for a match. ossec-logtest should be available on all server and local (not agent) installs.
Having a basic understanding of OSSEC's regular expression (regex) support may help with more complicated decoders. For the log message above we'll be using regex to pull out the username and IP address.
Before I discuss decoders, I should explain how OSSEC parses syslog messages. There are three main parts to OSSEC's log analysis engine: pre-decoding, decoding and signatures.
Phase 1, pre-decoding, extracts known fields like time, the hostname of the system where the message originated, the name of the program that created the message and the log message itself (without some of the metadata).
Here is an example of information gathered during the pre-decoding phase using ossec-logtest:
-----------------------------------------------------------
# /var/ossec/bin/ossec-logtest -D . -c etc/ossec.conf
2010/10/01 08:54:31 ossec-testrule: INFO: Reading local decoder file.
2010/10/01 08:54:31 ossec-testrule: INFO: Started (pid: 22955).
ossec-testrule: Type one log per line.
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
-----------------------------------------------------------
As you can see the timestamp has been removed. OSSEC uses an internal timestamp of when the message was received instead of the timestamp on the event. After the timestamp is the hostname, 'fedora11' (don't judge, this was an old log message). The program name, sshd, was pulled out of the message and the process ID (11773) was removed and discarded. Finally, the log message without the metadata is displayed.Phase 2, decoding, comes next. What information goes in? What information comes out? How do I get more information? And how do I write decoders for my applications?
Phase 3 will attempt to match a rule to the log message, and will be detailed in another post.
The decoders I will be using for these examples revolve around OpenSSH. OpenSSH is a complicated program, and has a number of logs. Some are more interesting than others, and we try to create rules for as many as we can. In a number of the logs information like usernames and source IP addresses are available, and this can be useful information. OSSEC can be configured to block IPs attempting to brute force an OpenSSH password, or disable users that are being targeted.
Just a quick note: Some distributions or operating systems have modified the logs in the default OpenSSH source code, so maintaining the sshd decoders has not been the easiest task. It can be a bit disheartening to see the same log information displayed in 3 slightly different and incompatible log messages on different Linux distributions. If anyone knows why package/OS maintainers feel a need to modify log messages please let me know.
Anyways, on to the decoders!
OSSEC decoders are written in XML, and have a very basic format. They can be found in /var/ossec/etc/decoder.xml, and custom decoders can be added to /var/ossec/etc/local_decoder.xml. The local_decoder.xml file will not be overwritten during an upgrade, but the system default decoder.xml will. The easiest decoders are three lines long, and don't include anything more complicated than using the "^" character to indicate that the string that follows is at the very beginning of the section. Many of the fields in decoders can utilize the OS_Match or sregex.
Here's the first example, the primary sshd decoder, followed by a line-by-line explanation:
<decoder name="sshd">
<program_name>^sshd</program_name>
</decoder>
- Each decoder starts the same way, with the "decoder," and "name" tags.
- The name must be a unique string, alpha-numeric and should identify the purpose of the decoder. In this case the name of this decoder is just "sshd."
- The next line indicates that this decoder will be applied to any log event produced by the "sshd" application. In this decoder the <program_name> field checks if the program_name begins with the string "sshd." If it does, OSSEC applies this decoder. Another option instead of <program_name> would be to include a <prematch> field, and this option will be detailed later in this post. All decoders must have either a <program_name> or <prematch> field, but <program_name> is often easier.
- The "</decoder>" tag ends this decoder entry.
The following is an example of the output of ossec-logtest without the sshd decoder in place:
-------------------------------------------
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
No decoder matched.
-------------------------------------------
Notice how no decoder is found in "Phase 2."
Now, for the output of the same log message with the basic sshd decoder in place:
-------------------------------------------
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
--------------------------------------------
Now the decoder is identified as "sshd".
With this information, OSSEC can focus on the rules that apply to the sshd decoder and not waste resources looking at rules that don't.
But this decoder does not give us much information. Many of sshd's logs include useful information like the source IP or username. This information may be useful to OSSEC admins. To get this information, child decoders can be used. Parent decoders can have any number of child decoders, and each child can have only 1 parent. This allows for a wide variety of log messages to be decoded properly without making it horribly difficult to understand and maintain decoders.
This next decoder pulls more information out of the log message (when available), and provides it in various fields that may be used in rules or active responses.
ssh-failed decoder:
<decoder name="ssh-failed">
<parent>sshd</parent>
<prematch>^Failed \S+ </prematch>
<regex offset="after_prematch">^for (\S+) from (\S+) port \d+ \w+$<;/regex>
<order>user, srcip</order>
</decoder>
- This decoder is a bit more complicated than the "sshd" decoder. It starts the same way as the "sshd" decoder, with the "decoder" and "name" tags.
- The second line (<parent>sshd</parent>) indicates that the "ssh-failed" decoder is a child of the "sshd" decoder. If "sshd" does not match, this decoder will not be checked, saving system resources. A log message that matches the "ssh-failed" decoder will be identified as belonging to the "sshd" decoder to make categorizations and administration easier as you will not need to know the name of the specific child decoder, only the parent.
- The third line (<prematch>^Failed \S+ </prematch>) is used by OSSEC to determine if this child decoder is the appropriate one to apply to this log message. As I said before, each decoder can have any number of children, so there has to be a method to select one for each log message. This line is using a simple regex pattern. This pattern is compared to the log message, if there is a match OSSEC continues with this decoder.
- The regex line is one of the more interesting lines in this decoder. I will break this line into two parts:
- <regex offset="after_prematch"> - This identifies the line as a regex entry, and indicates that OSSEC should try to match this regular expression to the log message after the prematch. That means, from the log message only the information after 'Failed password ' will be used.
- ^for (\S+) from (\S+) port \d+ \w+$</regex> - This second part of the regex line is a little more complicated. The beginning of the line should start with the characters 'for'. The "\S+" is a regular expression for "one or more non-whitespace characters." The parenthesis around this regex force the value to be kept in memory for use later. Next is "from (\S+)". Again, OSSEC looks for the string "from " and places the next one or more non-whitespace characters in memory. The last bit, " port \d+ \w+$" instructs OSSEC to look for the string "port", one or more numbers, and a word (a string of: A-Z, a-z, 0-9 characters) immediately followed by the end of the line.
- The <order> line is where the values pulled out of the log message by the regex operation will be categorized. Between the order tags are the values "user" and "srcip" (seperated by a comma). The first value from the regex line will populate the user field, and the second the srcip field.
- In the example log message this lines up with "ddp" as the user, and "172.16.51.1" as the srcip. There are a number of options available in the <order> field, and a list is at the end of this post.
In this example, using the ssh-failed decoder, you can now see the dstuser and srcip have been pulled out of the log message (note that the decoder is still identified as sshd, not as ssh-failed).
------------------------------------------
Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2
**Phase 1: Completed pre-decoding.
full event: 'Oct 8 14:15:33 fedora11 sshd[11773]: Failed password for ddp from 172.16.51.1 port 56588 ssh2'
hostname: 'fedora11'
program_name: 'sshd'
log: 'Failed password for ddp from 172.16.51.1 port 56588 ssh2'
**Phase 2: Completed decoding.
decoder: 'sshd'
dstuser: 'ddp'
srcip: '172.16.51.1'
---------------------------------------------
In the output from ossec-logtest, the "user" field is displayed as dstuser. I don't know why, but in this case it does happen to be the destination user.
I mentioned using offset="after_prematch" above, but there is another option. You can also use offset="after_parent". Using "after_parent" in the above example wouldn't help with the ssh-failed decoder, since the sshd decoder essentially stops after "sshd[11773]: ". If the parent decoder went further into the log message "after_parent" would be more useful.
So that's how you write a decoder in OSSEC. It can take a few tries to get a decoder right, especially when using regular expressions. The ossec-logtest program is indispensable in these situations. There have been plenty of times where I've had to break a regex line down to the basics, and slowly build it up until it worked the way I wanted.
And, as a final note, the order of decoders in decoder.xml is important. The first decoder OSSEC comes across that matches is applied to the log message, first match wins. I've spent time editing regex lines trying to get a decoder to work only to find out another decoder beat mine to the match.
I sometimes have trouble finding these, so here are the various options that can be used inside of <order>:
user
srcuser
dstuser
srcip
dstip
srcport
dstport
port
extra_data
status
protocol
url
id
action
Monday, October 18, 2010
Second Week of OSSEC Roundup: Day 2
So here are the day 2 blog posts:
2WoO Day 2: Abusing OSSEC by Michael Starks
WoO Day 2 : In The Beginning ... by Jason Frisvold
This Blog is Monitored by OSSEC by Xavier Mertens
Ossec – Architecture and Implementation – Best practices ? by K4l4m4r1s
Ossec Server Install on my Ubuntu machine by K4l4m4r1s
Deploying OSSEC for Windows with Bigfix by Shawn Jefferson (a recent convert to OSSEC)
And the self promoting link: Rule 1002 - Just a quick note about Rule 1002, kind of a prelude for another couple of posts.
mstarks also started a thread on the mailing list asking for stories on how OSSEC saved the day or saved money. You can read it here. Feel free to join the list and contribute.
And a quick thanks to marc.info. I've used the site to read and look for archives on other lists for years now. Only recently did I realize the ossec-list was being mirrored there as well.
So there's day 2, what do you think of the posts?
2WoO Day 2: Abusing OSSEC by Michael Starks
WoO Day 2 : In The Beginning ... by Jason Frisvold
This Blog is Monitored by OSSEC by Xavier Mertens
Ossec – Architecture and Implementation – Best practices ? by K4l4m4r1s
Ossec Server Install on my Ubuntu machine by K4l4m4r1s
Deploying OSSEC for Windows with Bigfix by Shawn Jefferson (a recent convert to OSSEC)
And the self promoting link: Rule 1002 - Just a quick note about Rule 1002, kind of a prelude for another couple of posts.
mstarks also started a thread on the mailing list asking for stories on how OSSEC saved the day or saved money. You can read it here. Feel free to join the list and contribute.
And a quick thanks to marc.info. I've used the site to read and look for archives on other lists for years now. Only recently did I realize the ossec-list was being mirrored there as well.
So there's day 2, what do you think of the posts?
OSSEC 2.5.1 released
Here is the official announcement.
There were a few bug and documentation fixes, with one new feature:
There were a few bug and documentation fixes, with one new feature:
- Added no_ar option to the rules (by Jeremy Rossi and Scott at atomicrocketturtle.com)
Also, there is a snapshot for HPUX users to try: here. Hopefully that will fix the issues that platform was having.
Second Week of OSSEC Roundup: Day 1
This is just a roundup of the Second Week of OSSEC (2WoO) posts on (or before) day 1:
Week of OSSEC: Day -2: Syngress released a few chapters of the OSSEC book, and is offering 30%.
PaloAlto Firewall Threat Monitoring Using OSSEC: @xme explains and posts a decoder and sample rules for a PaloAlto firewall.
2WoO Day 1: Crowdsourcing Log Integrity & Non-repudiation: Michael Starks discusses the idea of making hashes public to help prove logs weren't tampered with.
WoO Day 1 : Introduction: Jason Frisvold gives a quick introduction to what OSSEC is.
If I missed any, please let me know!
Week of OSSEC: Day -2: Syngress released a few chapters of the OSSEC book, and is offering 30%.
PaloAlto Firewall Threat Monitoring Using OSSEC: @xme explains and posts a decoder and sample rules for a PaloAlto firewall.
2WoO Day 1: Crowdsourcing Log Integrity & Non-repudiation: Michael Starks discusses the idea of making hashes public to help prove logs weren't tampered with.
WoO Day 1 : Introduction: Jason Frisvold gives a quick introduction to what OSSEC is.
If I missed any, please let me know!
Second Week of OSSEC Day 2: Rule 1002
Tuning intrusion detection systems, both network and host based, can be one of the bigger tasks associated with the devices. I've personally spent many hours of my adult life tuning IDSes and the systems that display the alerts to make them more useful (or to make them function). Too many alerts and none of them get the attemtion they deserve. Too few and it's your head on the chopping block after an incident is discovered.
You also have to worry about whether the system can handle the load you're placing on it. I've been handed snort sensors that were dropping 60% of the packets that crossed the wire, and the administrator wasn't sure why the IDS didn't seem to be working.
Luckily I haven't run into many problems with tuning OSSEC. The biggest task I've had is decoding "rule 1002" alerts. Rule 1002 reads as follows (Oct. 6, 2010):
Rule 1002 can catch a lot of log messages that are worth looking at, and probably shouldn't be removed or ignored. Instead use that opportunity to create a new rule. We understand that rules and decoders for internal applications probably couldn't (or shouldn't) be released to the public, but the OSSEC project welcomes new log samples and rules/decoders from the user community. Posting them to the user or dev mailing lists is always appreciated. The userbase can also provide feedback, or help debug if you have issues.
I'll be going through the steps to create and debug decoders and rules in the future. Until then here's a blog post that details writing decoders and rules. That blog post has also been incorporated into the OSSEC documentation here. Hopefully I can add something meaningful to it.
You also have to worry about whether the system can handle the load you're placing on it. I've been handed snort sensors that were dropping 60% of the packets that crossed the wire, and the administrator wasn't sure why the IDS didn't seem to be working.
Luckily I haven't run into many problems with tuning OSSEC. The biggest task I've had is decoding "rule 1002" alerts. Rule 1002 reads as follows (Oct. 6, 2010):
<rule id="1002" level="2">The $BAD_WORDS variable is:
<match>$BAD_WORDS</match>
<options>alert_by_email</options>
<description>Unknown problem somewhere in the system.</description>
</rule>
<var name="BAD_WORDS">core_dumped|failure|error|attack|bad |illegal |denied|refused|unauthorized|fatal|failed|Segmentation Fault|Corrupted</var>Rule 1002 basically looks for one of the bad words in all syslog messages that do not have a better match. If one of those words is found an alert is sent. This rule in particular has started a number of discussions on the user mailing list. There have been numerous instances where someone did not seem to understand the importance of Rule 1002 and wanted to ignore all Rule 1002 alerts.
Rule 1002 can catch a lot of log messages that are worth looking at, and probably shouldn't be removed or ignored. Instead use that opportunity to create a new rule. We understand that rules and decoders for internal applications probably couldn't (or shouldn't) be released to the public, but the OSSEC project welcomes new log samples and rules/decoders from the user community. Posting them to the user or dev mailing lists is always appreciated. The userbase can also provide feedback, or help debug if you have issues.
I'll be going through the steps to create and debug decoders and rules in the future. Until then here's a blog post that details writing decoders and rules. That blog post has also been incorporated into the OSSEC documentation here. Hopefully I can add something meaningful to it.
Tuesday, October 12, 2010
OSSEC 2.5 was released, and 2.5.1 is almost there
So here is the official release announcement of OSSEC 2.5. There are some neat new features in this version:
Get the source here: ossec-hids-2.5.1.tar.gz
Windows version here: ossec-win32-2.5.1.exe
I'd recommend using 2.5.1, or waiting for it to be signed and whatnot before upgrading from 2.4.1.
Anyhow, back to the new features list. There are some really great features available in the new version. A few of my favorites are at the top of the list above.
<directories check_all="yes" report_changes="yes">/etc</directories>
Depending on the system's function, I may also include other directories like "/var/www/conf and "/var/named/etc." It's good to know what has changed, along with the fact that it has changed. I don't seem to get a diff for every change to the file though. I'm not sure why not yet, but I am looking into it.
I think this required an addition to the named decoders, creating/configuring the list, and creating a rule to utilize the list. My list is simple, the domains are the keys, the values don't matter. It's a simple text file with keys and values separated by a colon (:).
I add the list to ossec.conf with the following in the rules section:
<ossec_config> <!-- rules global entry -->
<rules>
<include>rules_config.xml</include>
...
<include>attack_rules.xml</include>
<include>local_rules.xml</include>
<list>rules/list/blocked.txt.cdb</list>
</rules>
</ossec_config> <!-- rules global entry -->
And the rule to do the lookup is pretty simple:
<rule id="510005" level="10">
<if_sid>12100</if_sid>
<match>: query: </match>
<list field="url">rules/list/blocked.txt</list>
<description>DNS query on a potentially malicious domain.</description>
</rule>
More advanced options are available, and I hope to explain them a bit in the future. I'll include other list ideas when I do.
Decoders can be loaded in the same way, and I actually change the names in the same way.
<decoder_dir>loadables/decoders</decoder_dir>
<rule_dir pattern="_rules.xml">loadables/rules</rule_dir>
The 2 lines above load all files in /var/ossec/loadables/decoders as decoders, and all files ending in "_rules.xml" in /var/ossec/loadables/rules as rule files.
- Added support for “report_changes” on syscheck to show what was changed in the file modification alert.
- Added support for cdb lists inside the rules.
- Added support for drop-in rules and decoders directory.
- Added a Rule unit testing framework (in python) and inside logtest
- Added support for a generic multi-line log reader.
- Added granular Windows rules.
- Added option to restrict integrity checking to a set of files.
- Added alias option to the command monitoring.
- Added silent switch for windows installer.
- Added variable expansion in command output monitoring.
- Fixed several windows installer bugs.
Get the source here: ossec-hids-2.5.1.tar.gz
Windows version here: ossec-win32-2.5.1.exe
I'd recommend using 2.5.1, or waiting for it to be signed and whatnot before upgrading from 2.4.1.
Anyhow, back to the new features list. There are some really great features available in the new version. A few of my favorites are at the top of the list above.
Added support for “report_changes” on syscheck to show what was changed in the file modification alert.Using the report_changes option gives users something they've been asking for, diffs are now sent along with the alert email for modified text files. I've added the following option to all of my OpenBSD systems:
<directories check_all="yes" report_changes="yes">/etc</directories>
Depending on the system's function, I may also include other directories like "/var/www/conf and "/var/named/etc." It's good to know what has changed, along with the fact that it has changed. I don't seem to get a diff for every change to the file though. I'm not sure why not yet, but I am looking into it.
Added support for cdb lists inside the rules.This is an awesome feature that I didn't think I'd use as much as I do. CDB lists are a fast, compiled key/value store. I currently have a list of over 100,000 suspicious domains. I want to know when I, or someone else on my network, perform a dns lookup on one of these domains. I can do this in a number of ways, but the easiest is to turn on more logging in named and match domains in the lookups with domains on the list.
I think this required an addition to the named decoders, creating/configuring the list, and creating a rule to utilize the list. My list is simple, the domains are the keys, the values don't matter. It's a simple text file with keys and values separated by a colon (:).
I add the list to ossec.conf with the following in the rules section:
<ossec_config> <!-- rules global entry -->
<rules>
<include>rules_config.xml</include>
...
<include>attack_rules.xml</include>
<include>local_rules.xml</include>
<list>rules/list/blocked.txt.cdb</list>
</rules>
</ossec_config> <!-- rules global entry -->
And the rule to do the lookup is pretty simple:
<rule id="510005" level="10">
<if_sid>12100</if_sid>
<match>: query: </match>
<list field="url">rules/list/blocked.txt</list>
<description>DNS query on a potentially malicious domain.</description>
</rule>
More advanced options are available, and I hope to explain them a bit in the future. I'll include other list ideas when I do.
Added support for drop-in rules and decoders directory.This is the last new feature I'll comment on right now. Instead of including each rule or decoder file individually in the ossec.conf a whole directory can now be added. You'll have to be careful when using this option, because the order files are added may be important. If a rule in local_rules.xml relies on something in sshd_rules.xml (perhaps the local rule uses an <if_sid>5700</if_sid>), the sshd_rules.xml must be loaded first. I've changed the names for the rule files on my own system. Base rule rules are appended with "10_" at the beginning, rule files that require other rule files to be loaded first are appended with "20_." My local_rules.xml is now 90_local_rules.xml.
Decoders can be loaded in the same way, and I actually change the names in the same way.
<decoder_dir>loadables/decoders</decoder_dir>
<rule_dir pattern="_rules.xml">loadables/rules</rule_dir>
The 2 lines above load all files in /var/ossec/loadables/decoders as decoders, and all files ending in "_rules.xml" in /var/ossec/loadables/rules as rule files.
Monday, October 11, 2010
EuroBSD Con 2010
I'm on the wrong continent to go to EuroBSD Con. I frequently feel like I'm on the wrong side of the world for non-security conferences, but the right side for the security cons.
Anyways, it looks like OpenBSD had a good turn-out at this year's EuroBSD Con. At the time I'm writing this, 4 OpenBSD related presentations have been linked to on the OpenBSD site.
Marc Espie's presentation Entitled The long road to pkg_add -u was a good read. I think it shows some interesting details into the work he (and others!) have done to make OpenBSD's packaging system as good as it is. It's definitely my favorite packaging system.
The OpenBSD Update is short and sweet, detailing a lot of the changes made to OpenBSD in the past release (or two?). I forget how many cool changes there are from release to release since I use snapshots. They generally trickle in for me, and I get to enjoy them for up to 6 months before they make release. The smaller changes are easy for me to use, they just work. Some of the bigger things like new daemons may not get much use in my household because I don't (yet) have a use for them. I need a bigger lab.
One of the big highlights is the loongson platform. It's a small MIPS-like netbook. I'd love to get one, but I don't really have a use for it. So it'd be another toy without a real use (kind of like my Zaurus). The disk UIDs are really neat. I've started using them in some of my hotplugd scripts. Definitely makes things a lot easier. iscsid is neat as well, but I don't have a use for it (yet!).
I'm excited by iked and ldapd, and hope to play around with them in the future. Krautcomputing is the funniest thing I've heard in a while. I've done that plenty of times, just didn't know there was a name for it!
The placeholder - something OpenBSD networking related presentation also looks good. I got a chance to see one of Henning's previous presentations at DCBSDCon 2009 and had a good time listening to the talk. Henning has an interesting sense of humor. I'm guessing this one was probably just as good or better. I'm still going through the slides of this presentation, so no real comments.
I also haven't gotten through Marc Espie's other presentation, Efficient distributed ports builds. No comments on that either.
Eventually I'll get around to looking for some presentations from the other BSDs, but OpenBSD comes first. Hopefully audio or video of the presentations will be released. I'd love to be able to see some of these presented.
Anyways, it looks like OpenBSD had a good turn-out at this year's EuroBSD Con. At the time I'm writing this, 4 OpenBSD related presentations have been linked to on the OpenBSD site.
Marc Espie's presentation Entitled The long road to pkg_add -u was a good read. I think it shows some interesting details into the work he (and others!) have done to make OpenBSD's packaging system as good as it is. It's definitely my favorite packaging system.
The OpenBSD Update is short and sweet, detailing a lot of the changes made to OpenBSD in the past release (or two?). I forget how many cool changes there are from release to release since I use snapshots. They generally trickle in for me, and I get to enjoy them for up to 6 months before they make release. The smaller changes are easy for me to use, they just work. Some of the bigger things like new daemons may not get much use in my household because I don't (yet) have a use for them. I need a bigger lab.
One of the big highlights is the loongson platform. It's a small MIPS-like netbook. I'd love to get one, but I don't really have a use for it. So it'd be another toy without a real use (kind of like my Zaurus). The disk UIDs are really neat. I've started using them in some of my hotplugd scripts. Definitely makes things a lot easier. iscsid is neat as well, but I don't have a use for it (yet!).
I'm excited by iked and ldapd, and hope to play around with them in the future. Krautcomputing is the funniest thing I've heard in a while. I've done that plenty of times, just didn't know there was a name for it!
The placeholder - something OpenBSD networking related presentation also looks good. I got a chance to see one of Henning's previous presentations at DCBSDCon 2009 and had a good time listening to the talk. Henning has an interesting sense of humor. I'm guessing this one was probably just as good or better. I'm still going through the slides of this presentation, so no real comments.
I also haven't gotten through Marc Espie's other presentation, Efficient distributed ports builds. No comments on that either.
Eventually I'll get around to looking for some presentations from the other BSDs, but OpenBSD comes first. Hopefully audio or video of the presentations will be released. I'd love to be able to see some of these presented.
Sunday, October 3, 2010
First Annual Week of OSSEC recap
With the Second Annual Week of OSSEC (or as I like to call it "Week of OSSEC, The or "WoOT") coming up, I thought recapping last year's entries from Michael Stark on the Immutable Security blog would be a good idea.
So, here we go!
Week of OSSEC Day 1: Detecting World-Writable Files
Week of OSSEC Day 2: Detecting New Files
Week of OSSEC Day 3: Using Variables
Week of OSSEC Day 4: Using Groups
Week of OSSEC Day 5: Reusing Rule IDs
Week of OSSEC Day 6: Developing a Tuning Strategy
Week of OSSEC Day 7: Developing a Workflow
So, here we go!
Week of OSSEC Day 1: Detecting World-Writable Files
Week of OSSEC Day 2: Detecting New Files
Week of OSSEC Day 3: Using Variables
Week of OSSEC Day 4: Using Groups
Week of OSSEC Day 5: Reusing Rule IDs
Week of OSSEC Day 6: Developing a Tuning Strategy
Week of OSSEC Day 7: Developing a Workflow
Friday, October 1, 2010
Introductions are in order.
I've spent a couple of days thinking about the purpose of this blog. I haven't come to any firm conclusions, except for the goal of improving my own writing. I used to write. I used to write a lot. It's a passion I kind of lost over the years. Ihad never considered myself to be a good writer, but I enjoyed it. I'm not entirely sure why I stopped, and sometimes think about getting back into it.
So, I'm going to use this blog as a place to practice writing. I'll be posting technical content on a (hopefully) regular basis. There is a good chance most of my posts will be about OSSEC. It's currently one of my favorite Free/Open Source Software applications, and one I use daily.
You can generally find me on the OSSEC user list, as well as IRC (#ossec on freenode) when I'm able. I try to contribute decoders and rules, and frequently copy syslog messages from mailing lists when I find something interesting. I've also tried to help with the documentation, but I haven't done enough.
When I finally figure out how to organize it, I'm planning on releasing my rulesets (as well as the OSSEC rulesets) outside of the OSSEC source. I'm thinking of this being similar to how snort's rules are separated from the source code.
I currently have a post prepared for the Week of OSSEC, and one that needs to be written. I also have a number of little OSSEC tidbits hidden away in a text file waiting to be released. I'm hoping to convert some into (possibly short) blog posts.
Hopefully my rambling was readable and made sense.
*I didn't like my original introduction, so this should hopefully replace it*
So, I'm going to use this blog as a place to practice writing. I'll be posting technical content on a (hopefully) regular basis. There is a good chance most of my posts will be about OSSEC. It's currently one of my favorite Free/Open Source Software applications, and one I use daily.
You can generally find me on the OSSEC user list, as well as IRC (#ossec on freenode) when I'm able. I try to contribute decoders and rules, and frequently copy syslog messages from mailing lists when I find something interesting. I've also tried to help with the documentation, but I haven't done enough.
When I finally figure out how to organize it, I'm planning on releasing my rulesets (as well as the OSSEC rulesets) outside of the OSSEC source. I'm thinking of this being similar to how snort's rules are separated from the source code.
I currently have a post prepared for the Week of OSSEC, and one that needs to be written. I also have a number of little OSSEC tidbits hidden away in a text file waiting to be released. I'm hoping to convert some into (possibly short) blog posts.
Hopefully my rambling was readable and made sense.
*I didn't like my original introduction, so this should hopefully replace it*
Subscribe to:
Posts (Atom)