29 December 2006 Changes (HHH) ------------------------------- 1. Action: Adding ultsearch.com Added: BadDomains[i++] = ".ultsearch.com"; Reason: // INFO GATHER RULE This will be removed when find the rest of their servers. 2. Action: "tit" word Added: BadHostWordStarts[i++]="tit[^abhilmtu]"; Reason: // INFO GATHER RULE I need to find out what needs to be done to handle this gaping hole. I won't even consider a URL word. There IS a problem here though. We have "tits", "tittie" "titts" and "titty". But note that I am saying that "titt" is okay. That is because of words like "titter-totter" and "tittle-tattle". I will be experimenting in removing the "t" exclusion. THIS WILL BE A PERMANENT RULE AND HERE IS WHY: 3858 tit_Parts.txt 134 tit_Starts_and_Ends.txt 820 tit_Passed_All_Rules.txt ----------------------------- 4812 total 3. Action: Downgraded start / end "hard" rules From: BadHostURL_Starts[i++]="hard[(b|c|e|p|s)]"; BadURL_WordEnds[i++]="[^cs]hard"; To: BadHostWordStarts[i++]="hard[(b|c|e|p|s)]"; BadHostWordEnds[i++]="[^cs]hard"; Reason: digg.com/security/Marcus_Ranum_on_hard_disk_ ... I can already see the potential for LOTS MORE false positives. Enough is enough. 4. Action: Added GoodDomains rule Added: GoodDomains[i++] = ".edu"; Reason: No *.edu site is going to be in the business of distributing porn; however some come close. If that is the case we can blackball that school. 5. Action: Downgraded "submissi" rule From: BadURL_Parts[i++] = "submissi"; To: BadHostParts[i++] = "submissi"; Reason: Thu Nov 30 17:28:24 www.tv.com/css/user_submission.css Actually there are so few hsots with this I may remove this altogether and let the hosts file handle it. 6. Action: Temporarily added mqcdn.com as a good host Added: GoodDomains[i++] = "mqcdn.com"; Reason: tile18.mqcdn.com/map/Scale18000/8/401/4/177.gif If I get even more "18" in the host names I may have to drop the following rule: BadHostParts[i++] = "18"; I have added a comment // COUNTER "18" rule to all of the ones I am over-riding on this. This one is MapQuest. 7. Action: Added something for AVG Antivirus Added: GoodDomains[i++] = ".grisoft.com"; Reason: They had a "free.grisoft.com" that caused the false positive but just in case it is something else we do NOT want to stop it. I would like to be sure I stop NO AntiVirus or AntiSpy companies. 8. Action: Added livejournal.com as a good host Added: GoodDomains[i++] = "livejournal.com"; Reason: // COUNTER "live" RULES If I allow start: 2992 live_Parts.txt 1624 live_Starts_and_Ends.txt 718 live_Passed_All_Rules.txt ------------------------------ 5334 total If I allow end: 2992 live_Parts.txt 1908 live_Starts_and_Ends.txt 434 live_Passed_All_Rules.txt ------------------------------ 5334 total If I allow start and end: 2992 live_Parts.txt 1035 live_Starts_and_Ends.txt 1307 live_Passed_All_Rules.txt ------------------------------ 5334 total 9. Action: Allow grisoft.cz to do anything Added: GoodDomains[i++] = ".grisoft.cz"; Reason: The "free" may pose a problem. I am heading it off before it ever happens. The problem is I need ALL of the other ones that will cause problems. 29 December 2006 UNresolved False Positives (HHH) ------------------------------------------------- 1. Pattern: "exposed" Rules: BadURL_Parts[i++] = "exposed"; Reason: www.hackinglinuxexposed.com/articles/20031231.html 96 expos_Parts.txt 70 expos_Starts_and_Ends.txt 92 expos_Passed_All_Rules.txt ------------------------------ 258 total 47 of the 92 passing are exposed, the others are exposure, expose, and some variations of sexpost. Downgrading the rule does NOTHING. In fact I am thinking of shortening it to "expose" where it is usually used in the French pronunciation of eks-po-zay 2. Pattern: "hard" Rules: BadURL_WordStarts[i++]="hard[(b|c|e|p|s)]"; BadURL_WordEnds[i++]="[^cs]hard"; // Changed from "hard" to "[^cs]hard" Reason: digg.com/security/Marcus_Ranum_on_hard_disk_encryption So far this is the ONLY one I have encountered 29 December 2006 RESOLVED False Positives (HHH) ----------------------------------------------- 1. Pattern: "crazy" Rules: BadURL_Parts[i++] = "crazy"; Reason: Removing "crazy" we have: 256 crazy_Parts.txt 59 crazy_Starts_and_Ends.txt 154 crazy_Passed_All_Rules.txt ------------------------------ 469 total That is a lot of hosts, but I wonder just how many are porn hosts and how many are dead, etc. Since crazyegg IS doing gathering of information for clients even though they do have a good policy, I am inclined to let the rule stand as is for a while. Solution: Let the rule stand. 2. Pattern: "virgin" Rules: BadURL_Parts[i++] = "virgin"; Reason: www.cs.virginia.edu Solution: Added a clear all for *.edu sites. In reality, what I had was far longer than what was shown, but by adding the *.edu sites we have assured ourselves NO *.edu site will be blocked. 3. Pattern: "submissi" Rules: BadURL_Parts[i++] = "submissi"; Reason: To broad in scope Solution: BadHostParts[i++] = "submissi"; 4. Pattern: "18" Rules: BadHostParts[i++] = "18"; Reason: tile18.mqcdn.com/map/Scale18000/8/400/4/177.gif 776 18_Parts.txt 218 18_Starts_and_Ends.txt 929 18_Passed_All_Rules.txt -------------------------- 23 total Solution: We are going to have to LIVE with adding these until we have at a dozen or so that are countering the "18" rule. That rule is blocking at least over 300 hosts (600 if you count the optional [www.]badat18.com). It may be possible to be blocking over 1000 hosts. That is a LOT of protection to give up. We are also have the GoodDomains ".avast.com" rule for the same reason since it has a download18.avast.com for one of the AntiVirus download servers.