02 December 2007 Changes (HHH) ------------------------------ 1. Action: SPAM Added: BadNetworks[i++] = "192.187.196.244, 255.255.255.255"; // SPAM Reason: Some pesky spammers are filling my email box with unrequested stuff. So rather than informing them I am blocking them. 2. Action: instacontent.net Removed: BadNetworks[i++] = "4.79.120.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "64.191.192.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "65.216.116.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "192.147.176.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "204.0.99.112, 255.255.255.240"; // instacontent Reason: UNSURE ABOUT VALIDITY. If you ask me they cycle through a lot of them and you have to check them daily. 3. Action: instacontent.net Added: BadNetworks[i++] = "63.111.30.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "64.191.208.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "65.200.179.112, 255.255.255.240"; // instacontent BadNetworks[i++] = "65.206.60.112, 255.255.255.240"; // instacontent Reason: VERIFIED 2007-11-13. They usually start off with IP address ###.###.###.114 but if you call them in DNS often enough they eventually start to walk through their range of the last byte from 112 ... 126. They also shift the subnet they use every four to five days, but not en-masse. It is kind of like they are gradually letting one machine take over for another one and the lag is probably due to DNS caching. 4. Action: WHITE LIST RULES Added: GoodDomains[i++] = "ebonyjet.com"; GoodDomains[i++] = "openoffice.org"; GoodDomains[i++] = "pbskids.org"; Reason: "ebony", "butt", and "peekaboo" respectively. Wed Nov 14 12:29:51: documentation.openoffice.org/tutorials/daibutt/Base/Base.html Mon Nov 5 19:30:15: pbskids.org/images/programs/big-teletubbies-peekaboo.gif I don't want this to get carried away. First, I must emphatically say that NOTHING racial is meant by the "ebony" rule. It is just that from my warped perspective half of the Internet is nothing porn. People MUST be able to use an editor since that is the number one funciton of being on the Internet, and if PBS can have a white list, so can pbskids.org. I just don't want this white-listing to get carried away. I especially don't like the last rule because it is US-centric. I don't think anybody in the French speaking regions need either PBS or PBS Kids (I may be wrong. 5. Action: "angel" rule From: BadHostParts[i++] = "[^r^v]angel"; To: BadHostParts[i++] = "[^rv]angel"; Reason: OOOOOOOOOOOOOOOPS! 6. Action: "[^h]cock" From: BadURL_Parts[i++] = "[^h]cock"; To: BadURL_Parts[i++] = "[^hn]cock"; Reason: hancockbank.com johnhancock.com I toyed with the idea of GoodDomains rules but since this is a URL rule (and dropping it to host is out of the question), we have to live with a weaker rule. 7. Action: 2o7.net Added: BadNetworks[i++] = "128.241.21.14, 255.255.255.255"; BadNetworks[i++] = "128.241.21.15, 255.255.255.255"; Reason: These extra IPs showed up with this alias: metrics1.pricegrabber.com I will run everything through DNS again but the one thing I think needs to be done is hang on to the old IP addresses for at least a month or so before removing them if they stop using them. If you ask me, all they do is add, and add, and add. 8. Action: "tit" rules based on looking at host names Added: BadURL_Parts[i++] = "bigtit[^l]"; BadURL_Parts[i++] = "titten"; Reason: Since Rodney asked how the rule worked I started to indent the hosts that would fail, and I will probably change the BadHostWordStarts[i++] = "tit[^abhilmu]" rule to apply to the entire host, not just the start. The "[^l]" on the first rule is for "bigtitle" which I KNOW we are going to have. The indentation I made was for this rule, but I don't know whether to do it or not. 9. Action: "free" rule opened up From: // BadHostParts[i++] = "free[^d]"; To: // BadHostParts[i++] = "[^g]free[^d]"; Reason: http://drugfree.org/ 10. Action: IP RULES (Rodney) Added: BadNetworks[i++] = "64.255.172.50, 255.255.255.255"; // PORN 008 BadNetworks[i++] = "66.116.125.150, 255.255.255.255"; // PORN 009 THESE RULES ARE IN THE PROXY SECTION : BadNetworks[i++] = "82.98.86.163, 255.255.255.255"; // 303 BadNetworks[i++] = "82.98.86.167, 255.255.255.255"; // 235 Reason: I just hope they don't change but I KNOW at least the PROXY rules will - they change like MAD! 11. Action: BannerBank.ru Added: BadNetworks[i++] = "195.161.119.241, 255.255.255.248"; // BBRU1 BadNetworks[i++] = "195.161.119.248, 255.255.255.255"; // BBRU2 Reason: They have stayed at these IP addresses forever but we have almost NO aliases. I am pretty sure there are some more out there. 12. Action: YieldManager Added: BadNetworks[i++] = "208.67.70.27, 255.255.255.255"; // YMGR Reason: This is NOT to replace the hosts blocks but to catch any that they miss. WATCH OUT - IT BITES! I have had three IP addresses for ad.yieldmanager.com over a year or so. 13. Action: Panther Express Corp (*.panthercdn.com) Added: BadNetworks[i++] = "209.95.139.128, 255.255.255.128"; // PANTHER1 BadNetworks[i++] = "209.170.120.32, 255.255.255.224"; // PANTHER2 Reason: The *.panthercdn is like the instacontent.net in one way which is that that it walks through the ranges of 209.95.139.128 ... 209.95.139.255 and another one of 209.170.120.32 ... 209.170.120.63. There may be more subnets but without more, how do you know? 14. Action: Tightened two GoodDomains rules (thanks Rodney) From: GoodDomains[i++] = "live.com"; GoodDomains[i++] = "monster.com"; To: GoodDomains[i++] = ".live.com"; GoodDomains[i++] = ".monster.com"; Reason: Lots of bad hosts are allowed past. 15. Action: InstaContent ?STATIC? IPs Added: "// PRIVUS instacontent" RULES BadNetworks[i++] = "64.191.208.93, 255.255.255.255"; BadNetworks[i++] = "216.38.160.117, 255.255.255.255"; Reason: The first is for secure.instacontent.net. I have never saw it in my logs. OTOH, I think I have more than just the ec1.images-amazon.com host in my logs for the latter one which is just an alias to ant.mii.instacontent.net. We probably can and should block both. I just wished I could get my hands on the JavaScripts for the latter. If I can duplicate and prove they use the script but I can't pull it right after it shows up in the logs that is enough for me - I WILL block it! { 2010-01-23: Deprecated and removed. } 16. Action: Alteration of PASS alert message From: str = "Passed URL do to Good Domain in host: " + url; To: str = "Passed URL due to Good Domain in host: " + url; Reason: ungrammatical - does this mean that the URL does something to the Good Domain or to the host? 17. Action: *.2o7.net IP addresses From: WHAT WE HAD (SINGLE IPS) To: TO WHAT WE HAVE NOW (SUBNETS) BadNetworks[i++] = "66.150.208.0, 255.255.255.0"; BadNetworks[i++] = "66.150.217.0, 255.255.255.224"; BadNetworks[i++] = "66.151.152.0, 255.255.255.0"; BadNetworks[i++] = "66.151.244.0, 255.255.255.0"; BadNetworks[i++] = "70.42.134.0, 255.255.255.0"; BadNetworks[i++] = "128.241.21.0, 255.255.255.0"; BadNetworks[i++] = "128.242.125.0, 255.255.255.0"; BadNetworks[i++] = "216.52.17.0, 255.255.255.0"; Reason: A new IP showed up when I generated the IP addresses for the m.trb.com host: 66.150.217.4. I did a reverse whois lookup and it obliged with the range of 66.150.217.0 ... 66.150.217.31. So rather than adding just the one IP address I added the entire subnet. Rather than stopping there, I did it for ALL of their subnets. Oh yes, Omniture gave NINE IP addresses for m.trb.com. I did it again right now and only got SEVEN. Here are the rest of the IP subnets I came up with by picking IPs in the middle and then regenerating the new rules: 066.150.217.000 ... 066.150.217.031 066.150.208.000 ... 066.150.208.255 066.151.152.000 ... 066.151.152.255 066.151.244.000 ... 066.151.244.255 070.042.134.000 ... 070.042.134.255 128.241.021.000 ... 128.241.021.255 128.242.125.000 ... 128.242.125.255 216.052.017.000 ... 216.052.017.255 18. Action: Zango and ZangoCash rules. Added: BadNetworks[i++] = "64.94.137.0, 255.255.255.128"; // ZANGO1 BadNetworks[i++] = "66.150.14.0, 255.255.255.128"; // ZANGO2 Reason: I temporarily added a BadDomains .zango.com (but NOT a .zangocash.com) rules. That didn't handle the aliases. These rules handle ALL of them. 19. Action: PROXY rules Added: GoodDomains[i++] = ".facebook.com"; // PROXY BadHostParts[i++] = "facebook"; // PROXY // BadHostParts[i++] = "surf"; // YOUR CHOICE - PROXY BadHostWordStarts[i++] = "surf"; // YOUR CHOICE - PROXY Reason: $ grep -c surf add.Proxy ---> 857 $ grep -c surf add.PacProxy ---> 77 $ grep -c facebook add.PacProxy ---> 39 $ grep -c facebook add.Proxy ---> 16 20. Action: gator.com + belnk.com rule Added: BadNetworks[i++] = "64.152.73.0, 255.255.255.0"; // GATORBELNK Reason: Backup for these networks. Actually, there are an unknown sequence of aliases that they have. If they aren't being called any more then both gator.com and belnk.com are probably now defunct. 21. Action: "ass" PERSONAL rule From: BadURL_Parts[i++] = "[^blmprsvw]ass[^eiu]"; // PERSONAL RULE - 2007-11-10 To: BadURL_Parts[i++] = "[^blmprsvw]ass[^eiou]"; // YOUR CHOICE Reason: I added this on 11 Nov 2007. I said there would be some false positives and there was this one: Sat Dec 1 23:46:08: www.buildinggreentv.com/files/images/buildingmaterialreuseassoc.jpg Funny, I didn't even notice it; also, the only words in common use in English that start with "asso" are all derived from the Latin word associatus which is the past participle of associare (not used in English but used in some Latin derived languages). The other word in Latin related to it is sociare which of course does not cause any problems. 22. Action: imrworldwide.com Added: BadNetworks[i++] = "80.80.13.194, 255.255.255.240"; // imrworldwide Reason: I tried a BadDomains rule for this domain but got nothing I didn't already have. This rule takes all the way to 207, even though the IPs stop for them at 206. If we have false positives, then we will just need a one IP white list rule. 02 December 2007 UNresolved False Positives (HHH) ------------------------------------------------- 1. Pattern: "tgp" Rules: BadURL_Parts[i++] = "tgp"; Reason: www.tomshardware.com/Design/graphics/\ tomshardware/logo_tgpfoot.gif www.vmware.com/files/images/promos/\ ws_promo_tgp.gif www.symantec.com/content/en/us/enterprise/\ images/promo/ent-vista_sec_mktgpromo.jpg ANY IDEAS WHAT TO DO ABOUT IT? 2. Pattern: "chest" Rules: BadURL_WordStarts[i++] = "chest"; BadURL_WordEnds[i++] = "chest"; Reason: Wed May 16 10:11:41: images.bestbuy.com/BestBuy_US/en_US/images/global\ /features/gigrad_blueshirtchest_2007.jpg THE EASY SOLUTION WOULD BE TO ADD A GoodDomains ".bestbuy.com" RULE. THE PROBLEM IS: hope-chest drawer-chest, chest-of-jewels, treasure-chest, etc. Further, even the efficacy of the rule itself poses no reason to drop the rules from URL to HOST: 44 chest_Parts.txt 5 chest_Starts_and_Ends.txt 33 chest_Passed_All_Rules.txt 82 total 3. Pattern: "rape" Rules: BadURL_WordStarts[i++] = "rape"; (PROBABLY OKAY) BadURL_WordEnds[i++] = "rape"; Reason: Fri Jun 8 05:59:23: creativecommons.org/apps/scrape I found the following words that end with "rape" that have 1, 2, or 3 letters in front of that: crape drape grape scrape serape That gives the [^cdeg] in front of the rule at a maximum and a [^c] at a minimum. but remember that grape can be gangrape. SEE 15 Jun 2007 CHANGES IF THIS IS RESOLVED 02 December 2007 RESOLVED False Positives (HHH) ----------------------------------------------- 1. Pattern: "bbw" Rules: BadURL_Parts[i++] = "bbw"; Reason: Sat Jun 2 20:49:05: topics.nytimes.com/adx/bin/clientside\ /1dd00e15Q2F8!Q60VY6sQ2BQ3BXQ5B9L4LNTQ5BQ3BBwQ5BBQ5BXsN Solution: Live with it. I have a personal rule for the Washington Post so New York Times will have to do the same. This is only ONE of TWO false positives I have in my logs. I added the other one because they used NO spies: GoodDomains[i++] = "webbweavers.com";