27 Juillet 2009 Changes (HHH) ----------------------------- 1. Action: Added ad rule. Added: BadHostWordStarts[i++] = "ads1\."; // YOUR CHOICE ADS - 2009-06-21 BadHostWordStarts[i++] = "ads1\."; // VOTRE CHOIX ADS - 2009-06-21 Reason: It is identified as the next biggest ad rule after "ads." as identified by ABP + EasyList. 2. Action: Added BadDomains rules Added: BadDomains[i++] = ".mybeliefs.info"; // DNSWCD - WebBug - 2009-06-22 Reason: MDL removed a host named joerobertsrocks.com. I WILL NOT REMOVE IT. It's index.html file consisted of one huge line with calls to various scripts in this domain and the size of the file was 227,472 bytes. I wrote a fast and dirty C program and then used my fhttp program to extract the names in the index.html. Here they all are: mayo.mybeliefs.info mcallen.mybeliefs.info mckesson.mybeliefs.info mean.mybeliefs.info measurement.mybeliefs.info media.mybeliefs.info medications.mybeliefs.info meeting.mybeliefs.info meijyo.mybeliefs.info member.mybeliefs.info memphis.mybeliefs.info mendoza.mybeliefs.info mercedes.mybeliefs.info merlin.mybeliefs.info message.mybeliefs.info metallbau.mybeliefs.info metrosoft.mybeliefs.info mga.mybeliefs.info michael.mybeliefs.info michelle.mybeliefs.info mick.mybeliefs.info micro.mybeliefs.info microsoft.mybeliefs.info midge.mybeliefs.info mika.mybeliefs.info milf.mybeliefs.info milky.mybeliefs.info milwaukee.mybeliefs.info mini.mybeliefs.info minneola.mybeliefs.info mira.mybeliefs.info mise.mybeliefs.info mississippi.mybeliefs.info mitsubishi.mybeliefs.info mls.mybeliefs.info moab.mybeliefs.info mod.mybeliefs.info modern.mybeliefs.info moldy.mybeliefs.info money.mybeliefs.info monster.mybeliefs.info monte.mybeliefs.info moody.mybeliefs.info mormon.mybeliefs.info mortgage.mybeliefs.info mosquitoes.mybeliefs.info motels.mybeliefs.info motion.mybeliefs.info motorcycle.mybeliefs.info sub426.mybeliefs.info Each host had muliple calls for multiple PHP files. All of the PHP files look something like this: http://www.securemecca.com/public/ScienceProg_com.png That was just the first pass. I did a second wget because those names looked like they were generated by a PHP or other script. They aren't. I got the same stuff the second time around. The problem is, this is an exploit designed to gag a brower with do nothing PHP scripts. Accordingly, this rule was added along with joerobertsrocks.com not only not rocking on Windows but on Linux / Macintosh / other Unix as well. I even may add these pseudo hosts for a while ... I am going to add the sub426.mybeliefs.info since it is FIRST in the index.html file. After you have this rule and the host joerobertsrocks.com (and www....) blocked in the hosts file you could try going here: http://www.dit.gov.bt/cic/forumMessage.php?id=1111 I sent them several email messages and finally a snail mail letter. I don't know what good it will do. 3. Action: over-ride for tracker rule Added: GoodDomains[i++] = "securitytracker.com"; // tracker - 2009-06-23 Reason: It seemed to have stuff relevant for all security people which is what this PAC filter is for I guess. I thought I was creating it for normal people. 4. Action: Another SPAM IP Added: BadNetworks[i++] = "212.174.200.111, 255.255.255.255"; // SPAM-01-g - 2009-06-24 Reason: kceplh.ollectimon.com 2009-05-25 mvhusu.ollectimon.com 2009-05-25 ffhqnt.lisqunriep.com 2009-06-17 painmedspharmacyrx.com 2009-06-23 5. Action: One more tweak to try to reduce the DNS queries. Date: 24 Juin 2009 From: var HasIPv4Address = true; ===== if (! isResolvable(host)) { HasIPv4Address = false; } ===== tmpNet = GoodNetworks[i].split(/,\s*/); - if (isInNet(host, tmpNet[0], tmpNet[1])) { ===== tmpNet = BadNetworks[i].split(/,\s*/); - if (isInNet(host, tmpNet[0], tmpNet[1])) { ===== To: var HasIPv4Address = true; + var IPv4Address; ===== if (! isResolvable(host)) { HasIPv4Address = false; } + else { + IPv4Address = dnsResolve(host); + } ===== tmpNet = GoodNetworks[i].split(/,\s*/); + if (isInNet(IPv4Address, tmpNet[0], tmpNet[1])) { ===== tmpNet = BadNetworks[i].split(/,\s*/); + if (isInNet(IPv4Address, tmpNet[0], tmpNet[1])) { ===== Reason: I doubt it makes it any more efficient but at least it gives the appearance that no stone has been left unturned to tune this to go as fast as possible. 6. Action: Removed commented out rules in GoodDomains Date: 24 Juin 2009 Removed: // GoodDomains[i++] = ".hulu.com"; // thumb US-only // GoodDomains[i++] = "netscape.com"; GoodDomains[i++] = "netscape.com"; // GoodDomains[i++] = "network54.com"; // GoodDomains[i++] = ".pbs.org"; GoodDomains[i++] = ".pbs.org"; // VOTRE CHOIX // GoodDomains[i++] = "pbskids.org"; GoodDomains[i++] = "pbskids.org"; // VOTRE CHOIX // GoodDomains[i++] = ".shutterstock.com"; // thumb // GoodDomains[i++] = ".thumbshots.com"; // GoodDomains[i++] = ".thumbshots.org"; // GoodDomains[i++] = ".webshots.net"; // thumb Reason: We have not needed them. Netscape was removed from all files since it is a part of AOL now. pbs.org and pbskids.org were removed completely from the French files because they don't need it. The French and English files are going to star going their separate ways now. 7. Action: Widened the existing SPAM rules. Date: 26 Juin 2009 (06 Juillet 2009) From: BadNetworks[i++] = "60.191.221.123, 255.255.255.255"; // SPAM-01-e - 2009-06-15 BadNetworks[i++] = "60.191.239.164, 255.255.255.224"; // SPAM-01-b1 - 2009-06-15 To: BadNetworks[i++] = "60.191.221.117, 255.255.255.224"; // SPAM-01-e - 2009-06-26 BadNetworks[i++] = "60.191.239.150, 255.255.255.192"; // SPAM-01-b1 - 2009-07-06 Reason: New spammers. This seems to be their IP block for their new embedded URLs. 8. Action: Added two new SPAM rules. Date: 26 Juin 2009 Added: BadNetworks[i++] = "119.39.238.2, 255.255.255.255"; // SPAM-01-h - 2009-06-26 BadNetworks[i++] = "218.75.144.6, 255.255.255.255"; // SPAM-01-i - 2009-06-26 Reason: New spammers. This seems to be their IP block for their new embedded URLs. 9. Action: Added two new Traqueur rules Added: BadURL_Parts[i++] = "seostats\.php"; // Tracker - 2009-06-26 BadURL_Parts[i++] = "site_stats"; // Tracker - 2009-06-26 Reason: The first is because of mypagerank.net. The second is because of xslt.alexa.com. I have tried blocking both before and ran into problems. There are other web sites using the first term though. 10. Action: Once more - mods for the IP stuff. Date: 28 Juin 2009 From: What it was To: To what it is, basically if the host is an IP do not do a lookup and do only one check to make sure we do not have an IP address before looping through the IPs. I doubt the one check helps but the already have an IP WILL help. Anything that reduces DNS queries will help. Reason: To make it more efficient. But there may be future changes. 11. Action: Change to FTP string code which will go away, also any other changes I felt Danny made I could live with. Date: 29 Juin 2009 From: if (shExpMatch(url, "ftp:*")) { To: if (url.substr(0,4) == "ftp:")) { Reason: Danny suggested it as a speed up. 13. Action: Block Pseudo Park IP addresses (formerly DUMPERS) Added: BadNetworks[i++] = "61.143.211.187, 255.255.255.255"; // Pseudo-Park-1 - 2009-06-29 BadNetworks[i++] = "188.32.0.2, 255.255.255.255"; // Pseudo-Park-4 - 2009-06-29 BadNetworks[i++] = "221.231.137.94, 255.255.255.255"; // Pseudo-Park-2 - 2009-06-29 BadNetworks[i++] = "222.191.251.143, 255.255.255.255"; // Pseudo-Park-3 - 2009-06-29 Reason: There used to be no web servers where these things are. Now there are and they are ANYHING but normal. The list of hosts so far that were at each IP address are: - 61.143.211.187 dudpoe.cn, especialsinkbarrel.cn, goooog1e.cn, onemoo.cn (2) - 188.32.0.2 qualifyloud.js.cn, razorcorktame.sn.cn, relieveprocessiontribe.sh.cn, reproducearrange.sn.cn, restaurantcivilize.mo.cn, reviewloyaladmire.hi.cn, scissorseducate.jx.cn - 221.231.137.94 ip127.cn, kuaiqin66.cn (1), onemoo.cn (1) - 222.191.251.143 hotpornmovies.org, kuaiqin66.cn (2), meanwhileapple.cn, onemoo.cn (3) 14. Action: MORE SPAM IPS Added: BadNetworks[i++] = "222.241.150.146, 255.255.255.255"; // SPAM-01-j - 2009-07-06 BadNetworks[i++] = "222.186.12.113, 255.255.255.255"; // SPAM-01-k - 2009-07-08 Reason: What can I say? THEY ARE GROWING! 15. Action: Added temporary rule for myself Added: BadDomains[i++] = ".smartbizsearch.com"; // PRIVUS - 2009-07-12 Reason: This host named svetyivanrilski.com was one of several on this hacked Bulgarian web server. This along with about a dozen trackers including two dead clickers (brugeni.net, maislex.net) got into my way analyzing the web site to the point that I had to tame everything down some. I am also adding the host smartbizsearch.com into the hosts file. Some of the trackers that are now being added to my hosts file are click.search123.uk.com, rontraffic.com, www.rontraffic.com, thestatsdata.com, www.thestatsdata.com. I am also adding cgi.search123.uk.com to my own hosts file. Oh yes, for now svetyivanrilski.com is still being considered for being blocked. But now I got all this CRAP out of the way it seems okay, but for some reason none of it is being called any more. { 2010-01-23: Yes it was. Discovered and the host clicks.smartbizsearch.com is in the hosts file so the rule is no longer needed. Feel free to disagree, but I will NOT stop the blocking of the clicker / redirector because it could actually redirect you to a malware delivery host. } 16. Action: Added rules for Malware list sites Added: GoodDomains[i++] = "freepcsecurity.co.uk"; // 2009-07-17 GoodDomains[i++] = "malwaredatabase.net"; // 2009-07-17 GoodDomains[i++] = "malwaredomainlist.com"; // 2009-07-17 Reason: The first conflicts with the "free" rule when activated. The second had a hdporn.jpg file it conflicted with and since I had the third for myself I made it available to everybody. 17. Action: Added white list rules Added: GoodDomains[i++] = ".bp.blogspot.com"; // 2009-07-19 GoodDomains[i++] = "ddanchev.blogspot.com"; // 2009-07-19 Reason: I added both of them. Since the PAC filter is only being used by security people, why not have Danchev's blog allowed? We give up sume with the first rule but some compromises forced the issue. 18. Action: Whitelist rule for TrendMicro Added: GoodDomains[i++] = ".trendmicro.com"; // av2009 - 2009-07-21 Reason: av2009 - who would have predicted it? 19. Action: Removed SPAM rules Date: 2009-07-24 Removed: BadNetworks[i++] = "58.17.3.32, 255.255.255.240"; // SPAM-01-a - 2009-06-05 BadNetworks[i++] = "60.191.239.150, 255.255.255.192"; // SPAM-01-b1 - 2009-07-06 BadNetworks[i++] = "60.191.239.192, 255.255.255.252"; // SPAM-01-b2 - 2009-06-05 BadNetworks[i++] = "60.191.221.117, 255.255.255.224"; // SPAM-01-e - 2009-06-26 BadNetworks[i++] = "61.191.191.241, 255.255.255.255"; // SPAM-01-f - 2009-06-15 BadNetworks[i++] = "203.93.208.86, 255.255.255.255"; // SPAM-01-d - 2009-06-05 Reason: MOVED - shows the fallacy of depending on IP addresses to be static - host-names alone are volatile enough but these are HELL. 20. Action: Removed FTP exclusion Date: 2009-07-24 From: // Version: 2.9.4 if (url.substr(0,4) == "ftp:") { return "DIRECT"; To: // Version: 2.9.5 // if (url.substr(0,4) == "ftp:") { // return "DIRECT"; // } Reason: Unfettered FTP access causes a security risk. Thus I stopped recommending Privoxy and shifted to recommending NoScript. At the time, with Privoxy it was an acceptable risk and still is. But they need this ONLY if they use Privoxy (I think). If that isn't the case we will need to recommend a FTP program people can use. 27 Juillet 2009 UNresolved False Positives (HHH) ------------------------------------------------ NONE 27 Juillet 2009 RESOLVED False Positives (HHH) ---------------------------------------------- 1. Pattern: "teen" Date: Rules: BadHostParts[i++] = "teen"; BadURL_WordStarts[i++] = "teen[^y]"; BadURL_WordEnds[i++] = "teen"; Reason: The problem lies in the URLs, not the host names which almost always deliver porn, malware, or both. I need to study them a little more since what it means is I will probably just have to delete one of the URL rules. I am amazed it took me 2+ years to all of a sudden having false positives here. Oh well, I will handle it next month. Solution: DO NOTHING! Nothing has shown up since then. Let people either remove or white-list it as desired. The porn terms are waning as people are steering clear of them realizing that many infect their Windows machines. Either that or MalwareDomainList isn't picking up any new ones. I will have to take a look at the names again to decide if there are new patterns or old ones that may no longer be useful. I lean against removing ones that have worked well in the past - marking them and watching what happens works best.