It was with great interest that we read the release of the Spamhaus “The 10 Worst Top Level Domains.” The list claimed to provide statistics that identified “bad domains”—a subject that even caught the attention of the ICANN Board during one of its sessions at ICANN 55 in Marrakech.
As one of the registries that actually does take abuse VERY seriously, we were surprised not necessarily about what was in the report but about what was lacking in it.
What’s the proverbial saying … 98.75% of statistics are made up on the spot?
Let’s start by acknowledging that Spamhaus has historically been a solid source of information. For this reason their recent “World’s Worst TLDs” compilation is pretty difficult for registries to swallow.
Spamhaus’ stated intention for publishing the list was:
[to] help the “Good” Powers That Be (Starting with ICANN and its stakeholders) […] better focus their attention on network abuse issues, aiming for a better tomorrow for our Internet.
Although admirable, our issue lies not in this stated intention but rather in the execution of the report which lacks the following:
- NO measurable metrics
- NO explanation of methods
- NO indication of how Spamhaus defines “bad”
Ultimately readers are left asking this simple question, “How can this data actually help us understand the new domain landscape?”
Why this response?
For transparency sake, I want to place all my cards on the table—a small number of Rightside Registry TLDs ranked poorly on the list. For the purpose of this post, I want to use, as a core example, .NINJA, which at the time of writing was listed as our poorest performer at “31.2% bad.”
What is meant by BAD?
Other than indicating a strong case for being accepted into a ’50s gang from a Sondheim musical (the Jets or the Sharks?, I wonder), Spamhaus hasn’t even attempted to provide an actual definition. The ordinary and plain connotation of the word “bad” sounds dark an ominous, especially when linked to one of the new TLDs. At best, “bad TLDs” is alarmist and certainly not representative of the work registries have invested to manage a clean name space. At worst? Well, it paints an entire industry in a bad light.
The report provides a mere snapshot of a Spamhaus filtered statistic. We asked Spamhaus to provide details on their calculations and they responded that “the ratio is based on domains in the TLD seen by the DBL, in some way, in the last X weeks”. They also claim that the report was more of an indicator of problem areas, not a strictly statistically based endeavor.
Unfortunately, this explanation is just not good enough given that they released strongly-worded public content which has likely already tainted the reputation of any registry for purported “bad” domains. It may well be that the intention was to promote Spamhaus as a white knight, but given the huge spin involved, in reality, one is hard pressed to see the report as something other than a grasp for fame.
Although the Spamhaus report does not provide the actual statistics on which they have based their claims, they did identify their single source: the Spamhaus DBL. It is common knowledge that the details behind the Blacklist provider’s statistics has always remained a closely guarded secret, however the output of their algorithms, the blacklistings themselves, are available to all who want them.
ICANN’s requirement to the rescue!
As a new gTLD registry, we have specific obligations to ICANN whereby we are required to maintain actual statistics and records of any abuse reported within our TLDs. As we pride ourselves on being compliant (we’re responsible that way), we engage a third-party provider to source, correlate, and directly provide any such reports. Spamhaus has confirmed that the “bad” listing is sourced solely from their DBL (Domain Block List).
Our service provider has confirmed that they too provide us, as a registry, with all reports from the Spamhaus DBL involving our TLDs. Therefore, although Spamhaus does not reference any actual statistics grounding their claims, we also have access to the statistical source and so we can test their data to see if we can shed some light on the 30% “bad”rating for .NINJA.
Straightforward Statistics Presented in a Straightforward Manner
Using 2015 as the sample for statistics (as opposed to the helpful period of “X weeks” as confirmed by Spamhaus) we can note the following:
- The .NINJA zone grew by 29,480 domains
- During 2015, we received a total of 2,278 reports of alleged abuse associated with .NINJA (from all sources, including the Spamhaus DBL).
Using simple math (I’m not a statistician), based solely on the reports of alleged abuse, the data reveals a maximum percentage of 7.5%. To be clear, I do not claim that this is a good or a bad number, my point is simply that it is far less than the purported 30% as cited by Spamhaus.
The problem with simple statistics is that they do not provide context and detail. That being said, Spamhaus was accurate in some ways. They did state that promotional periods result in increased numbers of reported abuse. This is particularly true of .NINJA. During the year, we ran a promotion on .NINJA domains. Although hard to categorically quantify, this period resulted in roughly 20,000 new registrations. During that same period we received 1,754 reports of alleged abuse, equating to roughly 10% of the domains registered in that period and 77% of the abuse reported for the entire year.
There is an uncomfortable reality that lower prices are going to attract the baser elements of the internet, and such wrong doers will target every opportunity for their abusive schemes. But let’s not overlook the benefit here; 18,000 registrations were made by persons not intent on abuse. As a company, we reacted very strongly to the increase in reported abuse, and of course lessons were learned from the events. Unfortunately, none of this is reflected, or made clear in the Spamhaus statistics; an isolated spike in alleged abusive use of a TLD by registrants does not make that a “bad” TLD.
Furthermore, Spamhaus essentially confirms that the “bad” listing is not actually based on full verifiable statistics:
One must note that this list does not provide the worst TLDs in absolute quantity, other TLDs may have far more abusive domains, but they also have vastly more non-abusive domains. Instead the list shows the ratio of all domains seen by the systems at Spamhaus versus the domains our system profile as spamming or being used for botnet or malware abuse.
If the listings are not based on absolute quantities and they are not based on the statistics revealed by the DBL in the ordinary course of business, then what are they actually based on? Given the data in our possession, Spamhaus seems to have clearly engaged in driving the percentages up to 30%, as reported in .NINJA. Perhaps the actual percentages were not sufficiently alarming.
A large piece of the puzzle which Spamhaus seem to have disregarded in its report is mitigation—those actions taken by the registries to respond to abuses. Spamhaus, although alluding to some future report about levels of mitigation, have neglected to include such information in their initial report of so-called bad domains. Again to reference .NINJA, they have chosen to ignore the fact that of the 2,278 reports of alleged abuse received in 2015 just four .NINJA abuses were outstanding. This is a percentage rating of 0.01%. Again, it’s very hard to contemplate any justification of the current rating of 30% knowing the effectiveness of mitigation procedures.
Pride in our Process
From our point of view, Rightside reviews the information received for every instance of alleged abuse. Where it is possible to escalate or take action on any such report, we do. Admittedly, at times there is nothing we can do, but that is not from a lack of caring or trying. The important thing here is that we have and follow a process to combat bad behavior. A registry and registrar must, at all times, treat every registrant as a valued customer. Registrants are entitled to due process and fair and transparent procedures. We don’t have the luxury to just accept Spamhaus’ word that a domain is being used for the purpose of spam, and we certainly can’t just assume that the registrant is automatically responsible. Any taken action must be considered, measured, and when taken, it is done so by the most appropriate party, be that the registry, registrar, reseller, or even law enforcement. That is the truth of the situation, but again, Spamhaus have spun an exceedingly narrow and negative interpretation, stating:
Some TLD registries will claim it is not up to them, but to their sales agents (registrars) to deal with any misuse, but if theses registrars also do nothing, the problem remains.
The realities of fair procedure may be inconvenient to Spamhaus, but it is a necessity in the real world.
Spamhaus wants us to CARE more
Setting aside the difficulty we have with the actual statistics, of further ire in Spamhaus’ treatment of all registries is the statement:
A good number of TLDs succeed in keeping spammers off their domains and work to maintain a positive reputation; this shows that, if they wished to, any TLD registry can ‘keep clean’.
One of the benefits of being a Registry Operator with several TLDs in our care is that we have some excellent comparative reference points. Take for instance our .NEWS TLD; the zone contains almost 70,000 registrations to date and Spamhaus reports that .NEWS only has a “bad” domain percentage of 1.3%. Are we to believe Spamhaus’ inference that because.NINJA, has scored a 30% “bad” rating that, as a registry we don’t “wish” to keep our domains “clean”, and so therefore .NEWS should fare equally as “bad” or, indeed, worse? Of course, nothing could be farther from the truth.
We have no interest in sweeping legitimate concerns of abuse under the rug. The problems on the Internet are real, creating dangers and concerns that every stakeholder—from Registrars to hosting companies and beyond—must be involved in resolving, in order to effect a positive change. Rightside is committed to doing our part to resolve these issues, and have actively discussed concerns and problems in past reports, even going so far as to provide system access to prove minimal levels of malicious activity on our networks. It is precisely because we are committed to a better, healthier internet that we are offended by distractions including misinformation, bickering, and finger pointing. We are willing to meet with Spamhaus as well as any other interested organizations to discuss and address challenges and problems and ultimately advance the industry. We are here and willing to listen and collaborate, and hope to receive the same courtesy.
All quotes in this article stem from https://www.spamhaus.org/statistics/tlds/ or the related press release found at https://www.spamhaus.org/news/article/728/spamhaus-presents-the-worlds-worst-top-level-domains (last accessed 11 March 2016). For transparency we are using an independently compiled view of the Zone from http://www.nTLDstats.com.
Send comments to email@example.com.