<<

. 35
( 41 .)



>>

algorithm™s “S-box” implementations on the UltraSPARC processor by
ordering the events carefully and using extended logical instructions
provided on the UltraSPARC processor.
The result was another increase in speed”of ninety-seven percent
over the UltraSPARC clients released just the week before. An Ultra-
SPARC system always running the latest client would have gone from
testing 669,000 keys per second two weeks ago to testing 1.22 million
keys per second last week, and then up to 2.4 million keys per second
with this new client.
On the following afternoon, John Falkenthal at Sun ¬red up the
latest client on one of the 64-processor Star¬re systems. He gleefully
reported to the DESCHALL mailing list that the one machine was now
testing 239 million keys per second”more than the entire DESCHALL
project was before April 5.
Putting the new client on a dual-processor UltraSPARC desktop
(with 336 MHz processors), Falkenthal watched it sustain a testing
Terminal Velocity 245

rate of 9.716 million keys per second”roughly the processing power of
the project in the ¬rst week of March.
On Sunday, June 15, Virginia Tech computing engineering junior
Scott Coleman examined his university™s statistics through Graph-
O-Matic. With no small amount of pride, Coleman pointed out the
tremendous leap in their standings was due to a single machine run-
ning the new client.
In Seattle, Washington, Scott McDermott, a system administrator
for the King County Library System, was also running the UltraSPARC
clients. Con¬rming Coleman™s observations, he noted,
I™ve only got three machines running: a pair of [older SPARC
systems] and an Ultra 1. We were placed in the bottom half of
the 300s [near 400th place in keys tested] Friday. I switched the
Ultra to the new client and we jumped up to 213 on Saturday.
I was most impressed!
Also on Sunday, at Oklahoma State University in Stillwater, Colin
L. Hildinger noticed that DESCHALL wasn™t alone in making advance-
ments. Hildinger went over to the SolNET Web site and noticed that
they had just released a new, faster Windows client, boasting speeds
close to the speed of the Pentium clients o¬ered by DESCHALL. While
SolNET struggled to keep pace with DESCHALL on the Pentium pro-
cessors, the latest round of DESCHALL clients for 64-bit processors
further widened the gap in key testing speed.
DESCHALL clients were already running faster than many of us
would have thought possible at the beginning of the project. We had
used so many tricks to get the key-testing clients to run so quickly
that we simply could not rely on improving overall DESCHALL test-
ing speed by increasing key-testing software speed further. We needed
more clients running, which meant ¬nding more participants, but that
weekend, we weren™t going to worry about it. We just let the new su-
perfast clients do their work while we tried to unwind from a stressful
week of getting new software out the door.
DESCHALL developers could use the rest. Not only had we been
working to get the new UltraSPARC clients released, but we spent
extra time going through keyserver and gateway logs to ensure that
the key reports we were getting were legitimate. The alarm that drove
us to perform the extra work came up just as we were preparing to
release the ultrafast 64-bit UltraSPARC client for Solaris, when Justin
Dolske checked SolNET™s mailing list to see how they were doing.
246 CHAPTER 35

Thursday, June 12, 10:39 P.M.
The Ohio State University, Columbus

Dolske took interest in a thread of discussion on the SolNET mailing
list about a surge of activity. Michael Fahlbusch, a SolNET participant
from the Technische Universit¨t M¨nchen in Munich noticed a huge
a u
spike in SolNET clients from the Czech Republic. Excited to see so
many clients coming online at once, he asked how the Czechs managed
it, hoping to ¬nd the method they used to get more people participating
in the SolNET project.
Jan Pazdziora, a network administrator and graduate student from
the Czech Republic was also participating in the SolNET project.
Pazdziora looked at the SolNET statistics for the Czech Republic after
seeing Fahlbusch™s message and came to an unnerving conclusion: an-
other set of hostile clients had been launched to make false key testing
reports to the SolNET key server.
Dolske wondered if the attackers attempted to get the DESCHALL
clients before deciding to cause problems for SolNET. If that were the
case, they would have had to attempt to download a DESCHALL client,
so Dolske examined his client download logs to see whether we had been
getting any requests from the Czech Republic. There was one attempt”
which had (correctly) been rejected”three weeks earlier, but it wasn™t
from a university, much less the same one from which the attack against
SolNET was launched.
Dolske next wrote to Rocke Verser, Karl Runge, and me pointing out
that SolNET had been attacked again, and named the Czech network
from which the attacker hit SolNET.
After getting Dolske™s note, I checked the log for the client download
site under my control, and did not ¬nd evidence that our clients were
sought by SolNET™s attackers.
In addition to looking for evidence of attempted downloads, Dolske
also recommended that we do some additional analysis of the key-
server™s logs and check for any blocks that did not have the expected
reports of “half-matches” (see page 93) that legitimate DESCHALL
clients would make while testing keys. Blocks that had unusual patterns
or amounts of half-match reports could be indicative of a false-report
attack, so the extra step of detection would aid in recovering from an
attack should it occur.
We really didn™t have a lot of time to deal with things like extra
analysis, but integrity of the project was of the utmost importance, so
Terminal Velocity 247

when there was a particular need, we™d make an attempt to deal with it.
In the end, we don™t know why attackers chose SolNET and left DES-
CHALL alone. We speculated at the time that the attackers sending
false reports to the keyservers were engaging in the least interesting and
most blatantly annoying kind of attack against the keyservers”just for
a cheap thrill.
Because DESCHALL never published its source code or protocol
speci¬cation, an attacker would have had to get the client and observe
how it works, which messages it knew how to send, and how exactly
it would format those messages. The relative di¬culty of that reverse
engineering work combined with the di¬culty of actually getting the
clients if not based in the U.S. or Canada might have been what saved
DESCHALL from dealing with the attacks it did. Secrecy is not a
sustainable strategy, but it might well have kept us from being the sort
of “low-hanging fruit” that thrill seekers tend to target.
On the other hand, when DESCHALL did manage to get in the
attackers™ cross-hairs, the attack tended to be much more severe than
false client reports, as the mid-May attack against a DESCHALL de-
veloper demonstrated.
36
Duct Tape




When asked, Internet engineers”particularly when living through the
exponential growth in demand of 1997”will claim that much of the
global computing infrastructure is held together by duct tape and bail-
ing wire. Users rarely saw problems like delays in the mailing list when
something behind the scenes broke; more often, participants would not
notice or would only observe some strange side e¬ects. During the DES-
CHALL project, we had plenty of occasions to work around some part
of the infrastructure that decided to break for one reason or another.
Scott M. Hinnrichs, a participant from California went to the DES-
CHALL client archive on Saturday, June 14, to get updated client soft-
ware. When presented with the question, “Are you a citizen or na-
tional of the United States, a person who has been lawfully admitted
for permanent residence in the United States under the Immigration
and Naturalization Act, or a Canadian citizen,” he clicked “Yes.”
Next he was asked, “Do you agree not to export the DESCHALL
client software in violation of the export control laws of the United
States of America? Or, if you are a Canadian citizen, are you obtaining
the DESCHALL client software for end-use in Canada by Canadian
citizens, or return to the United States, in a manner permitted by
Canadian law?” He clicked “Yes.”
Finally, he was presented with, “Do you assert that you have an-
swered all of these questions truthfully?” Again, he clicked “Yes,” after
which, he moved his mouse over to click the “Submit” button.
Then he waited. Never having experienced a problem downloading
the client software before, Hinnrichs sat there drumming his ¬ngers,
¬guring that the server was just overloaded and that it would eventually
give him the link to the clients.


249
250 CHAPTER 36

At last, the server answered. Instead of providing the list of clients
available for download, it displayed a message: “Export status not de-
termined.” Our download software would show this message when it
was unable to determine from which country the request came.
Thinking that perhaps his computer™s name and IP address were
not mapping correctly, Hinnrichs checked his site™s DNS. His tests con-
¬rmed that DNS was working correctly in both directions: name-to-
address and address-to-name. That ruled out his site as the source of
the problem, so he sent a message o¬ to the DESCHALL mailing list
asking what might be happening.
After he and I exchanged a few messages, I looked at the logs for
the client distribution server. While far from overloaded, there was a
problem.
Our client archive was designed not just to ask people if they were
in the U.S. or Canada, but to make an attempt to verify the likelihood
of the claim. As described on page 145, the software did this by looking
at the veri¬ed domain name of the requesting machine and then ¬nding
that domain™s country of origin by querying the domain name registry
service provided by InterNIC.
InterNIC was established in January 1993 as a cooperative part-
nership among AT&T, General Atomics, and Network Solutions, Inc.
to develop and to operate various services for the Internet community
with funding from the National Science Foundation. These services,
collectively part of an Internet-wide network information center (NIC),
included whois, a service that would report the names and organizations
behind registered Internet domain names and networks. By looking at
the registration information for the domains and networks of the users
requesting the client software, we could reasonably verify that people
claiming to be in the U.S. or Canada probably were.
At the time that Hinnrichs was experiencing problems, our distribu-
tion server was timing out on those whois queries. The InterNIC system
simply never responded, and after two minutes, our server would give
up, and our software would report that export status could not be de-
termined. I tried some manual queries against the InterNIC™s whois
server, to no avail. The service was simply not working, and even the
web site for the InterNIC registry service was down. There was nothing
to do but wait, the last thing that anyone wants to do in a race.
On the same day that the InterNIC registry service was down, Rocke
Verser™s Internet service provider continued making network changes
Duct Tape 251

that had started the previous day. Details of the move were not made
clear, as usually happened for customers with “dedicated” connections
that were always on; Verser was only told that changes were in the
works.
Keeping DESCHALL operating under such conditions was like try-
ing to operate a business whose customers placed telephone orders at
the rate of three per second while the main number was being changed
and without the “owner” knowing what the new telephone number
would be once “changes” were complete. Without details on the net-
work changes, we could not work around any outages or other related
problems. The best we could do was sit around and wait for things to
break and hope that we could respond before any participants™ clients
needed to contact the keyserver.
Fortunately, Verser had the foresight to build a feature into the
client to handle a situation like this long before the problem arose: the
software would notice changes in the keyserver™s address and use the
new address in the event of a change. With nothing left to be done,
Verser kept in close contact with his Internet service provider so he
could update the keyserver™s IP address as soon as it changed. Once
that change was made, the feature in his client software would enable
it to adapt. In the short term, changes like this presented a signi¬cant
challenge”not unlike trying to write a document in a word processor at
the same that someone else was upgrading the operating system. Even
with these troubles, we did not worry about the long-term e¬ects, since
the Internet had time and again proved itself surprisingly resilient.
For applications like e-mail, an outage of several hours wasn™t a big
problem. Not that many people”particularly on Saturday afternoon”
were going to be responding immediately to e-mail. DESCHALL was
di¬erent, though. Our clients were testing tens of trillions of keys per
hour, and our project would experience a huge setback if a signi¬cant
portion of our clients couldn™t reach the key server for a couple of hours.
At the close of what turned out otherwise to be a relatively un-
eventful weekend, at least in terms of e¬orts needed to keep systems
running, Justin Dolske noticed another problem at about half past mid-
night, early on Monday morning. A network ¬le system (NFS) server
at Ohio State had died, taking down both of Dolske™s DESCHALL
T2U gateways that helped clients running behind ¬rewalls communi-
cate with the keyserver. Dolske dashed a message o¬ to Verser, telling
him of the urgent need to remove the Ohio State servers from the list
252 CHAPTER 36

of available T2U gateways. Verser quickly made the changes, leaving
only my T2U gateway at Megasoft Online in New Jersey.
As soon as the change had been made, Dolske sent a note to the
DESCHALL mailing list explaining what happened. He wrote:
The two gateway systems based at [Ohio State University] will
be dead for awhile, as their NFS server appears to have gone
belly-up for the night. The third gateway server should not be
a¬ected by this.
The DNS records for “deschall-gateway.verser.frii.com” have
been updated to remove these [two] systems, so everything
should be working smoothly shortly. Until your local DNS server
gets the new records, gateway users will continue to see a num-
ber of “connection refused” or “timed out” messages.
Unfortunately, these things happen.
The recovery time, measured by usual people-time was fast”only
an hour or two, but at the rate we were going, that delay would cost
DESCHALL a few trillion keys.
During the weekend of June
13“15, we had seen amazing
growth, thanks to the release of 620
600
the new, blindingly fast clients. 580
At the beginning of the DES- 560
540
CHALL e¬ort, the large cor-
520
porate and university campuses 500
with the sophisticated 64-bit ma- 480
460

<<

. 35
( 41 .)



>>