Go to Making Light's front page.
Forward to next post: De-localization of sex in art: a theory
Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)
— Herinnering aan Holland, by Hendrik Marsman
…en in alle gewesten
wordt de stem van het water
met zijn eeuwige rampen
gevreesd en gehoord.
…and in all quarters
is the voice of the water
with its eternal disasters
feared and obeyed.
Russel Shorto, the New York Times’ go-to Batavophile, has an interesting article up right now: How to Think Like the Dutch in a Post-Sandy World. It discusses the work of Henk Ovink, a Dutch water manager and senior advisor to Obama’s Secretary of Housing and Urban Development. Ovink, who appears to have taken on the role because he was bored out of his skull by how controlled water is back home, is helping HUD to create water policies and to plan for future flooding.
Obviously, there are culture clashes.
Dutch battles against water led his country to develop a communal society. To this day, Water Boards, which date to the Middle Ages, are a feature of every region, and they guide long-term infrastructural planning. American individualism, on the other hand, has yielded a system in which each municipality has a great deal of autonomy, making regional cooperation difficult. “The vulnerabilities are regional,” said Judith Rodin, the president of the Rockefeller Foundation, which is the main funding organization working with Donovan’s team. “Yet we have individual community rule, and very little incentive to get out of that.”
Shorto brings out the deep historical roots of the Dutch communal approach to water management. But he only briefly waves at another important element of the culture’s relationship with the discipline: the memory of past catastrophes, particularly the 1953 Watersnoodramp, the great flood that displaced tens of thousands of people and covered nearly a tenth of the Netherlands’ agricultural land. The disaster occurred when the North Sea, whipped up by storm winds and swollen with spring tide, overtopped the coastal dikes and ate them out from the vulnerable landward side. They hadn’t been built to resist water from the land. It was a critical vulnerability, the sort of thing that happens when there’s more risk than there is money to meet it.
In the aftermath of the disaster came some of the country’s most dramatic water engineering: the Delta Works, which shortened coastlines, moved the fresh/saltwater lines, and culminated with the massive Oosterscheldekering. It was a terrifically difficult and expensive project: people not only had to adapt to some substantial changes to the landscape, but also pay for their neighbors’ safety. If you live below sea level, the security of the dikes is everyone’s business in the end.
But while we’re clicking around, reading about Ovink and musing about 1953, the internet’s shared defenses have themselves been eaten out from the landward side. Four days ago, a critical vulnerability in OpenSSL, the open-source implementation of the web’s basic security protocols, was announced: Heartbleed.
OpenSSL is one element of the enormous body of open source and free software that keeps the internet going. Its failure is a big problem. Encryption and security matter, not only to keep our private business private and our finances under our control, but also to run our infrastructure. Heartbleed jeopardizes all of these things.
When I put the problem like that, it sounds like the solution is to move away from open source software. But the stuff is pervasive because it works; it’s robust, generally secure, and does what people need done. (And closed source software is not notably better.) OSS is as much a part of the internet as dikes are of the Netherlands. Likewise, online insecurity, like water in these times of climate change, is not going away. We’re going to have to learn to live with both.
I’ve been reading Heartbleed articles by techies as well as journalists, and they’ve been writing about it the way that Ovink talks about water engineering. This article by Dan Kaminsky is the one that really got me thinking about the parallels between the two.
There’s a lot of rigamarole around defense in depth, other languages that OpenSSL could be written in, “provable software”, etc. Everyone, myself included, has some toy that would have fixed this. But you know, word from the Wall Street Journal is that there have been all of $841 in donations to the OpenSSL project to address this matter. We are building the most important technologies for the global economy on shockingly underfunded infrastructure…
And so, finally, we end up with what to learn from Heartbleed. First, we need a new model of Critical Infrastructure protection, one that dedicates real financial resources to the safety and stability of the code our global economy depends on—without attempting to regulate that code to death. And second, we need to actually identify that code.
It’s a good read, even if you don’t know the technologies he discusses. And I think Kaminsky’s thesis is sound: this is critical stuff, and we need to treat it like critical stuff without breaking what put it into that position in the first place (the OSS culture). Which brings me round to Ovink again, in another way: resistance to the cultural foundation upon which the tools to protect us are built.
Samuel Carter, an associate director at the Rockefeller Foundation, underscored that the very concept of regional planning is still a work in progress in the U.S. “A lot of people feel that it goes against the American character,” he said. Ovink experiences that pushback on a regular basis. He told me that not long ago he was in New Jersey talking with residents hit by Sandy who were raising their houses on stilts. He laid out for them a future situation in which, rather than have each homeowner undertake such difficult and expensive work, the community would embrace measures to protect an entire region from flooding. The response, he said, was, “That would be a socialistic approach.”
OSS culture doesn’t get called ‘socialistic’, but it’s self-organizing and anti-capitalist in its own way. Creating a bridge between that and the businesses and regulators who are tasked with managing critical infrastructure is going to require an Ovinkian charm offensive. Patrick McKenzie’s article on What Heartbleed Can Teach The OSS Community About Marketing looks like a useful start. And I’m sure there’s much more smart writing that I haven’t stumbled across; I’m just skimming the community.
The final quote in Shorto’s article seems like a good way to end this one, too:
“It’s a long shot,” Eric Klinenberg said. “The only reason to think it will work is that we know if it fails, we’re essentially doomed.”
(As for Heartbleed? Take it seriously. Test your key sites, and change your passwords when they’re patched. Don’t share passwords across sites. Watch your bank statements and your email notifications of purchases and registrations.)
Thanks to Laura Mixon for the Shorto link and Jan Lehnardt for the Kaminsky and McKenzie ones. Eclectic Twitterfeed is definitely the name of my next band.
It teaches us a variety of things, including the idea that "many eyes will surely catch problems" has as much to do with "somebody else will take care of the problem; I don't need to care" as anything else.
The thing about Heartbleed that made my ears prick up was a line in some news article that said it was (1) a simple coding bug, and (2) part of a software release about two years ago. (Specifically 14 March 2012, if I’m reading these release notes right.)
Remember that Apple SSL bug, the “goto fail” bug, that was big computer industry news a few months back? That was also a simple coding error, and it went live in September 2012. At the time, it was noted that this was right around the time that the NSA had been talking internally about having gained the ability to capture data directly from various big companies — Apple, Microsoft, Google, Yahoo, Facebook, and several others. The slides that were leaked last summer were dated October 2012.
And the EFF is talking about possible evidence that a large botnet was actually using Heartbleed to spy on Freenode and other IRC networks in November 2013. “This is an activity that makes a little more sense for intelligence agencies than for commercial or lifestyle malware developers.”
abi, I think you've identified some great commonalities there.
I'd correct you on one point: OSS culture - particularly the GNU/FSF/Linux orbit - does indeed get called "socialist", primarily by big corporations who hope to make money off scaring everybody into running their proprietary code. Microsoft of course is one of those, and the late and entirely-unlamented incarnation of SCO was another.
2
The guy that made the mistake has owned it, but hasn't yet said why it was left in for (at least) two years.
P J Evans@6, it was left in for two years because it was a mistake; once it got noticed, it got fixed and then announced (and then everybody realized how potentially and unauditably bad it could be.)
But there have been comments by other people (notably OpenBSD's Theo de Raadt) that the reason the bug was so serious was that OpenSSL insisted on doing its own memory management by default because the standard malloc() was too slow on some kinds of machines, so there was likely to be interesting data sitting around right near the vulnerable locations where the Heartbleed bug was grabbing adjacent buffers, and so any standard library tools for zeroing free()'d data or operating system tools for blocking access wouldn't get used. It's kind of like having procedures to keep the bank deposit bag on the radiator in the back room of your store so nobody loses it, and later discovering that the window doesn't lock so anybody can reach in and grab stuff.
Back when I was in high school, we had a teletype with time sharing on a PDP-11 at the university, and somebody discovered a similar trick in BASIC that let you grab random chunks of unallocated data on the disk. Mostly it was unintelligible bits, but we also found an old copy of the password file that way.
> change your passwords when they’re patched.
This is also an excellent opportunity to improve your passwords. Consider switching to a password manager.
If you've been using the same password for many sites, consider switching to a rule that varies it per site but is memorable to you: for example, use a constant prefix, like "u7fGdddd", and then append the second letter of the site name, shifted one key rightwards on the keyboard.
Reviewing the basics: passwords that mix uppercase, lowercase, and non-alphabetic are more secure. (If the sole uppercase is the first character, it doesn't make it more secure. If the sole non-alpha is the last character, it doesn't make it more secure.) Passwords that are 9 characters or longer are more secure. Correct horse battery staple. Writing down your passwords on a piece of paper is completely fine, provided the threat model does not involve attackers who might know where the piece of paper might be.
Also, if you use the same password on many sites, you should still use a different password on your banking and your email (because someone who cracks your email can use the "forgot your password?" link on any site to get in).
re 7: Back in my undergrad days at UMCP they changed the student's machine's version of EXEC8 so that it would not zero memory out when it was allocated, to keep them from relying (unintentionally) on uninitialized memory being zero. Of course, current data security issues were not an issue then.
xccd does a typically excellent job explaining Heartbleed, if you want to understand what the bug actually does.
The thing about these kinds of bugs is that they don't sit around marring otherwise perfect software. Pretty much all code has bugs in it, with effects ranging from "totally invisible" to "press this button and everything dies". There are bugs arising from incomplete understandings of the problem space, of the solution to use, of how people use the software, and how the world will be in the future. There are beautifully-constructed algorithms that run delightfully on their own but combine to create a smoking hole where your data used to be.
So it's rarely useful to ask, "how did this bug get into the software?" (though it sounds like the developer was Coding While Tired, which is a risk factor). The questions are, "why was this not caught?", "what other effects does this have?", and "what do we do now?"
I tend to think of it like evolution: we all have the effects of random mutations in our bodies, either quirks we inherited or ones we've developed on our own. Most of the time, they're invisible or trivial (I have some kind of spinal abnormailty that makes it hard to give me an epidural.) But sometimes they will present an evolutionary advantage, or suddenly kill us at the age of 40, or something equally unpredictable.
Dan Kaminsky's article mentions Matthew Green. Some of his views on Heartbleed and OpenSSL are here:
http://blog.cryptographyengineering.com/2014/04/attack-of-week-openssl-heartbleed.html
"more risk than there is money to meet it."
Though with OSS, the key resource isn't necessarily money, it's often attention. How many people would have been capable of spotting that bug? How many of those actually knew about the OpenSSL project? Or did look at the code, or probed the interface hard enough to find such bugs?
Similarly with wikis and other crowdsourced projects -- for any given task, stuff happens if there's a person who's found the project, and wants to do that task, and is able to do it. That includes maintenance against spammers, pranksters, vandals, and occasional saboteurs, which is why wikis are so famously uneven in quality.
By the way, that "test" program says there's a certificate issue for www.nielsenhayden.com: "x509: certificate is valid for *.hmdnsgroup.com, hmdnsgroup.com, not www.nielsenhayden.com".
OpenSSL is a pretty famous project, since it seems like about half the world uses OpenSSL code. But there's a big difference between knowing about/using something and carefully going through it looking for bugs. As Dave said, there just aren't that many people looking.
OpenSSL is a pretty famous project, since it seems like about half the world uses OpenSSL code. But there's a big difference between knowing about/using something and carefully going through it looking for bugs. As Dave said, there just aren't that many people looking.
Dave, #12: "...attention..." just what do you think the salaries of software engineers and QA experts pay for? I keep remembering David Byrne's remark on music, “Do you really think people are going to keep putting time and effort into this, if no one is making any money?” We've had a free ride in FOSS, because of particular historical circumstances. But that ride is ending.
I think the remarks about the use of C as a development language for security-critical software have some validity. If this were English-language text, we would recognize this as the sort of error that line-editing would catch. We ought not be using such vulnerable tools and practices. Over, perhaps, to Go, or C++. Hey, if Go makes it, maybe Plan 9 will yet conquer the internet.
I also think the criticism of open-source software has no validity here. Such bugs are equally possible in closed-source software, and there is an enormous incentive for a closed-source vendor to conceal them. Think of the Chevrolet ignition switch problem, which was concealed for a decade while it was causing fatal accidents. This, at least, is far less likely with open development processes.
From this article:
All [Canadian] federal departments using software vulnerable to the so-called Heartbleed bug have been ordered to immediately disable public websites.
Which means, among other things, that those who haven't completed their tax returns are unable to file them electronically, for the moment. I haven't seen whether they'll extend the April 30th deadline or not.
My return is already complete, so at least I don't need to worry about that. On to changing every password I have, I guess.
Randolph @16:
I also think the criticism of open-source software has no validity here. Such bugs are equally possible in closed-source software,
Yes, very much so.
If the claim is that attention is a problem for open-source software, that's far more true for closed-source, where you have maybe two or three people other than the original coder going over it in code review before it gets merged in. If you've written nontrivial code you've had bugs get through code review, and if you've reviewed nontrivial code you've missed bugs. I include myself in both respects here.
Cheryl @17:
Make sure that the sites are clean (use one of the checkers mentioned earlier in the thread) before changing the password - if they're still vulnerable, your new password could be stolen.
Aaaaaannnnndd to no one's surprise, the NSA has known about Heartbleed for two years:
http://www.bloomberg.com/news/2014-04-11/nsa-said-to-have-used-heartbleed-bug-exposing-consumers.html
@18 lorax
Make sure that the sites are clean (use one of the checkers mentioned earlier in the thread) before changing the password - if they're still vulnerable, your new password could be stolen.
Of course; I should have said that I had done so.
In fact, I tested them in three checkers. Because I'm slightly obsessive that way. I like the output of LastPass best, I think.
Thank you for the reminder, though.
Stefan@19, the NSA and Heartbleed...
(Expanding rant from twitter)
That article starts "The NSA knew [about Heartbleed], two people familiar with the matter said." That's the entirety of the sourcing in the article.
Am I totally misunderstanding the conventions of journalism, or is that just not journalism at all? Shouldn't there at least be some acknowledgement that sources refuse to be named?
(Let's concede from the start that this claim is *plausible*. I want to know why I should think Bloomberg didn't just invent it out of thin air.)
Randolph #16: I keep remembering David Byrne's remark on music, “Do you really think people are going to keep putting time and effort into this, if no one is making any money?”
Actually, yes. Because people make music for their own reasons. For those in whom the creative impulse wells up strongly, creativity is "what they do", as surely as the caged bird sings.
We've had a free ride in FOSS, because of particular historical circumstances. But that ride is ending.
This overstates things, I think. The "free" creative effort of the public is a common resource, which various projects can draw upon (crowdsourcing) according to their appeal. It's possible to boost a project's appeal (see the discussion of Mozilla's "brand" of activism), but any crowdsourced project is also limited by the supply of willing people with suitable skills, which is also affected by how many people can find out about the project. (Thus, social media at any level is an enabler for crowdsourcing.)
Managing such a project to avoid the pitfalls of crowdsourcing (e.g., nobody wants to clean the sewers) is itself a major project in human relations. That applies on multiple scales -- trawling through networking code (especially "working, accepted" code) probably counts as "cleaning the sewers" for most programmers.
Cheryl #20: That actually brings something to mind... I'm considering changing my password keeper. Does anyone have comparative notes among the various "brands"?
Actually, I think looking for security flaws has an advantage over other forms of "cleaning the sewers"; maybe more like "hunting alligators in the sewers". That is, there are people who consider finding security flaws "cool", and so some of those people make a hobby of looking for such flaws. There are also people who make a living at it, but then share their findings (especially about open-source code) with the rest of society. My impression is that this is how Heartbleed was found. Some of the people who found the bug were working on a closed-source "fuzzing" testing tool, but there's no reason the open-source community couldn't create their own "fuzzing" tools. I don't see any evidence in this incident to support "the ride is ending".
Hmm. Not only does Apple not have a statement on their site, but a search in their searchbox for "Heartbleed" yields no results, as of a few minutes ago. WTH? Not even forum posts?
Apple made official comment: http://recode.net/2014/04/10/apple-says-ios-osx-and-key-web-services-not-affected-by-heartbleed-security-flaw/
Why only to a news site, and not on their own site, I don't know.
(MacOS and iOS don't include OpenSSL, but rather a different (open-source) SSL toolkit. As we recall from the *last* SSL bug, which was Apple-only.)
Dave, #22: But this all depends on FOSS developers having time and resources to work on FOSS projects. The major FOSS developers in the past have been funded researchers (Dennis Ritchie or Stallman, as two examples) or students without major financial needs (Torvalds at the beginning of his career.) The funding is drying up.
I suppose future engineers and artists are supposed to live on light and eat air while they are practicing their art.
21
I haven't seen either names for sources (not even 'unnamed source') nor any credible claims that NSA did anything other than, probably, take advantage of it. (Because I'm sure that they have people whose entire job is finding security holes.)
Randolph @ 27 -
Still we work on the code, open source code,
To seek what problems shall be,
And we all shall fare on the light and the air
As our provender and sal'ry.
Heartbleed was a big part of my last week. I think this was a wake-up call to the big Internet players like my employer, and we're beginning to act on that -- this is us talking about the custom software enhancements which largely prevented our customers' private keys from being exposed, and we've now made the patch public.
As the mailing list posting says, "OpenSSL is important to us, and this is the first of what we hope will be several significant contributions in the near future."
Blah, brain is fried, can't think fast enough.
Dave Harmon @25: I highly recommend 1Password assuming you don't need a Linux desktop client. My coworkers who moonlight as cryptographers trust them more than anybody else (and lord knows there's a ton of bad crypto in password safe apps out there). They've also got the user experience basically nailed.
(I do need a Linux desktop client, so right now I'm using the classic encrypted text file. Not pretty, but it works all right.)
I write both open source software and proprietary software. In both cases, I try to write the best code I can, but there are obviously bugs. The proprietary software tends to get more formal testing, though not necessarily any more code review (it depends hugely on the client); the open source software gets more third-party patches.
As for the available developer effort, certain kinds of open source software are healthier than ever: GitHub has something like 5 million repositories, and many companies happily release "infrastructure" libraries as open source. The usual argument is something like: "We sell training software, not multimedia tools. So why don't we open source the multimedia tools, and hope that one of our competitors wants to collaborate?"
But when it comes to end-user software, the incentives are different. It's too tempting to sell mobile apps in an app store, or to create a startup selling web-based software, so these projects do tend to be the focus of fewer open source developers. Also, honestly, it's often more fun to write open source software for other programmers than for end users—other programmers can actually chip in and help out, and certain kinds of end users can be a bit ungrateful. (Handy tip: Before telling an unpaid volunteer that their software sucks, and asking them to provide free tech support, it helps to say, "Thank you." Well, unless the software just destroyed all your data or something horrible.)
Which brings us to OpenSSL: A volunteer effort, with basically no funding. Honestly, this happens all the time. Like any other non-profit, if you want money, you generally need to work for it: you need to run fundraisers, you need to contact potential big donors, and you need to respond to certain tech support emails with, "For this kind of complicated work for a corporation, I think it would be a good idea to set up an official support contract." There are some incredibly specialized end-user tools which raise $20,000/year using a Kickstarter-like model.
Now, as somebody who has written security-critical networking code in C, I agree that certain bugs are almost nightmarishly hard to prevent. It can be done, or at least approximated, but it requires huge amounts of self discipline, paranoia and specialized testing. The Apache webserver, for example, has thousands of moving parts, but it has had relatively few security advisories over the years. And Daniel J. Bernstein, for example, has written a number of very secure programs in C.
But honestly, wouldn't it be nice if our languages made certain mistakes impossible? I'd love it if my compiler could tell me:
1. This program accesses no uninitialized memory.
2. This program never writes data beyond the end of a buffer.
3. This program never allows an array index to overflow because the value is too large.
4. (And for cryptography:) This subroutine always takes exactly the same amount of time to run, no matter what inputs you give it.
Actually, we do have languages which can guarantee most of these things. But they tend to be specialized, and they tend to come with awkward engineering tradeoffs. We need a drop-in replacement for C that can somehow do all this. It's going to take a while to sort out all the details, make it popular enough that people can seriously consider it, and rewrite tons of software that appears to work just fine.
re 32: Well, there's no drop-in alternative for C which could apply test 3, because using arrays as pointers-you-can-do-math-on is something used down in the deepest guts of Unix code.
Dave Harmon @25: I strongly recommend against SplashData's SplashID password manager. At least for the Mac, it's buggy and unreliable. Nothing like opening your password keeper to find that records are displayed out of alphabetical order, and not searchable! So far, it's always recovered, eventually, and I've been too lazy to migrate my data to a good one. Since I'm updating passwords for Heartbleed, and deleting old records as I go, I'll probably (finally) move.
When they added a "cloud" feature, they tried forcing everyone to switch to it. They backed out of that fast! What was appalling is that they introduced it without second factor authentication. They have since added the authentication, but ... really? Store all your passwords in the cloud, accessible with just 1 password, and don't even offer better authentication?
Don Simpson, #29: Obviously that's the chorus of "Code-o-Bedlam's Song."
One of the not-really-talked-about issues is hidden behind the "how to test a site" advice: that this apparently has to be fixed on a site-by-site basis, and that therefore it is likely not to ever get fixed in lot of places.
C. Wingate, #37: Obviously, we need internet traffic tickets. You know, $25 fine for running a server with poor security, $100 for an accidental open mail gateway, $200 per spam...
Take it seriously. Test your key sites, and change your passwords when they’re patched.
A handy list of major sites & their status can be found here:
http://mashable.com/2014/04/09/heartbleed-bug-websites-affected/
(Just so you don't need to test biggest sites.)
Amateur cryptographers are ruthlessly mocked. Programmers get drummed into their skull that cryptography is only to be left to the experts, and rolling your own is a waste of time.
Therefore the number of people reviewing crypto code and spotting obvious coding errors is, basically, zero. The "many eyeballs" thing doesn't apply because it doesn't _get_ many eyeballs, because experts like Bruce Schneier and Dan Bernstein have successfully convinced open source programmers that rolling our own crypto code is a bad idea, and contributing to existing crypto projects isn't something we're qualified for.
This isn't the first time somebody screwed up OpenSSL in an obvious way and nobody noticed for a while: https://lwn.net/Articles/282038/
Stephen Frug @39: A handy list of major sites & their status can be found here:
http://mashable.com/2014/04/09/heartbleed-bug-websites-affected/
I like that list overall, but I find myself reeeeaaaaally skeptical that none of the financial institutions listed were ever vulnerable over the lifetime of the bug. Extraordinary claims, extraordinary evidence, all that jazz.
C. Wingate @37: One of the not-really-talked-about issues is hidden behind the "how to test a site" advice: that this apparently has to be fixed on a site-by-site basis, and that therefore it is likely not to ever get fixed in lot of places.
We can extrapolate backwards, too. We know ancient SSL vulnerabilities remain in routers, switches, remote management cards, cell phones -- devices which are hard to update and whose manufacturers have stopped support or gone out of business or just can't be fucked enough to care.
Therefore the number of people reviewing crypto code and spotting obvious coding errors is, basically, zero. The "many eyeballs" thing doesn't apply because it doesn't _get_ many eyeballs, because experts like Bruce Schneier and Dan Bernstein have successfully convinced open source programmers that rolling our own crypto code is a bad idea, and contributing to existing crypto projects isn't something we're qualified for.
While I agree that we need more eyes on OpenSSL and crypto code more generally, I think the thesis here is flawed -- even when there's a bug in OpenSSL, at least this way we only have to patch it once and then distribute it, rather than going through everybody's different shitty homegrown crypto library auditing its heartbeat support, and then getting all the users whose libraries forgot bounds checking to take updates.
We've got a reasonable amount of diversity in SSL stacks -- OpenSSL and GnuTLS and NSS and whatever Windows uses -- which granted us a fair bit of protection here. It's not a monoculture. But there aren't so many implementations that we'll go mad fixing them all.
@40, and further to @42,
This wasn't an error in the crypto. This was a basic coding error understandable by anyone with moderate competence in C.
I couldn't tell you if the crypto algorithms in OpenSSL are any good, or even if they've been implemented properly. I can see what's wrong with this particular bit of code.
J Homes
Kevin Riggle @41: I was a bit surprised by that too, but then I learned that many of my employer's sites have SSL/TLS handled by our hardware load balancers, which aren't susceptible to Heartbeat. Large financial institutions strike me as less likely to use open-source crypto, and more likely to pay a lot of money for some "Enterprise" solution, which has its own, different, failure modes.
For those following along here -- sadly it turns out those software enhancements didn't protect all the sensitive data, and our customers' private keys might have been exposed.
Following up to my #17, Canada Revenue Agency has reopened its site, meaning that people may now file their taxes again. They've extended the deadline to May 5th.
However.
Regrettably, the CRA has been notified by the Government of Canada's lead security agencies of a malicious breach of taxpayer data that occurred over a six-hour period. Based on our analysis to date, Social Insurance Numbers (SIN) of approximately 900 taxpayers were removed from CRA systems by someone exploiting the Heartbleed vulnerability. We are currently going through the painstaking process of analyzing other fragments of data, some that may relate to businesses, that were also removed.
::sigh::
The CRA is sending out registered letters to inform affected people of the breach. They'll have an 800 number they can call.
"The CRA will also provide those who have been affected with access to credit protection services at no cost. And we will apply additional protections to their CRA accounts to prevent any unauthorized activity."
The Privacy Commissioner of Canada has been informed, and the RCMP is investigating.
And I am selfishly sitting here thinking, please don't be me, please don't be me. I have no spoons for this, man.
Obviously, there are culture clashes.
But isn't one of the basic truisms of development assistance that you've got to work with the local culture as you find it and not as you might wish it to be?
Maybe you can nudge the local culture a bit, but coming in to a situation and saying "The deep historical roots of your culture are no good; you've got to adapt the tenets of my culture" is not generally path to success, at least not in the absence of a huge disparity in resources.
Local municipal authority in that region probably dates back to the beginning of European settlement in the area. An expert imported by the Rockefeller Foundation isn't going to change that in a week or a month, and he or the Foundation thinks he is, they're in the wrong line of work here.
One of the good things about uptake of open-source software ("free" and otherwise) is that more people are doing it, more companies are willing to pay people to both use it and fix it (and contribute back to it), and things can settle into "well, there are 8 ways to do it, but this one is becoming consensus".
When, of course, consensus is just picked up because "running a LAMP stack for your website is zero-cost", you elevate the risk level of any bug in that zero-cost/zero-attention stack.
But a downside to the uptake of open-source software, a downside to the scope of software in general, is that while "many eyes make all bugs shallow", products are proliferating (and the size of each of the products are also proliferating) faster than eyeball time, so there aren't "many eyes" on any particular part of the sample space any more.
But as not-"many eyes" >> zero (outside a company who may decide that passing the risk on to customers is cheaper than fixing something), it's still worthwhile, in the main.
Mycroft:
I don't think many eyes make the bugs shallow. In practice, few people look, and few of those look carefully or well enough to matter. Many of the bugs that have gotten headlines lately were not really very subtle at all. But nobody was looking very closely.
Doug @47:
Some of the local culture (like municipal independence) is genuine, historical fact. But then, so is the existence of wider authorities (like the federal government) that cross those sub-boundaries.
Other elements, like the "every man for himself" mentality that causes people to reject all collective solutions, are much more recent choices. The frontier was also a place of barn raisings and community support; the decision to de-emphasize that is very, very recent, recent enough to be reversed.
America has done large collective-good things, from the transcontinental railroads through the WPA. Collective solutions that keep the Mississippi flowing in its own bed rather than the Atchafalaya's. (This may or may not be a good idea, but it's certainly the fact on the ground.)
Shorto's article is making a point, but it's not necessarily the complete operating truth in this case.
So many rants, so little time. Before I start the banshee keening, one addtional problem added to the current fiasco: Heartbleed allows the theft of security certificates; as many as 500,000 certs on god alone knows how many servers are potentially compromised. Imagine the effect of invalidating that many certs on network traffic. Contrariwise, imagine the effect on security and end-user confidence of not invalidating them.
Now, rants. I spent a large part of the 35 years I worked in the computer biz advocating for better and more reliable software tools (including languages), and even got paid to do that part of the time. Most of what we need has existed (or in some cases, used to exist) for some time, but short-term focus, faulty cost-benefit analysis, programmer culture, and the ever-popular premature optimization waltz kept them from becoming well-known and used.1
I don't think there's much difference between open & proprietary software in this respect. Most people don't particularly like reading other peoples' code, and even when they do read it, they rarely look at it with the kind of concentration required to find significant problems. The mindset necessary to be a good QA engineer is rare, and its value is above rubies.
By the way, just because a problem is well-funded doesn't mean it will be well-built, reliable and robust, or even finished (even in the area of security and law-enforcement; ask the FBI). Ultimately we need a fundamental change in the culture and logistics of software development and maintenance, on the order of creating regional flood planning.
1. Ask me about capability-based systems when you've got some time to spare.
abi @ 50 "Shorto's article is making a point, but it's not necessarily the complete operating truth in this case."
Indeed it's not. Though I still hate seeing rookie mistakes in what's supposed to be the best newspaper the US has.
Bruce @ 51: I'd also say that Heartbleed is an example of the kind of bug that's really hard to find. It's in a part of the system that nobody uses. If you feed it well-formed input it behaves perfectly.
Thus, there are no bug reports. No issues. Nobody searching through that code looking for other bugs.
It's an even trickier to find bug than the average C stack-smashing buffer overflow security hole, because accidental triggering of the bug, if it ever happened, doesn't crash anything. It's a read, not a write.
Comments containing more than seven URLs will be held for approval. If you want to comment on a thread that's been closed, please post to the most recent "Open Thread" discussion.
You can subscribe (via RSS) to this particular comment thread. (If this option is baffling, here's a quick introduction.)
HTML Tags:
<strong>Strong</strong> = Strong
<em>Emphasized</em> = Emphasized
<a href="http://www.url.com">Linked text</a> = Linked text
Spelling reference:
Tolkien. Minuscule. Gandhi. Millennium. Delany. Embarrassment. Publishers Weekly. Occurrence. Asimov. Weird. Connoisseur. Accommodate. Hierarchy. Deity. Etiquette. Pharaoh. Teresa. Its. Macdonald. Nielsen Hayden. It's. Fluorosphere. Barack. More here.