I have no idea, and that's one of those things which has a hint of orwell... if we are all keeping huge chunks of things online, and Google, et al., have the ability to make them invisible....

i share your concern.

i've used the Usenet archives for doing research on programming topics, for years and years. and now Google's gone and hidden-away all that information - decades of programmers discussing programming.

f-ers.

I was always under the impression that copyright in Usenet posts was held by the original poster. The only alternative I can think of is for the rights to be held by "Usenet", and there was no "Usenet" entity beyond "the collective of all users" to the best of my knowledge. (That, or for all posts to be public domain, but the few copyright discussions I remember seemed to take the position that that was absolutely not the case.)

What were the circumstances under which Google acquired the Usenet archives?

Interestingly, this appeared in my aggregator at the same time that this did...

Slashdot has the story, here.

A posting says there's a bug in the advanced search form and gives a workaround.

I am able to follow that link ( http://groups.google.com/group/net.sf-lovers/msg/988b7faf00af1308 ) in a browser with no Google cookies. I'm not signed into Google in any way.

So while you're getting wacky behavior, it doesn't seem to be a policy decision.

(Related, although not evil as far as I can tell: the Google Groups "advanced search" page was rewritten recently. This is what I typically use to search Usenet. I think it lost the date range constraint for a while, but if so they screwed their heads back on and replaced it.)

That's really weird. I vaporise all of my cookies regularly, and I've never run into this.

I was able to follow the link with all cookies deleted and it worked (in IE 7, Firefox 3.0 and Opera 9.6). How very odd. Maybe it's because I'm coming from outside the US?

Google's behavior toward Usenet has been antisocial at best for years -- their posting software used to break long-established Usenet standards, and they never seemed to care. I drifted off Usenet years ago, so don't know if they ever fixed it.

I've noticed that sometimes Google will give you different answers on the same search a few minutes apart, and occasionally give severely truncated results. They do optimize for speed rather than completeness, so if you happen to hit a server that doesn't have a full index or isn't able to get to parts of its indexes, you may miss things you'd otherwise hit.

(I don't know exactly how www.google.com resolves, but I do know that it not only varies by location but does seem to go to different servers at different times. Given the number of hits they get, they have to be smart about spreading out the load.)

I don't know if this has anything to do with the missing Google results, but it might be worth trying the same search at different times if something seems unavailable initially.

I've run into this a couple of times when not explicitly signed into Google, even though their cookies are all over my machine.

IIRC, I first noticed it when I *was* signed into Gmail, but wasn't explicitly signed into Google as whole. I tried to follow a link from a post in James Nicoll's LiveJournal to a specific post in the Google Usenet archive, but in that case I got a message telling me that I couldn't see it unless I set up a GoogleGroups account.

There have been a few other times when I was searching for something I was sure was in there, and failed to get any hits, but without the "please log in first" message appearing. It didn't occur to me at the time, but I think it's the same behaviour Teresa's describing.

For those who are able to get to it, here is one discussion about this very topic that is taking place, ironically enough, in one of my favorite Usenet haunts, alt.callahans. I've linked there to the discussion here, by the way.

For those of you who, like me, use a newsreader or email program to access Usenet, I encourage you go look up alt.callahans and check it out that way.

And for those who didn't know and are wondering: Yes, that Callahan's. The community in question is modeled on and populated largely by fans of Spider Robinson's Callahan's Crosstime Saloon.

I can access the post in question from the direct link here, and I'm not signed into Google in any way, shape or form.

Needless to say, this makes the whole thing only more confusing.

"While I honor Google greatly for the work they’ve done in searching out and compiling old archives from the frontier days of the internet, that doesn’t mean they own the old Usenet message base—does it?"

I am not sure how much work was Google's, and how much Dejanews'.

The copyright on a work originally belongs to the author. Did anybody ever sue Google over massive copyright infringement of their Usenet postings?

Before Dejanews, Usenet used to be a largely ephemeral affair. I am not sure we have a right to tell Google how to keep its own database accessible. Me, I am more worried about the lying bit. If they'll lie about Usenet postings, they'll lie about any search domain.

Google's USENET archive in no way complete. The further back you go, the lower percentage of USENET postings they have. That's not their "fault" - USENET archives from 20+ years ago were erratic to put it mildly. And Google's only been existence a little over ten years.

I generally like Google's USENET interface, but as USENET was pretty much overwhelmed by trolls years ago, I don't follow many newsgroups these days.

I don't have a problem with Google's cookies as I use Google quite a bit. The only feature I pretty much avoid is gmail. I have a gmail account but it's only there to collect USENET spam.

Much of the first few years of Usenet is in the public domain. In the US, a copyright notice used to be required on works, or they'd enter the public domain upon publication. The notice requirement ended in March 1989, as part of the US adoption of the Berne Convention (which explicitly prohibits any notice requirement or other "formalities" for copyright).

I joined Usenet in 1987, when the notice requirement was still in effect. Some folks were putting copyright notices on their posts, then, but most posts didn't have them. (And I suspect lots of posters didn't care about whether their posts were copyrighted.)

I work for a competitor to certain bits of Google, so praising them is something I do but grudgingly, and defending something I rarely do at all.

But this sounds more like a bug than Evilness.

Basically, Google's interest is in giving you all your results because then you come back and they get more ad revenue (and they also get more data about behavior and related topics, generating more targeted ads, which generates more ad revenue). That's their main business model.

Chomping off search results thus makes no sense, outside of caching and slow propagation, since Google doesn't charge for their free services. You might think this is a gimmick perhaps to get people to sign up for GMail, but that would take too much work.

So yeah; bug.

Google did improve the Google Groups interface, but it's not something I use with any great amount of enthusiasm. This is mostly because I hate email lists and Usenet, though, not so much the interface (which I do like better than, say, Yahoo's).

On the off-chance that he can do something about it, I forwarded the link to this post to a friend who works in user experience design at Google.

As far as I know, Usenet contributions are owned by the contributors. There's not been much (if any) litigation on the subject, but the rough-and-ready rules have been that one can republish postings in public electronic form, but if one wants to reprint them for profit, one has to get the contributors permission. As for the non-recognition of Usenet, this has to do with the failure of Usenet as a political entity, something I've been writing about for about 25 years. There was not, in the early days, any willingness to get together and give the old co-op a formal presence; the arguments made against this were exactly the same arguments libertarians make against economic regulation. In reality, I think, the arguments masked a near-total unwillingness to engage the problem and now it is probably far too late; co-operative social networking is no longer typical of the net.

As for evil...I say Google is the not-so-evil empire. They are not a co-op. They act, so far as I can tell, by and large decently, but that depends entirely on their management's goodwill.

On Teresa's observations and question in more or less reverse order:

Re Question:

I hesitate to give advice to you about anything publishing-related, but here goes!

One can claim a valid compilation copyright in a work comprised solely of original material created by others - but I believe the upshot of the various court cases involving phone books, directories, atlases, databases, et al. is that some element of creative work must be involved in the selection process. You've edited anthologies, so you know what that's about. If, like a phone book you simply throw in everything you're able to find, there is no selection process, no creativity, and nothing which can be copyrighted. My conclusion is that Google can hold no copyright on its Usenet archive, no matter how diligently they worked to assemble it.

Obviously, you should check this with a copyright lawyer before relying on it.

Observations:

3, 2: I believe your complaints 2 and 3 stem from the same problem - Google Groups is now primarily in the hands of people who have never used Usenet and are hardly aware of its existence. Everything I've seen of it in the last few years says they don't begin to understand Usenet. They're modelling its operation on private mailing lists like Yahoo's groups, but they don't seem to understand those very well either.

The Google people who originally built it up from their acquisition of the Deja News archive clearly understood its importance and were Usenet users (though IIRC they had some bad policy choices regarding posting from Google at times.) I am guessing they're no longer involved with the project.

At some point a while back Google Groups changed the interface in ways that really obscured that there was an underlying, pre-existing network of newsgroups. Then they changed it some more. As I recall, Deja News had done pretty much the same thing shortly before they lost all remnant of interest from the public and folded.

Unfortunately (in this respect) Google is unlikely to run out of funding for it no matter how much they manage to confuse and turn away potential users or to obscure valuable information.

1. As it happens, I have a fairly cookie-free computer myself because Friday night's automatic update managed to kill my home Windows installation somehow; I had to reinstall Windows and then start reinstalling all the programs I use, so I am fairly sure I have no Google login cookies myself. Going directly to the Google Groups page takes me to a "please sign in" page, like you.

The link nonetheless works for me. I conclude it's a bug and probably a transient one.

Not Google-evilness, but IIRC, in connection with a crackdown on Usenet child p*rn groups (said crackdown initiated by Andrew Cuomo), Time Warner decided to block customer access to all Usenet groups.

TW is my service provider, but I don't track any Usenet groups that I could check and see if this is in fact true. It was widely reported a couple of months or more ago.

I own the copyright on everything I wrote on Usenet.

For stuff posted to moderated groups, the moderator owns a compilation copyright on the selection.

DejaGoogle could reasonably claim that it's providing a Usenet server with a custom interface, and therefore no more of a copyright infringement than any other server.

Google recently changed the advanced groups search page, and apparently screwed up some stuff.

"Aaahm... medium evil."

(Not to drag anyone kicking and screaming back to the '90s or anything.)

Maybe I don't understand. Even with cookies blocked and deleted I can dig as far back as there are archives to dig.

(Also not States-based.)

I've typed over a hundred thousand words into Usenet, so I am very interested in being able to read my own stuff. I have made heavy use of the Dejanews and Google searches to read my own words and the conversational context around them. I am strongly interested in preserving, or improving, access to that body of text.

(Usenet also carried the postings of my late brother, and of other friends who have passed away.)

In the past couple of years, Google's Usenet search has degraded considerably. I don't understand why, but I will follow the present thread with great interest.

It's working for me, too --- in a browser profile with all Google cookies blocked, a groups search for "nielsen hayden" gets me lots of results; the one that winds up on top of the heap happens to be from 1999. So, from where I sit, no systematic walling off of the archives is obvious.

On the other hand, one of Google's known (and arguably bad) habits is testing out new, and quite possibly buggy, versions of their stuff on a small, randomly selected portion of their normal userbase. So, it's at least possible that Teresa was seeing a new version of Groups query processing which, for whatever reason, doesn't deal well at all with diabetic browsers rejecting their cookies.

In this case I think it's incompetence and indifference rather than slight evilness. (Though one might reasonably argue that indifference is evilness.) I had the same problem a week or so back - it demanded I log in to access an article; shortly after this I moaned about it to a few friends; said friends accessed it without a problem; and I, upon checking again, could get to it without a problem.

Google Groups' search feature has been ridiculously buggy for more months than I have fingers, their browsing interface gets worse every time they change it, and they're not even bothering to filter all the spam that their portal lets people spew across Usenet. If there weren't a number of decent people relying on it to access rasfc, I'd pray that Google would just admit the experiment's been a failure and turn the archives over to someone else.

Did anybody ever sue Google over massive copyright infringement of their Usenet postings?

AIUI, there's a very good chance that the act of posting to a system that you understand at the time you do so is intended to distribute copies of your message to anyone who wants to read it implicitly grants "the system" a license to do that. Usenet is a very loosely defined system, and it's quite clear that google are as likely to be beneficiaries of that license as anyone else.

If you notify them that you do not wish them to distribute your messages, they may be legally obliged to cease doing so. OTOH, I doubt this matters very much because I believe they honour such requests anyway.

As to the completeness of their archive, a lot of the stuff I posted between 1994 and 1996 is missing. I was a heavy poster in that time frame, but searching for my email address from the period only turns up 49 threads, many of which are lacking my messages but include people quoting them. I guess this is about 10% of what I posted. At least this thread appears to be mostly intact. That was history being made. Or at least plotted.

On the general copyright issue, Google's reputation is less than lily-white. Most recently, there was the fuss about Google Chrome.

The details of that copyright grab were so similar to those of other companies, scattered back over my experience of the Internet, that I suspect that few lawyers know anything of the legal history. Worse, they cannot didstinguish between the copyright there might be in the design of a web page, or the design of a book cover, and the words than another person has written. Somewhere, it seems, there is a piece of aggressive catch-ll copyright boilerplate which is used by everyone.

I'm also told that there are clauses in standard publishers' contracts which competent agents strike out on sight, and the publishers don't argue the point.

How many internet users have an agent? What ISP would accept a customer sending back a contract with an amended copyright clause? It may be nothing more than inertial stupidity, but it all feels part of the way in which an ISP seems able to buy and sell customers as if they were slaves. And Google is tainted by being a part of that world, and apparently having less control over its lawyers than it does over its programmers.

Zeborah #26: If there weren't a number of decent people relying on it to access rasfc, I'd pray that Google would just admit the experiment's been a failure and turn the archives over to someone else.

The Internet Archive would be a logical choice to manage an archive of USENET.

When I first saw the thread title, I though, oh no, what repressive regime have they added to serve out their collection of censored search engine results?

"They" being Google (to clarify).

Here's a ceg - a comment beg, unfortunately, not a misspelled offer of beer :(

What newsreader software do people use?
What if you read news on more than one computer?

The question of whether ownership of the only copy of something gives you more than permission to read it yourself or allow others to read it locally in your presence has been pretty well settled by various cases involving the letters of living authors.

Now here's question: because Google sells ads against usenet, does this unambiguously establish a value for those texts, so that someone could finally could be liable under USC 1801--whatever for unauthorized distribution of copyrighted material worth more than $1K?

Hanlon's razor: Google is a Big Corporation, and thus full of Stupid; Usenet is a marginal issue for them which they don't know how to monetize. The result is an interface which is getting worse by season (and user support that I'm sure will be a large part of my stay in Hell).

I think Google is much less full of stupid than its predecessor search engines.

I worked for Lycos for nearly five months in early 1996 when they were still based in Pittsburgh. A computer scientist at Carnegie Mellon came up with Lycos, and it was practically the first Web search engine in the mid-90s. It was then sold to a company that was basically marketing company.

This company had no clue about the Internet. Nada. It was really pretty appalling.

At times, they did not update the Lycos search catalog for two solid months. I remember that in particular because I was the person answering the "letters to the Webmaster" E-mail. As you might exepect, many people asked us why their sites were not listed. That was why. Not that I could say that. I could only give them generalities along the lines of "We will be updating the catalog soon." Gaak.

I made a bunch of proposals about things they should be doing. One of the Lycos guys (a Pittsburgh-based programmer) and I often made the exact same proposals. Two things we both pushed them on were archiving and offering a USENET groups interface and archiving things like CNN, AP et.c. (as ClariNET was doing on the Internet in those days). Lycos ignored every suggestion we made. I quit in utter disgust a few months later.

Google pretty much implemented every suggestion we ever made to Lycos (and a whole lot more).

Rob #20:

TW did close down their *own* servers, but I'm a TW subscriber and I most certainly can access other Usenet servers, such as news.individual.net .

The way you phrased it, it sounded as if there were some Great Firewall of Time Warner. Not so.

I've saved all the "log me out" links I see to a folder, and on occasion click "open all in folder"-- it's always surprising.

How come there isn't a simple transparent tool available for keeping track of who's keeping track of you, on a realtime basis?

______________________________________________
"He found himself gazing into a glittering, interested brown eye ... The chimp was looking through the keyhole to see what the psychologist was doing."

-- Murray Leinster, "Keyhole"

joann @35: Thanks for clarifying.

Barry @#31: I use "pan", but only from within my private network (so I can use "ssh -X" to reach it from another of my computers). I'm not completely happy with it, though -- it staggers on big message lists, and it can't killfile spam based on Message-ID.

Barry @#31: I use Agent Forte. I'm only using it on one computer, though, so I have no idea whether it would work in a situation where you're accessing the same groups from multiple computers.

I do know that I like it, though.

paul @ #32: The question of whether ownership of the only copy of something gives you more than permission to read it yourself or allow others to read it locally in your presence has been pretty well settled by various cases involving the letters of living authors.

That is true, but IMHO the established law for private letters doesn't really match the fact pattern for Usenet postings.

Posting to the Usenet medium inherently implies the request that it be copied electronically to as many participating machines as possible, with no control over how they may redistribute it or how long they may keep it. (If you don't assume that's implied by posting, then you're fundamentally asserting that Usenet should never have been allowed to work.) The limits and boundaries of that implied permission could be debated endlessly and indeed have been, but in my non-lawyer's opinion, its presence should be enough to make Usenet postings a very different case than private letters.

Building on your suggestion differently, though, I wonder if there's any copyright precedent involving reproduction of "open letters" published as advertisements or issued with a request to circulate them?

Clifton @40

Posting to the Usenet medium inherently implies the request that it be copied electronically to as many participating machines as possible, with no control over how they may redistribute it or how long they may keep it.

Or, in other words, UN-altered REPRODUCTION and DISSEMINATION of this IMPORTANT information is ENCOURAGED!

(I'm very sorry. Usenet nostalgia and all.)

"slightly evil" as in the Prince of Insufficient Light? (character in Dilbert)

Jules, #27, but sensible people don't post to Usenet via Google, so they would have no right to our words. They're just another server on the net.

Barry, #31, I haven't been on Usenet for some months, but I use Agent for news and email. Using two computers is a problem because the reader resides on the computer -- there's no synch command. You need something resident on a server if you want to use more than one computer.

Using Agent on two or more computers is easy: installl it on one of those neat 1 or 2 gig USB sticks.

The problem here is that USENET is more of a transport protocol with a culture than an application. It's a rather good implementation of a basic multicast content delivery system, but it doesn't inherently provide for either a nonshitty user interface or any persistence, archiving, linking etc.

I remember on SHWI someone asking what if USENET had become the WWW; I still think that wouldn't be such a bad idea if we could solve the globally-unique stable link problem.

Barry@31: I use trn on a UNIX system at UC Berkeley. Whether I'm at home or travelling, the computer I'm typing on is essentially a terminal over the net, so different computers isn't an issue for me so long as I have net access.

I use Thunderbird on Windows XP.

Martin @ 44 -- I was going to suggest exactly the same thing.

There are lists of free Usenet servers which are ongoing concerns out there. (Google them, perhaps?) I found my current one, a Polish server, using one. I used to use the German cis.dfn.de, but my German has since deteriorated sufficently that I can no longer work out how to set it up.

Rob Kerr @ #47: Thunderbird for Usenet access isn't bad, except that it really doesn't lend itself to filtering out the trolls easily. I used to use it, but switched to Agent when alt.callahans was overrun for a while with trolls and spammers (it's since gotten somewhat better, but I still love the ability to killfile the garbage posters).

Martin, #44, and Rob, #47, now I feel stupid. But you know why I didn't think of that? My laptop is too old for USB. When I traveled, I used to set both the laptop and desktop up and clear the laptop to-date and then read on the desktop. When I pulled news the next day on the laptop, I would have missed only a message or two. Then I had to reverse that when I came home. Now the laptop's version of Agent is too old to use for email so I'm just keeping it as an emergency machine.

I'd like to get an Eee and use it both as desktop and laptop and then I wouldn't have to worry about news & email when I travel. Of course, I'd like money to fall from the sky, too.

Marilee@ 49: What's an Eee, when it's not the sound my partner makes when she spies a mouse? (And I am not kidding about her sound, either. You should hear her. The best one was a "HeyEEK!" and I've never let her forget it.)

Anyway, I digress. What's an Eee and why do you prefer it? I thought all the computer mavens preferred iBooks and such?

Ginger, Marilee means the Eee PC. I haven't actually seen one of them in real life yet (I guess I should get out more), but people who like them usually say that they're cheap compared to other notebooks (especially Apple-made ones), that they take up less space, and sometimes that their software is less bloated (that is, they don't have that many "bells and whistles" that do little aside from slowing down the computer).

Ginger, #50, Raphael is right -- it's a subnotebook and for about $525 has lots more memory and space than my nine-year-old desktop. I'm actually thinking of the Linux version with a solid-state drive. I know a lot of people who have them, and the only complaints I've heard are that you need to put a better version of Linux on, and some people's fingers are too big. I would probably need some help with the first, but there's a Linux Users Group here, and for the second, I'm sure my fingers are small enough. I'd have to get used to having the trackpad in front of the keyboard, even though I'd probably plug my trackball in for actual use.

Oh, and from a closed journal, I here reprint "The Depths of Evil":

Google, the not-so-evil empire,
Amazon, the medium-evil empire,
Microsoft, the evil empire,
Autodesk, a more evil empire,
Exxon-Mobile the eeeeeevil empire,
Haliburton, the spawn of hell

Marilee@ 52 (and Randolph @51): Wow! That's a useful little machine. I just might have to get one for my son. He's in 7th grade, and has dysgraphia, so they issued him an AlphaSmart. It might make more sense to get him this, and he can do his homework in our wireless networked house, then email it to his teachers.

Ginger, most people are using them for writing in cafes and things like that, but it's so much better than my current desktop that, if I can get it, I'll hook all the desktop stuff to a powered USB hub and just plug that into the Eee. When I want to take it somewhere else, I'll unplug. That way, at home I'll still have the big monitor, keyboard, mouse, printer, room for stickdrives, and probably the old computer as another drive. The Eee doesn't come with a CD drive, so I'd have to spend some time ripping my CDs on the old drive, which will run faster without Quicken and other space eaters on it.

Getting back to Google for a moment, they've figured out how to track the progress of a flu epidemic faster than the CDC.

[Spam from 59.98.112.182]

Nice try at staying on topic, #57, but really.

Robert at 47: There are lists of free Usenet servers which are ongoing concerns out there. (Google them, perhaps?) I found my current one, a Polish server, using one. I used to use the German cis.dfn.de, but my German has since deteriorated sufficently that I can no longer work out how to set it up.

If you need any help translating anything from German to set that server up, I'd be glad to help.

While I find Google generally evil (only slightly more evil than, say, Microsoft, in that they deny it), my girlfriend says that she, like Usenet, can only be found if I bear cookies.

FWIW, the same day I posted @17 above, my friend at Google looked at this and said he figured it was an outright bug. He e-mailed me today to say it looks like it's been fixed.

My colleagues in usenet search tell me that this bug is fixed now. If you're still seeing it, email me at kevinmarks@google.com Sorry for the problems