Go to Making Light's front page.
Forward to next post: In Brooklyn, about a mile south of us
Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)
We’ll be implementing the long-awaited upgrade to Movable Type 4 tomorrow morning. As a result, for about three hours, commenting will be disabled. The front page and individual archive pages should all remain readable.
Assuming all goes well (knock wood, touch a belaying pin, throw a goat over one’s left shoulder, bury a statue of Giordano Bruno upside down) (the best superstitions are the ones you make up yourself), the no-commenting period should last about three hours. It’ll begin at 9 AM European summer time, which is 8 AM in the British Isles, 3 AM on the east coast of North America, and midnight on the west coast. The rest of you can work out the time differences for yourselves. Don’t forget the goat.
Update from Martin at 12:30 CET: This is not going as well as I had hoped. Commenting is still down; investigation ongoing.
Update from Patrick at 8:30 EDT: While Martin investigates, he’s reverted us to MT 3.33, and commenting has been re-enabled. We’ll try again when he’s reasonably sure it will work. Carry on…
I assume that it's a coincidence that you're doing this on Sunday the 14th, rather than Saturday the 13th?
Are you sure a goat will be enough? A bull might be safer.
Let's leave the goat for LJ, shall we?
A llama, on the other hand... :-)
janetl, "Flag Day" used to be slang in the software industry for a big change done all at once, after which users have to get used to the new system. Usually involves some downtime.
Whether it's a coincidence or not, it's fortuitous, isn't it? Writing this before any answer has been given, but it is kind of cool.
Patrick, I've often declared that rubbing a bald man's head is good luck. I may have gotten that from somewhere, but I promote it because I like it.
There's enough bull to go around. We just have to be careful that no one gets our goat.
Might I enquire as to whether the goat should be a Nubian or an Arabian? These things matter.
Xopher @4:
We always called those "Big Bang" implementations. But we're a tad late to do it on that day, no matter what else.
I could tell you why it's to be tomorrow, but it's much more fun to see everyone speculate, to be honest.
#6: and, what is the goat's air-speed velocity?
Sadly I only have the one statue of Giordano Bruno, so I have to retrieve it every time once the event has passed.
Fragano@6: A Nubian for hardware upgrades, and an Arabian for software upgrades.
For changes to the user interface, a sheep.
Xopher, "I've often declared that rubbing a bald man's head is good luck. I may have gotten that from somewhere. . ."
Um. That was GWB's gesture.
Not to say you appropriated it from him.
Debra Doyle #10: A black-belly sheep for customer support?
Or perhaps a rubber chicken? Spouse keeps one in his office, attached to a toilet plunger handle, and spins it round over his head when things get really dicey.
Xopher @ 4: I have heard quite a variety of different terms for a scheduled ship date, but I'd never heard "flag day" before.
As to good luck talismans, we have a piece of 2x4 sitting on the bookcase in the area where we have our daily standup. Whenever anyone says anything that might tempt the gods to punish us for hubris, we knock on it. It recently acquired a sparkly, pink unicorn sticker. Depending on your point of view, this may enhance it's effectiveness, or not.
janetl #15:
It's not a ship date, it's a change date. Usually involves heavy-duty maintenance and/or systems tweaking. Not what you're sending to customers, but rather what you're doing to yourself.
"Don't worry, the change will be transparent to the users...."
"Much like a helicopter's rotor blades."
Xopher @ 4... I've often declared that rubbing a bald man's head is good luck
It is a truth universally acknowledged that nothing but very good luck can ensue from rubbing the not-quite-bald head of a French-Canadian who sounds like Christophe Lambert.
Me fishing for head rubs?
Jamais!
joann @ 19... Non, rien de rien, non je ne regrette rien.
James McDonald @ 17... Did that one originate with you? No matter what, next time someone tells me "this is going to be totally transparent", I'll have to quote you.
I'm reminded of a recent outage notification which started with "unplanned" and included as the cause "unplanned" ;)
That said, may your goats be entertaining (perhaps fainting ones, and a few handy loud noises), and there be no need for the invocation of cowboys to resolve a rodeo situation...
Serge @ 11: Who's to say it won't be?
Wyman Cooke @ 23...
"So. Invisible killer drones."
"Yeah. Keeps getting better."
Linkmeister #12: Rubbing the head of bald guys was around long before Shrub! Remember Steve Austin on the Jerry Springer show? It's more likely derived from customs surrounding Buddha statue(tte)s.
Also, instead of Bruno, shouldn't you be using a statue of Charles Babbage? Or even Alan Turing?
:-)
Damn! I read Patrick's post on the front page, and wanting to do anything I could to help, I sprang into action. Now I see I read too quickly, and need to head outside and dig up Bruno Giordano!
I hope he's still breathing.
David Harmon @25: Rubbing the head of bald guys was around long before Shrub!
I understand there is a bronze plaque at MIT of George Eastman, with a well polished nose. It has been the tradition for students to rub his nose for luck before taking an exam, and the thousands of test-taking rational engineers have kept George's nose buffed to a shine.
abi: Y'all are doing it for my birthday. D'uh (which I share, with the flag, as mentioned and the Army).
Re statue rubbing. The nose of Rickover is bright for the saame reason. The statue of the Naval Acadamy has a region kept polished by sending plebes to caress it, and the nipples on the mermaids at the MGM (which were made by a friend of mine) have had to be repatinad several times, and repaired once.
At UCSB there was a bust of Storke which had a polished nose, for the same reason. It was in the UCen (essentially the student union, which at the time included the bookstore and university branch post office).
Fingers crossed that it will/has go/gone well.
Fragano, #6: Would that be an African or a European swallow?
Knock wood that it will be 3 hours and not 24-48 before ML is back on the intertubes. Kthnxbai!
Knock wood that it will be 3 hours and not 24-48 before ML is back on the intertubes. Kthnxbai!
Plz delete the duplicate. Kthnxbai!
Could "making up a new supersition yourself" be good luck?
There should have been an earth-shattering kaboom. Where was the earth-shattering kaboom?
More like a damp squib. We're looking into it; meanwhile, as my update said, carry on.
The next attempt may entail somewhat less notice.
Carnegie Mellon also has a plaque with a shiny nose... except I never saw a student rub it, and I *did* see maintenance staff polishing the thing on their late-night rounds. So, an artificially-maintained good-luck tradition, exempli gratia.
Andrew Plotkin @42:
An alternative explanation is that the nose is lucky, not for students, but for maintenance staff?
No matter why the shine, today's news were a good nose. My many thanks to Martin, Abi and everybody else.
Serge @ 44 ...
It's allergy season ... no nose would be good nose...
[but yes ... most excellent news!]
Guys,
The congratulations must be returned with thanks. Read Patrick's note again.
The upgrade failed; we fell back to the previous version. We're going to be investigating why and trying again at another time.
Abi... Oh. I guess I'm so used to panicky it-didn't-work!'s here at the office that it didn't register as a curses-foiled-again. Well, look at the congrats as an advance payment.
xeger @ 45... It's allergy season
It'd appear that the nose was something of a nostrum.
You man need to bury Giordano Bruno more thoroughly, and ensure he is facing North (True, not Magnetic).
A rubber chicken may also be needed in case of network issues.
Cadbury (Who has only ever seen goats recommended for SCSI problems)
abi #46: Well, a graceful fallback/recovery is a pretty good outcome!
abi @ 46 ...
The congratulations must be returned with thanks. Read Patrick's note again.
The upgrade failed; we fell back to the previous version. We're going to be investigating why and trying again at another time.
Not at all! You may not have succeeded with the upgrade, but the rollback -worked-! Given the number of things I've been involved in, where the backout plan boiled down to:
"Run around, scream and shout
Hope to god we sort things out"
... I'm definitely congratulatory! You tried the upgrade... it failed... but the contingency plans (a) existed, and (b) worked!
[paraphrased from a different event, just for an idea...]
00:00 "Hi! We're going to take the application down for maintenance. It should only take an hour, three at the worst."
02:00 "Uh, the maintenance is taking a bit longer than expected.
03:00 "We can't get the application going again"
04:00 "We think there's a problem with the database"
05:00 "There's definitely a problem with the database -- one of the tables is corrupted"
06:00 "We can't fix the table -- we need the offsite backups, and it'll take them 2 hours to get here"
08:00 "The application is still offline, and we're still waiting for the backup tapes"
09:00 "The tapes finally arrived, and we're starting to restore from backup"
10:00 "The restore is taking a long time. The next update will be in 4 hours"
14:00 "The restore is done, but we still can't get the database going. We're trying something else."
16:00 "We got the database going, but we're missing data for the last two days. We're going to try and import the missing data."
17:00 "Importing the missing data didn't go so well. We're going to stop doing that, and just bring the app up with the older data."
19:00 "The App is back up again, but missing 2.5 days of data"
19:30 "The App is down again."
20:00 "The App is up again."
21:00 "We're throttling connections to the app so it doesn't fall over again."
22:00 "We're tentatively saying the app is working for now, but we're all watching and crossing our fingers"
24:00 "The app is still running ... and we still need to import the missing data..."
[... and it went on for another day-or-so ...]
Xeger (52), I'd say "Gosh, you never told me you worked for formerlyemployedatsitewhichshallremainnameless," except you said the process only went on for another day or so.
Weeks, I tell you. Months. Ongoing. And on some issues, not far short of the next plural in that sequence.
Cadbury Moose #49: (Who has only ever seen goats recommended for SCSI problems)
Andrew* @#42: I always wanted to find a way to quickly tarnish metal, so I could tarnish the nose and polish the rest of the plaque.
*Very appropriate, that name
xeger @ 52...
"...we need the offsite backups..."
"What offsite backups?"
I was afraid of this. Standard technology advice: always double (at least) the number of goats the specs recommend.
#52
That sounds like the database compression they did a couple of months ago on our stuff. It was only supposed to be down for the weekend, but it turned into most of two weeks.
Now they just run the compression every weekend, and tell us we can't do anything between Friday evening and Tuesday morning, and we hope they're right about how long it will take. (It's a big database.)
#56: "Okay, what about the onsite backups?"
"They're handled by another department. You need to submit a formal request."
"How long will that take?"
"No idea. The person that used to do them was an engineering coop student, and he graduated last year. I don't think he's been replaced yet."
(true story)
CarrieS: For most metals a weak acid will do the job. For copper, brass, and bronze there are specific chemicals designed to patina statues.
So apply a resist to the parts you want to keep bright, apply the acid to the whole thing. Rinse, an when the effect has the affect you like, buff to an even tone; removing the resist in the process.
This does not bode well if I carry on with my plans to upgrade from MT 3.15 to MT 4. ML is a heckuva lot more complicated than my blog, though, so maybe I shouldn't read that much into it.
My old boss liked to say "Transparent means you don't see it until you hit it."
I think something has gone ascrew with the ad column over on the right. The first link is the title of the halal cafe post, and indeed feeds back to the post via a rather circuitous route.
I don't recall seeing this sort of thing before just now?
At WhereIWorkWhichShallRemainNameless we create and test the rollback for every upgrade. Someone will occasionally make noises about this being a waste of time for "small, low risk changes". The other people look at them with cynical expressions, and they pipe down.
I'm not a superstitious person, but somewhere down in my bone marrow, I believe that doing this is why we haven't had to do a rollback in a long time.
Joann #68 -- No, actually, the "Book Blogads Network" ad includes a running feed of headlines from blogs inside blogads.com's "Books" network, and we're one of those blogs.
Blogads.com is pushing their "Books" network with a bunch of initiatives, including asking member blogs to run the ad. Since the ads they get us pay for our server expenses, we said sure.
Patrick #70:
Thanks for the explanation. I had been afraid there had been dueling stylesheets leading to messed-up tables or some such.
Jon Meltzer #59 :
I was afraid of this. Standard technology advice: always double (at least) the number of goats the specs recommend.
... even when you take this rule into account.
Jon Meltzer @ 61 ...
"No idea. The person that used to do them was an engineering coop student, and he graduated last year. I don't think he's been replaced yet."
I'll trade you the person who got their backup tapes (and mine, and another coworkers, in both cases anxiously awaited, to restore a down service), and promptly put them into their tape jukebox and overwrote them [0]...
[0] Clearly the fact that they had more tapes returned than they were expecting meant that the tapes were spare/extra, not that they -might belong to somebody else- [1]
[1] Descriptives continue to fail me, but that wasn't the most unique thing that particular coworker did...
Carrie, #55: Liver of Sulfur.
janetl, #69: AKA "The difficulty for which one is prepared never occurs." It's not infallible, but you'd be surprised how often it appears to work.
Re statue rubbing: in Budapest there's a bonze statue of a Hussar general. The horse he's sitting on is a stallion and its scrotum is bright and shiny, unlike the rest of the statue, becaue the local students rub this for luck before exams...
"Bronze", not "bonze" - why do typos always become visible just after I've pressed the "Post" button?
dcb: I was working around that in ref the US Naval Acadamy.
The mascot is, "Billy the Goat". One of the allowed things to do to plebes is to say, "Billy the Goat's Balls", at which point they have to run to the goat, slap the scrotum, and run back.
dcb @76:
I'm convinced that the typos don't actually exist until the POST button is pressed.
Maybe in the upgrade they should rename the Post button to "Reveal Typos." THAT would sure steepen up the learning curve for commenting here!
Wait, we don't want to do that, do we? I got turned around somewhere...
Since Giordano Bruno didn't help... Has anyone thought of using Dick Giordano instead?
And Bruno Giordano didn't help either. But I'm happy to report he's still breathing, though somewhat dirty.
John Houghton @ 78: I like that theory.
Terry Karney @ 77: They have to climb a bit to reach the statue in Hungary - it's on a plinth. But it's nice to see that these rituals are spread far and wide...
In a cemetery in Paris is the tomb of Victor Noir, which it is considered lucky to rub. In part.
This is not the same as a head rub.
Re: backups/tests/having a plan B/other IT insurance methods and those who scoff at them...
Whenever one of our users gripes about such "wasted" efforts, I usually respond with my take on the Murphy/Finagle laws.
Long version: You're insufficiently paranoid; the universe will mess up your plans unless you do something about it, and that mess will probably be very painful to clean up. {Can be expanded as needed into an hour long lecture about Murphy, Finagle, statistics, failing gracefully v. failing spectacularly, learn from my mistakes, etc., etc. Sometimes the mere threat of such a lecture is sufficient to make folks behave.}
Short version: Which way do you want to be wrong (about your expectations)? {Because sooner or later you will be!}
dcb: Billy the Goat is on a lower plinth. I think it requires a step and jump to slap them.
Comments containing more than seven URLs will be held for approval. If you want to comment on a thread that's been closed, please post to the most recent "Open Thread" discussion.
You can subscribe (via RSS) to this particular comment thread. (If this option is baffling, here's a quick introduction.)
HTML Tags:
<strong>Strong</strong> = Strong
<em>Emphasized</em> = Emphasized
<a href="http://www.url.com">Linked text</a> = Linked text
Spelling reference:
Tolkien. Minuscule. Gandhi. Millennium. Delany. Embarrassment. Publishers Weekly. Occurrence. Asimov. Weird. Connoisseur. Accommodate. Hierarchy. Deity. Etiquette. Pharaoh. Teresa. Its. Macdonald. Nielsen Hayden. It's. Fluorosphere. Barack. More here.