Scripts, assuredly :) CSS? Not unless you drug me...

I can whip up a web page like other people play tetris, but only enough experience server-side to install a webserver and be acquainted with some of the standard server processes, and that was 10 years ago. If all the people with more of the server experience you need are too busy carrying their IT departments, I wouldn't mind taking a look at your set-up, establishing some important questions, and researching their answers. I'm currently being processed by a non-profit to start some part-time work next week, but I'm otherwise available. resume: http://www.thedevilsdictionary.com/resume.html

I have both PHP and CSS experience, but none with the MT framework. I would be happy to answer questions regarding either, if there is a problem you cannot solve or if you need some advice, but I do not feel qualified to do the upgrade myself.

Server side experience I have plenty of -- ask Ken :)

Why do you say "more probably to Movable Type 4"? Because you expect the migration to be easier from one MT to another, or are there other reasons to prefer MT?

Do you have approximate numbers for the site's traffic? Or do you bother quantifying such things?

Is the site running on a single server that handles both database and web serving duties?

Are there any other pages being hosted by this server or this codebase? For example, what's running the rest of nielsenhayden.com?

I don't have recent experience with MT or WP (my day job is Drupal) but helping out with this would be a fun way to learn.

If I were planning to port this site, I'd probably start by making up a list of the design and functional elements that you'd want to capture in the new site. Let me start off by listing: Posts in reverse chronological order. Comments that can be added by anonymous users (that is: users that don't have passwords). At least three page layouts that I've found so far: The home page, the individual thread pages, and the entertainingly named "Making Lighter". Subscribing via RSS to particular comment threads. An existing scheme to block comment spam (which needs to be explored in detail, since deploying something that doesn't work as well as the current system could be... bad). It's important to preserve all existing content at its existing URLs. Et cetera...

The big thing would be how the data is currently stored. If the data can be migrated the engine running it becomes less of an issue from that perspective.

What you do need to think about is what you can currently do, what has to stay, and what you could, if needed, lose or change. I am pretty familiar with WP, I'm actually working on a plugin for it. If you have an questions or need help please feel free to contact me.

WP w/ Askimet has some really nice spam filtering and protection and WP has a number of plugins around aimed at mass level administration.

Like Mike suggested I recommend you make a list. There are a lot of ways you could also tune things regarding the site. In addition to what he said I'd add the following:

1. What features you currently have that you need, like, and don't care about.
2. If there is anything you need that you'd like if possible.
3. How is your data currently stored. (This will dictate how easy it is to migrate things over).

I've not used MT in ages, I moved to WP and stayed there because I like it a bit more.

Hi... long time reader, infrequent commenter here. I'd be happy to do a migration to WordPress, if that's the route you choose, and am very open to an alternatively means of payment than my going rate.

Okay, I'm a better web designer than I am a sentence composer apparently. "Open to an alternative means of payment other than my going rate."

I desperately want to offer to do something helpful, but aside from knowing that what you want to do is eminently possible, and having (strong) opinions on what tech you should use to do it with*, I'm no help at all.

*wordpress, with the latest iteration of WP Super Cache plugin to make most pages load as if they are static, the akismet default plugin for spam, etc, and the import from MT "just works". Except, what you want is someone who is not you to iron out all the inevitable bugs when that setup doesn't work right.

To quickly answer a couple of questions:

I'm mildly more inclined to stay with Movable Type simply because lots of our existing templates are full of Movable Type's own markup tags. But I'm open to being convinced that this isn't a big problem, or that the merits of WordPress make the hassle worthwhile.

Making Light is hosted on a single server that handles both database and web-serving duties, yes. I believe we (which is to say, nielsenhayden.com, which also includes our home page and all the old Electrolite stuff) are the only thing on the box. The database is a big old ball of MySQL. The server OS is some variety of Linux, and we have ssh, sftp, cpanel, and phpMyAdmin access.

Several questions above deserve longer answers and I'll try to get back here later today, when I no longer have a stack of work-related work to deal with.

Upgrading to the latest Movable Type from earlier versions is supposed to be pretty painless. (We all know there will always be some problems.)

I'd offer to help upgrading MT, but I don't have the required experience. I'm not sure you want someone green and keen treating your site as a learning experience.

I do have experience with upgrading Movable Type (I've taken my own sites from 2 to 3 to 4) and I'd be happy to help. I'd think your best bet would be to at least try the upgrade to MT4 first and see if it helps the performance issues (it did for me), because if so, great, and if not, at least not much time would be wasted and then you can move on to looking at reimplementing the whole kettle of fish in WordPress or such. And it's pretty easy to test the upgrade on a sandbox version of the site.

Anyway, I'm contactable via the email on this comment. Cheers.

I'm a professional web programmer/PHP guy. I'd be happy to help in exchange for books. Sweet, delicious books. Contact me at the email attached to this comment.

I'd love to help, but have no relevant experience or knowledge.

I run a couple of WP blogs and we could probably handle MT too. Our firm specializes in systems integration but also does a lot of OSS/Web Design work. Both I and my main web programmer are big fans and would be glad to help. We have a big web design project brewing but could probably work it in over the next couple of weeks and we are already "in the zone." We'd gladly take glory as payment :D

#14 to #7; Xopher can still help indirectly, as may be able to provide Jeremy with an "alternative means of payment other than my going rate".

...no, I meant chocolates. Honestly, you people.

If I had the skills, I'd do it in exchange for free books.

Heya,

I'm willing to volunteer if you move to wordpress. I run everything through wordpress and I'm pretty handy at moving content into from other blogs.

I write html and css in text editors, and most sites I maintain are in wordpress whether they are blogs or not. I work with php and mysql. (your sever seems to have the right stats anyway).

Wordpress is clean, functional, free and modular. I'm pretty good at fixing code so it will work as a wordpress theme.

Why will I volunteer my time? Cause I like you people :D. I'm a web designer anyway and I love my job so much I do it in my spare time too.

ajay, you are EVIL. I like that in a person.

You totally got me with that. I was thinking "well, what makes you think Jeremy would...oh, chocolates."

I've done a bunch of wordpress migrations, but from things way more obscure than MT. I've got server & sysadmin experience, so I'll keep an eye open for things that need to happen and help where I can.

I have MT and WP experience, am a web developer, and would be happy to assist. I migrated Mr. G's site from an outdated CMS, a mass of plugins, and a heap of unused files at every level to an entirely new server, rational directory structure, and a custom CMS with various custom widgets. Compared to that, creating a copy of the MT database and Making Light (so as not to affect the site in the interim, and allow for multiple attempts if things go awry) and then attempting to upgrade to the latest MT and update all of the hacks and plugin-ins as necessary should be* relatively painless.

* It always "should be" and rarely "is", but never the less I'm willing.

I see a lot of tech people raving about a host called Squarespace.com. I dont know much about it, or hosts in general, but everyone says its easy to migrate your stuff to, and is great at allocating resources to prevent outages.

I've moved several blogs and sites from MT to wordpress...even my own in the past.

It's very easy imo.

So, Patrick...how are you going to choose from among the many offers of experienced help? This is great! All knowledge is contained in the Fluorosphere, indeed.

Hmm... depending on the timing Patrick's looking for, it could be entertaining to do a makinglight hackathon at worldcon...

I'd be glad to help; although I've no experience with MT I've been working on server-side apps for years, and have some MySQL experience. On the other hand, I have a lot of time available these days.

xeger @ 25... it could be entertaining to do a makinglight hackathon at worldcon...

And, since Abi would be telepresent, we'd get to see her react.

Serge @27:
And, since Abi would be telepresent, we'd get to see her react destroy everything we did.

Fixed it for you.

I speak html, ssh, & some css and would be delighted to help in return for the karma and the experience.

Put me on the list of people willing to contribute to the pool of valuable gifts to be sent in the direction of people generous enough to help out in this effort. I'm too busy to offer my time and limited expertise at the moment. (Day job is in one of its periodically non-linear states this month. That, and I stupidly joined a writing group, for which I have short story due in two weeks that I only just outlined.)

I offer assistance as a scape goat for if things go horribly wrong.

I'm really not the right person to make any technical changes, but I am really good at owning up to mistakes and taking the abuse for things going wrong.

So, whatever happens, it's my fault. I'm sorry, I just didn't expect things to go this way. I thought we had taken all the necessary precautions, but as you can see, it wasn't enough.

Abi @ 28... we'd get to see her react destroy everything we did.

And this is how we'd respond.

I've written a book about WordPress and have installed and maintained MT for a couple of big name companies. I'd be happy to chat with you, if you're interested.

I must admit I'm surprised to here you're suffering from so much downtime. MT pushes out static HTML files, which is one of its big strengths, so unless you have a wacky plugin working on every page (or lots and lots of traffic) your blog should be pretty stable.

Perhaps moving Abi from Scotland to den Nederland didn't increase the distance enough, and you actually need to add some super-special-high-class-very-expensive Abiveld shielding™©®℠℗.

I will now go and have my name officially changed to Jhn Hghtn, it'll make things easier.

John @34:

Ain't now power in the 'verse that can stop the Abiveld. Believe me, I've tried.

(Really...you think it only works when I'm trying to break things? Imagine my travel luck.)

Serge @32:
Yes, the facial expression at about 10 seconds is entirely familiar to me. My colleagues make it a lot.

Scott @33 (whose name sounds familiar) says:

your blog should be pretty stable.

Indeed. I assume the issue is either related to excessive comment traffic causing lots of page regenerations (particularly in light of the storms of comment spam referred to above) or it's a matter of lots and lots of static page loads. In which case MT might not be the bottleneck at all, and I believe it's time to go and google "reverse proxy cache" or "Apache tuning".

Shrug...not a fan of MT because over time it proves again and again to not be stable.

There can be lots of other reasons. Can't really do better than assume unless I look at the backend.

Abi @ 36...

"The shields are dropping!"
"Well, raise them."

The latter sounds like the kind of answer that the non-tech manager of our tech group would come up with.

If you're having problems with Moveable Type, I can't imagine that Wordpress would improve things. Is the problem flaky code or flaky server, though?

Has anyone here had experience with the current flavor of Wordpress and/or available mods on a high volume site? I mean, *really* high volume, like thousands of PV/hour or greater?

Unless they've improved the code a lot in the last few years, WP is not workable for high volume without some mods. It's a resource hog and I had issues with the database corrupting when when things got busy. Better code may be available now, I dunno. I was last working with WP about three years ago and threw my hands up eventually after it crashed and burned the umpteenth time.

For point of reference, when I had WP on the site, FFNews was getting 2500 pv/day or so. And I had to regularly fix it. Traffic was spikey and whenever we got a spike it went boom. This was in 2006, though, so the code might be better now.

(Firefox News runs on Article Live. I am not sure that would be appropriate for Making Light, but it's fairly stable unless we, say, get hit with xx,xxx pv/hour. The databases themselves rarely corrupt, but sometimes the server goes boom.)

@40

Like I said I use Wordpress in all my development. My most volume heavy site wordpress holds very well (they improve the code all the time.) We get about 1000 unique hits day hits a day much more with page reloads and such.

Three years ago is a long time in WP development. I'm using the most updated version right now.

Either Way THERE IS NO WAY to tell without mucking in the code.

It seems that most comment spam shows up on older threads. Would it help things to simply close threads to further commenting after a certain amount of time?

@ 42 Erik

I'm a Captcha fan as far as that goes but good comment spam filters would help.

No MT or WP experience, but I would be quite willing to help with Apache web server admin or with any shell or Perl hackery required. I used to be pretty BSD-centric, but I do a fair bit of Linux these days; I've even done some Apache setup on Windows Server, which is pretty much the same.

I've done a modest amount of performance-intensive webserver work, though not blog-type content.

Erik @42:
It seems that most comment spam shows up on older threads.

Actually, when you look at the size of the site, that's not the case. We have over 2500 entries on the database, but a remarkably large portion of the spam hits threads a few months old or younger.

Would it help things to simply close threads to further commenting after a certain amount of time?

It might. That's the call of the real NH's, not the small rag-tags that dangle at their tails.

----
* Not to be confused with dangleberries

Patrick M. (#31): I think you could make a fair amount of money freelancing that service. I know there are days I'd be willing to pay for it....

I think the MT problem is that Making Light is a write heavy weblog, especially with spam. I'm seeing a few hundred non-spam comments in the last day, and that's not a particularly heavy day. Couple that with a bunch of regens on pages with high hundreds of comments and you have load. Add spam to that and I can see how it would knock over a server.

I sure Making Light is an outlier. Waaay out there. I'm seeing standard deviations whoosh by. And personally, I don't think I'd put it on wordpress unless I had a really stout server to put it on. And even then...

And holy page size batman, Open Thread 124 is 800k right now. And that's with (only) 744 comments. (and yslow reports that it's not gzipped, nor are e-tags enabled. I think that would be a prime thing to try. Especially with the staticness of the pages. Hmm, and the style sheets aren't set with expires tags.)

I'd be interested in seeing if it's apache falling over or if it's perl causing a storm with the DB. I would venture a guess that apache is using a bunch of memory for all the connections that it needs to support, and that's starving perl and mysql for memory, and then things thrash and fall over. Unless the perl loads are just too much. A bit of data might help here, but throwing ram at a server is rarely an expensive or bad idea.

this is just to say

i have read the exchange
in #16 and #19
which you probably intended
only as casual commentary

forgive me
this is why i come here
so cute
and so funny

(it is surprisingly hard to write without capitals when you're used to using them)

I guess since everyone thinks of "performance intensive" differently, I should qualify that: the runner position search functionality for the Honolulu Marathon website for race day. Implemented with mod_perl and a Berkeley DB and tested to around 20-30Mb/s of CGI query results (mostly text/html) without caching. I forget how many queries/minute or hour that translated to; the actual peak load was about 1/6 of that.

I'm seeing a few hundred non-spam comments in the last day, and that's not a particularly heavy day. Couple that with a bunch of regens on pages with high hundreds of comments and you have load. Add spam to that and I can see how it would knock over a server. [...] I sure Making Light is an outlier.

That load probably isn't all that different from, say John Scalzi's blog, and various others.

eric @ 47 ...
I think the MT problem is that Making Light is a write heavy weblog, especially with spam. I'm seeing a few hundred non-spam comments in the last day, and that's not a particularly heavy day. Couple that with a bunch of regens on pages with high hundreds of comments and you have load. Add spam to that and I can see how it would knock over a server.

A few hundred entries is really trivial from the standpoint of most apps that backend into databases. We're not even talking about an entry a minute...

And holy page size batman, Open Thread 124 is 800k right now. And that's with (only) 744 comments. (and yslow reports that it's not gzipped, nor are e-tags enabled. I think that would be a prime thing to try. Especially with the staticness of the pages. Hmm, and the style sheets aren't set with expires tags.)

Heh. It's always interesting to see what different people think of as 'large' and 'load'.

I'd be interested in seeing if it's apache falling over or if it's perl causing a storm with the DB. I would venture a guess that apache is using a bunch of memory for all the connections that it needs to support, and that's starving perl and mysql for memory, and then things thrash and fall over. Unless the perl loads are just too much. A bit of data might help here, but throwing ram at a server is rarely an expensive or bad idea.

I'd say that there's no real sanity to speculating wildly until there's more information available. We don't know if this is a shared or dedicated environment, if it's virtualized, what the specs of the environment are, which versions of what are being used (and given mysql, the type of storage engine being used), what add-ons and customizations are in play... if there are any environmental issues, like network latency that could result in odd surges of traffic combined with cache wait...

... or if you prefer, my current WAG is that something unhappy is going on somewhere, and it needs to be fixed or worked around :P

I speak HTML, CSS, WordPress templating, some PHP, and some mySQL. I am willing to help for the chance to see people who know more than I do do their thing!

Not really good with the migration, but in the midst of mucking about with Drupal, discovered two technologies that supposedly help spam blocking -

http://recaptcha.net/ - An apparently pretty secure version of the infamous captchas. Free.

http://mollom.com/ - Advanced website spam filtering tech. Has free & pay options.

Both work with WordPress, purportedly.

Patrick, I offered at the time, and offered again during election season -- and the offer is still open. I ported all the data into the current one, so porting it back out is a no-brainer.

Xeger -- If apache has 20 megs resident/process, and 20 processes, that's half a gig there. Remember, each apache child is tied up for the duration of the download, and an 800k download takes more than a second or two for a lot of the world. You need also need cache for mysql, and you need file system cache for apache to not hit the drive for every hit. If you're on a 1gig machine, you're in for a world of hurt. Or you switch to something that doesn't tie up that much memory per long running hit. (nginx, lighttpd, etc)

Then they get hit by a comment, which will regen the front page, the three last N comments pages, and the page for the comment. Which is 400k, 800k, and 1.8m, 200k for the front page, and something large for the article page. That's all going to take memory and thrash the cache. Now, think about doing 10 of those at a time from a spam cluster.

And once you're not all in memory, you're hosed.

Also, Scalzi doesn't run his own blog anymore, it's the wordpress.com guys. They've got _infrastructure_.

At one point, I ran a 10k hit per day phpbb installation on a 64meg virtual machine. There was normally a daily thread that would hit the better part of 1000 posts. It was really touch and go until I switched to lighttpd and fastcgi. Once I made that change, all was well, because the working set would fit well enough into the memory I had.

I forgot to mention that you can run Wordpress and Moveable Type locally on your windows machine using XAMPP*.

I use this setup for web development and testing backups to see if they are good.** I would suggest importing a ML backup into a vanilla install of the latest Moveable Type and Wordpress versions and just see what happens. Maybe nothing will break...?

*XAMPP = the Apache/MySQL/Perl/Php backend that wordpress runs on. The Mac version is called MAMPP, I think. XAMPP is now available as a portable app, so you could even run it off a usb drive if you wanted to.
**I would do better to install XAMPP on a virtual machine and work from that, but I'm too lazy to redo the work I've already got going.

IMHO, the way to keep threads open without incurring the database overhead would be to cache older comments and get them all in one go. If there's nothing new on a thread, it's one access, no database hits. You'd have to dig into the code to make it work, but ... it would be pretty easy.

Also, the other reason I should do this upgrade is that I'm well-known for taking on hopeless renovations of lovable old monsters. It's part and parcel.

If we only had a wheel barrow. Or a holocaust cloak.

(or even vmstat 10 from when the server is busy)

Michael @57:

I find myself picturing an IM conversation sometime in the near future:

PNH: Michael's been working on the site.
Abi: How's it looking?
PNH: Great, actually. He says he's sorted out the last of the memory leaks and...OH, SHIT, there's water pouring out of my computer!

(Yes, I'm enjoying your blog, please keep posting.)

@56

It's Mamp for Mac

For windows I prefer Wamp.

Now I could use a Namp.

(Sue me, I couldn't help myself).

WordPress is dead easy to stand up, and I believe there are some decent antispam plugins for it, as well. If I were doing this, I would get a WP instance running on a local machine, grab a copy of the site's database and try to use WP's import feature to slurp in the existing content. Seeing what fails will give you a sense of the scope of the work you're looking at... not to mention how well the formatting survives the transition.

I'm not a blog management expert, but I have run and designed several large internet service hosting infrastructures over the last decade.

Questions from the infrastructure point of view:

Do you intend to or need to change hosting providers?

Do you need to separate out database and web functionality onto separate servers?

Is it run now on a physical host at a provider, a virtual host dedicated to Making Light, a shared host with other services on it, or not sure?

What types of logs / performance statistics do you have for the load and system performance?

This will help bound the solutions space...

@#40 Leva,

Actually, Scalzi moved the Whatever to Wordpress several months ago. I would be willing to bet his traffic is similar to ML's... plus or minus one order of magnitude :-)

I have heard of way more problems with MT than WP, and no code-related problems with WP scaling.

I blog at a mystery writer’s site called Murderati.com. We (or more accurately, our long-suffering leader JT Ellison) recently moved to a host called SquareSpace and it was quite painless. Don't know how she did it, but the host might be worth looking into.

@61 you can import to wp without touching the existing database (though you should always back up before non-touch or touch).

Scalzi on Wordpress vs Movable Type
http://whatever.scalzi.com/2008/03/15/wordpress-and-movable-type/

He uses a local installation NOT wordpress.com

I keep being surprised how many of the genuine talk.bizarre oldbies pop up here from time to time.

George Herbert's suggestions for what data to collect are wise, particularly taken with xeger's point about not theorizing in advance of the data.

"He uses a local installation NOT wordpress.com"

Michelle:

"He uses a local installation NOT wordpress.com"

Not anymore. I DID use a local installation but eventually began having significant load issues, at which point I was invited by the folks at Wordpress.com to join their VIP program, which happened in October 2008.

It should be noted the load issues were an artifact of hosting the blog/database on a shared server; I was considering getting a dedicated server, but the Wordpress VIP program offered some significant advantages for me.

@ Scalzi

I stand corrected then!

Thanks for the note as well.

A while ago I updated my own website to MT4 so I could implement a comment-spam plug-in, one of those "captcha" thingies. I discovered that MT4 is nothing like MT3--it's all widgets and what-have-you. After considerable frustration I did get it working and figure out just a few things, so I might be able to help a bit, although my own website is a "factory" theme, and fairly simple. I'm like you, though--I figure it out, remember just long enough to get it working, then forget everything.

I'll help where I can; I've gotten pretty familiar with WP (check out the three blogs, particularly www.68caliber.com - all WP) and would recommend it as the platform is supported by thousands - many of whom are more than happy to share their technical expertise.

But I'll be happy to do whatever I can to help, even if it means sorting through lists of data or other drudge work.

Abi, I hardly ever have water pouring out anywhere now. That was yesterday.

My real goal, though, is to get the entire basement into memory at once. I'm convinced it would run much faster. In 1923, a guy went down there and stayed the night, and when he came out, twenty years had passed. It pages something awful. And of course, it's even worse now, because I really don't think anybody ever reindexed it. I've upped the swap space, and I can at least go down there to do plumbing -- but it still always takes far longer than I anticipated, and my cell phone drops off the network because it can't synch with the protocol. Too slow.

mary @ #69, you've just explained why I have not gone from MT 3.15 to MT 4. I like the templates I have and don't see why I should have to learn such a single-function-specific set of protocols just to keep up-to-date. The comment spam is going to force me to do it sooner or later, I suspect.

If you're doing a big move anyway, I definitely recommend WordPress. I moved from MT to WP a little while ago, but unfortunately MT had become so buggy I wasn't successfully able to export all my posts (and I don't have the time for blogging to write my own app to do it). WP has certainly been a lot easier to use and has had less problem. I did get shut down the first time I had a big traffic influx, but a widget to cache pages solved that and I haven't had the problem again.

Christopher Davis @46

If you paid me, then people would assume I deliberately screwed everything up. Really, I try my best to make it better and solve the problem, but then you know what happened...

It rained.

It's my fault. You might as well just blame me. I'll try to keep it from raining next time, though.

I'm teaching myself Drupal, which I think is pretty cool, but would prefer not to dump that learning curve on people I actually liked.

ajay, #16, I was thinking I could offer jewelry as alternate payment, if wanted.

I am willing to knit or crochet something(s) small as a contribution to alternate payment. Wristwarmers, or a small stuffed animal. I could also monogram something, or send over some genuine Buffalo hotsauce.

FWIW, I'm pretty sure that "about an order of magnitude larger" is a roughly-correct description of young Mr Scalzi's traffic compared to ours.

Well then. You are both outliers.

And also, speaking from experience, going from something old and hacked together to something that's relatively standard and supported is a big win. Otherwise you're just putting off the inevitable pain. (which is sometimes a perfectly valid strategy, if the pain you're putting off is a lot larger than the effort it takes to put in another couple sticks of ram)

I was led here by a plea for help on my site....

I can't offer tons of time, but I can tell you from experience that wordpress scales very well to large users.

For instance, although it is not on just one server, the popular site PerezHilton is a wordpress site and I believe that gets literally millions of hits a day.

I own my own server, with about 75 sites on it. I have one in particular that is wordpress that gets between 15-20K unique hits a day and it doesn't hardly tick the resources at all.

Speaking of Akismet, one datum I'd like to throw in is that Movable Type supports Akismet (look for the MT-Akismet plugin). I'm on 3.33 and MT-Akismet is blocking over 99% of the incoming spam; I'd be astonished if MT 4.x doesn't also support this plugin. Highly recommended.

Michael Roberts @ 71 ...
My real goal, though, is to get the entire basement into memory at once. I'm convinced it would run much faster. In 1923, a guy went down there and stayed the night, and when he came out, twenty years had passed. It pages something awful. And of course, it's even worse now, because I really don't think anybody ever reindexed it. I've upped the swap space, and I can at least go down there to do plumbing -- but it still always takes far longer than I anticipated, and my cell phone drops off the network because it can't synch with the protocol. Too slow.

You owe me new nose plumbing ... not a new eyboard, since the old keybod may be a tad slow and gummed up, but it's reliable enough, and I harly eer mis leers...

I just mailed a submission package to Tom Doherty Associates, LLC. If I help, will it be considered a bribe?

I'm not a programmer, but I setup a custom Wordpress Blog on my http://1and1.com/ account (about $5 a month.) What made it easy was that MySQL was a 1-click setup. From there it was only a matter of customizing two entries on the WordPress configuration file. Only a few years ago I would have been sweating blood trying to do this.

I have no idea what would happen under large amounts of traffic, but 1and1 does not (I believe) have extra charges for heavy traffic.

Sounds as if you have more professional IT help, but if you want easy and cheap, I might be able to help.

Here's the link to my blog: http://unselfishgene.com/blog/

Robert burns

I see two important issues here -- your current MT infrastructure is highly customized, out of date, and hard to support; and you sometimes have reliability problems that you think are load-related (you're probably right).

To solve the performance and reliability problems, it's vital to understand them. It's very common in site development in general for people to spend a lot of time, effort, and money fixing the wrong thing, because they didn't analyze the situation properly first. So count me as one extremely loud vote supporting the people asking for specific information and pushing to diagnose the problems better before proceeding with the solution. Reverse-proxy or caching or a different http server or some combination might well be the solution there.

I'm happily running WP, but in such a different environment that I have no clue how it would work for you. And I don't know MT at all. So I'm not your conversion guy (luckily it sounds like several good candidates have already raised their hands).

Michael Roberts: If you'd reindexed your basement, it would be easier to find the pipe you're looking for.

eric @ 86 ...
Only if he selected the right key...

Actually, you know what I think you need most? (And I hope you will listen to me, Patrick.)

You need to pick a project manager. The situation here is that you have lots of people who are willing to volunteer effort, money, or work for money, with widely divergent skill sets. You have some people volunteering who have genuinely extensive and specialized skills, some at a national level, and probably some who vastly overrate their own skills. (Probably you will have more volunteers than the actual optimal-sized work team.) You need to find and select one person with good people skills, and with some technical knowledge but no technical axe to grind, who can sort out the real experts, make a plan which gets them together to decide what the real problem is and what should be done, then pull together the right group of people to implement that plan whatever it turns out to be.

I could throw out ideas about the problem right and left, based on my own experience, and they might well be very good guesses as to the problem, but they'd be guesses. Ditto for solutions - I hear a problem description and I start wanting to implement an optimal solution for it, but any solution at this point would be a wild guess. You need to get a fact-based plan pulled together, and for that you need a fact-oriented person in charge of it.

Charlie Stross @ #82, MT-Akismet sounds wonderful. But then, from its manual page under requirements: "MT-Akismet requires Movable Type 3.2 or higher."

See? I'm at 3.15. I know I should upgrade, but it all works right now. I get 1/10,000 the comments that our hosts or Mr. Scalzi or Mr. Stross get, and even the comment spammers are in waves that have large troughs in between, so the incentive just hasn't gotten large enough yet.

If you haven't already made the decision, I have migrated several large sites from MT 2.63 or MT 3.x to MT 4.x. And am willing to help.

I really like MT 4.25 and find that the new commenting system (as of the last year) is very good at taking care of spam. There are a number of other features in the new MT that make it great for communities.

Ms. Jen @ #90, I may ask you for assistance. I've hesitated moving to 4.x because of the template changes required/recommended.

@58 eric wrote "If we only had a wheel barrow. Or a holocaust cloak. (or even vmstat 10 from when the server is busy)"

vmstat doesn't always help :(

We've got a web server or two at work that are running like slugs with a migraine. So we moved them to shiny new Solaris boxes with 16Gb of real memory (count it!) and lots of multiple CPUs (and even more virtual ones). Or rather I moved them and its all my fault.

And the buggers slowed down. vmstat is fine. And I can look at all the vmstat I want because I installed the servers. I unpacked them out of the boxes and put them on the racks and installed the OS and installed apache & MySQL and basically they are my babies. But it does no good.

Sometimes, in a really busy period, one of the many CPUs gets up to 10% busy over almost a whole minute - though never much longer. And there is no paging at all and the memory is hardly touched except for about half an hour a day. And that's usually in the wee small hours when its rebuilding indexes. And disk activity is minimal (in fact its below the threshold at which disks perform better when they get more load - that used to be important back in the early 1980s, sometimes you could improve database performance by generating more work) and the network is uncongested (& there are up to six network cards we could use if it wasn't - its a chunky server)

I blame python myself. I think it sees all those CPUs and, like a second-rate footballer who has been passed the ball unexpectedly, looks round to see what to do with it. (a first-rate player does the right thing instantly - a third-rate player does the wrong thing instantly - Manchester United were second-rate on Wednesday...)

Ken, if the python stuff is one single python instance trying to multithread, then I'd expect exactly what you're seeing. Python's interpreter is largely single-threaded: most things grab the dreaded Global Interpreter Lock, so anything CPU-bound is going to be CPU-bound on one thread only.

Python is a lovely language, but its interpreter is intentionally really simple, and this is one of the places where that shows.

I have almost no experience of MT or WP, but I have a fair bit of experience of herding unix boxen (and not having any prior experience didn't stop me from merging multiple separate vBulletin databases into one and have it working afterwards, but it took me a few days of poking about at off-line backups before it was seamless), so if you want some poking about and general "look at what MAY be causing the performance problems", I'm game.

Clifton Royston @ 88 ...
You need to pick a project manager. [ ... snip ... ] You need to find and select one person with good people skills, and with some technical knowledge but no technical axe to grind, who can sort out the real experts, make a plan which gets them together to decide what the real problem is and what should be done, then pull together the right group of people to implement that plan whatever it turns out to be.

Absolutely, and in spades! [0]

[0] For all my and other folks various skills, a good project manager is worth all of our weight put together in the highly valuable substance of your choice, to put it mildly!

Eric @86 - true! I pulled a lot of old pipes out from under the kitchen floor, and it did take me an hour to find the right one. But the house was built in 1890 and never reindexed. I shudder to think of how long a full reindex would tie up core; I really need that basement to keep running. Without the foundation API, how would the house execute at all?

But the basement's insufficient clock speed is the least of my worries. Garbage collection in the carriage house is today's task. It's a real mess, because the core code is all some ungodly assembler written in 1920 along with the rest of the structure. I don't think they even had garbage collection at that point -- that code was written in some deviant FORTRAN/COBOL mixture hastily bolted on sometime during the 40's after the advent of higher-level languages -- I admit, that did make masonry a lot cheaper -- but the interface leaks. That's really hard on brickwork. I mean, I can comprehend tuck pointing in Perl -- but FORTRAN? It's going to take me a while to really grok the situation over there.

I wouldn't bother with mark and sweep garbage collection. Just toss the whole damn heap in the dumpster, preferably from a great height.

There may be small rodent daemon processes that complain, but they weren't much help anyway.

eric @ 97 ...
Maybe Michael Roberts needs a better mouse driver...

I have been referred to by some clients and employers as a very good project manager. I'd be willing to undertake the coordination on this if asked.

A lot of what a PM does is maintain a framework that everyone else can check when they need something, as well as oiling various gears and wheels, and asking hard questions.

Please note that I work full time (West Coast USA), but if we moved forward with this I would definitely give it good attention. Ideally, though, the PM would be someone in the same TZ as the primary site caretakers, or even local, for ease of coordination.

Remote PM is routine nowadays (my work team is based in Mountain View, SF, Seattle, Singapore, and the Netherlands) but it's SO much easier if the project sponsor and the PM share an office or TZ.

Oh, and glory only, I haz day job.

Earl @75: Ditto the Drupal coolness. And double ditto the learning curve... I really think the NHs don't need to go there from where they are.

Would volunteer to help but have no MT or Wordpress experience. Have been accounted a d*mn fine project manager, but have insufficient experience of managing a wholly distributed, volunteer team (I just know my whip wouldn't reach far enough).

Could offer cakes as incentive, but don't think they'd travel well from the UK to the likely location of most volunteers.

Earl @75, arkessian @100; thirded on the Drupal coolness and steep learning curve.

And yes to the project manager - good ones can make all the difference. Bad ones, no matter how well meaning they are, are as much hindrance as help.

Hi! Here's some opinions!

0. I agree with Clifton Royston #88: make sure you have someone who understands all the requirements and all of the plan, whether or not they actually do the work.

1. I agree with David Dyer-Bennet #85: determine the actual problem. In particular, moving to completely different software is highly likely to be troublesome, and possibly silently mangle old posts. Find out what the real problem is, and see if it can be straightforwardly fixed.

2. If you move to other software:

2a. Do not use threaded comments, unless the default view can be linear; I believe thread trees would change the character of the conversation for the worse.

2b. It would be really nice if the replacement supported deleting (due to spam) and inserting (due to moderation approval) comments without changing the numbering of comments.

3. I have heard people complaining about WordPress's post input format (for purposes like posting source code, which admittedly is not what you usually do here), in that it has auto-formatting that doesn't do what they actually want and can't be turned off.

Kevin Reid @ #102 re: threaded comments

Hush your mouth! Don't even mention them, lest they infiltrate!

Blech to the whole concept, at least for this community.

imo any thing can be moved given the tools.

Whether you choose mt or wp upgrading is best either way

What Linkmeister said.

I can't help with any of the building (save to say that getting photography site up with Drupal was painless. I was, however, creating ab initio not moving), but for the look/feel aspect I won't mind a bunch of changes, save that.

Threaded comments is bad, IMO.

ML folks: I've recently ported a few MT installs to WP (most notably PeterDavid.net, which gets a hefty amount of traffic and abuse); it is indeed pretty painless and I'd be happy to help. Most of the functionality of your current site is keepable right out of the box, and I suspect the rest can be had with a bit of searching for the right plugins. You have my numbers and email.

Kevin @102

You mention an issue with Wordpress comment formatting.

Here, we don't worry about source code, but what about poetry?

Hey, I like threaded comments! That was one of the best things about Usenet, and I've missed them ever since. It's also one of the reasons I generally prefer LJ to other blog formats. Here, I adapt (by using reference numbers), but I'd like to know if there's any argument against threaded comments that boils down to more than "it would be Different, and Change is Bad".

Lee @ 108:

I think the general argument is that threaded comments can work well in certain circumstances, but not here. In more free-wheeling conversations, threads tend to get split, untied, woven, tangled, and knotted as the situation warrants. Threaded comment systems allow for divergence, but not convergence.

Lee @108:
The pattern of ML discussion includes a lot of crossing of subthreads. By the end of a thread (in ML terms), there are usually enough parallel themes running that people start finding synergies, echoes, and contrasts. Threaded conversation kills that, because you have to decide which conversation you're in, which element has primacy.

It's also useful to read thread comments in time order. Reading multiple parallel threaded conversations loses the time element, in that one reads later comments in an earlier thread first. Reading an ML thread in conversational order means you get to watch a mood develop. Indeed, I sometimes go to the back end and read all the site comments in chronological order, because conversations, relationships, and losses of temper spill between browser tabs all too easily.

Basically, the tone and timbre of this community grew in an environment without threads. I think that threading would change the community, and yes, it's a change I don't think we need or necessarily want. (This is not to say that the community should not, or will not, change. But I think we're more comfortable with those that are created by the ways that interactions evolve over time, rather than imposed by a different software platform.)

You could say "it would be Different, and Change is Bad". But I think the subtext there is a bit unfair to the reality of the situation.

----
* I'm of the belief that there is a shorter-term version of zeigeist that grips the internet from time to time, when suddenly everyone is talking about the same stuff. I love watching it emerge in conversations here.

Abi @ 110... Threading is fine for LJs, where there are (usually) few people commenting. When many people comment though, it becomes a mess. In other words, I agree with you about ML.

Lee @ 108:

The other aspect, which I realized after I'd gone, is a technological one. abi covered the sociological side in more detail than I gave.

Any newsreader worth its salt knows which posts you haven't read and which posts you have. Blogging software doesn't. In a linear comment system, you just find the last post that you remember reading and move forward from there. Contrast this to something like a high-traffic LiveJournal community or (egad!) Slashdot, where you have dozens of new posts spread out all over the place.

Please don't take this as a pile-on, because I don't mean it to be. I used to think that threaded comments were superior. I still think they are in certain contexts, but not here.

Please note that I'm not arguing in favor of changing to threaded commenting here. I was a little uncertain whether the (rather vehement) objection to it was specific or general; it appears that it was specific, and that people have well-thought-out arguments about why threading would be inferior in this community, and that answers my question. (And no, I don't feel piled-on -- I asked for information, and I got it, so I'm happy.)

alt.callahans had 2 conventions for dealing with the thread-drift issue. One was header changes, which wouldn't work here. The other was the (XthreadX) notation, which people would insert into a post which referenced something going on in another thread. But both of those were community-specific, and without them, I agree that a lot of synergy would have been lost.

Lee @113:
And no, I don't feel piled-on -- I asked for information, and I got it, so I'm happy.

Happy is good.

(This is not actually a message brought to you by The Committee for the Propagation of Boiled-Down Truisms and Niceties, no matter how much it does sound like it.)

abi gave my reasons for my knee-jerk reaction to threaded comments, far better and with more thought than I've ever formulated.

Why, you'd never know about dinosaur sodomy if it were mentioned in a different thread than the one you were perusing! Is that right? No, your honor!

KeithS @112: I agree; and furthermore I like what Making Light's software does better than what Scalzi's blog does. My browser remembers the URL of the last comment I clicked on and changes its color to "this url has been read"; on Whatever the last comment I clicked on doesn't look any different after I've clicked on it. I like being able to click on the sidebar, scroll up to the last differently-colored date, and read down from there, without having to actually read my way up.

Agreed on needing a project manager.

One possibly-cool feature for the new home would be quick-link buttons for comment references -- say, you click on a "reply" button in the comment header, and that inserts a link like "Cally Soukup #116:" in the comment box.

Originally posted in what may or not be the wrong thread, but:

If your very old MT install is screwed up, are you actually saying that your dinosaur has been sodomized?

Agree on the importance of clear requirements and a project manager.

Haven't mentioned the need for good QA, because you've got Abi. Should she want some less-skilled assistance, I volunteer.

Also, someone on the open thread suggested the ability to edit posts (after suffering a typical ohnosecond ;-) ). Definitely needs a time limit, though -- perhaps half an hour or so.

David Harmon, 120,
Also, someone on the open thread suggested the ability to edit posts (after suffering a typical ohnosecond ;-) ). Definitely needs a time limit, though -- perhaps half an hour or so.

I think a 5-10sec "oh sodomy!" timer would be good enough. Hit post, and you see a spinner run. Mash the button to abort the post and drop back to the editor.

They also serve, who only stand and wait.

Jack Siolo #121: I get your idea here, but I think that's too little time. Not only will people just "wait out the hourglass", but for a lot of typos, the author really needs a few minutes before they can spot the error.

A comment punitively edited by a moderator (such as for disemvoweling) could be marked as locked and ineligible for editing. ML would have to support two-tiered member login accounts (or perhaps an OpenID interface) to prevent unauthorized editing by third parties of comments they didn't create.

#120
firedoglake and its sister-sites have something like a five-minute limit for editing after posting. It's good for the 'oh-no' situations. (You have to hit 'save' to make the edit work.)

All@ #120-125: Do the cool kids not read their text after they hit "Preview"? Moderate snark here: RTFP.

Brenda Kalt (126): I carefully read the preview (and sometimes alter or altogether abandon the comment, because it sounds stupid), but I still sometimes see an error right after I hit Post.

Brenda Kalt #126: Do the cool kids not read their text after they hit "Preview"? Moderate snark here: RTFP.

Thank you so much for your thoughtful advice.

Why Brenda... it's so nice to read you don't suffer from having petty mistakes, such as missing comma, corrected for by your proofreading mind; just after you've written them.

Then again, if you were to be treat people with more charity, we might not have to work so hard to be charitable in return.

The comment itself was moderately offensive (as I suffer greatly from not being able to effectively proof my own work for a day, or so, after I write it, because of that very quirk. That quirk, of course is completely unrelated to noting a tone of typer problem, once something is public).

The Acronymized aside, was rude. It may have been meant as snark, it came across as smug, and insenstive; with a decided touch of holier than thou.

I don't read my text before I preview. I'm too cool.

Brenda, #126: *checks View All By* That level of nastiness is out of character for you. Are you okay?

Reading the FABULOUS preview (like RTFM) is no guarantee of catching obvious or subtle bits of information.

Similarly, claiming a particular tone for a post is neither defense nor guarantee that others will read the post thusly.

That aside, a quick perusal suggests that Brenda prefers the drop-and-run method of posting, so I'd suggest wandering back to things of interest, like the correct way to sort ones sock drawer... [0]

[0] Colour? Size? Texture? Season?

xeger, I handle that one by having identical socks. (It works for me better than 'preview' does. Usually.)

Brenda Kait @126: Do the cool kids not read their text after they hit "Preview"?

As experienced blog-commenters know, there's a class of typo that's invisible up until that brief span between when you hit the "Submit" (or "Post") button and when your browser reloads the page. They seem to have evolved Preview-resistance.

Its not just bloggers! There is a class of typo that is invisible until you see your writing in print on paper.

That's why publishers and newspapers used to emply copy editors. And why the proof copy was invented.

xeger @ 122

I sort by migratory predilection: pairs likely to disappear spontaneously as a pair in one group, socks which only disappear in the dryer in another, and unpaired socks in the front of the drawer where the mates can easily slither in for a reunion.

Ken Brown: I discovered that it's persistent too. I mentioned the amount of time because, back when I was writing newspaper copy, that was about how long it took me to be able to see my own copy as if someone else had written it (i.e. spot the silly mistakes).

To crossthread with the color discussion... I have a lot of socks that I have to carry into the bathroom to sort under the bright light over the sink, because some ranges of my color vision drop out under low lighting. I believe I've posted the ultimate sock-sorting solution here before.

If you need to sort sockpuppets, that's a different problem.

Thirding David Harmon #117 - I've been saying that for a long time: No threading has its merits, but with typical comment number in hundreds one badly needs a way to follow/refresh individual overlapping conversations where replies are separated by 10-20 other comments.

P J Evans (#133) Do you have hot'n'sweaty feet in summer or frigid feet in winter?

I s'pose your winter shoes could be a size larger so you could wear 2 or 3 sets of socks. Or your climate is relatively mild and invariant. Or your lifestyle keeps you insulated from your climate. Or you have natural/zen/whatever bodily control. <SFnal thoughts on insulation, temperature control, etc.>

Where's that darn cat? <sound of vacuum cleaner>

Jan Vaněk jr. @139: I think David Harmon @117 makes a helpful suggestion of a 'reply to' button (which I simulated here with a little tedious cut-and-paste and 'Copy Link Location'). You could at least follow a chain of reference backwards.

It would probably be a bigger problem to append posts with forward links; but I see various blog postings adding links for pages that linked to them.

Pardon, I am pretty much saying what you said...

Mez, southern California. Not much different from yours, barring the six-month offset for the change in hemisphere. In the summer getting off the train usually feels like stepping into an oven. When it's hot (by my definition, at least 35C) it's more like stepping into a blast furnace. And in a Santa Ana condition, you can almost hear the entire area dehydrating (I think the record low humidity is in single digits, something like 5%).

I'm wondering how the colony of bees under the small tree by the train station is going to do during the summer. (Literally under the tree: it's in a hole in the ground!)

P.J.: The bees will be fine. The ground is a wonderful insulator. One of the tricks to emergency desert survivial is to dig into the ground, or the bank of an arroyo; for the cooler air.

P J Evans @ 143 & Terry Karney @ 144:

Have either of you ever heard of the Forestiere Underground Gardens in Fresno? I've never been but have always wanted to go, simply because it's supposed to be such a wonderful example of how insulating the ground can truly be. Since Fresno tends to be bitterly cold in winter and blistering hot in summer, Forestiere decided to build his extensive house underground (it's supposed to be about 10,000 sq. feet!).

Looks like the work is being done right now, right? Kind of fun to watch it happening right in front of your eayes. (Allthough a bit weird.)

Rob Rusick #141: Yes, those things are two sides of the same coin.

Typically, I come across a comment starting "Joe Fan #(20 comments back)." To recall / find out what it responds to, I then have to scroll (or use the browser's Find function for the given number/name) to the earlier post, and after reading it find again where exactly I was so I can continue downthread.

Now, a script button to automagically add a link to the replied-to comment's anchor (only a few kind souls even more obsessive than I am bother with coding them manually) means that I can click at the link, getting there instantly, AND when I'm done I just use the browser's Back function, which returns me to the reply. No wandering around, LOTS of time (and mental clarity) saved.

this is all very interesting because I'm getting ready to set up a foodie/restaurant/recipe/cooking advice blog somewhere.

Blogspot is looking like a good starting place because it can support advertising there. I'm still unemployed but getting unemployment pay for a bit longer so every bit of additional money helps.

I'm also looking to start a "downsizing/decluttering/divorce" assistance service in the Kansas City, MO area, if anyone is interested you can contact me. I have been helping a friend sort his stuff out and it has been very interesting and fun.