Back to previous post: Explosion in Oslo

Go to Making Light's front page.

Forward to next post: International Bog Day

Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)

July 23, 2011

Software for Self-Publishers
Posted by Jim Macdonald at 09:43 PM *

I am by no means an expert, nor am I famous (nor rich) for (or from) self-publishing. But I have been doing it for a while, starting with a Xerox machine and a saddle stapler in 1978. I started POD publishing in 2005, and e-publishing for the various ebook readers starting in September of last year. I’m putting my backlist back into circulation, which is a perfectly dandy use of the technology.

I use a Windows machine, so the programs I’m going to mention are Windows programs. Mac, Linux, and other versions of the same or similar software exist.

The price for all of these items is free.

So: A list.

  1. Open Office (Productivity Suite) writes MS-Word compatible files and converts them to HTML
  2. LibreOffice (Productivity Suite) writes MS-Word compatible files and converts them to HTML
  3. Notepad++ Reads and manipulates HTML/CSS files
  4. AceHTML Reads and manipulates HTML/CSS files
  5. Calibre Designed to manage your e-book collection, but will convert books to epub from a variety of formats
  6. MobiPocket Creator Makes .prc files. (Note: Crashes if you have MSIE 9 installed.)
  7. Sigil Edits epub files
  8. GIMP (Gnu Image Manipulation Program) A Photoshop work-alike for creating covers
  9. Threepress Consulting Epub validator to check epub files.
  10. Epubcheck Epub validator to check epub files.

Of these, all you really need is something that will create MS-Word compatible .doc files, and something that will create cover images.

Handy things to have read:

Things that you probably won’t do:

  • Get rich (divide income by hours to see what I mean)

Still, it’s good for a laugh. As hobbies go it’s cheaper than photography, safer than skydiving, and takes up less room than model railroading.

Doesn’t self-publishing violate Yog’s Law? No. Self-publishing is a subcategory of commercial publishing. The publisher is still paying for everything. The author is still getting the income. That the publisher’s pocket and the author’s pocket are in the same pair of pants is interesting but unimportant.

Google

Comments on Software for Self-Publishers:
#1 ::: P J Evans ::: (view all by) ::: July 23, 2011, 10:46 PM:

I have OpenOffice and Calibre.

There are some MSOffice things that OpenOffice won't do, but if it's a really big file, I'd try OpenOffice first.

#2 ::: Evan ::: (view all by) ::: July 23, 2011, 10:52 PM:

Good stuff, Jim.

If you're a Python nerd (or even willing to dip your toes in the water a little) you can write in reStructuredText and run it through Sphinx, which produces EPUB. Both are open source & free.

#3 ::: Kevin Riggle ::: (view all by) ::: July 23, 2011, 10:54 PM:

What's your general workflow for converting a story to an electronic format, Mr. Macdonald?

#4 ::: Erik Nelson ::: (view all by) ::: July 23, 2011, 11:04 PM:

One of the few things that forces me to leave the wonderful world of open source is InDesign. If there were a decent open source page layout program I would hardly ever leave my Linux partition. Scribus does not quite cut it.

#5 ::: mjfgates ::: (view all by) ::: July 23, 2011, 11:16 PM:

It might be worth noting that LibreOffice is the next version of OpenOffice, or maybe that OpenOffice is the old version of Libre. Oracle somehow obtained ownership of the "OpenOffice" name, and pretty much stopped development, so the real contributors just made a new project with a new name and ran with it.

#6 ::: James D. Macdonald ::: (view all by) ::: July 23, 2011, 11:21 PM:

Page design pretty much doesn't happen with e-books. The size of the text, the typeface, the page-width -- all of those are determined by the reader, not by the publisher (as I understand it).

Okay, my workflow:

Create a cover with GIMP. Save two versions: One 500x800 for Smashwords and Lulu, one 800x1280 for Kindle.

Create original document with WordPerfect. (They'll take away my Reveal Codes command when they pry it from my cold, dead fingers.)

After creation and correction and proofreading and all: Save a copy in .rtf format.

The WordPerfect document gets headers and page numbers added, then formatted to a paperback book size and saved as a .pdf for publication via POD through Lulu.com (Price: free).

The .rtf version gets opened in Open Office, and saved in .doc format.

A second copy gets the standard Smashwords text block added to the copyright page, and saved in .doc format.

The first .doc file is converted to .prc with MobiPocket Creator.

Upload the .doc file to Smashwords using the smaller cover. Upload the .prc file to Kindle using the larger cover. Upload the .pdf to Lulu using the smaller cover.

And that's pretty much it.

#7 ::: Paula Helm Murray ::: (view all by) ::: July 23, 2011, 11:22 PM:

I agree with P J Evans. The One Thing that I lovea about MSWord is that it allows one to open just about any files. Even if the file is totally batty nonsense,

In my prior life I was a database manager for a tradeshow publisher. For whatever reason, a lot of people at the show management side who claimed to be their "database managers" were marketing people who couldn't find a clue with both hands and a flashlight about making databases. There were too many occasions where I tried to open a file that claimed to be an Excel file or some other such organized format, switched to Word and found out I had a stinking pile of unorganized ASCII text that would never parse as data no matter how hard one tried.

Without Word this process would have been much more painful. And with the same, 'you need to re-export your data or try again."

#8 ::: P J Evans ::: (view all by) ::: July 23, 2011, 11:37 PM:

7
At work, I have Word, Excel and Access 2003. Because I deal with databases. (I'm working in a GIS project. It's stretching everyone including our data-management software provider. There are maybe a half dozen people in the entire company who were around for the previous try, twenty years ago.) 25-megabyte Excel files, 500-megabyte Access files....

#9 ::: Christopher Wright ::: (view all by) ::: July 23, 2011, 11:40 PM:

I find that there's a big gap between the number of styles an epub editor will support and the number of styles an epub reader will display...

In both Sigil and Jutoh I created a test ebook (a few chapters long) and I noticed that in the editors the chapter headers were centered -- and in my pc ebook reader that displayed fine... but on my nook (rooted, running aldiko) not only were the headers aligned to the left, but all "extra" formatting (bold and italics) were also not displayed.

I know these readers can display centered text and italicised text because I've seen them in other e-books, but it seems that the styles that the epub editors are using aren't being read or parsed by the e-reader software on the tablet. It's a little bewildering at this point, but every tool takes a while to get used to...

#10 ::: thomas ::: (view all by) ::: July 24, 2011, 12:15 AM:

The size of the text, the typeface, the page-width -- all of those are determined by the reader, not by the publisher (as I understand it).

That doesn't seem to be completely true for the Kindle. Although I haven't seen any variation in typeface, there is some variation in text size. For example, Eric Frank Russell's Wasp, which I recently read on Jo Walton's recommendation, has a larger font than the usual (when set to the smallest possible iPhone size).

#11 ::: C. A. Bridges ::: (view all by) ::: July 24, 2011, 02:36 AM:

Some ebooks have defined fonts and sizes, some do not. If the font is defined, your ereader settings will not affect it.

Font control set by publisher pros: complete control of the look, more professional appearance, will look the same everywhere.
Cons: some fonts too small to be easily read on a smartphone such as an iPhone or Droid, some readers prefer larger text due to eyesight, wanting to read while working out, etc.

#12 ::: C. A. Bridges ::: (view all by) ::: July 24, 2011, 02:44 AM:

If you have an ePub ebook with fixed-sized fonts or odd margins, you can occasionally fix that with Calibre. Right click on the book in your Calibre library and select Tweak ePub. Then click on Explode ePub. You'll see a list of the various components of your ePub ebook. Look for one ending in .css and open it with Notepad, Notepadd++, etc. Save a copy of this file somewhere. In this one, look for any instance of "font-size: XX;" and delete those lines. Save the file, and click on Rebuild ePub.

Now you can View the ebook to see if you like the changes. If you have margin issues, do the same and look for the appropriate margin styles.

You can also edit ebook style sheets with Sigil.

#13 ::: James Moar ::: (view all by) ::: July 24, 2011, 06:26 AM:

"If the font is defined, your ereader settings will not affect it."

In the cases I've seen on the Kindle, the defined font size seems to be "minus one","plus one", "plus two" etc. of whatever I have the font size set as, since I can still change the size.

#14 ::: John Chu ::: (view all by) ::: July 24, 2011, 06:51 AM:

Well, page design can happen with ebooks even though the user controls typeface, font size and page size. Web page designers have the same sorts of constraints and the ePub format is essentially a ZIP file with some HTML and CSS files. (Yes, this is simplifying things a bit, but close enough.) This doesn't stop many of the ebooks I read from being poorly designed anyway.

It's still early days. The automated flows that create ePubs don't always do a good job. e.g., I have an ebook where they inserted a line break after the drop cap that opens each chapter. It's possible to force page breaks and, conversely, keep elements together but most automated flows won't know enough to do that.

Some of the stuff a book designer might want to do can't be done in an ebook yet. (Actually, not just book designers. I'm told that the Kindle can't display non-Latin characters. I guess Amazon isn't interested in the Chinese market then.) In any case, we're still at the point where a really good ebook has to be marked up and styled by hand. (Not that this matters for Smashwords since you have to give them a Word file.)

Responsive web design is all the rage these days. i.e., designing web pages that accommodates devices of all sizes. I hope that some of that savvy migrates into ebooks.

#15 ::: Christopher Wright ::: (view all by) ::: July 24, 2011, 07:41 AM:

So essentially it's early web browser wars all over again.

There's a program called Jutoh that allows you to create epub files and can use Amazon's KindleGen file to create a .mobi file from the same Jutoh project...

The project uses fairly basic formatting (in my opinion) but it does use some: Headers are centered,there are words in italics, and some paragraphs have padding between them.

The .mobi file didn't recognise all of those formats but it managed to convert 85% of them. The advantage is that one company controls how the reader displays, so if I can figure out how to customise that last 15% it'll work every time (Jutoh does allow you to use a single project and define formats that are applied only to .mobi and others only to .epub).

This makes .mobi the Internet Explorer of ebooks.

Over in .epub land there are a lot of different readers out there. I'm testing with the Nook reader, Aldiko (which is rather popular among Android tablet and phone users) and FBReader.

The Nook reader formats everything perfectly. No deviation whatsoever. By all appearances it's a 100% standards-compliant browser. I can find no analog to early 90s-era browsers, but this makes me nervous, because the 100% standards-compliant software that works beautifully is usually doomed.

Aldiko, on the other hand:

- won't use the styles I created for headers or for text (all font sizes and styles are pre-set and applied)
- won't display italic text
- won't recognise any kind of padding applied to paragraphs
- automatically fully justifies text, even to paragraphs that were deliberately left justified

FBReader has exactly the same limitations as Aldiko, but their custom display settings are completely different. So even though you're working with the same constraints you'll wind up with a completely different looking book.

So they are Mozilla on Windows and Mozilla on Linux. Early to mid 90s browser technology.

Meanwhile, I have read during my nosing around that the iPad reader will not recognise the center command and in order to get around it you have to put span tags inside the style tag that actually centers the text. I haven't been able to test ebooks on an iPad because I don't have one.

What this tells me is that the only way to make an ebook that can be universally read without issues is to make the layout as sparse as possible and accept that it's going to look different depending on what ereader the reader is using.

#16 ::: James D. Macdonald ::: (view all by) ::: July 24, 2011, 08:44 AM:

John Chu #14 (Actually, not just book designers. I'm told that the Kindle can't display non-Latin characters. I guess Amazon isn't interested in the Chinese market then.)

Not just the Chinese market. I was unable to use Romanian characters in my own "Philologos; or, A Murder in Bistriţa."

#17 ::: Randolph ::: (view all by) ::: July 24, 2011, 08:47 AM:

Far as I know you can include fonts in an ePub file, however licensing must be addressed, which means either use free fronts or DRM. The Google Web Fonts are free and some of them are not bad. Beware of missing characters, however!

#18 ::: James Moar ::: (view all by) ::: July 24, 2011, 08:52 AM:

Randolph@17:
"Beware of missing characters, however!"

Is there a Making Light free font which is missing all the vowels?

#19 ::: Randolph ::: (view all by) ::: July 24, 2011, 09:24 AM:

James Moar, #18: obviously, it would be named "Dsmvwld."

#20 ::: Debbie Notkin ::: (view all by) ::: July 24, 2011, 10:01 AM:

I dropped in to say that "the publisher pays for everything and the author gets the income" is amusing, but the version of it I use when I'm negotiating contracts on my day job (representing a large nonfiction publishing house is), "the publisher takes all the financial risk" and "the author is responsible for the content." The publisher takes a significant share of both the income and the profits, which is why self-publishing, if profitable at all, can be more beneficial to the author.

Then I saw John Chu (#14), so I also wanted to say "Don't assume that Amazon 'isn't interested in the Chinese market' unless you know what the obstacles are to ebooks in Chinese characters (simplified or complex?) and whether or not Amazon or someone selling to them is working on the problems." I don't know about that, but I do have a sense of how many good minds with real experience are working on the design issues. Those will be solved, and soon, but they're clearly nontrivial.

#21 ::: Larry ::: (view all by) ::: July 24, 2011, 10:32 AM:

On the mac I'd look into Scrivener, it's a really nice program and does epub export.

#22 ::: Patrick Connors ::: (view all by) ::: July 24, 2011, 10:45 AM:

#21, Larry:
And I've heard that Scrivener is developing a Windows version. I haven't seen it yet; it's not in public beta.

This will be good for me, since I swing both ways: Mac at home, Windows at work and on my tiny little travel netbook.

#23 ::: dcb ::: (view all by) ::: July 24, 2011, 11:04 AM:

Request for assistance: I'm still using a Psion 5mx (I still haven't found a modern machine to match it). However, the newer Mobi books can't be opened with the current Mobi reader I have on the Psion (even when I change them from .mobi to .prc).

Does anyone know whether a recent Mobi reader will work on the Psion 5mx and/or have a way of convering files from e.g. .doc, .mobi or .rtf into an older-style /prc file?

I'm wary of trying to load the newed Mobi software then finding out the Psion doesn't like it and I can't return to the old version (okay, I could do a backup just beforehand, and Restore afterwards if it messed up).

#24 ::: Paul Duncanson ::: (view all by) ::: July 24, 2011, 11:04 AM:

Larry @ 21 & Patrick Connors: I'll second the recommendation of Scrivener. The Windows version is available as a public beta. Download links and all the relevant disclaimers that come with beta software can be found here.

#25 ::: James D. Macdonald ::: (view all by) ::: July 24, 2011, 11:38 AM:

For readers that take multiple formats, you might consider Stanza.

There's also Adobe Digital Editions, for checking to see if the file turned out looking like you wanted it to look.

Firefox has an epub reader add-on.

#26 ::: grackle ::: (view all by) ::: July 24, 2011, 11:49 AM:

Patrick @ 22:
I have a windows version of Scrivener installed but I haven't used it yet. Lyx is another nicely designed document processor (also free).

#27 ::: James D. Macdonald ::: (view all by) ::: July 24, 2011, 12:17 PM:

One sorrow in my life is that I don't own all of the various e-book readers, so I have no idea what the actual reader-experience of someone who gets one of my books from one of the on-line e-bookstores is actually like.

#28 ::: Kip W ::: (view all by) ::: July 24, 2011, 12:22 PM:

Jim, there are simulators, like Adobe Digital Editions, Calibre, and a Kindle app. They don't give the exact experience, but then no two readers will show things the same way anyway.

I got in early and made some bucks converting books to .epub and Kindle formats, but the industrial guys are moving in fast with one-size-fits-all services that also promise to put your precious book onto every sales platform in existence for one low price. It's my guess that they don't give books much individual attention, but maybe I'm a sorehead.

#29 ::: James D. Macdonald ::: (view all by) ::: July 24, 2011, 12:35 PM:

Yeah, I have Adobe Digital Editions, and Calibre, and Kindle for PC.

Still, I suspect it isn't the same. And I wonder, is it possible that my Kindle stories are really set in Courier New, or is that some setting that I didn't know I had in this particular reader that's making 'em appear so? (I've been trying for Palatino Linotype, but ... maybe not?)

#30 ::: Christopher Wright ::: (view all by) ::: July 24, 2011, 12:43 PM:

Here's another thing that can throw a wrench in to reader experience...

... I was trying to puzzle out why some readers looked like they were stripping away some of the ebook formatting commands, and it turns out that they actually have a setting for stripping away ebook formatting commands. In Aldiko there is a checkbox called "Advanced formatting" that, when checked, overrides the "publisher formatting" with a set of defaults. When I unchecked it my test project rendered perfectly.

Aldiko leaves the override setting on by default... and since I've used Aldiko for months without ever noticing it, I might as well treat the reader as if the setting isn't even there, from a usability perspective.

FBReader doesn't even give you that option.

I wonder if ebooks are going to have to have their own version of the colophon that, instead of listing production notes, is a how-to guide on what you need to do to all the more popular readers to get it to display right.

#31 ::: Christopher Wright ::: (view all by) ::: July 24, 2011, 01:42 PM:

I have no idea if any of these revelations are at all useful to anyone here but since James MacDonald happened to post this at exactly the same time I was trying to to suss this out it's become too tempting not to post. Apologies if it's cluttering up the thread.

I've learned that some Ereaders don't actually support cascading style sheets. This is interesting, since the epub format is html driven and uses css for all major formatting decisions.

FBreader, for example,is a free ereader for Android. It parses html but not css, so it will detect h1, h2, h3, p, em, strong, etc tags and will render those but will not display css classes associated with it.

(It also defaults to displaying all text with the Droid Sans font, which is the default font on all android phones. This turns out to be a double-whammy because Droid Sans doesn't actually display italics, so if your epub uses italics in any way shape or form the reader has to deliberately switch to another font in order to correctly view that.)

This convinces me further that the best way to make sure an ebook can reach the widest audience possible is to use as little formatting as possible. This was something I had believed when I first started this project, but as it turns out my assumptions of what qualified as "very little formatting" was still more formatting than where it actually needs to be...

#32 ::: Larry ::: (view all by) ::: July 24, 2011, 01:43 PM:

Patrick@21: Did not know that, it's a good program so should be good for Windows users.

#33 ::: John Chu ::: (view all by) ::: July 24, 2011, 04:45 PM:

Debbie@20: Technically, there are no obstacles to displaying Chinese characters in ebooks. Unicode has been around for a while now and contain practically any Chinese character one might want to use. (Caveat: Some personal names use obscure or newly created characters that Unicode does not support.) How we represent Chinese characters in digital text is, in the common case anyway, a solved problem. (Whether a book wants to use simplified or traditional characters is, of course, its choice. Unicode contains both sets.)

What they need is an ebook format which supports Unicode, modify their readers to support non-Latin characters, and include the relevant fonts. All of these are solved problems. Amazon just needs to do implement those solutions.

As someone else pointed out, ePub allows embedded fonts. (Note: Not all ebook readers support embedded fonts.) Amazon, of course, doesn't use ePub. It started with a version of the Mobi format. It has since transitioned to another proprietary format called Topaz. Since it's proprietary, I have no idea whether it's capable of non-Latin characters or not. If it is, can your typical self-publisher create a Topaz file to give them? Does it support embedded fonts or does the Kindle come with fonts capable of displaying non-Latin characters?

This is all stuff that has been done before. No, I don't know whether Amazon has any interest in this or whether they are working on it or not. However, these are not hard problems. They have already been solved by other companies in this domain and in closely related domains. If, after three iterations of Kindle and two different ebook formats, Amazon still can't support Chinese text, they are either incompetent or disinterested.

(Note: I don't own a Kindle. I'm going on what I was told about the Kindle's abilities. I'm surprised it doesn't support non-Latin characters because, in the grand scheme of things, there are harder problems to resolve when it comes to ebooks.)


#34 ::: Dave Bell ::: (view all by) ::: July 24, 2011, 05:06 PM:

Scivener seems to me to be one of several alternatives for organising the writing of a book. It might not suit everyone—that sort of program has far more differences, program to program, than do word processors or text editors.

I find there can be pitfalls with different epub readers and the choice of how to encode characters. HTML entities such as — seem more reliable that the unicode equivalent.

#35 ::: Bruce E. Durocher II ::: (view all by) ::: July 24, 2011, 05:29 PM:

The expert in e-book formats is Dave Howell: he made a living from it for years. Unfortunately the Bush Recession wiped out his company, so I don't know if he'd be willing to sound off on the subject...

#36 ::: thomas ::: (view all by) ::: July 24, 2011, 05:36 PM:

John Chu@33

As I understand it, the problem is that the Kindle doesn't come with fonts for more than Latin-1, but is otherwise Unicode-friendly. There have been hacks to install other fonts on the Kindle hardware, allowing display of a wide range of characters.

Topaz allows bitmap fonts to be embedded, but the quality is variable, the file size is much larger, and it comes with extra DRM.

#37 ::: J Greely ::: (view all by) ::: July 24, 2011, 06:23 PM:

Amazon has significantly improved their international font support in the current generation. I have the Kindle edition of The Rough Guide To Japan, and all of the kanji are smooth at all font sizes. The embedded web browser also renders Japanese correctly, so there's at least one decent international Unicode font built in, in addition to the Topaz format's embedding support.

(indeed, the only reason I finally bought a Kindle was that they added full support for Japanese, including their free 3G networking all over Japan, so I could search for restaurants, etc, while walking around)

-j

#38 ::: Sandy B. ::: (view all by) ::: July 24, 2011, 06:47 PM:

Not much to add but ObXKCD from last week.

#39 ::: Charlie Stross ::: (view all by) ::: July 24, 2011, 07:06 PM:

Mac/Windows/Linux note:

1. Scrivener for Windows and Linux is in beta at present -- should be out later this year. It allegedly produces the best epub output on the Mac.

2. The Mac text editor BBEdit just went to release 5.0, a major upgrade. 5.0 adds full HTML 5.0 and XHTML editing support and the ability to open and edit files inside zip archives ... which is what epub is: a .epub file is a zipped container of Other Stuff, most notable among which is the text of the ebook in the shape of a file containing a limited subset of XHTML. Thus, BBEdit apparently gives you a syntax-colourising (but non-WYSIWYG) editor for non-encrypted epub files.

(I haven't upgraded to BBEdit 5.0 yet so this is just reading off the press release at present -- but a combo of Scrivener and BBEdit 5.0 sounds like a seriously powerful setup for producing epub ebooks on the Mac, and Scrivener for Windows is ... well, if you haven't tried Scrivener yet, you'll find it an interesting, or even enlightening, experience. It's like a writing tool that was actually designed for novelists! Who knew?)

#40 ::: Charlie Stross ::: (view all by) ::: July 24, 2011, 07:12 PM:

dcb @23: the machine to replace the Psion 5MX is, to my mind, clearly the Viliv N5. (TL;DR is that the N5 is exactly the same size as a Psion 5MX but packs in a basic-spec netbook running Windows 7 with a 1024x600 screen, wifi, and 3G.) However, Viliv went bust last week, so if you want one you'll have to grab fast before stocks run out -- and it will cost you as much as a new Psion 5MX, back in the day.

Yes, it runs the Windows version of Scrivener. And OpenOffice. I've tested it.

#41 ::: Paul Duncanson ::: (view all by) ::: July 24, 2011, 07:42 PM:

Charlie Stross @ 39: BBEdit has just gone to release 10.0.

#42 ::: Charlie Stross ::: (view all by) ::: July 24, 2011, 07:58 PM:

Paul @41: So it has! What a brain fart ...

Yes, it's 10.0 that comes with the funky epub editing stuff.

#43 ::: ErrolC ::: (view all by) ::: July 24, 2011, 08:12 PM:

I did find reading the Hugo Packet ePub of The Dervish House suboptimal due to how some Turkish characters were rendered. It is possible that there are setting/font combinations on my Stanza+iPhone that would have been better, but trial-and-error got tedious after ten minutes.

#44 ::: SamChevre ::: (view all by) ::: July 24, 2011, 09:17 PM:

Not epublishing specific, but for text handling I really like TextPad. It will open much bigger files than notepad (I've opened 300 meg files and it wasn't bogging down), it will switch encoding sets among any of the common ones (both variants of Unicode/ASCII/ANSI/UTF-8, PC/UNIX/Mac), and it has really good column indexing. (So error at line x, column y is fast to find.) (All very useful when working with big, buggy files off of old mainframe systems.)

#45 ::: Ingvar M ::: (view all by) ::: July 25, 2011, 07:17 AM:

My toolchain (to both printable files, although currently not in any handy "fiction size" and EPUB files) is:

- Text editor (pick one, I use emacs, but that is because 20+ years of experience is not cheaply replaceable)
- LaTeX (markup-light format, mostly there's no markup at all, the occasional line starting with "\chapter{Insert a chapter title here}" are the bulk of it)

From there, I can generate print-ready PDFs (pdflatex), HTML, plain text or EPUB (via a tool wot I wrote myself).

I wouldn't necessarily recommend that tool-chain, though. It's woefully incompatible with most (dare I say "all"?) publishing houses. It is idiosyncratic. It requires a while of effort invested before it's remotely useful.

But, I already knew a text editor and learned LaTeX for a different purpose, so the marginal cost was pretty low. All in all I have probably spent more time cobbling together something that emits valid EPUB files than on anything else.

Now for some comments:

James D Macdonald @ #6:
Layout, font size and the like should be determined by the reader (as in the human operating the program of confusingly the same name). Unfortunately, some EPUB conversion toolchains see it fit to include font sizes in every single paragraph, something that actually spurred me to complete my conversion tool, so I could unbreak an ebook that I wanted to read, but had a font that was small enough that reading on the phone was full of eye-strain.

thomas @ #10:
At least for EPUB, the recommended "internal format" is xhtml, with some restrictions on what is valid formatting.

C. A. Bridges @ #12:
Or it's specified as a style tag on every <p> element in the body. At that point, ripping the EPUB apart, find the file, do a bulk search-and-replace, then pack it up again... Not for the faint of heart, I'd say.

#46 ::: PurpleGirl ::: (view all by) ::: July 25, 2011, 12:47 PM:

Jim: Many thanks for this post and to everyone writing a comment. I've typed for someone for 20-odd years and now I'm learning about ebooks so we can transform a novel she wrote into an ebook format. This post and comments is full of information. Again, everyone, thanks.

#47 ::: Evan ::: (view all by) ::: July 25, 2011, 12:56 PM:

Ingvar: A few years back I experimented writing stories in LaTeX (using sffms). What held me back was that while LaTeX -> PDF conversion is obviously excellent, LaTeX -> HTML was pretty weak and LaTeX -> EPUB non-existent. So the fact that you've written an EPUB converter is very exciting! Have you tried publicizing the tool anywhere? Is the source code available?

#48 ::: Christopher B. Wright ::: (view all by) ::: July 25, 2011, 01:05 PM:

Is there any way for someone on the PC side of the great digital divide to verify that an epub will display appropriately on a mac/iPhone/iPad? I have heard, for example, that iPad's do not correctly display centered text unless you use a kludge while making the epub file. Jutoh actually offers to put that kludge in for you but I'd really like to find a way to verify that it works.

#49 ::: Christopher B. Wright ::: (view all by) ::: July 25, 2011, 02:16 PM:

The answer to my own question is:

If it displays in the kindle app for Android, it will display just as well in the Kindle app for Mac.

If it displays in the Nook app for Android, it will display as well or perhaps slightly better in the e-reader for iPad if you remember to put span tags inside of any tags that are used to center text.

This doesn't answer questions about the Mac but it's more info than I had before...

#50 ::: dcb ::: (view all by) ::: July 25, 2011, 02:32 PM:

Charlie Stross @40: That's really tempting - as you say, a practical replacement - the £650 price tag is a bit steep, unfortunately.

#51 ::: Charlie Stross ::: (view all by) ::: July 25, 2011, 08:48 PM:

dcb @50: I paid £400 + VAT for mine, not £600. Suggestion? Check ebay. Also, if you get one, consider a 32Gb SSD (not the expensive 64Gb one) and a 32Gb micro-SDHC card (around £50) formatted as NTFS and mounted as your home directory.

(NB: Ubuntu apparently "mostly works" on the Viliv N5, but requires some chicken-waving.)

#52 ::: James D. Macdonald ::: (view all by) ::: July 25, 2011, 09:10 PM:

#45 ::: Ingvar M Layout, font size and the like should be determined by the reader (as in the human operating the program of confusingly the same name).

Did I say anything different?

As far as I know it is, and it's different for every reader, so I don't even bother trying to set layout, font size, or anything else.

One thing I do -- I put a hashmark in lines deliberately left blank, because there's no telling where that line will wind up on the screen, and I'd rather have the readers know when I've changed the scene.

#53 ::: P J Evans ::: (view all by) ::: July 25, 2011, 09:45 PM:

52
I wish more authors would do that for their printed books. It's disconcerting when you start the next paragraph and discover it's the start of a new scene, even though it looks like a continuation of the previous one.

#54 ::: Dave Howell ::: (view all by) ::: July 26, 2011, 02:14 AM:

Oh, man. I had to stop reading this thread about 15 messages in. I saw all this coming back in 1997. That's why we built the RosettaMachine. You mark up your text once, using XML, and feed it to the Machine, and it would manufacture PDFs (Letter, A4, and 4:3 for screens in 3 different font sizes), HTML, MobiPocket, Franklin eBookMan, MSReader, AportisDoc, and (later) Kindle. When a platform improved (Look! Now we can have italics on a Palm!), then the translation tables were updated, and you could re-run the same master file to get an improved version.

We'd upgraded it to handle building ePub files, and covers. Just upload one large file of a minimum size with a certain amount of sacrificeable space around the edges, and the RosettaMachine would downconvert, shrink, crop, grayscale, and/or otherwise do what was needed to include the image in the target edition when possible.

Handling interior illustrations was, IIRC, just about done when the banking meltdown swept our funding out from underneath the company (Rosetta Solutions). The technology still exists: the RosettaMachine is currently found at netgalley.com. We managed to keep all the development work from being wasted, but I no longer have any financial connection to it.

I know the long term plan was to allow general access to the RosettaMachine, but teaching people how to mark the text up correctly in the first place was tricky enough that we needed a situation where a small number of users would process a large number of titles, thus minimizing the tech support overhead, while we continued to improve the interface.

Maybe that will happen some day. Or maybe all the gizmos will support a single file format. Or maybe flying pigs will handle it all for us. Who knows . . . .

#55 ::: Ingvar M ::: (view all by) ::: July 26, 2011, 05:35 AM:

Evan @ #47:

It's only been finished since "sometime this past Sunday". And that's "finished" in the sense that the tool works, but is rather reliant on hand-driving parts of it, rather than being a suitable end-user-usable application.

It also only deals with a subset of LaTeX (a different subset than Pandoc), due to the fact that I'ev only implemented what I needed. One of the things I did need, because I rely on it to structure narrative flow, is \import and \include (each chapter being a LaTeX file of its own, I can if I want re-order chapters by simply changing inclusion order into the "master" document) and that's one of the few things Pandoc doesn't support.

The current user-interface consists of passing finicky parameters to a function in a read-eval-print loop (although that should be a solvable problem for a linux release).

Wrapping it up to be usable is... trickier. Primarily because there are (now) external dependencies (should be shipping a patch to a dependency shortly) and while the dev environment is quite supportive of 'snag dependent code from the external world", that's not 100% ideal for an application.

Do try Pandoc and see if it fulfills your needs. If not, please do not try to hold your breath waiting for my latex-mangler to be released, as it's only partially there.

#56 ::: Ingvar M ::: (view all by) ::: July 26, 2011, 05:46 AM:

James D. Macdonald @ #52:

I think I was primarily trying to point out the difference between the reader (the device or application) and the reader (the individual presumably operating the other reader).

And, alas, venting at the fact that font and size MAY be hard-coded, but doing so is annoying at best.

#57 ::: Kip W ::: (view all by) ::: July 26, 2011, 09:45 AM:

My case is probably different from the thrust of the post, because I'm converting books that (usually) have already been published through someone like Lightning Source. I've been trying to make them as much as I could like the originals, but with all the roadblocks the hosts throw in (so much for all your specified fonts! so much for your carefully sized artwork!) I'm puzzled about how to retain any trace of personality, especially since different platforms will do their own thing, and different readers on the same platforms will do their own thing as well.

What kinds of formatting does it throw out? Fancy-pants stuff, like centering and bold! Eggheads love that stuff, but for regular Joes, just throw it all to the left and make it the same size. You want your table of contents included? Try sacrificing a beloved pet; that sometimes works.

According to documents helpfully provided by providers, you just have to have a complete knowledge of HTML, and from there it's simple. That's lovely.

#58 ::: Tim Walters ::: (view all by) ::: July 26, 2011, 10:13 AM:

Kip W @ 57: PDF files will do what you need, but the tradeoff is that they will be very hard to read on devices that aren't close to the target size, or if the reader resizes the text, or if the device doesn't support PDF, as some don't. And since you're obliged to do the typesetting, you should be reasonably good at it, or the result might be no better, or even worse, than auto-flow.

A large selection of well-designed books would make me a lot more interested in a Kindle or iPad, but since resizable text is the main draw of e-books for a lot of people, I doubt if it will happen any time soon. Given the choice, I'd rather have things made easier for visually impaired people than indulge my design snobbery anyway.

#59 ::: dcb ::: (view all by) ::: July 26, 2011, 02:05 PM:

Charlie Stross @51: Unfortunately the cheapest I can see on ebay is £560+ from South Korea... Your recommendations for SSD and micro SDHC are noted and appreciated (and will be followed if I get one of these).

I suppose I'll have to wait and hope that if one company has produced something like this, someone else will do so also, in the relatively near future.

#60 ::: Craig R. ::: (view all by) ::: July 27, 2011, 08:22 PM:

Jim __

in re the issues with creating .prc files: the free Mobipocket Reader has an "import" function that will take an MS-Office, PDF, HTML, text, CHM or ePub document and create the .prc file that the reader can use.

I don't use it enough with non-mobi/non ePub imports to tell just what idiosyncracies (sp?) it may force.

#61 ::: Charlie Stross ::: (view all by) ::: July 28, 2011, 11:22 AM:

dcb: there was one (on ebay.co.uk) going for £470-odd yesterday when I looked. I think you're just going to have to keep your eyes open; they're going for silly money right now as the supply chain drains.

#62 ::: Russ ::: (view all by) ::: July 28, 2011, 11:54 AM:

James D. Macdonald@29

Well, I have a kindle and have been meaning to pick up something of yours to read, so I could put that one to the test. Was it something in particular, or everything? If the latter, is there somewhere you'd recommend to start?

#63 ::: MichaelC ::: (view all by) ::: July 28, 2011, 06:18 PM:

Take Control Books uses Apple's Pages to create their EPUB editions (I know this for a fact, because I'm the pixel-stained wretch who does that part of the job).

But for books that are primarily flowing text, such as novels, I have to say that Scrivener is really first-rate at producing EPUBs. And it's a great tool for writing, too, of course.

#64 ::: James D. Macdonald ::: (view all by) ::: July 28, 2011, 08:54 PM:

Russ #62 ... Was it something in particular, or everything?

Anything, really. I have a bunch up at $0.99, so ... start small.

(Actually, the $0.99 ones were converted by my son. Bad Blood: The novel and Judgment Night are ones that I converted. Those would be more useful to me now, to see how my conversions worked. Of those ... Bad Blood is the first in the series; Judgment Night is number three. I don't have volume 2 converted yet.)

#65 ::: dcb ::: (view all by) ::: July 29, 2011, 04:59 AM:

Charlie Stross @61: Okay, I'll keep checking then - it's very tempting.

#66 ::: Russ ::: (view all by) ::: July 29, 2011, 06:14 AM:

James D. Macdonald@64

Looks fine to me. Although I wouldn't be able to identify the actual font, it's legible and not noticeably different to any other book I have on there.

I scrolled through to about 30% of the book and didn't see any obvious format problems - obviously when I get around to reading it through* I'll let you know if anything does look screwy.

* Hmm...werewolves, eh? I'll slot you in right after Ghost Story**; I should be just in the mood :D

** Which I'll get to just as soon as I finish Across the Nightingale Floor and then The Book of The Dragon. Dammit, where does the time go?

#67 ::: Charlie Stross ::: (view all by) ::: July 29, 2011, 08:17 PM:

dcb @65: dynamism.com (a grey-market importer of far eastern tech toys) is pumping them out for £430, ex. VAT and shipping. Don't know how long the supply will last. (I have used Dynamism in the past and they were legit then.)

#68 ::: Glenn Hauman ::: (view all by) ::: July 30, 2011, 12:50 AM:

#54: 1997? Howell, you young whippersnapper...

#69 ::: James D. Macdonald ::: (view all by) ::: July 30, 2011, 10:28 PM:

Russ #66

Thanks for looking. The indents look okay, which is good. I'd heard a lot about how indents in people's self-pubbed books came out ... very badly.

BTW, I now have all three of the Bad Blood books up.

If you don't like it, you can tell me. If you do like it, tell your friends....

#70 ::: Daphne B ::: (view all by) ::: August 01, 2011, 08:25 AM:

I just wanted to say that Calibre/Sigil is a great team. I have a text file that I had converted to a pdf, and tried reading it on my iPod, with little success -- tiny fonts. But Calibre made it into an epub, and Sigil is helping me make it a well-designed epub. When I'm done I'm going to ask the author if they want to put it up on their site (along with their rtf, txt, and Word versions, which are free to the public).

#71 ::: Kip W ::: (view all by) ::: August 01, 2011, 11:19 PM:

Fiddling and more fiddling, followed by the light bulb of inspiration, have brought me to a discovery of how to have formatting preserved, a proper-ratio cover, and a table of contents: I output the book (with table of contents created already) from InDesign to MOBI with no front cover, and then add the cover as I convert from MOBI to EPUB using Calibre. Almost disgustingly simple, and I keep expecting to find "the catch." So far, there's no catch.

#72 ::: Mitch Wagner ::: (view all by) ::: August 13, 2011, 03:34 PM:

Jim, this is great. Thanks for posting it.

Can you point to any good information on the business of self-publishing, including where to post work, how much to charge, and best ways to market?

I'm planning to self-publish four science fiction and fantasy novelettes.

#73 ::: James D. Macdonald ::: (view all by) ::: August 13, 2011, 04:21 PM:

Can you point to any good information on the business of self-publishing, including where to post work, how much to charge, and best ways to market?

Gee, Mitch, you aren't asking for anything small, are you?

Let me think about this. I do have opinions, but am by no means an expert.

#74 ::: Mitch Wagner ::: (view all by) ::: August 13, 2011, 05:46 PM:

I was just hoping for a link or two. But if you want to write something up, I'd be grateful.

#75 ::: Mitch Wagner ::: (view all by) ::: August 15, 2011, 12:39 AM:

... Or not. Not is fine too.

#76 ::: James D. Macdonald ::: (view all by) ::: August 15, 2011, 06:51 AM:

Your wish is my command, Mitch. Got something for you, in my bid for the E-Pubbing Langsford W. Hastings Memorial Award

#78 ::: Mary Aileen suspects spam ::: (view all by) ::: August 31, 2011, 05:03 PM:

First-time poster, non sequitur post, non-human name.

#79 ::: Mitch Wagner ::: (view all by) ::: September 03, 2011, 07:50 PM:

Thanks for the link to the how-to guide, Jim.

#81 ::: Jim Macdonald ::: (view all by) ::: December 02, 2011, 09:36 PM:

More: Amazon Kindle Publishing Guidelines (.pdf)

I've had reasonable success using the following series of actions:

Create a Word .doc file. Save as .HTML (filtered). Convert to .prc using Mobipocket Creator. Upload.

Since I don't have a Kindle I can't really tell if things like the Tables of Contents are working. I hope they are.

Dire legal notice
Making Light copyright 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 by Patrick & Teresa Nielsen Hayden. All rights reserved.