While last time I focused on the initial contact with the publishing company (just referred to as "Packt" from now on), this issue is about the process of putting together an outline proposal and coming up with things to write about in the first place. As from this point on in the process Udo was involved with everything, I will be referring to "us" and write "we" most of the time from now on.
The Publisher's Expectations
The only thing we knew about the books would-be content was a chapter template for Packt's cookbook series as well as the general description provided by Sarah earlier:
As I'm sure everyone is aware, MySQL is a relational database management system. Administrators of MySQL will be tasked with things such as maintaining the database, tuning the server, managing users etc etc.
This cookbook will have all the MySQL recipes an administrator could dream of, spanning from creating tables to managing views, from improving performance to securing the database, from using monitoring tools to using storage engines. DBAs of all levels will be catered for with recipes of varying difficulty, allowing the reader to administer MySQL to their hearts' content.
Before actually beginning the outlining process there were a few more emails sent back and forth, mostly questions on our part, for example whether we should include programming related materials or tool descriptions in addition to more "real" database themes, what spectrum of experience to expect from potential readers and so on.
Packt did not provide a list of topics they had in mind - instead we were completely free to make suggestions for what we deemed interesting for MySQL administrators. There were no hard rules on what topics would be out-of-scope, but in general we were asked to find a balance and suggest roughly the same number of topics that would be interesting for relatively inexperienced readers, people already familiar with MySQL and experienced, professional DBAs - with a general tendency toward the latter. At the time I was a little disappointed as to how little guidance they provided, but in hindsight I believe this was definitely for everyone's benefit. Packt's position on this apparently is that their authors generally are closer to their designated audience than someone who can only have so much of an insight into a lot of technical topics; something I can certainly agree on :).
As for the MySQL versions targeted, they naturally asked us to concentrate on the most up-to-date version at that time - which was MySQL 5.1. In our day jobs we are still stuck with 5.0, but it was a good opportunity to get a feeling for what changes we will have to make to our processes and practices when eventually we decide to tackle the upgrade. Considering the fact the writing the whole thing would take its time this was of course a reasonable decision. Midway during the process we thought about looking at 5.4, too, but decided against it - checking everything on 5.0 and 5.1 was enough work already.
The cookbook format as it was described to us would allow for "spin-off" recipes: A general recipe with detailed instructions, followed by shorter, more concise ones that described a variation of the original baseline, sometimes more advanced, sometimes just with a different focus. So in general Packt recommended to go for a greater number of shorter individual recipes, because that would allow the reader of the book to skim through the table of contents and not have to interpret too much into the titles in order to grasp what each recipe would do for her.
While we actually ended up with a few of these in the final book I did relate to the idea too much. As a matter of fact I think we could have better used those pages for genuinely different content, but that might simply be a matter of personal preference.
On January 23rd, not just yet one month after the initial contact, we sat down together in front of Google Docs and held a little brainstorming session on topics that occurred to us while talking about our experiences with MySQL. Of course some of the "bigger" milestones from our day jobs sprang to mind first, but over the course of a couple hours with some snacks and drinks we came up with quite a lot of individual topics.
Looking back at it I think we might have better used a mind-mapping application like FreeMind, because when working with text documents one is always tempted to format or move around stuff, trying to find structure where there usually is none intended in brainstorming. On the other hand, brainstorming sessions are usually short, so maybe that made up for it :-)
There was lots of random talk about other stuff, too, but we consciously decided not to try and concentrate and focus too hard. Instead whenever someone had an idea - however stupid it might seem at the time - we added it to the list in a very rough sketch sort of way, not thinking about it at all.
Structuring the results
When we were finished, we had compiled a list of about 125 ideas, some being duplicates of others on second thought, some being broad enough to be split again into two or more more specific ones. We went through them all and tried to categorize them logically into rough areas such as "replication", "schema related", "indexing" and so on. We did not try to come up with a list of categories or chapters first, but just made them up on the fly. I think, that worked out very well. The result - after some more minor tweaking - can be found in the final book now as the 9 chapters and the appendix.
The latter one was not really a part of the cookbook format's description, but after combing the list a few items remained that proved very resistant to the idea of being formulated as a "How to do xxx" recipe.
For a short moment we considered dropping these ideas as being "not right" for the cookbook, but quickly decided to propose the idea of an appendix titled "Good to Know" as sort of a collection of interesting and relevant ideas one might find useful despite them being not strictly step-by-step instructions.
During the course of this sorting phase we also spent some time thinking about what level of experience a reader would be expected to have for each of the proposed recipes. We categorized everything as "A" (novice) through "D" (professional) to get an idea if we were anywhere near a uniform distribution among these. After that first pass we saw we had come up with only a very few "A" level topics, roughly the same (higher) amount of "B"s and "C"s and also quite a few "D"s. We took one or two more passes, sometimes splitting up more complicated recipes into two more simple ones to get a few more novice level recipes. In the end we reached an approximate bell curve with a slight tendency toward the more complicated stuff. We decided against trying to cut on the more complicated topics, because we came to the conclusion that this was what we would expect from a book, were we to buy it ourselves.
The end result was compiled into an email and sent to Sarah shortly afterwards on January 26, 2009. At that point we considered it to be sort of a pick list for Packt to choose from. It did not contain estimated page counts at that time and we surely expected a reasonable number of items to be combed out by Packt.
Just a day after I had sent the email to Sarah, we got this reply:
Thanks for the outline. I've had a look through and its looking good. You have a good mix of basic and advanced recipes which will keep all readers happy.
I don't really have any suggestions to make at this stage. I am happy with the outline and so have presented it to my colleagues who will give us their feedback and suggestions which will help us to finalise the outline.
If possible, could you provide me with the expected page counts of each chapter. This only has to be a rough estimate at this stage, but you will find it useful to know what to aim for when writing each chapter.
That same email had a contract proposal attached, but I will spare that for a later post, because that's a little story of its own.
Page Count Estimate
The page count issue turned out more difficult than we had imagined. The initial description of the project called for about 300 pages. That's one of the reasons why we assumed the publisher would cut down the number of recipes from our proposed outline quite a bit. Even though we had no per-recipe estimate at that time, we had sort of a "feeling" going in the direction of around 400+ pages, just from the number of topics.
Even before we had sent the estimate Packt came up with a proposed schedule. Unfortunately there was little information on what that was based on, so we assumed it was calculated based on 300 pages with one page per author per day. That would have meant almost no buffer and a writing phase of approximately 5 months.
We had sat down again, thinking about how long each recipe would probably be. This turned out to be quite difficult with no experience to draw from. So we first were quite generous with the estimates. When looking at the final page count we were at about 500 and decided that this was a little over the top, especially when every recipe was intended to provide a quick solution to a clearly defined problem. So we thought about every single one again, this time considering how much we would be willing to read ourselves were this someone else's book. In the end we came up with around 420 pages.
So if the number of pages were to come closer to the 400 we had in mind, that would have meant about 7 months of writing, still with basically no buffer.
I decided to write a sample piece to get a feeling for how long an actual recipe and a chapter introduction would be, and if we had the same level of detail in mind as Packt did. If you'd like to read it, just go ahead: It ended up almost unchanged as the introduction of Chapter 3 (Tools) and the recipe on Sharing MySQL GUI Tools' connections.
In the meantime we had received another email, informing us about the acceptance of the outline proposal we had sent. No word on any cuts or redactions. So along with that content sample we sent our concerns about the schedule and the roughly 100 page difference we expected.
This is what we got back:
All recipes have been accepted, but if you feel that you need to remove some of them, that would be fine. The outline was approved by the editorial team here, so none of us have extensive technical experience. Therefore, we rely on an author's technical experience when talking about the specific details of the content. We would ideally like to keep the page count as close to 300 pages as possible, as this is what all books in the cookbook series are based on.
Do you imagine that all the recipes in the book will be as detailed as your sample one? I expect that different levels of recipe may need different amounts of explanation, depending on the topic. If you feel that you may need to delete some less-important recipes from the outline, then that would be fine. As long as the page count didn't go over 350 pages, that would be ok.
The schedule is just based on the first drafts of the chapters. Once a first draft chapter has been written, we will send it to a technical reviewer and, once all first drafts have been written, you will start writing the final drafts based on mine and the reviewers' feedback. [...] On the whole, we estimate the first draft writing stage to take 6-7 months, whilst the final draft stage takes 2-3 months. [...]
However, we do allow 30 days 'breathing room' in our editorial timeline and the schedule will be fairly flexible if needs be.
At that time we were reluctant to accept the schedule, because we were not quite sure the "1 hour per day and page" rule of thumb would hold very well and we did not want to reserve every moment of free time for writing.
Some more emails flew back and forth.
On February 10, 2009 I wrote:
We definitely see your point regarding the general 300 to 350-page rule for the cookbooks [...]. Unfortunately we do not see a way to keep all recipes in the outline and stay below that mark. Leaving out details is a path we would be reluctant to follow, because we fear that would mean giving up the really important stuff.
[...] option might be to cut a lot of the easier recipes (probably most if not all marked A and several designated B) and concentrate on the more ambitious ones that are not covered elsewhere at a comparable level of detail or in similar contexts. To do this we would generally assume a slightly higher level of expertise regarding the readership.
On the very same day I got this reply:
I would be reluctant to lose the easier recipes as it would cut out a great deal of the audience. I think it's important to include a wide range of recipes to cater for all levels of experience.
Therefore, if you agree, I would like to keep all of the recipes that you mentioned in the outline. I have spoken to the cookbook series editor and he agrees that all the recipes are invaluable and so would be happy to extend the page count if necessary. Obviously, this would mean extending the schedule too. [...]
I'm glad that you both have put so much thought into this and can see that you are committed to quality, which is exactly what I like to see! :-)
So in the end getting the outline and page count agreed on took more than four weeks, which we would not have believed up front. In retrospect I believe our initial estimates were always a little too high and conservative. Probably if we were to tackle another one of these, both the estimates would be more accurate and the whole process of agreeing on the outlines and schedules would take a lot less time to get done with.
The next part will probably concentrate on the matters of getting our contracts ready - a rather dry and boring experience, but an important one nonetheless.