This part will be about more software used in the process of writing the book. The last episode covered writing tools, file/version management and backups. What's up now is graphics programs, virtualization and PDF handling.
For outlining and structuring thoughts I like mind-maps. I know they are not for everyone, but if you like them and do not want to spend a lot of money on MindManager, have a look at FreeMind. It is a free, open source, Java based mind mapping application with a large set of features and a really nice UI. In fact, I am using it right now to structure the writing of this series of blog posts. It comes in readily installable versions for Windows, Mac OS X and Linux, which makes it really a good fit if you regularly switch platforms and want to be able to add a thought or two to an existing map without having to dual-boot or fire up a virtual machine.
Graphics and Illustrations
As with any technology, some things in database land are much easier to show as a diagram or schematic drawing than explaining them verbosely in text. Not to forget the occasional – well, the regular – screenshot to show GUIs or console printouts.
As a matter of fact in the first drafts of most chapters Udo and I had a lot of textual presentations of query results and plain text output from the mysql command line client, but the publishing company wanted more screenshots instead to make the book appear less text heavy. There have been mixed reactions to this, but generally I think less would have been more. Several of the screenshots merely show console windows which are not always good to read due to scaling and anti-aliasing.
To take screenshots, Mac OS X is pretty well suited without any additional tools. Even though there are some nice programs that offer advanced features like Layers which can automatically create a PSD file with one layer per open window, usually screenshots were taken of a single Terminal.app window or specific dialogs. For that the built-in features were completely sufficient:
- ⇧+⌘+3 produces a snapshot of the whole screen
- ⇧+⌘+4 produces a crosshair that you can drag over a specific area of your screen
- ⇧+⌘+4 followed by the space bar turns the crosshair into a camera to snapshot a single window
The only adjustment needed was to disable the drop shadow that is otherwise put behind the windows in the screenshot files by default:
defaults write com.apple.screencapture disable-shadow -bool true
That change gets effective after logging out and back in again or by killing the SystemUIServer process.
The screenshots are placed on the desktop by default. Depending on the OS X release the filename and format may differ. For the book we wanted PNG files, because JPGs would have caused compression artifacts. PNG is the default in Mac OS X 10.6 Snow Leopard. Generally this command allows you to change the format of screenshot files:
defaults write com.apple.screencapture type png
Instead of png you can also choose from tiff, gif, jpg, pdf, if I am not mistaken.
For Windows I used is a handy little tool called Screenshot Pilot which has a slightly “nostalgic” GUI style, but it works really well. It is far more pleasant to use than the otherwise necessary Windows style screenshot taking by hitting
Alt-PrtScr to copy the current Window to the clipboard and then paste it into
mspaint.exe and save it as a file.
Very early in the process we asked Packt about how to go about illustrations. They told us that anything would be fine that worked for us, because they had professional illustrators who would re-draw anything we sent them to give the book a uniform style. For example they would accept drawing made with any of the Office tools, bitmap graphics or just scans of hand-drawn sketches. I was very relieved when I heard this, because if there's one thing that eats up time even more quickly than text formatting it certainly is graphics work. I think of myself as nothing close to an artist, so knowing that someone else with probably lots of experience and routine would take care of this was very comforting and would relieve us of the necessity to find a set of good graphics tools.
Or so we thought at the time... To give you an idea, this is one of the sketches I sent along with a chapter of text:
While certainly no masterpiece I think the idea is brought across well enough for someone to redraw this with a little style. I scribbled together a few of these and sent them off in good faith.
Later in the process, this is what I got back for the sketch above:
Frankly, I was very much disappointed and certainly did I not want graphics of this style and quality to go into the book. Just look at the tunnel entrance and exit symbols… So in the end I decided to take care of the graphics myself. I fooled around a little with OpenOffice's drawing tool but did not manage to create anything that I even remotely liked. Fortunately a friend of mine owns a copy of OmniGraffle, a professional vector based diagramming tool for the Mac, and he let me spend some time on his machine remotely. With this I managed to create new versions of the illustrations in a style that I liked and with a little more attention to detail. This is my version of the same idea:
Making these took quite a lot of time. Once finished with the bulk of the work I sent it off to the publisher in EPS format, because I hoped that would be the easiest way of making sure they had the best possible quality to work with without requiring them to organize a Mac. Turns out the EPS is not as portable as I thought and had hoped. There were lots of troubles with line and arrow styles, transparencies etc. In the end I had to download a trial version of Corel Draw for Windows, because that's what Packt's graphics people use.
With trial and error I managed to get the illustrations to Corel Format with the correct fonts and no strange rasterizing artifacts for partially transparent areas. All in all, had I known I would have to take care of this myself from the beginning, I would have asked for at least one more month in the schedule and prepared them along the way. The same goes for the very few charts included in the book. I made these with a trial version of OmniGraphSketcher, also from OmniGroup. The idea behind this tool is to actually draw diagrams, instead of having them generated by a spreadsheet tool from pure numbers. While I generally like the idea, the tool is still lacking in my opinion. For example getting the X- and Y-scales right was more complicated than I would have thought. Anyway, I will keep an eye on the program - it surely has the potential to become a valuable tool and I will most certainly check it out again if/when the need arises.
A few more words on Corel Draw: I have no option but to rant about it! The last version I remember to be really good was Corel Draw 5, about a millennium ago or so. After that I had a look at Corel Draw 7 and did not like it at all. In my opinion it has suffered form the same feature-creep that most applications that were once lean, fast and powerful, getting bigger and especially buggier all the time (Firefox, Nero anyone?). I am glad I have no need to work with this software on a regular basis. From real crashes - which I saw quite a few of - to annoying user experience problems; this is not a program you would enjoy to work with. The stupidest thing in my opinion was that for virtually any file operation the open or save dialogs would open in my home directory, instead of remembering which directory I had last used. An estimated three hours of my lifetime could have been saved, had I not had to spend them navigating to the right path time and again. Also what is it the myriad of file format versions? As you save a file you have to decide what version (7, 8, 9, 10, 11, 12) you would like to use. Of course they are all called ".CDR", and the program does not remember what you chose for the last file... Better make sure to include the format version in the filename, because once on the disk you have to easy way to tell which is which!
Test setups / Virtual Machines
As it is easy to imagine, for a database cookbook lots of experiments and trying out stuff is required in order to make sure that everything you claim actually works. Anyone who kept reading up to this point is probably technical enough to understand that no matter how well you plan and how carefully you write down any sort of script, source code sample or the like, it will not work unless you copy and paste it from the actual command line window or source code editor. There just is no other way. And also it is (almost) guaranteed that you will break any code by making "one last beautifying change" in the word processor; be it a missing semicolon, a space introduced into a regular expression or a capitalization change, something will go wrong!
The only chance to make sure your readers are not too likely to find a bug in your scripts or command lines is to try them all - each and every one of them. Of course, some of them by design make permanent changes to the test setup, so if anything goes wrong and you cannot just take a screenshot and put it into the manuscript you might be in for varying amounts of resetting the machine and trying again. More than once I made a simple mistake in that, too and ended up with an ever so slight difference in the test setup, messing up the test again. In that light, one wonders why any functioning production systems exist in the first place...
Fortunately this is not 1995 anymore and there are very capable virtualization solutions available for reasonable prices to ease the pain with repeated tests and different setups.
As Mac OS X is not the primary platform most MySQL administrators use - neither as a client, nor as a server - I need a way to test with different versions and setups of Windows and Linux. One way would have been to use bootcamp to dual-boot into Windows, but that would have caused unacceptable roundtrip times. Because performance was rarely an issue, I decided to buy VMWare Fusion 3. Though not very cheap it certainly is a product worth its money. Here you can see my Virtual Machine Library - some of the test VMs have been deleted since the book was finished - but you can still see the Ubuntu, Vista and Windows 7 VMs I primarily used to take screenshots on.
Apart from being able to run different operating systems and MySQL versions, the killer feature for me was snapshots. Basically they allow you to clone a virtual machine and all its state into one or more named copies and reverting back to these "save-games" at a later time, discarding any changes that happened in the meantime. Though not particularly suited for long term operations - performance is degraded quite a bit - they are ideal for what I call "destructive" tests: Those that actually modify system state, like configuration files, database contents, software installations etc. For every VM I would first set up a well-known "good state" and take a snapshot of that. If in the course of writing a recipe I needed to make several attempts to get everything just right I could mess everything up with no worries and go right back to the beginning, without any risk of accidentally missing something important.
If you regularly have to do any sort of software testing you will appreciate the benefits virtualization immediately. The fact that I chose VMware is not too important. As it is often with software, personal preference plays a huge role. Udo got a good deal on Parallels Desktop which is another commercial virtualization solution for the Mac. However you could just as well go with the free VirtualBox from Oracle, available for Windows, Linux and the Mac. They all offer very similar feature sets and would all have been equally suited for the tasks I performed.
In the later stages of production annotated PDF documents superseded the previously used OpenOffice and Word document files. The reasons for and problems with that will be focused on in a later installment of this series.
Mac OS X incorporates very strong PDF handling facilities right in the operating system's core. Much of the Mac's graphics are based on PDF internally, so it is not too surprising to find advanced PDF handling features in the built-in "Preview.app". Out of the box any Mac can open PDF documents, inspect their meta-data and also create new PDFs from any application without the need for special PDF printer apps or the like. Before the book writing project I had never even bothered to install the Adobe Reader - I just did not have any use for it. Considering the fact that compared to Preview.app it takes ages to load and is a never-ending source of security problems I had no intention of changing that.
But then Packt started to send us PDF versions of the first formatted and laid out chapters, complete with annotations and comments. I blogged about a rather unpleasant episode regarding Preview.app's shortcomings earlier.
I won't repeat everything here, but suffice it to say that you should not trust it to show all the annotations and comments a document contains!
So I needed to download and install Adobe Reader to make sure I could at least see all annotations and meta-information embedded in the PDFs from the publisher. I have not checked again since, but when I last downloaded the installer from their website it was not the latest version. In order to keep your system (reasonably) secure against the many Adobe Reader based exploits circulating the net, make sure to manually run the updater again and again until it tells you there are no more updates left! If there had been updates like this in 1995, I am sure they would have been implemented like this...
The same goes for Acrobat Professional which in the further course of getting the book done was needed as well - again, details will follow later. For reasons beyond logic there is no trial version on Adobe's page for Acrobat Pro as a standalone application on the Mac - only for Windows. As a Mac user you need to download the full CS4 suite to try it... Fortunately I did not have to jump through these hoops, because a version 8 DVD was included in the software bundle with Fujitsu's ScanSnap. That one, too, had to be updated in what felt like 100 individual runs of the Adobe Updater.
While being considerably slower than the Preview.app accompanying Mac OS X, Acrobat and Adobe Reader were much better at handling document commenting, revising and especially comparing! Yes, it is possible to compare two PDFs and have Acrobat highlight changes between the two - a feature that became indispensable in the later project phases:
With that I will conclude this second part of the tool descriptions. I have probably forgotten one or two little utilities that came in handy, but these were the companions that accompanied me along the way constantly. In the upcoming episodes I will refer to them as needed, but you should have a general idea on what kinds of tools it took to complete the project. Be advised though that once my little saga here is complete you will find that quite a few of them shouldn't have been necessary for me to have, but that is for another time.
The next part will resume the more chronological style of describing my experiences. Stay tuned :)