Windows Live Writer

During the transition from JRoller to Blogger one of the factors that drove my decision were the available editors. Both offer some sort of WYSIWYG HTML editors to compose blog posts. However I was not too pleased with either one of them. Both tended to produce somewhat bloated markup with lots of divs and spans, combined with inline style attributes - in the end not really much better than <font> tags...

In the end I gave up on the "rich edit" modes and wrote the markup by hand, usually with Bluefish.

While I was generally fine with that I still kept looking for some decent blogging tool a little more sophisticated than either the browser's text-area control or Bluefish. I tried ScribeFire for a very short time, but was not pleased with it. Maybe it suits people better who usually blog about stuff they find while surfing the web, but for me it just did not work. Finally - I cannot really remember when or how I got to it - I stumbled across the Windows Live Writer.

It has a lot of nice features, one of them being the ability to download the layout from your actual blog (BTW I just tried this with my Blogger account, even though there are other types supported and I just assume they offer the same functionality). This allows you to write stuff with a very close approximation of what the final posting will look like.

You also get a "preview" mode in which you can see the full page layout complete with graphics and style-sheets. This is really nice to get a final look before publishing, because frankly the preview mode you get in Blogger's web interface plain sucks. JRoller at some point upgraded to a newer version of the Roller engine apparently and got that better from then on.

And apart from the plain GUI niceties, the HTML code that gets generated is really nice and clean. You may even choose between standard HTML mode (in which e. g. <p> tags are not closed) or XHTML compliant code (which I prefer).

Images you want to insert are automatically uploaded, so no hassle there, either. And you can put borders around it in different styles as well:

The one thing I missed though was the ability to insert custom snippets of HTML code, e. g. a <pre> tag with a specific class attribute. To do I had to switch from "rich editing" to the HTML mode and insert that manually. Today I found a nice template plugin (did I mention there is a plugin-architecture? ;-)) that allows me to do just that. It can do even more, because you can include snippets of C# code inside the templates. E. g. the following code was first written, then selected and finally the template with that code in it applied.

<pre class="prettyprint">
<%= _selection %>
</pre>

I have come to like this tool so much that even when working on Ubuntu (which is most of the time) I fire up a VirtualBox'ed instance of it.

So in case you have not found your favorite editor yet, are always ready for something new and - that one might be a show-stopper ;-) - you are not completely anti-Microsoft, go have a look and try for yourself.

Follow-up: Strange InnoDB Locking Behavior

On March 4th I wrote about an observation we made concerning InnoDB locking behavior in combination with foreign key constraints.

Since then we have been in contact with MySQL support and they in turn with the InnoDB developers. One of the conclusion I can already draw from the whole thing is that another level of indirection (InnoDB being a part of Oracle, i. e. an external resource to MySQL, too) does not tend to speed things up...

But be that as it is, here is a more thorough explanation we got about how the more aggressive locking comes to pass in case a foreign key constraint references a parent table's primary key. As the original response from the support staff spans several emails and includes some internal stuff I will paraphrase to protect the innocent (i. e. me ;-)).

First - just to remind myself - some notes on how the foreign key checks work. For any update you do to a child table, the DBMS needs to make sure that you cannot concurrently remove the parent table's row the FK refers to. Otherwise you could end up with a request to roll back the transaction that was intended to change a child row and be left without a suitable parent value. To do so, locks are placed on the index records being referenced by a child table's foreign key.

In the example transaction 1 locks a row in the parent table to update it. It does so by locking the primary key. It uses an X (exclusive) lock on the particular value of the PK for one row in this table. An exclusive lock blocks access to the entire row, because the primary key is a clustered index in InnoDB. That means the key itself is part of the record - in contrast to non-clustered indexes where a copy of index columns' values is placed in a location outside the actual records. So regardless of whether or not the key values themselves are changed, the record is locked completely.

Again: Because of InnoDB's structure all primary key locks are X locks, as the key itself is part of the row. In contrast to that an S (shared) lock only locks entries in a secondary index, which are stored outside the actual records.

Now, transaction 2 intends to update a record in a dependent table. It has to take a lock on the foreign key there whose reference key is the primary key in the parent table. However the exact row the child record refers to in the parent table has already been locked by transaction 1, hence operation 2 has to wait.

Transaction 3 needs to change a column in a row belonging to a child table, too. This column has a foreign key to the parent table, but not one whose parent is a primary key. It instead references a secondary key of the parent table. That means transaction 3 does not have to wait, because it does not require X lock on the entire row in the parent table. It only requires an S lock on the secondary key. As no other transaction has taken such a lock on that index entry, transaction 3 can proceed.

While all this explains perfectly why we experience a locking situation in one case, but not in the other, it still leaves the question open on how to go on from there.

Basically the one conclusion that can be drawn here already is this:

Either you get undesirable locks when foreign keys reference their parent tables' primary keys, our you have to define superfluous indexes that cost disk space and slow down write operations. In some cases you cannot even choose, because the optimizer might still decide to go for the primary key, even though you defined a secondary one, depending on the specific conditions for any given statement.

The latter part of that is especially true if you go for the - usually suggested - short primary keys in InnoDB tables, e. g. single numeric key columns. While those are generally a good idea especially for InnoDB they work against you in the scenario at hand.

Right now, the only suggestion we got from the MySQL folks is to rather go for less indexes, because the cost they introduce is usually significant in contrast to the locking problems. This is hard to quantify however and may depend heavily on your application's access patterns. Making matters still worse is InnoDB's current (5.0) limitation that it will lock all rows that are examined while executing any particular DML statement. Only in 5.1 only rows that get actually changed will be locked, alleviating the situation to some degree.

There might still be something coming here, however right now I do not think there is a big chance of this being changed any time soon. The one thing I will ask for however is to improve the documentation on foreign keys to have people know about this stuff in advance.

Enable Explorer Window Titles on Vista

Something struck me recently about the appearance of the Vista desktop. I did not consciously realize it at first. What's "wrong" with this picture:

If you do not see it, compare it with this one:

See the difference? The second screenshot was taken with "AeroBar" loaded. The only thing it does is make use of the otherwise wasted space in the title bar by displaying the folder name. Does anyone know why Microsoft would not do this by default?

iPhone SDK - Sweet with a smack of bitterness

When Apple released the iPhone SDK to finally allow 3rd party applications on the iPhone and the iPod Touch, I was immediately excited. Even though Apple claimed that web applications were a great way of developing software for their devices, I did never think so. It may be a feasible way for some simpler types of software, but not being able to leverage the full potential of a "fat client" (even on a device as thin as that ;-)) always struck me as a severe limitation.

However the sun does not shine so brightly after all. The Register's Bill Ray writes about his view on things in "Cometh the hour, cometh the iPhone SDK". I tend to largely agree with him. When I watched the "March 6 event" video I didn't even realize that the $99 fee for the development tools was supposed to be a yearly subscription, but got the impression it was simply the price for a license to use the tools in the SDK. This is still what I understand from the iPhone Developer Program website. But maybe its just hidden somewhere in the fine-print I did not care to look at.

However much more important to me is the fact that the Apple "App Store" will be the one and only way to distribute software to end users. This has huge implications, even though at first glance you might be tempted to follow Apple's portrayal of this being a good thing: a very easy way for software authors to get their stuff deployed to the largest number of customers possible. They underline this will be a free service for free applications. And maybe even the 30% of the price for non-free products is a fair deal, compared with what others claim. Nevertheless this makes each and every software developer - be it a company or an individual - dependent on Apple's good will.

Jobs already made clear that there will be restrictions on what kinds of software will be sold and which will not. Of course he presented examples that most people will tend to agree with: porn, illegal and malicious apps and those compromising your privacy. One might possibly argue that this may lead to a higher quality standard. But think about it a little longer: it also means that Apple takes the role of the "guardian of the public morals". This might even be seen as a good thing, but this also means, that they have full control over the software market. They can easily delay or even reject anything that is not to their liking, i. e. is a competition to their own products.

Think of a company that specializes in iPhone apps and becomes really successful. Maybe at some time in the future Apple decides that 30% of the revenue is not enough and wants to sell a similar product of their own. In a regular kind of software market, the customers still have the choice whether they use the one or the other vendor's products. But with the App Store Apple could just decide that the competing product cannot be sold though their channel any more, effectively killing the other company. Maybe it will need a little tweaking of the terms and conditions, but hey: what are you gonna do if you don't agree with them then? The only thing you can do is leave with your product, which is exactly what you were supposed to do in the first place. Now, that's what I call a monopoly.

I definitely like Apple's products, they are really great. Seeing what people were able to do with the beta-version of the development kit was really impressive. I expect a lot of great new stuff to come out in the near future. From what I see going on here people should consider if shouting "foul" at e. g. Microsoft is always a call in the right direction.

Nevertheless, maybe this is just the beginning of a new round of jail-breaking. In the past you couldn't run non-Apple software, so smart people found ways around that. Now you can have 3rd party software, but only through Apple. I guess someone will work around that, as well...

iGoogle for iPhone

While iGoogle is nothing really new, I only recently (and by accident) noticed, that there now is an iPhone optimized version of it. While the regular Google home page for mobile devices has seen some improvements specific to the Apple mobile devices as well (tabs at the top of the screen), I really like the combined iGoogle and small display optimized combination iGoogle/i offers.

Unfortunately the latter one does not have the nice tabs to quickly switch between GMail, Calendar, Google Reader and the other apps they offer. This would be the last thing to make it close to perfect for me.

I wonder how this got past me without noticing it earlier...

For some screenshots see tuaw.com (iGoogle/i) or TechCrunch (Google/m). It seems you have to use an iPhone or an iPod to see the differences, the "regular mobile version" appears to be unchanged when opened with e. g. Internet Explorer.

Monsters of the Programming World

Through a comment on my post about my latest encounter with unexpected/unexplained locking phenomena David Linisin's blog caught my attention. Looking through some older postings there I found a post called "The Cruel World Of Programming" which contains some hilarious drawings of "bug species" everyone knows. Go have a look, a hard copy of this is now pinned to the wall behind my desk :-)

Importing Audiobooks to iTunes on Windows

I may have found a new passion: Audiobooks. While for quite some time I was rather skeptical, even when my girlfriend gave me "The Hobbit" as a Christmas present a year ago I hardly tried it. I just could not imagine sitting around and listening to someone reading a story to me.

However back then I did not have an iPod (or any other portable music player for that matter), so now - listening to podcasts most of the time I drive, walk or do my household chores - I reconsidered when I got hold of a "Glenkill" set of CDs.

My first shot was to just import them into iTunes. However that way all 5 CDs with all their tracks showed up in my "Music" library section where I really did not like them. Even though you can change the files' properties to exclude them from shuffling this was not really acceptable for me.

A colleague told me that he had found "Doug's AppleScripts for iTunes" when he was trying to do the same thing for his children's audiobooks and they served him well. However there is one slight detail that makes them (at least for now) unsuitable for me. Let's just say there is a part of the site's name that does not lend itself to a Windows or Linux user...

There are some suggestions, but most of them involve re-importing the CDs as a single file per CD into iTunes - something I did not really want to do if not absolutely unavoidable, because I consider swapping media a rather boring task.

So I went and looked for something similar to the above mentioned Apple scripts for those being stuck with the Windows based version of iTunes. Eventually I found just the right tool for the job: Mp4Box or rather its GUI front-end YAMB.

Once I had downloaded and installed it I could join all the single m4a files I already had into a single large one. The GUI is a little rough around the edges (try and re-order the tracks to see what I mean), but that's alright. After some time (the progress indicator is not really helping) I ended up with a new MP4 file that contained the concatenated tracks I had fed into YAMB.

Importing it into iTunes once the file suffix had been changed to "m4b" (the file extension for audiobooks) automatically put it into the "Audiobooks" category. I have not found out yet, if I can define chapters or bookmarks - that (and syncing it to the iPod) will have to wait until tomorrow.

InnoDB Primary Key and Secondary Index Locking Strangeness

Recently we noticed a strange locking problem within our application. While the previous release worked flawlessly the current version produced a number of Lock Wait Timeouts.

The code itself did not change in the area we experienced the problems, however our persistence framework had been updated to a new release in conjunction with our app.

One notable difference was the schema generation strategy (the schema is generated based on an UML class model). In the previous version primary key columns were always created alphabetically. Because that is not necessarily a desirable thing it always created a secondary index, too, that contained the same columns in a user-specified order. In case you really wanted the alphabetical order you ended up with two identical indexes. Especially for InnoDB this is a very inefficient thing to do. (See my Index Analyzer tool for more info on why.)

Often we ended up with something like this:

CREATE TABLE `parent` (
    `A` char(3) NOT NULL default '',
    `B` char(2) NOT NULL default '',
    `C` decimal(12,2) default NULL,
    PRIMARY KEY (A, B),
    KEY `IDX_P` (B, A)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

In this table the Primary Key and IDX_P are both referring to the same columns, just in a different order.

With the new persistence framework release those secondary indexes are not generated anymore. Instead, the primary key order is used as defined in the object model.

Although this is generally an improvement we appreciate a lot, it turns out this did have a negative side-effect of accidentally introducing a different locking behavior we would not have expected:

create database if not exists testcase;
use testcase;
drop table if exists child1;
drop table if exists child2;
drop table if exists parent;

CREATE TABLE `parent` (
    `A` char(3) NOT NULL default '',
    `B` char(2) NOT NULL default '',
    `C` decimal(12,2) default NULL,
    PRIMARY KEY (A, B),
    KEY `IDX_P` (B, A)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `child1` (
    `X_parent_A` char(3) default NULL,
    `X_parent_B` char(2) default NULL,
    `id` bigint(20) NOT NULL default '0',
    PRIMARY KEY (`id`),
    CONSTRAINT `child1_parent` 
       FOREIGN KEY (X_parent_A, X_parent_B) 
       REFERENCES parent (A, B) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `child2` (
    `X_parent_A` char(3) default NULL,
    `X_parent_B` char(2) default NULL,
    `id` bigint(20) NOT NULL default '0',
    PRIMARY KEY (`id`),
    CONSTRAINT `child2_parent` 
       FOREIGN KEY (X_parent_B, X_parent_A) 
       REFERENCES parent (B, A) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO parent (C,A,B) values (11.95, '111' , '11'); 

This sets up three tables and populates the "parent" table with a single row.

Now we go ahead and try some work on those tables in two separate transactions:

Note that this transaction has not been committed yet. It makes a change to the single row we have in that table. However it does not change the primary key value!

Now in a second transaction try something with a child table:

To me this does not really seem like a difficult task for the database to perform: Insert a new child row, referencing the parent table. Even though there is a concurrent update on that parent row in progress, it does not intend to change the primary key value which means the referential integrity is going to alright, no matter what transaction gets committed first.

In our app this is a very common pattern and we would definitely have noticed if that was something MySQL/InnoDB had problems with.

This is the relevant portion of the InnoDB status output on locks, when the UPDATE statement above has been issued in the first transaction:

As you can see, there is a lock on the primary key of that single row we are updating.

When we try to insert into the "child1" table, this is what we get regarding locks:

This notoriously hard to read output (see Xaprb's Maatkit) says that the INSERT INTO "child1" is waiting for a record lock on the master table to be granted. Because this lock is being held by the update transaction which has not been committed yet, the INSERT will eventually time out and report the error we have seen.

So far, this might not be obviously strange. But look at this now ("transaction 3"):

Compare transactions 2 and 3 carefully. You will see that they are identical in terms of the values. The only difference is that the former inserts into "child1" and the latter into "child2".

Here are the relevant parts from the DDL of tables "child1" and "child2":

CREATE TABLE `child1` ( ...
    CONSTRAINT `child1_parent` ...
        REFERENCES parent (A, B) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE `child2` ( ...
    CONSTRAINT `child2_parent` ...
       REFERENCES parent (B, A) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

As you can see the foreign key constraint defined in "child1" references the primary key columns in the parent table. "child2" uses the same columns, but in the order they are contained in the secondary "IDX_P" index.

According to the manual you need not reference the primary key in foreign key definitions, but any index that begins with the referenced columns in the right order will do. But obviously there is an important difference: Modifications to all child tables will block in case a referenced record's PK has been locked, even though its value is not being changed, whereas using a secondary index will happily allow you to run both statements concurrently without conflicts.

For completeness' sake we made some more tests, e. g. with a secondary index that's defined in the same order as the primary key. In that case you will get the locking in both transactions, because then InnoDB always uses the Primary Key and hence locks updates in both child tables. And of course you will get a locking situation on the child tables when you are going to modify the primary key value in the parent table.

Currently MySQL/InnoDB are looking into this, because from what I hear from their support staff, they find it curious as well. I'll follow up on this one for sure :)

Vista (Home Premium)

Last weekend I decided to buy a license of Vista to replace the XP install I still kept for the occasional necessity. I went for the Home Premium version which I got from Amazon.de for €89.

As I usually use Ubuntu I decided to do a clean install to replace the XP partition completely. All documents are on the Linux disk anyway, so nothing to lose there.

The install worked flawlessly, even though I do not know how long it took, because I let it run over night after I had entered the product key. In the morning it was asking for a user name and allowed me to log in shortly afterwards.

At first glance I did not notice any striking differences compared to the RC1 build I had tried out some time ago. Nevertheless I quickly realized that the computer was beeping using the internal speaker. A look at the device manager (Ctrl-Pause still works) revealed missing drivers for my sound card.

Windows update pulled about 170MB and wanted a reboot. After that I had new graphics drivers (nvidia Geforce) and sound drivers (Creative Labs Soundblaster SB Live). While the graphics worked fine, the audio sounded somewhat garbled - just as if the volume was too high for the speakers. Only when reducing the volume to around 5% the sound became clear. Looking around the net I found out that Creative Labs has announced the end-of-life for its Audigy and Live series. This means that a perfectly fine sound card that has served its purpose well for is now utterly useless, because of missing drivers.

While I can understand that Creative may have new products to sell, I can tell you that this is not going to drive me into their arms again. For now I have settled with the onboard Intel sound chip that was disabled via BIOS setup up to now.

Apart form that driver issue there seems to be a little more work to do with the sound subsystem. When I played the "refurbished" Solitaire game and switched to a different window the game sound were gone until I closed the game completely and re-launched it...

The "new" start menu (is it still named like that, even though the "Start" caption is gone?) has a nice search feature that allows me to just enter some characters and see all matching entries. However I would have liked it even better had they not removed the old program group cascading menu structure. The way it is now almost forces me to switch between mouse and keyboard, while the old style was a little easier on mouse only usage.

So far I cannot tell I am terribly impressed - only the Aero GUI is nice to look at. So I guess I will continue to use Ubuntu as my main work OS and just go to Vista to test Windows specific stuff and maybe to play from time to time.