Sunday, April 30, 2006

Change default printer via command line

An old application I still need to use has no way of printing to a printer other than the Windows default. It is the only program that we need to print on an ink jet with form paper. All other applications usually print on a laser.

In order not to have to change the default printer manually before starting that application I wrapped it up in a little batch file. It changes the default printer appropriately with this command line:

@RunDLL32.EXE printui.dll,PrintUIEntry /y /n "Canon i560"

The printer name has to be the value displayed in the control panel.

All available options can be displayed like this:

RunDLL32.EXE printui.dll,PrintUIEntry /?

Wednesday, April 26, 2006

Java Source File Encoding

Some time ago we switched our CVS Server from Debian to RedHat and ran into issues with file name encodings. Now we switched to a new build machine, changing platforms from Windows to RedHat.

The ant build process had run in a Cygwin environment on Windows, so most scripts could be used without changes on the new machine, too. So far the change went pretty smoothly.

However our application for some reason failed to resolve some I18N text values that are stored in property files. Not all of them, but some were suddenly messed up in the GUI, as if the resource's key could not be found in the property file.

It turned out to affect only keys that had German umlauts in the key part. While this is not exactly good style it had worked up to now, so we wondered what might have caused it to suddenly fail. After a while we compared the .class files (produced on the Windows and Linux build servers) of an affected dialog and indeed found a difference in the binary representation of the string constant's value that holds the resource key.

As most of our developers use Eclipse on Windows the source files were saved in Win1252 encoding (Eclipse's default on that platform). In 1252 the German characters are in the range between 128 and 255. When compiling the source on Windows it was transferred into the classfile "as is".

However on RedHat the compiler did not interpret the .java files as Win1252 encoded (who would have guessed) but as something else (probably some misled UTF-8, as this is RedHat's default for all locale settings). So in the resulting .class files the umlauts were represented differently, while the property files went into the final JAR unchanged. At runtime the program looked for the key as found in the class and of course did not find an appropriate entry in the .properties file, thus not displaying the GUI as expected.

Luckily the javac compiler allows a command line switch to specify the source files' encodings. We added the "-encoding Win1252" option which solved the problem.

Even though this works it shows the bug-producing potential non-ASCII characters have in software development. We will now go through the source and change the occurences of umlauts to their ASCII-replacements, just to be sure.

Monday, April 24, 2006

Awt_Robot and File Handles

In January I wrote about translucent windows with Swing. I found it in O'Reilly's Swing Hacks book and was pretty pleased with the results.

However for some time we have been experiencing problems with our application in production (running Sun's JRE 1.4.2 on RedHat 9). They were not obviously GUI related; for some strange reason a barcode reader attached to the machine randomly stopped working. We traced and debugged a whole lot, but could not find any reason apart from a bug in the native driver layer for these (JavaPOS) devices. As a result we went to the driver vendor and had them look into it on the driver level.

After a while they came up with some results which revealed that there really was a problem in the native part of the driver; however they could not find a way to reproduce it using their test tools, but only when using our application.

Basically what they found was a process called awt_robot that had a file handle on a device node in the /dev filesystem used to communicate with the barcode scanner. However that process had not issued an open call to the file but had started using the already open file handle right away. When our main application tried to close the handle at some point, that close call froze until one killed the awt_robot manually. Only then would the close call return and the application continue normally.

So that explained where the problem was, but now how it came to pass. Armed with that knowledge about the awt_robot we started looking around and found it to be a binary in $JAVA_HOME/jre/lib/i386/. Apparently it is part of the implementation of the java.awt.Robot. No one here really knew that there was a separate program to back the Robot class, but when we debugged through the translucent-windows-part of our code we could see an instance of this program being forked by the java process upon first invocation. Having been started once it would not go away until the VM itself was shut down.

Up to then we had (at least I had) believed that the native robot stuff was part of the JVM itself. As they say: You never stop learning...

Anyway (learning even more here), as any process forked under Linux inherits the file handles of its parent (found this page from IBM with a quick Google search) the awt_robot also inherits the (already open) handle on the device node. It seems that as long as the child process does not close this handle, the parent also cannot do so and has to wait for it. So if awt_robot does not usually exit before its parent java process stops the handle will actually never be closed during the application's runtime.

So in the end our hardware problem turned out to be really very well hidden GUI problem. We commented out the code for the fancy windows and everything ran smoothly again. We did not try if this problem also occurs on the Windows version of Java, but I believe one should at least be aware of the situation.

These comments originate from my old blog and I find them interesting enough to repost here:

Oh, wow, that's an interesting observation. I wrote the initial version of that code. We had to use an external process because of locking issues in AWT. That is, in 1.3 anyway, there were several cases where AWT_LOCK was held over long periods of time. AWT_LOCK is meant to prevent multiple concurrent calls to Xlib, because the Xlib of that time frame was not thread safe. It took several iterations and we ended up with having a separate process so that the Xlib of the child process could not in any way make concurrent Xlib calls. The child process is supposed to inherit a couple pipe file handles, and communicate with the parent over those. It absolutely does not need any other file handles than the two pipe's and the Xlib connection to the display. Have you filed a bug on this???

Posted by David Herron on April 24, 2006 at 06:50 PM CEST #

Not yet. First priority was to get the workaround jar for the customer ready. But I will post a report tomorrow.

Posted by Daniel Schneller on April 25, 2006 at 12:22 AM CEST #

I have now filed a bug report with Sun. The problem occurs when the file in /dev gets opened exclusively by the driver module. In that case it cannot be reopened by the main application once it has been closed, because awt_robot still keeps a handle on it. This prevents opening it again in exclusive mode.

Posted by Daniel Schneller on April 28, 2006 at 05:32 PM CEST #

This is the bug report I filed: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6450359 I was closed as "not reproducible". However there is a reference to this one: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6287244 Basically it says that the awt_robot child process is no longer needed in Mustang (Java 6). While this is not an option for us, at least there is hope for the future :)

Posted by Daniel Schneller on August 15, 2006 at 12:17 AM CEST #

Sunday, April 23, 2006

Insultingly Stupid Movie Physics

Surfing about the German c't-magazine's web site I found a link to a site about movie physics being so dramatically wrong it is almost insulting for the audience.

Go have a look for yourself (the site is in English). They first offer some common mistakes that are shown in virtually any action movie and an explanation what is wrong with what Hollywood's film makers present us. After that they have a long list of movies they "reviewed" in respect to physical correctness.

I really had to hold on to my chair reading some passages. Great fun and you may eventually even learn something new (as did I; see the section about throwing lit cigarettes into puddles of gasoline.)

Saturday, April 22, 2006

Drive-through-tax-declaration

Just a couple of days ago I returned from a vacation in the San Antonio, TX area. One day the postal office was closed, police vehicles blocking the entries to the parking lot. Afterwards I saw in the news that someone had placed a package labelled "BOMB" at the front door which led to an evacuation.

Some days later I drove by there again, and again saw the police blocking the entries. However this time the center lane was also closed and there were lots of employees of the postal service taking envelopes from passing cars. It produced quite a traffic jam in both directions and I was somewhat unnerved because it took like forever to get through there.

Suspecting something similar to the bomb incident (which luckily turned out to be a bad joke) I scanned the news and found that Postal workers were aiding tax filers on deadline.

Has anyone ever heard of such nonsense? Why could people possibly have to wait till the very last day to file there taxes? I would very much like to know how much fuel was spent uselessly on that road that day, because of all the vehicles waiting there, just because some folks could not have their taxes ready one or two days before...

Sunday, April 09, 2006

Mind Deprecation Warnings

One of our projects allows to "emergency dump" some database contents to XML files. There are multiple reasons for this, one of them being unreliable networks. However, that's not my point here.

When the code detects its needs to dump out data, it creates XML files that mainly contain base64 encoded CDATA regions, because this proved to be the least problematic way to handle certain types of content. At a later time that data needs to be reinserted into the database. This works just the other way round, calling the corresponding tool in a "reverse" mode to put the decoded base64 data back into the database.

The only thing the people using this stuff do not like is that they have no idea what's encoded in the base64 region. So I started to write a second output module that displayed the data from the XML to the screen instead of putting it directly into the database. To make it look nice I wanted to simply apply a stylesheet after having decoded the base64 in memory.

On my way there I used a java.io.StringBufferInputStream without thinking about it. When I then had the transformer begin its work, it started to complain about

SAXParseException: Character conversion error: "Unconvertible UTF-8 character beginning with ..." (line number may be too low)

This drove me almost crazy, because I did not see the problem, because the file was definitely correctly UTF-8 encoded. Only after two hours of surfing and searching it struck me like lightning: The StringBufferInputStream's constructor was underlined with a nice yellow line in Eclipse. Hovering that line Eclipse happily showed me the deprecation warning that This class does not properly convert characters into bytes. As of JDK 1.1, the preferred way to create a stream from a string is via the StringReader class.

And in fact, after I had removed the offending line, it worked like a charm. Basically what happens is described in the javadoc of the read-method: It returns the low eight bits of the next character in this input stream's buffer [...]. So instead of handing the XML parser the next valid character, it just tore the UTF-8 sequence apart, making the parser fail.

Again I see that deprecation warnings may sometimes be more than just warning; using the marked methods anyway can bring you serious trouble. However I do not get it that Sun tends to mark things deprecated and replace them with something else instead of repairing them in the first place. Backwards compatibility is one thing, but leaving the old and broken stuff lying around clearly seems "Microsoftesque".

Distaster Recovery

What's the difference between a backup and a restore? Right:

  • A backup is made to preserve a known-good state for later, after something has gone wrong (so called "disaster").

  • A restore is the backup being re-installed after something went wrong (so called "disaster recovery")

So far so good. But what about a restore that is installed when nothing went wrong and the server is just running nicely, handling lots of requests... Call that a "home made disaster". Yes: THAT'S REALLY BAD!

Last week we suddenly got lots of failure reports from one of our production systems (decentralized structure). People complained about all clients failing with somewhat random error messages. We first took the stacktraces and looked at the code in question to find out why it might suddenly start failing in places that had been working fine for months. We quickly realized that data corruption must be the cause. So the next step was to do some integrity checks on the database. It revealed that several hundred thousand records had vanished from a very important table (among other things, we only found out about later).

As our application does not provide any means of deleting records from that table, apart from deleting something else that has delete-cascade rules, we suspected either a hardware malfunction (raid controller, RAM etc.) or a severe bug in the application that triggered a lot of the cascading deletes.

In the end our theories turned out to be wrong, but let me elaborate a little further.

First thing we did was take the application server and the database offline and create a full copy of the MySQL data directory and the binlogs (the binary representation of every transaction that changed data in any way) just to be sure we could analyze later on. Then we began restoring the last full backup of the database from the night before. Importing this SQL dump took about two hours. After that we then tried to apply the binlogs in the correct order using the instructions from the MySQL documentation. Basically everything you do is pipe the binlog files through the mysqlbinlog tool and from there into the mysql command line client. The first step creates regular SQL statements that get then executed against the database.

Unfortunately this did not work very well. After a few seconds of work the database started to complain about lots of foreign key violations, because it could not find rows being referenced. It took us (and the MySQL support) some time to find out that the mysqlbinlog tool did not create a vital "set names utf8;" statement at the beginning of the transformation. That led to the data being interpreted as some variant of latin1 which in turn caused the broken references.

At that time we had to restore the full backup again (another two hours waiting), because we could not know how much of the binlogs had already been applied until the first visible errors occured. In the meantime we created one SQL file per binlog and prepended it with the "set names utf8;" statement to be able to replay them easier later.

When the full backup had finally been restored (again), we began applying the incremental binlogs (again...), this time without any problems. We restored the database to a point in time about half an hour before the error report. Just to be sure we took another full backup after that; in the meantime production had been resumed using some stand-alone systems that could perform the most critical tasks and gained us some time.

Still not knowing what had caused the problem (and still suspecting a cascading delete) we started rolling further forward, after each single binlog-SQL running the consistency checks again. We were very confident the error must be contained in the binlogs, because several read-only replication slaves had obediently recreated the corrupted database state.

We finally found the offending binlog; after it had been applied the consistency check failed. We took a closer look at the SQL generated from that binlog, hoping to find the delete statement that triggered the cascade. What we did find however was not a simple delete statement, but a much worse "set foreign_key_checks=0; drop table ..." statement. This really shocked us, because nothing like this is anywhere in the application code.

It turned out, that by accident somehow a restore-script had been triggered (we are still investigating the cause, but suspecting some sort of scheduled task the called the wrong script) and begun restoring a full backup in the middle of the day, knocking the application server of its feet.

In the end we were able to restore the data to a point in time about three minutes before the crash and identify what needs to be recreated manually.

Lesson learned: Do not place the restore script onto the production servers by default, but only bring them there when you really need them.

Wednesday, April 05, 2006

Adobe Reader without Ads

While Adobe Reader 7.0.7 seems to be faster than 6.0 it still has this nasty little ad-display in the upper right corner of the window. This annoys me very much, but I found this tweak to turn it off.

Launch regedit.exe and navigate to "HKEY_LOCAL_MACHINE/SOFTWARE/Adobe/Acrobat Reader/7.0/FeatureLockdown". Create a new DWORD value named "bShowAdsAllow". The default value "0" is ok, it disables the ad-display.

Head First HTML with CSS and XHTML

It's been a long time since I have actually written HTML pages... Back then ASP was state of the art (no flamewars please) and HomeSite was the editor of choice. Dreamweaver was version 2 and most image editing software was beginning to learn about GIFs and their transparency.

Since then a lot has happened, and while I have been doing some XSLs at work I somewhat lost contact with state of the art website building. When I started this blog I was pleased with the pre-defined themes from a purely aesthetic point of view; but looking at the sourcecode I was even more impressed. Ok, I know what CSS classes are, how to define a background color for all td's and some other things. But seeing what's possible and how elegantly so, I really feel the urge to try some of that again.

Incidentally I recently read a short summary about O'Reilly's new Head First HTML with CSS and XHTML and after reading the sample chapter I ordered a copy.

Maybe this page will look different in the future :)

As soon as I have read some of the book I will tell you about my impression with it.