Wednesday, February 20, 2008

Blogger Backup

Tonight for the first time I realized that all my posts to JRoller and Blogger are all just present on their respective websites, but not as a copy on my local machine or some other backup. While I have full trust in Google's engineers to keep Blogger up and running as well as my data safe, it is never a bad idea to make a copy of your stuff.

So I just googled for "Blogger Backup" and found just that: A Blogger Backup tool. It is a .NET based tool that utilizes Google's C# data access libraries. Probably it would have worked with Mono, but as I had a VirtualBox'ed XP instance up anyway I used it there.

Once I had downloaded and installed the tool, it was just a matter of a few clicks to get all my posts saved in Atom/XML files locally. One thing has to be noted though: Make sure your blog feed is set to "Full posts", not just the summary. I first got this wrong, so all the backup files just contained at most the first paragraph.

Now all 99 posts I posted to my blog reside in a local disk folder which makes me feel a little better :)

As for JRoller I so far have only found some special template based backup ideas, however none look really too appealing. Maybe I will just use a website grabber to snapshot the whole thing as HTML.

Sunday, February 10, 2008

Back to text mode

Over the past couple of weeks my machine repeatedly just froze - mouse not moving, keyboard dead. I do not know if it was just X11 crashing, because I do not have any remote services active on the machine, so I could not log in remotely. Just as a precaution I did not boot into X11, but now I am in text mode (80x25), backing up my home folder to an external hard disk. After that I will go for a fresh install of Ubuntu 7.10. The current install has seen upgrades from 6.06 to 6.10 to 7.04 to 7.10, spiced and seasoned with some manual tweaks, especially when something did not work in one of the previous Ubuntu versions. So I cannot really tell if something that was need back then now tends to destabilize my machine. It almost feels a little like this "again Windows syndrome" - where people claim that Windows just "needs a reinstall once a year" to remain fast and stable. I am a little reluctant to admit this, so a part of me hopes this is really a hardware problem; that would mean of course I would have to make a decision on staying with the PC and Linux or buy a Mac for the first time in my life... As the backup takes some time, I thought I might just post this here. I am writing with the elinks browser in plain VGA text mode. I did not really remember what a PITA this extremely limited amount of screen real estate is. Usually I would have put in links to my earlier posts about the Toshiba external hard drive (which I finally switched to HDD only mode, hence the new full backup) and the Ubuntu upgrade articles. However I do not even have mouse support here, and copying over the post URLs by hand from another console is a little too tedious. Right now I feel like I am back to DOS - only back then the monitor was a little smaller. I can definitely tell I have no desire to go back to a real text mode interface ever again. Don't get me wrong - I really love command line interfaces. But as one of my professors once said: "Graphical user environments' sole purpose is to provide multiple terminal windows with good fonts and a nice background image.". While I would not go that far, right now I would really like to have just that :)

Friday, February 08, 2008

String.replaceAll() caveat

Sometimes I wonder how it is possible to write Java programs for as long as I have now and still stumble across little gotchas I did not know about.

When writing a class that is intended as a utility to write well-formed queries for our persistence framework, I was bitten by a nice little subtlety of the String class. Just a little background: In our application we use a persistence framework with a SQL-like (but still different) query language. Statements in that language are easier to read because they are closer to the business object model, but in the end they get translated into "real" SQL.

Unfortunately the query parser is based on simple query strings without support for something like prepared statements which makes it susceptible to injection attacks if you put user-entered values into a query directly. While there is a way to escape the input accordingly it is a little cumbersome and makes the code to assemble the query conditions into the final String hard to read.

I decided to build a little wrapper that would allow support for a sort of prepared statements by having people put placeholders into their queries and escape the values upon insertion.

The code for such a query might look like this:

StmtHelper h = new StmtHelper("SELECT firstname FROM customer WHERE name == #n#");
h.replace("n", txtfield.getText());
String queryString = h.assemble();
Inside the StmtHelper class' "replace" method I first escape the parameter value as needed and store it with its placeholder "n" in a map. Upon "assemble" I create a new String based on the template passed to the constructor where all placeholders are replaced with their escaped values.

This is the "assemble" method:

public String assemble() {
String result = template;
for (Map.Entry<String, String> entry : replacements.entrySet()) {
 result = result.replaceAll(entry.getKey(), entry.getValue());
}
return result;
}
At first glance this does what it is supposed to do. Unfortunately there is a bug in this code that will only show its ugly face with proper values to be filled in. For example this code will lead to a problem at runtime:
StmtHelper h = new StmtHelper("SELECT firstname FROM customer WHERE name == #n#");
h.replace("n", "O'Reilly");
String queryString = h.assemble();
This should lead to a query string as follows:

SELECT firstname FROM customer where name = 'O\'Reilly'

The single quote in the name must be escaped with a backslash. This is taken care of inside the "replace" method and works reliably - inspecting the replacement Map proves that. Nevertheless the final query string will be wrong: The single quote will not be escaped, effectively breaking the syntax for this particular query and name, but practically opening a gaping security hole for injection attacks:

SELECT firstname FROM customer where name = 'O'Reilly'

I was rather surprised when I came across this, because the replacements worked in all the test cases I had built into the JUnit tests. Unfortunately those did not contain a replacement value with a single quote, even though this is of course one of the first things someone would try in an attack... Looking into this more closely I learned something about String.replaceAll() I had never noticed before. This is from the Java 1.5 API documentation:

Replaces each substring of this string that matches the given regular expression with the given replacement. An invocation of this method of the form str.replaceAll(regex, repl) yields exactly the same result as the expression Pattern.compile(regex).matcher(str).replaceAll(repl)

IMHO this does not suggest anything special concerning the replacement, as long as the regular expression for the first parameter is valid. In my case this was just a simple name surrounded by hash characters, so no problems to expect here. However the backslash in front of O'Reilly's single quote disappears after the call to "replaceAll", leaving the quote unprotected in the resulting String.

Only looking at the Java 6 documentation makes it more clear:

An invocation of this method of the form str.replaceAll(regex, repl) yields exactly the same result as the expression Pattern.compile(regex).matcher(str).replaceAll(repl) Note that backslashes (\) and dollar signs ($) in the replacement string may cause the results to be different than if it were being treated as a literal replacement string; see Matcher.replaceAll. Use Matcher.quoteReplacement(java.lang.String) to suppress the special meaning of these characters, if desired.

That added paragraph suggests that more than one person must have fallen into that trap. While in the Java 5 version of the docs there was a mention of the result being exactly the same as if you used a Pattern/Matcher combination, at least to me it was not obviously hinting at a special behavior concerning the replacement string, but only the first parameter called "regex".

Strangely enough only after that experience I noticed that I must have been lucky all the time before, because I practically always used the "replaceAll()" method instead of the simple "replace()" I not use to replace the key/value pairs. As I do not really need the regular expression capabilities for the search term, this also saves some overhead.

Monday, February 04, 2008

Sun javac / Eclipse compiler difference

Our primary development environment is Eclipse 3.3.1.1, set to Java 5 compatibility. However during the nightly builds we use Sun's javac, currently version 1.5.0_12 I believe.

Today I noticed one of the builds had failed in this code (reduced to a small test case):

public interface Task<V> {}
public interface RunnableTask extends Runnable, Task {}
public class ProgressMonitorUtil {
public <V> V execute(final Task<V> aBackgroundTask) {
return null;
}
public void execute(final RunnableTask aBackgroundTask) {}
}
public class Test {
public static void main(String[] args) {
new ProgressMonitorUtil().execute(new RunnableTask() {
    public void run() {}
});
}
}

Put these classes into an Eclipse project, either with Java 5 or Java 6 compatibility settings. You will see no problems. However compiling this with Sun's javac (also either 5 or 6 will do) leads to this:

Test.java:4: reference to execute is ambiguous, both method <V>execute(Task<V>) in ProgressMonitorUtil and method execute(RunnableTask) in ProgressMonitorUtil match
new ProgressMonitorUtil().execute(new RunnableTask() {

Note: Test.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
1 error

Apparently method dispatching works differently in both compilers. In Eclipse the call goes to the second method (void execute...) which seems reasonable since RunnableTask is a more specific type.

However you can still convince Sun's compiler to compile this if you get rid of the "unchecked" warnings:

public interface RunnableTask extends Runnable, Task<Void> {}
public class ProgressMonitorUtil {
public <V> V execute(final Task<V> aBackgroundTask) {
return null;
}
public void execute(final RunnableTask aBackgroundTask) {}
}
public class Test {
public static void main(String[] args) {
    new ProgressMonitorUtil().execute(new RunnableTask() {
        public void run() {}
    });
}
}

This compiles without warnings or errors on both compilers and leads to the same dispatching that Eclipse does. I guess I will post this as an Eclipse bug nevertheless, because responses there are usually way quicker than at Sun...

Edit: Making RunnableTask generic is not very useful in this context, but instead just declaring it as "extends Task<Void>" is sufficient as well. I updated the samples above.