Mike's Software Development Blog

The Java Collections Framework - Queues

2012-01-28T14:58:00.000-05:00

The Collections Framework is a group of classes that allow you to group objects together in various ways and perform operations against those groupings. It is contained within the core Java API, so you don't have to install anything extra to use it. Its features are so basic and powerful that pretty much every piece of software written in Java makes use of this framework in one way or another. The framework has been around since Java 1.2 (1998) and has been improved upon with every subsequent Java release. There are four major collection types: Lists, Sets, Queues, and Maps.

The four basic collection types (List, Set, and Queue all inherit from the Collection interface).

A queue is an ordered collection of elements in which only the "top" element can be accessed. When the top element is removed, or popped, the next element is revealed. The order in which the queue's elements are arranged depends upon the type of queue. Also, note that the term push is used to refer to an element being added to a queue.

A FIFO (first in, first out) queue orders its elements according to how recently they were added to the queue--elements that were pushed first will be popped first.
A LIFO (last in, first out) queue, also known as a stack, is the opposite of a FIFO queue--elements that were pushed first are popped last.
There's also a priority queue, in which each element has a priority associated with it. The elements with the highest priorities are popped first.

A dequeue (double-ended queue) allows the programmer to push and pop elements from the bottom of the queue as well as from the top.

"Exception vs. special return value" convension

The Java Collections Framework has a number of implementations for different types of queues (all inheriting from the Queue interface). They all follow a special convension: for each operation that you can perform on the queue, there are two methods. Each method does the same thing, but they differ in the way in which they handle edge cases. One method will throw an exception, while the other will return a special value. Here are some examples of these kinds of methods from the Queue interface.

add() - pushes an element on the queue, throwing an IllegalStateException if the queue is full.
offer() - pushes an element on the queue, returning false if the queue is full.
remove() - pops the next element off the queue, throwing a NoSuchElementException if the queue is empty.
poll() - pops the next element off the queue, returning null if the queue is empty.

Example

Here's an example of how to program a queue in Java. This program simulates a drive-thru window at a fast-food restaurant. Customers place their orders at the drive-thru window and then the kitchen makes their food. The kitchen makes the meals in the same order in which they were received, so it is a FIFO queue (if you are the first car in line, you will be the first to receive your order).

I use a BlockingQueue, which extends the Queue interface. It provides the ability to block the method call if you try to pop from an empty queue or push to a full queue. I use the take() method to pop the next element off the queue. If the queue is empty, then it waits until an element is pushed onto the queue before executing the next line of code.

The BlockingQueue imlpementation I use is the ArrayBlockingQueue, which requires that a max queue size be defined. In the case of my drive-thru simulator, this is the maximum number of cars that can fit into the driveway. I use the offer() method to push orders onto the queue. If this method returns false then it means that the driveway is full of cars (the queue is full). The would-be customer then has to drive away and find someplace else to get their food (nothing is pushed to the queue).

import java.text.MessageFormat;
import java.util.*

/**
 * Drive-thru simulator for a fast-food restaurant.
 * @author Mike Angstadt - http://www.mangst.com
 */
public class DriveThru {
  private static Random rand = new Random();

  public static void main(String args[]) throws InterruptedException {
    //the number of cars that can fit in the driveway
    int drivewaySize = 5;

    //the food orders are placed in a FIFO queue with a max capacity
    BlockingQueue<String> orders = new ArrayBlockingQueue<String>(drivewaySize);

    //the drive-thru window takes orders from the customers and pushes those orders onto the queue
    DriveThruWindow driveThruWindow = new DriveThruWindow(orders);
    driveThruWindow.start();

    //the kitchen pops the orders off the queue
    Kitchen kitchen = new Kitchen(orders);
    kitchen.start();
  }

  private static class Kitchen extends Thread {
    private BlockingQueue<String> orders;

    public Kitchen(BlockingQueue<String> orders) {
      this.orders = orders;
    }

    @Override
    public void run() {
      int orderNum = 0;
      while (true) {
        try {
          //get the next order
          //if the queue is empty, then wait until an element is pushed onto it
          String order = orders.take();

          //make the food
          System.out.println("Order " + orderNum + ": Making order \"" + order + "\"...");
          Thread.sleep(rand.nextInt(9000) + 1000); //sleep for 1-10 seconds
          System.out.println("Order " + orderNum + ": Order \"" + order + "\" finished.");
          orderNum++;
        } catch (InterruptedException e) {
          break;
        }
      }
    }
  }

  private static class DriveThruWindow extends Thread {
    private List<MenuItem> menu = new ArrayList<MenuItem>();
    private int orderNum = 0;
    private BlockingQueue<String> orders;

    public DriveThruWindow(BlockingQueue<String> orders) {
      this.orders = orders;

      //create the menu
      MenuItem menuItem = new MenuItem("Quarter-pounder", new String[] { "cheese", "tomatoes", "onions", "pickles" }, "{0} with {1}");
      menu.add(menuItem);
      menuItem = new MenuItem("coke", new String[] { "Small", "Medium", "Large" }, "{1} {0}");
      menu.add(menuItem);
      menuItem = new MenuItem("chicken nuggets", new String[] { "6", "10", "20" }, "{1}-piece {0}");
      menu.add(menuItem);
      menuItem = new MenuItem("Apple pie");
      menu.add(menuItem);
    }

    @Override
    public void run() {
      while (true) {
        //wait for the next car
        try {
          Thread.sleep(rand.nextInt(6000) + 1000); //sleep for 1-7 seconds
        } catch (InterruptedException e) {
          break;
        }

        //get the customer's order
        int index = rand.nextInt(menu.size());
        String order = menu.get(index).generate();

        //push the order onto the queue
        boolean inserted = orders.offer(order);
        if (inserted) {
          System.out.println("Order " + orderNum + ": Car placed order for \"" + order + "\".");
          orderNum++;
        } else {
          System.out.println("Driveway full...customer lost!");
        }
      }
    }
  }

  private static class MenuItem {
    private final String name;
    private final String[] modifiers;
    private final String format;

    public MenuItem(String name) {
      this(name, null, null);
    }

    public MenuItem(String name, String[] modifiers, String format) {
      this.name = name;
      this.modifiers = modifiers;
      this.format = format;
    }

    public String generate() {
      if (format == null) {
        return name;
      } else {
        int modifier = rand.nextInt(modifiers.length);
        return MessageFormat.format(format, name, modifiers[modifier]);
      }
    }
  }
}

The Java Collections Framework - Sets

2012-01-26T23:14:00.000-05:00

The four basic collection types (List, Set, and Queue all inherit from the Collection interface).

Another type of collection in the Java Collections Framework is the set. A set is a collection of elements where duplicates are not allowed. If the programmer tries to add a duplicate element, the set will see that the element already exists and nothing will be added. Sets also do not allow random access like lists do. The only way to retrieve elements from a set is to iterate over all the elements using an Iterator or a foreach loop (so, for example, you cannot directly get the "fourth" element of a set).

Buckets

Sets are really fast when it comes to determining whether an element exists in the set or not. The way sets do this is by making heavy use of the Object.equals() and Object.hashCode() methods. When an object is added to the set, the set uses the object's hash to find a bucket to store the object in. Later, when the set is asked if it contains a given object, the set uses the hash of the given object to find a bucket, and then iterates over each element in that bucket. It calls the equals method on each element in the bucket to determine if the given object is the same as any of the elements in the bucket. This is a lot faster than a list because a list has to iterate over every single element in order to determine if it contains an object. Sets just have to iterate over the elements in a single bucket, which contains only a fraction of the total number of elements in the set.

Importance of hashCode()

However, the efficiency of the set's search operation relies heavily on the quality of the object's hashCode() implementation. If the implementation is good, then the set will spread all of its elements evenly amongst all buckets. However, if the implementation is not good, then the elements will not be spread evenly amongst all buckets. This will cause some buckets to be larger than others, which means they will take longer to search over and decrease the overall performance of the set.

Probably the worst possible hashCode() implementation you could create is one that returns a hard-coded value. It's a completely valid implementation, but it will cause all elements to be stored in a single bucket, making the set's search performance no better than that of a list.

public class BlogPost {
  private String content;
  private Date created;
  private List<String> comments;

  @Override
  public int hashCode(){
    //the worst-possible implementation
    //DO NOT USE!!
    return 1;
  }
}

What you want to do is create a hashCode() implementation that will generate a wide range of values. Luckily, if you use Eclipse (or probably any other IDE), you can have one generated for you. In Eclipse, right-click on the code and go to "Source > Generate hashCode() and equals()". It will prompt you for what class fields you want to include in the hash calculation and then generate the proper code automatically.

public class BlogPost {
  private String content;
  private Date created;
  private List<String> comments;

  @Override
  public int hashCode() {
    //the implementation generated by Eclipse (much better)
    final int prime = 31;
    int result = 1;
    result = prime * result + ((comments == null) ? 0 : comments.hashCode());
    result = prime * result + ((content == null) ? 0 : content.hashCode());
    result = prime * result + ((created == null) ? 0 : created.hashCode());
    return result;
  }
}

Relationship between hashCode() and equals()

Note that Java expects the equals() and hashCode() methods to follow certain rules (read this three times to let it sink in):

If two objects are equal, then they must have the same hash.
But if two objects have the same hash, they are not necessarily equal.

It is your responsibility as a programmer to follow these rules. If you do not, then sets (as well as other collection types from the Collections Framework) will not function correctly. The code that Eclipse generates complies with these rules.

Example

Here's a short code example of how to use a set in Java.

Set<String> users = new HashSet<String>();
users.add("Mark");
users.add("Steve");
users.add("Mark");
users.add("Kelly");
System.out.println(users.contains("mark"));
System.out.println(users.contains("Mark"));
for (String user : users){
  System.out.println(user);
}

On line 1, I create the Set object. I'm assigning the object to a variable of type Set instead of type HashSet so that I can more easily change implementations if I ever need to in the future.

On lines 2-5, I add elements to the set. On line 4, I try adding a duplicate element, but nothing will be added because sets do not allow duplicate elements. No exception will be thrown or anything--the add() method will just return "false" instead of "true".

Line 6 will print "false" because the String "mark" is not the same as the string "Mark". Line 7 will print "true".

On lines 8-10, I iterate over the entire set, printing each user name. This is the only way to retrieve elements from a set.

Set Implementations

There are three implementations of the Set interface. They differ in the way in which they iterate over their elements.

HashSet - The most basic set implementation, iterates over elements in no particular order.
LinkedHashSet - Iterates over elements in the order in which they were added to the set.
TreeSet - Iterates over elements in their sorted order. The way in which the elements are sorted can be defined by having the elements implement the Comparable interface, or by passing in an implementation of the Comparator interface to the TreeSet constructor.

The Java Collections Framework - Lists

2012-01-23T08:33:00.000-05:00

The four basic collection types (List, Set, and Queue all inherit from the Collection interface).

In this blog post, I'll discuss the List type. Lists are defined as being an ordered collection of elements where duplicate elements are allowed. Probably the most used list implementation (and probably the most used class in the entire Collections Framework) is the ArrayList class. This class is meant to behave just like an array, with the added ability of being able to add, insert, and remove elements (which cannot be done with normal arrays).

List<String> names = new ArrayList<String>();
names.add("Mark");
names.add("Steve");
names.add("Jill");
names.remove(1);
names.remove("Jill");
names.add(0, "Zach");
Collections.sort(names);
System.out.println(names);

On line 1, I create a new instance of an ArrayList. Using generics, I specify that the list will only contain String objects (generics support was added to the Collections Framework in Java 1.5). If I try to add anything other than a String to my ArrayList, a compilation error will be thrown.

Also notice how I'm assigning my ArrayList object to a variable of type List instead of type ArrayList. List is an interface that ArrayList implements. The reason for doing this is that there are multiple implementations of the List interface. By assigning my ArrayList to a variable of type List, I'm making my code more flexible so that it's not tied to a specific List implementation. The only thing the code cares about is that the List variable satisfies the definition of what a "list" is supposed to be within the Collections Framework (an ordered collection where duplicates are allowed). It is good practice to apply this technique throughout the Collections Framework to maximize code flexibility.

On lines 2-4, I populate my list by adding some elements to it. The add() method appends the element to the end of the list.

On line 5, I remove an element by specifying the element index (Lists are 0-based just like normal arrays, so "0" corresponds to the first element, "1" to the second, etc). This will remove the "Steve" element. Then, on line 6, I remove an element by passing in a String object. When an element is removed in this way, it iterates over each element in the list, using the object's equals() method to determine which element to remove. If it doesn't find any such element, nothing is removed.

On line 6 of the code sample, I insert an element into the List at a specific index. The index I specify is "0", which means it will be inserted at the beginning of the list.

Then, on line 7, I sort the list so that the names are ordered alphabetically. This method requires that the class of the elements in the list implement the Comparable interface, which defines how two objects of that class are compared against each other when they are sorted. The String class already implements this interface, defining that Strings be sorted alphabetically. If you want to sort the list differently, you can pass your own implementation of the Comparator interface as a second argument of the sort() method. For example, to sort the strings by length, you could write:

Collections.sort(names, new Comparator<String>(){
  @Override
  public int compareTo(String element1, String element2){
    //sort Strings by length
    if (element1.length() < element2.length(){
      return -1; //"element1" should come before "element2"
    } else if (element1.length() > element2.length()){
      return 1; //"element1" should come after "element2"
    } else {
      return 0; //"element1" and "element2" are equal
    }
  }
});

Finally, on line 8, I print the list out so I can see what it looks like. Passing an object into the System.out.println() method will print the return value of that object's toString() method. The list's toString() method generates a string representation of the list by calling the toString() method of every list element. It surrounds the entire list in brackets and separates each element with a comma.


[Mark, Zach]

Adding a StatCounter tracker to your website

2012-01-14T13:50:00.000-05:00

StatCounter is a free service that tracks the number of hits your website receives. It records a wealth of information about your visitors, including the type of browser they use, their screen resolution, and their geographic location. It can also tell you whether the user came to your website from a search engine, showing you the exact search query that the user entered.

Adding the tracking script to your site

After creating a StatCounter account, click the "Add Project" link in the menubar at the top of the screen. Fill in all the information and select "Invisible tracking" for the counter type on the right side of the page. This option will make the counter invisible to visitors of your webpage. I use this option because it makes my website look more professional.

The "Invisible tracking" option will make your counter invisible.

It then brings you to a page that contains a long list of different installation instructions for a variety of platforms, frameworks, blogs, and hosting services. Choose the one that best fits your particular setup. If you have a plain, "vanilla" website that doesn't really use anything extra, then click the "Default Installation Guide" link on the right. That's what I'll be doing.

Use the default installation guide if none of the other guides apply to your setup.

The next page displays the code that you'll need to add to your website. The code inside the "Standard" tab is the best overall choice. If you're a stickler for writing websites in well-formed XHTML, use the code in the "Standard (xhtml)" tab. If you don't want to use Javascript on your website, use the code in the "Basic" tab. And if you want your code to be both XHTML-compliant and Javascript-free, use the code in the "Basic (xhtml)" tab. Note that the "Basic" versions do not record as much information about the visitor. Javascript has to be used in order to get certain information, like screen resolution and geographic location.

The "Standard" option is the recommended choice.

I'm going to use the "Standard" code. Copy and paste the code right before the </body> tag on your website. By placing the code here, the StatCounter script won't run until the entire page is loaded. If it were to be placed "higher up" in the HTML, like in the <head> tag, then the rest of the page wouldn't load until the StatCounter script finishes downloading and executing. Sometimes, this can take a few seconds, so it's best to put the script at the end of the page so it doesn't keep the rest of the page from loading.

Once you've added the code to your website, you can check to make sure it works by clicking the "Check Installation" button at the bottom of the installation guide page.

StatCounter can check to make sure you've installed the code correctly.

Creating a blocking cookie

Lastly, you're going to want to add a blocking cookie to your computer. This cookie will prevent your own visits to your website from being tracked so that the project statistics aren't skewed by your own activity. Go to the main "Projects" page and then click the "Blocking Cookie" link in the menu bar at the top of the screen. Then, click the "Create Blocking Cookie" button. This will create a blocking cookie that applies to all of your StatCounter projects.

A blocking cookie will prevent your own visits from being recorded.

Remember that if you have multiple browsers on your computer, you'll have to do this for each browser, because each browser has its own set of cookies.

GWT Image Bundles

2012-01-08T14:18:00.000-05:00

Often times, websites contain a number of small, icon-sized images. These images don't take long to download individually, but collectively, they can slow down the load time of a webpage because one HTTP request has to be made for every image. Plus, the HTTP protocol states that browsers can only make two simultaneous requests per domain at a time. This means that no matter how fast your Internet connection is, only two images can be downloaded at a time.

CSS image sprites help to get around this bottleneck. An image sprite is a single image that has multiple images inside of it. CSS styling is used to "cut out" an individual image from the image sprite and display it on the page. By consolidating all the images into a single image, the browser only has to make one HTTP request, as opposed to making multiple HTTP requests for each image. This decreases the load time of the page.

An image sprite from Amazon.

GWT gives you the ability to combine your application's icons into an image sprite automatically. In this tutorial, I'm going to show you how this is done by adding some images to my Twitter Search application. I will be adding:

An icon for the "Search" button.
An icon for the "Privacy Policy" button.
An icon to display along with error messages.

The TwitterSearch application with the new images.

Creating an image bundle interface

First, create an interface that extends the ClientBundle interface. This is called an image bundle and it's what we'll use to access the individual images from our image sprite.

package com.acme.twittersearch.client;

import com.google.gwt.resources.client.ClientBundle;
import com.google.gwt.resources.client.ImageResource;

public interface Images extends ClientBundle {
  @Source("search.png")
  ImageResource search();

  @Source("error.png")
  ImageResource error();

  @Source("document.png")
  ImageResource privacyPolicy();

  @Source("loading.gif")
  ImageResource loading();
}

The image bundle contains one method per image. Each method must return a ImageResource object and can be given any name. Each method is also annotated with the path to the image file. This path is relative to the location of the image bundle interface. Since my image bundle is in the com.acme.twittersearch.client package, this means that all my images are there too (the images cannot be in the war directory, they have to be inside the classpath, alongside the source code). The GWT Eclipse plugin will tell you if the path to an image is incorrect by throwing a Java compilation error.

I also add the loading animation image to my image bundle. Normally, animated GIFs cannot be added to image sprites, but GWT still allows you to add them to image bundles. They won't be added to the image sprite that GWT generates, but programmatically, they are treated as if they were part of the image sprite.

Adding the images to the application

Now that I've created my image bundle, I can add the images to my application. First, I'll create an instance of the image bundle via deferred binding:

public class TwitterSearch implements EntryPoint {
  private final Images images = GWT.create(Images.class);
  [...]
}

Instances of ImageResource can be passed into the constructor of the Image class, so that's what I'll do for the error and loading images.

public class TwitterSearch implements EntryPoint {
  [...]
  private Image errorImage;
  private Image loadingImage;
  [...]

  private void createWidgets() {
    [...]
    errorImage = new Image(images.error());
    loadingImage = new Image(images.loading());
    [...]
  }
}

The "search" and "privacy policy" images are a little more complicated because of the way the Button widget works. After creating an Image object for each icon, I generate raw HTML code, which will be passed into the Button constructor (note: calling the toString() method on any widget object will return its HTML code).

public class TwitterSearch implements EntryPoint {
  [...]
  private Button privacyPolicyButton;
  private Button searchButton;
  [...]

  private void createWidgets() {
    [...]

    Image icon = new Image(images.search());
    SafeHtmlBuilder builder = new SafeHtmlBuilder();
    builder.appendHtmlConstant(icon.toString());
    builder.appendEscaped(" Search");
    searchButton = new Button(builder.toSafeHtml());
    searchButton.addClickHandler(new ClickHandler() {
      [...]
    });

    icon = new Image(images.privacyPolicy());
    builder = new SafeHtmlBuilder();
    builder.appendHtmlConstant(icon.toString());
    builder.appendEscaped(" Privacy Policy");
    privacyPolicyButton = new Button(builder.toSafeHtml());
    privacyPolicyButton.addClickHandler(new ClickHandler() {
      [...]
    });

    [...]
  }
}

And that's really all there is to it! GWT will create the actual image sprite for you. To see what the image sprite looks like, compile the GWT application and then navigate to the war/twittersearch folder. The image sprite will be somewhere inside this folder (it will have a crazy name like "F0B5712E038983254B46645D77476DA7.cache.png").

GWT automatically creates an image sprite from the images in our image bundle.

More information on GWT image bundles can be found in the GWT Developer Guide.

The complete source code from this tutorial can be downloaded here.

CORRECTION (1/12/2011): Even though GWT generates an image sprite from an image bundle when compiled, it actually only uses the image sprite for IE 6 and possibly IE 7 (thanks Boris). What GWT really does is encode the images as data URIs. A data URI puts the actual image data directly inside the HTML page, instead of referencing the image via a URL like normal (I wrote a brief blog post on data URIs a while back and also wrote a data URI generation tool). I'm not sure why GWT does this, but I'm sure it has its reasons. You should avoid using data URIs for large images--since the image data is embedded directly inside the page, the browser can't cache it, so the image basically has to be downloaded every time the page is loaded. --Mike

Custom border images in the Highslide Editor

2012-01-02T19:24:00.000-05:00

Highslide is a Javascript library used for creating "lightboxes", or Javascript-based popups, for your website. The Highslide website has an editor that you can use to customize the look and feel of your lightbox. It's actually pretty powerful, allowing you to customize everything from the popup animation to the font size of the image caption. Once you've customized the lightbox to your liking, you can view the HTML, CSS, and Javascript code required to make it work on your own website.

However, if your lightbox has the "graphic outline" setting enabled (this setting can be accessed by clicking the "Border and outline" button on the "Style" tab), then the hs.outlineType configuration variable in the Javascript code will be set to custom.

The "custom" outline type will not work with your own website.

This is not an outline type that comes with the Highslide installation, so if you try using this on your own website, it won't work. What the Highslide Editor does behind the scenes, is make an AJAX call to a custom.php script, which dynamically generates a border image based on the settings you've defined in the editor. As far as I can tell, the editor doesn't provide a way for you to download this image, so you have to get the URL to the custom.php script yourself, and download the image that way. Here's how to do that using Chrome:

Click the Wrench button, then go to "Tools > Developer Tools". This opens a window that shows additional information about the current browser tab.
Opening the "Developer Tools" window in Chrome
Click the "Network" button at the top of the screen. This screen displays all HTTP requests that the browser tab makes, including the images that it downloads and the AJAX requests that it makes. When you first open this screen, it will be blank. But if you refresh the Highslide Editor webpage, the Network screen will flood with all the requests the browser has to make in order to load the page.
The "Network" screen shows all the HTTP requests that were made.
Now, click the sample image in the Highslide Editor so that it expands into a popup. This causes the browser to make an AJAX request to the custom.php script (for some reason, it makes the same request twice).
An AJAX request to the custom.php script is made when the sample image is clicked.
Right-click on the custom.php request in the Network screen and select "Open Link In New Tab". This will open the border image in a new browser tab, allowing you to save it to your local Highslide installation.
Load the image in a new tab.
Finally, save the border image to the "graphics/outlines" folder of your Highslide installation. Whatever name you give the file (minus the file extension) will be what you'll use in the hs.outlineType setting. For example:
```
//uses the "graphics/outlines/rounded-corners.png" file
hs.outlineType = 'rounded-corners';
```

Making an image transparent with GIMP

2012-01-01T22:08:00.000-05:00

I'm not a graphic designer, so if I need an image for my website, like a loading animation or an "error" icon, I'll often just do a Google image search. One quality that I look for in images is that they have transparent backgrounds. Images with transparent backgrounds tend to look nicer because they allow the background color or background image of the website to show through the image. Images without such backgrounds are surrounded by an ugly white square.

An image without a transparent background.

However, it is possible to add transparency to a non-transparent image. I'll show you how to do this using the free, open-source GIMP image editor.

After opening the image, the first thing to do is add an alpha channel. This is a special layer in the image that defines exactly where the image is transparent. Go to "Layer > Transparency > Add Alpha Channel"
Adding an alpha channel.
Next, select the parts of the image that you want to make transparent. This can be done using the Fuzzy Select Tool. This tool will select all parts of the image that are similar in color to the part of the image that you click on. You can increase or decrease the sensitivity of this tool by adjusting the Threshold setting (this is helpful if the background is similar in color to the foreground).
The Fuzzy Select tool
Then, go to "Edit > Clear". This will delete everything that you've selected with the Fuzzy Select Tool and expose the transparency layer.
The image after the background was cleared.
You're now ready to save your changes. If the image is a JPEG, you'll have to save it in a different format because JPEGs do not support transparency. PNG supports all levels of transparency, so, for example, you can make one part of your image 100% transparent and another part 30% transparent. The GIF format supports complete transparency, but not partial transparency like PNG does. GIF images do not support many colors, however, so if you are making a JPEG image transparent, it's best to save it as a PNG because JPEGs tend to have a lot of colors in them.

The webpage background can be viewed through transparent images.

A Javascript script with query string parameters?

2011-12-19T21:30:00.000-05:00

TopUp is one of the more popular so-called "lightbox" libraries that allows you to create Web 2.0 popups for tasks such as viewing full resolution images and playing videos. It works like this: Instead of opening a new browser window, like traditional popups do, TopUp uses Javascript to modify the webpage's DOM and display a popup window inside the webpage. This creates a more fluent user experience because it doesn't push the user away from the webpage they're currently viewing, like traditional popups tend to do.

Lightboxes are popups that are part of the webpage itself.

I was reading the library's documentation and noticed that you can pass configuration settings via query string parameters in the URL to the Javascript file. For example, to enable "fast mode" (a setting that improves support for IE 6 and 7), you would import the TopUp script into your page like so:

<script
  type="text/javascript"
  src="path/to/top_up-min.js?fast_mode=1">
</script>

The question I asked myself was, "How are they passing query string parameters into a Javascript file?" If top_up-min.js was really a server-side script that returned Javascript code, then I could understand. But it's an ordinary, plain-text .js file! The file can't be aware of its own URL, so it can't see the parameters!

I opened up the source code to have a look. The author queries the DOM to get a reference to the <script> element that imported the file, and then parses the element's src attribute to get the query string parameters. Pretty clever!

But how does he get a reference to the <script> element? There's nothing unique about this element that you can use to pull it out from the DOM. No unique ID. No specially-named class attribute. How does he do it? What he does is he adds a dummy element to the DOM as soon as the script loads. This guarantees that the dummy element will be added immediately after the <script> element, so he can just get the dummy element's previous sibling and wham...you've got your <script> node.

var scriptElement = (function deriveScriptElement() {
  var id = "tu_dummy_script";
  document.write('<script id="' + id + '"></script>');

  var dummyScript = document.getElementById(id);
  var element = dummyScript.previousSibling;

  dummyScript.parentNode.removeChild(dummyScript);
  return element;
}());

Some people say that programming is not a skill that requires creativity. I would ask these people to look at this code sample and tell me that it's not creative!

GWT History Mechanism

2011-12-15T19:01:00.000-05:00

GWT provides a way to let you interact with the history of the client's browser. This helps to better integrate your GWT application with the browser and provide a more streamlined experience to the user. It allows the application to respond to the user clicking the "back" and "forward" buttons.

In this tutorial, I'm going to add history support to my TwitterSearch application that I created in some previous blog posts. This is a simple app that allows the user to perform tweet searches, as well as view the Twitter privacy policy. By adding history support, the user will be able to navigate back to searches that she made previously. Feel free to download the complete source code so that you can better follow along with this blog post.

What the TwitterSearch application looks like

Because GWT apps never navigate away from the main, HTML page, the way GWT implements history is by adding a fragment identifier to the URL. The fragment identifier starts with a hash (#) and comes at the end of the URL. You often see fragment identifiers being used on large webpages. It allows you to jump around to different parts of the same page by clicking links that have fragment identifiers in their URLs. GWT, however, uses them to identify the "state" that the application was in at a specific point in history.

Enable history support

The first thing to do is make sure your GWT application has history support enabled. This is done by adding an <iframe> element to the HTML page (if you've created your GWT application using Eclipse, then it will be enabled by default):

<html>
  [...]
  <body>
    <iframe src="javascript:''" id="__gwt_historyFrame" tabIndex='-1' style="position:absolute;width:0;height:0;border:0"></iframe>
    [...]
  </body>
</html>

Adding history tokens

Next, we must figure out when to add entries to the browser's history. In our case, we want to add an entry when the user performs a search and when the user views the privacy policy.

searchButton = new Button("Search");
searchButton.addClickHandler(new ClickHandler() {
  @Override
  public void onClick(ClickEvent event) {
    String query = searchQueryTextBox.getText();
    History.newItem(query);
    doSearch(query);
  }
});

privacyPolicyButton = new Button("Privacy Policy");
privacyPolicyButton.addClickHandler(new ClickHandler() {
  @Override
  public void onClick(ClickEvent event) {
    History.newItem("_privacyPolicy");
    doGetPrivacyPolicy();
  }
});

Here, we're adding a new history token when the "Search" and "Privacy Policy" buttons are clicked by calling the static History.newItem() method (new instances of the History class are never created--only its static methods are used). This method simply takes a String as an argument, which is the text that will appear in the fragment identifier. We use the search query as this string for tweet searches, and we use the hard-coded string _privacyPolicy for when the privacy policy is viewed. The doSearch() and doGetPrivacyPolicy() methods call the Twitter API and then update the UI with the results (download the complete TwitterSearch project to see how this is done, or read my previous blog posts).

Try performing a search now--you'll see the search query appear in the fragment identifier of the URL. But if you click "back", nothing will happen! We must explicitly tell our application how to respond to these events.

The history token is added to the fragment identifier in the URL

Handling history change events

Now, let's handle what happens when the user clicks the "back" or "forward" buttons. We'll do this in our onModuleLoad() method, so that the handler we define is registered as soon as our app loads.

@Override
public void onModuleLoad() {
  History.addValueChangeHandler(new ValueChangeHandler<String>() {
    @Override
    public void onValueChange(ValueChangeEvent<String> event) {
      String token = event.getValue();
      if ("_privacyPolicy".equals(token)){
        doGetPrivacyPolicy();
      } else {
        searchQueryTextBox.setText(token);
        doSearch(token);
      }
    }
  });

  [...]
}

As you can see, we call the History.addValueChangeHandler() method and pass in an implementation of the ValueChangeHandler interface. This handler will be invoked every time the user goes back or forward in their browser history. In the handler's only method, onValueChange(), we get the history token (which is the fragment identifier in the URL), and then use it to update our UI accordingly.

Handling history tokens on app startup

Besides giving the user the ability to use "back" and "forward", another good thing about using history is that the user can save bookmarks of our app. For example, a user might want to frequently check Twitter for news on her favorite celebrity, Brad Pitt. She might want to bookmark her search, so that she can reopen the bookmark at a later time and immediately see the latest search results.

To do this, we need to check for an existing history token on startup. We must check for it manually because the presence of a history token on startup will not fire a ValueChangeEvent (this event is only fired when the user clicks "back" or "forward").

@Override
public void onModuleLoad() {
  [...]

  String token = History.getToken();
  if (!token.isEmpty()){
    if ("_privacyPolicy".equals(token)){
      doGetPrivacyPolicy();
    } else {
      searchQueryTextBox.setText(token);
      doSearch(token);
    }
  }
}

The History.getToken() is called to get the current history token. The method returns an empty string if there is no token, so we first check to see if the string empty. If it's not, then we update the UI according to the token. Note that this should happen at the very end of the onModuleLoad() method because the UI widgets must be created first in order for the UI to be updated.

Does this code look familiar? It should because much of it is identical to the code we wrote for the History.addValueChangeHandler() method, so let's refactor it out into its own method:

@Override
public void onModuleLoad() {
  History.addValueChangeHandler(new ValueChangeHandler<String>() {
    @Override
    public void onValueChange(ValueChangeEvent<String> event) {
      String token = event.getValue();
      handleHistoryToken(token);
    }
  });

  [...]

  String token = History.getToken();
  if (!token.isEmpty()){
    handleHistoryToken(token);
  }
}

private void handleHistoryToken(String token){
  if ("_privacyPolicy".equals(token)){
    doGetPrivacyPolicy();
  } else {
    searchQueryTextBox.setText(token);
    doSearch(token);
  }
}

Changing the window title

It's also helpful to change the window title of the web page to reflect the history token. That way, when the user views their entire history, they can have a better idea of what Twitter searches they performed at a glance. To do this, we'll modify two sections of code. First, we'll modify the place where we add new history tokens.

searchButton = new Button("Search");
searchButton.addClickHandler(new ClickHandler() {
  @Override
  public void onClick(ClickEvent event) {
    String query = searchQueryTextBox.getText();
    History.newItem(query);
    Window.setTitle("Twitter Search - " + query);
    doSearch(query);
  }
});

privacyPolicyButton = new Button("Privacy Policy");
privacyPolicyButton.addClickHandler(new ClickHandler() {
  @Override
  public void onClick(ClickEvent event) {
    History.newItem("_privacyPolicy");
    Window.setTitle("Twitter Search - Privacy Policy");
    doGetPrivacyPolicy();
  }
});

Here, we've added calls to Window.setTitle() to set the title of the browser window. It's very important that you call this method after adding the history entry. Otherwise, the titles will not be synced properly with the history entries.

Second, we'll modify our handleHistoryToken() method to set the proper Window title when a history token is loaded.

private void handleHistoryToken(String token){
  if ("_privacyPolicy".equals(token)){
    Window.setTitle("Twitter Search - Privacy Policy");
    doGetPrivacyPolicy();
  } else {
    searchQueryTextBox.setText(token);
    Window.setTitle("Twitter Search - " + token);
    doSearch(token);
  }
}

Now, when you view your history, you'll get a good glimpse of all your past searches.

What the browser history looks like before and after setting window titles.

I hope you've enjoyed my GWT History tutorial. For more information, please see the GWT History page in the GWT Developer's Guide.

GWT RPC (Part 3 of 3) - Take a ride on the client-side

2011-12-11T12:23:00.000-05:00

GWT is all about creating web applications. And a web application wouldn't be a web application without client/server communication. A mechanism called GWT RPC (remote procedure call) is used for this communication in GWT. In this series of tutorials, I'm going to show you how GWT RPC works by building a simple GWT application that uses the Twitter API to perform tweet searches and retrieve the Twitter privacy policy.

The process involves three steps. I'll be covering step 3 in this blog post.

Create the synchronous and asynchronous interfaces that define each RPC method
Create the server-side implementation of each RPC method
Create the UI (user interface) that the client will use perform the searches

You can download the entire sample project used in this tutorial here.

In this blog post, I'll show you how to design a basic UI that will call the RPC methods we created in the previous post. This UI will give us the ability to search for tweets and to view the Twitter privacy policy. Along with explaining how to call the GWT RPC methods from client code, I'll also give you a brief introduction into UI programming.

EntryPoint


package com.acme.twittersearch.client;

[imports...]

public class TwitterSearch implements EntryPoint {
[...]
  @Override
  public void onModuleLoad() {
    createWidgets();
    layoutWidgets();
  }
[...]
}

The first thing to notice is that the class implements the EntryPoint interface. Every GWT application must have one class that implements this. It tells GWT where to start when a user loads the application in her web browser. The interface consists of only one method, onModuleLoad(), which can be thought of as the GWT-equivalent of the public static void main(String args[]) method you see in command-line Java programs.

In our class, onModuleLoad() calls two private methods (that I've created), createWidgets() and layoutWidgets(). The createWidgets() method initializes our buttons, text boxes, etc. The second method, layoutWidgets(), adds those widgets to the page, positioning them in just the way that we want. You are not required by GWT to organize your code in this way, but I find it helpful to do so because UI code is very verbose and can quickly grow in size as your application becomes more complex.

Also, the fully-qualified class name of our entry point class must be added to the GWT module definition file (TwitterSearch.gwt.xml), as shown below:

<module rename-to='twittersearch'>
  [...]
  <entry-point class='com.acme.twittersearch.client.TwitterSearch'/>
  [...]
</module>

Widgets

I've defined my widget objects as class-level fields at the top of the class:

public class TwitterSearch implements EntryPoint {
  private Button privacyPolicyButton;
  private Button searchButton;
  private TextBox searchQueryTextBox;
  private Panel resultsPanel;
  private Label errorLabel;
  private Image loadingImage;
  [...]
}

privacyPolicyButton: This button will call the getPrivacyPolicy() RPC method and display the returned privacy policy on the page.
searchButton: This button will call the searchTweets() RPC method and display the search results on the page.
searchQueryTextBox: This is where the user will enter her Twitter search query.
resultsPanel: We'll display the results from each RPC call here.
errorLabel: If an error occurs while calling one of the RPC methods, we'll put the error message in here.
loadingImage: We'll display an animated loading icon when the client sends an RPC request to let the user know that their request is being processed. It's a good idea to always display some sort of loading image or message while an RPC request is being sent. Even the simplest requests can take several seconds to complete, especially if the user is on a slow network or the server is experiencing heavy load. In our case, we actually end up having to make two network calls per request, (1) the call to the GWT server and (2) the call to the Twitter server. So for us, having a loading image is essential (by the way, I got mine from loaderinfo.net).

The HTML page

Every GWT application has an HTML file that is loaded when the user navigates to the application in her browser. Here, I'm creating the "Twitter Search App" header and adding an image of the Twitter bird. I also have a CSS file which adds some extra formatting. The Javascript file that is being included is what the browser will use when the GWT application is compiled and run in production. The widgets that we load from our Java code will be added to the page dynamically at run time and be placed below the <h1> header (at the end of the <body> tag).

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
    <link type="text/css" rel="stylesheet" href="TwitterSearch.css">
    <title>Twitter Search App</title>
    <script type="text/javascript" language="javascript" src="twittersearch/twittersearch.nocache.js"></script>
  </head>

  <body>
    <img src="twitter-bird.png" align="right"/>
    <h1>Twitter Search App</h1>
  </body>
</html>

What the app will look like.

Calling the RPC methods

Calling the RPC methods involves us using the asynchronous interface we created back in step 1 of this blog series. You might be thinking, "But TwitterSearchServiceAsync is an interface and we never created an implementation class for it." You are correct. But this is actually not a problem--GWT uses a technique called "deferred binding", which means that GWT will create a implementation for us during runtime. This is done by calling the GWT.create() method:

public class TwitterSearch implements EntryPoint {
  private final TwitterSearchServiceAsync service = GWT.create(TwitterSearchService.class);
  [...]
}

Even though we want an instance of the asynchronous interface, we pass it the class object of the synchronous interface, TwitterSearchService. The method then returns an instance of the asynchronous interface, TwitterSearchServiceAsync.

Remember how each method in the asynchronous interface took an extra AsyncCallback parameter? We will be using this now. Let's write the code that will do the Twitter search:

searchButton = new Button("Search");
searchButton.addClickHandler(new ClickHandler() {
  @Override
  public void onClick(ClickEvent event) {
    setLoading(true);
    String query = searchQueryTextBox.getText();
    service.searchTweets(query, new AsyncCallback<List<Tweet>>() {
      @Override
      public void onFailure(Throwable caught) {
        errorLabel.setText(caught.getMessage());
        errorLabel.setVisible(true);
        setLoading(false);
      }

      @Override
      public void onSuccess(List<Tweet> result) {
        resultsPanel.clear();
        for (Tweet tweet : result) {
          SafeHtmlBuilder builder = new SafeHtmlBuilder();
          builder.appendHtmlConstant("<b>User: </b>");
          builder.appendEscaped(tweet.getFrom_user());
          builder.appendHtmlConstant("<br /><b>Created: </b>");
          builder.appendEscaped(tweet.getCreated_at());
          builder.appendHtmlConstant("<br /><b>Tweet: </b>");
          builder.appendEscaped(tweet.getText());
          builder.appendHtmlConstant("<br /><br />");
          resultsPanel.add(new HTML(builder.toSafeHtml()));
        }
        setLoading(false);
      }
    });
  }
});

As you can see, we add a ClickHandler to the search button to define what happens when the search button is clicked. When the button is clicked, we first set the page to a "loading" state by calling setLoading(true). This method disables the buttons so the user can't click them and displays the loading image. We then grab the search query from the search text box and make a call to the searchTweets RPC method using the asynchronous interface.

A loading animation appears when a RPC request is made.

The onFailure() method of the AsyncCallback class is called when the RPC method throws an exception. The thrown exception is passed in as an argument to the method. If this happens, we put the exception message in our error label so the user can get an idea of what went wrong. We then set the page back to its normal, "non-loading" state by calling setLoading(false), which will re-enable the buttons and hide the loading image.

An error is thrown when no search query is specified.

The onSuccess() method of the AsyncCallback class is called when the RPC method completes without error. Here, we clear the results panel of past searches, and then loop through the list of returned tweets, adding each one of them to the page. After all tweets are displayed, we call setLoading(false) to re-enable the buttons and hide the loading image.

Search results for "#christmas".

I'm using the SafeHtmlBuilder class here to generate the HTML that is used to display each tweet. This class is like the StringBuilder and StringBuffer classes in that it allows you to efficiently create a large string in a peace-meal fashion. However, SafeHtmlBuilder specializes in making strings of HTML code. The appendEscaped() method escapes all HTML special characters in the string before appending it (for example, converting all < characters to <). The appendHtmlConstant() method does not escape HTML special characters, so you can use this to add HTML tags to the string.

The RPC method call for the getPrivacyPolicy() method follows the same pattern:


privacyPolicyButton = new Button("Privacy Policy");
privacyPolicyButton.addClickHandler(new ClickHandler() {
 @Override
 public void onClick(ClickEvent event) {
    setLoading(true);
    service.getPrivacyPolicy(new AsyncCallback<String>() {
      @Override
      public void onFailure(Throwable caught) {
        errorLabel.setText(caught.getMessage());
        errorLabel.setVisible(true);
        setLoading(false);
      }

      @Override
      public void onSuccess(String result) {
        resultsPanel.clear();

        // convert newlines to  <br / >
        SafeHtmlBuilder builder = new SafeHtmlBuilder();
        builder.appendEscapedLines(result);

        resultsPanel.add(new HTML(builder.toSafeHtml()));

        setLoading(false);
      }
    });
  }
});

I hoped you've enjoyed my three part series on GWT RPC. Creating and calling GWT RPC methods is very easy to do once you become familiarized with these three steps. Simply create the synchronous/asynchronous interfaces, create your server-side RPC implementation, and then call the RPC methods from the client using the asynchronous interface.

For more information on GWT, check out the GWT Developer's Guide.

GWT RPC (Part 2 of 3) - The server-side RPC implementation

2011-12-09T17:39:00.000-05:00

The process involves three steps. I'll be covering step 2 in this blog post.

Create the synchronous and asynchronous interfaces that define each RPC method
Create the server-side implementation of each RPC method
Create the UI (user interface) that the client will use perform the searches

You can download the entire sample project used in this tutorial here.

Since we've already defined our RPC methods in the TwitterSearchService synchronous interface, we're now ready to write the implementations of those RPC methods. We'll create a new class named TwitterSearchServiceImpl (the naming convention is to use the synchronous interface's name with "Impl" appended to the end). It will be located in the "server" package because it's meant to only run on the server. It will extend the RemoteServiceServlet class and implement our TwitterSearchService synchronous interface (it should not implement the asynchronous interface).


package com.acme.twittersearch.server;

[imports...]

public class TwitterSearchServiceImpl extends RemoteServiceServlet implements TwitterSearchService {
  [...]
}

We do a little bit of error checking in the searchTweets method. If the search query is empty, then an IllegalArgumentException is thrown. With GWT RPC, exceptions are sent over the wire to the client. So the exact same exceptions can actually be handled in the client code (as you'll see in my next blog post). Exceptions can be used to return validation errors, as well as return unexpected server-side errors to the client.


@Override
public List<Tweet> searchTweets(String query) throws IllegalArgumentException, IOException {
  query = query.trim();
  if (query.isEmpty()) {
    throw new IllegalArgumentException("No search query specified.");
  }

  // see: https://dev.twitter.com/docs/api/1/get/search
  String q = URLEncoder.encode(query, "UTF-8");
  URL url = new URL("http://search.twitter.com/search.json?q=" + q);
  HttpURLConnection connection = (HttpURLConnection) url.openConnection();
  InputStream response = null;
  try {
    response = connection.getInputStream();
    return parseSearchResponse(response);
  } finally {
    if (response != null) {
      response.close();
    }
  }
}

Also in the searchTweets method, notice how the search query is passed into the URLEncoder.encode() method. This URLEncoder.encode() method encodes the search query so that it can be safely included in the query string of the Twitter API URL. You should always encode query string parameters in this way (just like you should always encode data before passing it into a SQL query string to avoid SQL injection). If a value happens to contain a character that has a special meaning in a URL (like an equals sign or ampersand), then you're likely to run into trouble.

We use the HttpURLConnection class to make the calls to the Twitter API. This class is not on the JRE Emulation list, but that's OK because we're only using it on the server. If we were to use it on the client, we'd get an error because GWT does not know how to convert this class to Javascript.

We also use the google-gson library to parse the JSON-encoded responses that are returned from the Twitter API. The way google-gson works is that you create a class that mirrors the JSON data you are unmarshalling. Each field in the class is named after a field in the JSON data. Google-gson looks at these field names to determine how to unmarshall the JSON message. So, we've created a SearchResponse class for our tweet search API call and a PrivacyPolicyResponse class for our privacy policy API call to do just that. However, for this demonstration, we don't need to unmarshal all the data in each response, so I've only included fields for the data that we need. You can see examples of the full JSON responses for the search and legal/privacy methods on the Twitter API documentation page.


private List<Tweet> parseSearchResponse(InputStream response) throws IOException {
  Reader reader = new InputStreamReader(response);
  SearchResponse searchResponse = new Gson().fromJson(reader, SearchResponse.class);
  return searchResponse.results;
}

private class SearchResponse {
  public List<Tweet> results;
}

Under the covers, TwitterSearchServiceImpl is a plain old Java servlet, which means that only one instance of this class is created to handle all requests. Therefore, be sure to keep thread-safety in mind.

web.xml

Our TwitterSearchServiceImpl class must also be added to the web application's deployment descriptor (web.xml).


<web-app ...>
  [...]

  <servlet>
    <servlet-name>twitterSearchServlet</servlet-name>
    <servlet-class>com.acme.twittersearch.server.TwitterSearchServiceImpl</servlet-class>
  </servlet>  

  <servlet-mapping>
    <servlet-name>twitterSearchServlet</servlet-name>
    <url-pattern>/twittersearch/search</url-pattern>
  </servlet-mapping></span>

  [...]
</web-app>

As you can see, the url-pattern for the servlet is /twittersearch/search. The first part, twittersearch, is the name of our GWT module, which is defined in the module definition file (TwitterSearch.gwt.xml). The second part, search, is the value of the @RemoteServiceRelativePath annotation in the TwitterSearchService interface that we created in the previous blog post.

Tweet class

Let's finish by creating the Tweet class. Because we're unmarshalling JSON data into this class using google-gson, the field names in this class must be named after the fields in the JSON message. Also, GWT requires that all objects that are sent over the wire implement Serializable, so let's do that too. The class must also be put in the "client" package, because it will be used in client code.


package com.acme.twittersearch.client;

import java.io.Serializable;

public class Tweet implements Serializable {
  private String id;
  private String from_user;
  private String created_at;
  private String text;

  [getter/setter methods...]
}

In the next and final blog post of my GWT RPC series, you'll learn how to create a basic UI and how to call these RPC methods from the client.

GWT RPC (Part 1 of 3) - Synchronous and asynchronous interfaces

2011-12-08T20:34:00.001-05:00

The process involves three steps. I'll be covering step 1 in this blog post.

Create the synchronous and asynchronous interfaces that define each RPC method
Create the server-side implementation of each RPC method
Create the UI (user interface) that the client will use perform the searches

You can download the entire sample project used in this tutorial here.

Synchronous interface

The first thing to do is figure out what kind of communication you need to do with the server. What actions do you need to perform? What data do you need to retrieve? In this case, we need to do two things: perform tweet searches and get the Twitter privacy policy.

These actions are defined in the what's called the synchronous interface. I'll create two RPC methods in my synchronous interface (there's no limit to how many you can define). The interface must extend RemoteService and have a @RemoteServiceRelativePath annotation (which we'll revisit later). It must also be in the client package.


package com.acme.twittersearch.client;

[imports...]

@RemoteServiceRelativePath("search")
public interface TwitterSearchService extends RemoteService {
  List<Tweet> searchTweets(String query) throws IOException, IllegalArgumentException;
  String getPrivacyPolicy() throws IOException;
}

The first method, searchTweets, will take a search query as input and then return a list of tweets in the response. Tweets contain a lot of information (such as an ID, author, and creation date, not to mention the actual tweet message), so I'm going to create a Tweet class that encapsulates all of this information. I'm not going to worry about creating this class right now, though. For now, just assume that the Tweet class holds all the information about a tweet that we need it to.

Also notice how searchTweets throws two exceptions. An IOException will be thrown if there's a problem querying the Twitter API. We'll also have it throw an IllegalArgumentException if the search query is empty. Even though IllegalArgumentException is a runtime exception and doesn't have to be defined in the method's throws clause in order for the Java compilation to pass, GWT requires it to be declared here. Otherwise, it will send a generic "500 error" exception to the client instead of the real exception if that exception is thrown.

The second method, getPrivacyPolicy, doesn't need any parameters. It will return the Twitter privacy policy, which is a block of text that we will hold in a String object. As with searchTweets, it too will throw an IOException if there's a problem calling the Twitter API.

Asynchronous interface

Now that we've defined our RPC methods in the synchronous interface, we're ready to create the asynchronous interface. This interface will contain the exact same methods that are in the synchronous interface, but in a slightly different form:


package com.acme.twittersearch.client;

[imports...]

public interface TwitterSearchServiceAsync {
  void searchTweets(String query, AsyncCallback<List<Tweet>> callback);
  void getPrivacyPolicy(AsyncCallback<String> callback);
}

The interface can be named anything, but I've named it TwitterSearchServiceAsync because the convention is to use the synchronous interface's name with "Async" appended to the end.

As you may have noticed, unlike their counterparts in the synchronous interface, the methods don't return anything. They also take an extra parameter--an AsyncCallback object. This object will be used by the client to handle the response when the response is returned from the server. Since this interface will be used by the client, it must be located in the client package. Also, note that, while the methods in the synchronous interface throw exceptions, the methods in the asynchronous interface do not.

Both of these interfaces must remain in-sync. By that I mean, if one interface changes, the other must also change. For example, if I change the return value of a method in the synchronous interfaces, I must make the same, corresponding change to the asynchronous interface (by changing the generics parameter of the AsyncCallback parameter). The Eclipse GWT plugin is helpful in this regard. If it notices that the two interfaces are not in-sync, it will throw a Java compilation error.

In my next blog post, I'll talk about creating the server-side implementation of the RPC methods.

GWT Introduction

2011-12-03T17:54:00.000-05:00

The Google Web Toolkit allows you to write Javascript-based web applications without having to write any Javascript yourself. It translates Java code that you write into Javascript automatically. In this informational overview, I'm going to walk you through creating a GWT application and then describe the basic components that make up a GWT application.

Install the Eclipse plugin

Google provides an Eclipse plugin you can use for developing GWT applications. This is what I'll be using here, but the raw GWT SDK is available as well for command-line junkies.

The Eclipse update URL for the plugin is:

http://dl.google.com/eclipse/plugin/<version>

where <version> is your Eclipse version (3.7, 3.6, etc).

You'll need to install these modules:

Google Plugin for Eclipse
SDKs > Google Web Toolkit SDK 2.4.0

Create a new project

Once the Eclipse plugin is installed, create a new "Web Application Project".

Enter "SimpleApp" for the project name and "com.acme.simpleapp" for the package. The package name will be the "base" package for all of the packages in your project as you'll see in a minute. If you haven't installed the Google App Engine plugin, then uncheck the "Use Google App Engine" checkbox (I was on my netbook when I wrote this, so it's hard to see in the screenshot). The Google App Engine is a cloud-based service that lets you upload your application to the web, but that's out of the scope for this article.

Running the project

When you create a new GWT project in Eclipse, it creates a sample application, so let's run it. Right click on the project folder and select "Run As > Web Application". A "Development Mode" view will appear, showing a URL.

Let's quickly take a look at the URL:

http://127.0.0.1:8888/SimpleApp.html?gwt.codesvr=127.0.0.1:9997

The web server (GWT uses jetty, a light-weight web container) runs on port 8888 (remember that "127.0.0.1" is the same as "localhost"). The "gwt.codesvr" query string parameter is how GWT is able to run the application without compiling the project into Javascript. This is called running it in "hosted mode" (which is what you'll always want to do while developing and debugging a project). If this parameter is removed, it will instead use the Javascript code that was generated the last time the project was compiled (more on compilation later). This is called running it in "web mode". We haven't compiled the project yet, so running it in web mode will throw an error because there's no Javascript code to use.

Now, load the URL in your browser. In order to run the application in hosted mode, you have to install a browser plugin, so install the plugin and then refresh the page.

After a few seconds the web page will appear, asking you to enter your name. Enter your name and click send.

A popup message will appear with some information. What it's doing here is sending a message to the server using GWT RPC, which is the main mechanism GWT uses to communicate between the client (a Javascript-powered web page) and the server (a Java web application).

Debugging the application

You can set breakpoints and debug the GWT application just like any other Java application. Just be sure to select "Debug As > Web Application" instead of "Run As > Web Application" when running the application.

Translatable vs non-translatable code

One thing to keep in mind when developing a GWT application is the idea of distinguishing between Java code that should be converted to Javascript, from the Java code that shouldn't be converted to Javascript. This is defined in the "SimpleApp.gwt.xml" file, which is a configuration file for the application.

<?xml version="1.0" encoding="UTF-8"?>
<module rename-to='simpleapp'>
  <!-- Inherit the core Web Toolkit stuff.                        -->
  <inherits name='com.google.gwt.user.User'/>

  <!-- Inherit the default GWT style sheet.  You can change       -->
  <!-- the theme of your GWT application by uncommenting          -->
  <!-- any one of the following lines.                            -->
  <inherits name='com.google.gwt.user.theme.clean.Clean'/>
  <!-- <inherits name='com.google.gwt.user.theme.standard.Standard'/> -->
  <!-- <inherits name='com.google.gwt.user.theme.chrome.Chrome'/> -->
  <!-- <inherits name='com.google.gwt.user.theme.dark.Dark'/>     -->

  <!-- Other module inherits                                      -->

  <!-- Specify the app entry point class.                         -->
  <entry-point class='com.acme.simpleapp.client.SimpleApp'/>

  <!-- Specify the paths for translatable code                    -->
  <source path='client'/>
  <source path='shared'/>
</module>

As you can see, the <source> tags mark the "client" and "shared" packages as Javascript-compatible, which means they will be converted to Javascript code when the application is complied.

Keep in mind that, GWT can only convert certain Java classes to Javascript (for a complete list, see the JRE Emulation Reference). So, for example, even though creating a FileOutputStream in the Java code won't throw any errors when the Java class is complied, an error will be thrown when the project is compiled to Javascript or run in hosted mode. To help you remember which Java classes are translatable, just remember that you're writing code for Javascript and there's certain things you can't do in this environment (like save files to disk, for example). As long as you keep this in mind, you won't need to refer to the JRE Emulation Reference very often.

Packages that are not defined as translatable code will run on the server as real Java code, so you can do whatever you want with them.

Source code

Let's just drill down each of the files in the "src" directory and describe what they do:

SimpleApp.gwt.xml: This file is called the "module definition". It contains the project's configuration settings including which Java classes are translatable to Javascript, the "entry point" class (like the "main" method for the client), and the application's look and feel.
GreetingService.java: An interface which defines the GWT RPC methods that are used by the client to communicate with the server.
GreetingServiceAsync.java: An alternate version of the GreetingService interface that the client uses to perform asynchronous communication with the server. Note that these two interfaces must remain in-sync, so any time the GreetingService interface is changed, the GreetingServiceAsync interface must also be changed.
SimpleApp.java: Contains all of the client-side GUI code. It is also the entry point class.
GreetingsServiceImpl.java: The server-side code that is called when the client sends requests to the server via GWT RPC. It provides the implementation of all of GWT RPC methods defined in GreetingService.
FieldVerifier.java: Contains validation code that is used on both the client and server. This is one of the advantages to using GWT--because the same code can be used for both the client and server, only one version needs to be maintained.

Client-side code

Let's quickly take a look at the client-side code. In this sample application most of the code is in one class, SimpleApp.

public class SimpleApp implements EntryPoint {

  private final GreetingServiceAsync greetingService = GWT.create(GreetingService.class);

  public void onModuleLoad() {
    [...]
  }
  [...]
}

The SimpleApp class implements the EntryPoint interface and thus must implement the onModuleLoad() method. This method is where the application "starts" when the client loads it in her browser (like a "main" method). To communicate with the server over GWT RPC, an instance of the GreetingServiceAsync class is created.

Server-side code

When the client communicates with the server via GWT RPC using the GreetingServiceAsync class, the code in GreetingServiceImpl class is executed.

public class GreetingServiceImpl extends RemoteServiceServlet implements GreetingService {

  public String greetServer(String input) throws IllegalArgumentException {
    [...]
  }
  [...]
}

Under the covers, this class is just a plain Java servlet, but with some GWT additions. Notably, it does the work of converting all requests and responses to JSON when sent over the wire.

The only GWT RPC method here is greetServer. It accepts a String as the request and returns a String in the response.

Compiling

When you're ready to deploy your GWT application to production, you'll want to compile it. This will translate all the client-side Java code into Javascript code, making it a fully-functional web application that you can run from a web container like Tomcat.

To do this, right click on the project folder and select "Google > GWT Compile". When it's done, just copy the "war" directory into the web container.

Apache Commons IO Introduction

2011-12-02T20:48:00.000-05:00

The Apache Commons project is a collection of over 40 individual Java libraries that provide convenience methods to eleviate the developer from having to write tedious, boiler-plate code. The libraries try to stay as close to the core Java API as possible, so they have few or no dependencies on other libraries.

One of the libraries that I use a lot is the IO library. It contains features that build upon the core Java IO API and also make it easier to interact with the filesystem. I'm going to list three of the methods that I use a lot (and that you should use too). Looking at the API though, there's a ton more stuff this library can do, so don't let my tiny blog post keep you from exploring it.

IOUtils.copy()

The IOUtils.copy() method is great for copying data from an InputStream (or Reader) to an OutputStream (or Writer). No more creating arbitrarily-sized buffers and parenthesis-laden while loops! What it does here isn't rocket-science--any programmer worth their salt could code this up themselves. The point of using it is to make your code easier to read, less error-prone, and more consistent with the broader Java programming community.

These two code samples show just how much less code you have to write if you use this method:

Plain Java:

InputStream in = ...
OutputStream out = ...
byte buffer[] = new byte[4096];
int read;
while ((read = in.read(buffer)) != -1){
  out.write(buffer, 0, read);
}
in.close();
out.close();

Commons IO:

InputStream in = ...
OutputStream out = ...
IOUtils.copy(in, out);
in.close();
out.close();

IOUtils.closeQuietly()

The IOUtils.closeQuietly() method eliminates the boiler-plate required to close a stream in Java. It (1) checks to make sure the stream isn't null, (2) closes the stream, and (3) catches the IOException that is thrown when the stream's close() method is called. Again, what it does here isn't rocket-science. Its purpose is to reduce the amount of fluff in your code:

Plain Java:

InputStream in = null;
try{
  in = ...
  ...
} catch (IOException e){
  ...
} finally{
  try{
    if (in != null){
      in.close();
    }
  } catch (IOException e){}
}

Commons IO:

InputStream in = null;
try{
  in = ...
  ...
} catch (IOException e){
  ...
} finally{
  IOUtils.closeQuietly(in);
}

FileUtils.writeStringToFile()

Whenever I switch from coding in PHP to coding in Java, my heart sinks a little bit when I have to write some text to a file. In PHP, you can do this with a single function call. But in Java, you need to write a dozen or so lines of code. The FileUtils.writeStringToFile() method, however, allows you to write text to a file in one fell swoop.

Plain Java:

String text = "some text";
File file = new File("myfile.txt");
Writer writer = null;
try{
  writer = new PrintWriter(file);
  writer.write(text);
} catch (IOException e){
  ...
} finally {
  if (writer != null){
    try{
      writer.close();
    } catch (IOException e){}
  }
}

Commons IO:

String text = "some text";
File file = new File("myfile.txt");
try{
  FileUtils.writeStringToFile(file, text);
} catch (IOException e){
  ...
}

I hope you've enjoyed my little introduction to Commons IO. Using this library can greatly increase the readability and reliability of your code.

Book review: Tomcat 6 Developer's Guide

2011-11-23T12:33:00.001-05:00

The book is very wholistic in its approach to educating the reader. It provides a well-written overview of JavaEE and HTTP in Chapter 2. And whenever it mentions a technology that Tomcat uses in some way, it immediately gives a brief overview of that technology. I found this helpful because it gave me the knowledge I needed to continue reading without having to switch to a search engine. For example, in Chapter 5, when discussing the JNDI functionality that Tomcat comes packaged with, the book pauses for a moment to explain the basic concepts of JNDI itself. The book will also often describe the historical origins of the technology, like in Chapter 1, where it discusses the origins of Ant (spoiler alert: it was created as a custom build system for Tomcat).

The instructions for downloading the Tomcat source code and building it are very clear. It provides a specific branch you can checkout from the Tomcat Subversion server, which is the exact version of Tomcat that was used when writing the book. This allows you to follow along without worrying about dealing with differences between what you see in the code and what you see in the book. And it also includes instructions for getting the project properly configured with Eclipse.

The later chapters of the book go into a lot of detail about the various classes that make up the Tomcat source code, so be prepared for some serious code spelunking. Anyone who is interested in doing serious coding in the Tomcat codebase will gain a lot from these chapters.

I'd recommend this book to people who want to work on the Tomcat source code and need something to guide them through it. I'd also recommend the book to JavaEE developers who use Tomcat on a regular basis and who want to expand their knowledge.

Microsoft Word: Disabling spellcheck for a block of text

2011-10-15T13:19:00.000-04:00

Often times, when I'm putting together a piece of documentation for work, it helps to include code samples and console commands in the document. I like to use Word to write documentation because it allows you to stylize the text (such as increasing the font size of headers, making things bold, etc) which I think makes it easier to read (when used in moderation). It's also nice for code samples because when you copy and paste code from Eclipse, it includes the syntax-highlighting of the code. Creating plain text files is simpler, but you loose that readability aspect I think is important.

Anyway, one annoying thing is that Word spell checks and grammar checks these code samples and console commands, so you get all these red and green squiggly lines under the text.

However, it is possible to disable spell and grammar check for selected blocks of text! First, select the text that you want to exclude from being checked. Then, select the "Review" tab from the ribbon at the top of the screen and click on "Set Language".

In the dialog window that appears, click the "Do not check spelling and grammar" checkbox and click "OK". If your document happens to have multiple written languages in it, you can also use this dialog to tell Word which blocks of text are written in which language.

Now, those squiggly lines should disappear!

Book Review: Play Framework Cookbook

2011-10-08T14:10:00.000-04:00

Play Framework Cookbook is divided into seven chapters, each of which contains a dozen or so recipes. The recipes range in skill level from beginner to advanced and are pretty much organized in this order, with the more basic recipes at the start of the book and the more advanced ones at the end. Some examples of the more basic recipes include "Defining your own controllers" and "Basics of caching". Advanced topics include "Understanding bytecode enhancement" and "Implementing your own persistence layer". Nearly all of the recipes are self-contained so you don't have to read the book from start to finish--you can jump around at your leasure and read whatever looks interesting to you.

Each recipe is usually divided into four sections. "Getting Ready" describes how to prepare your development environment for the recipe. "How to do it..." covers the steps you have to take to complete the recipe, like what code you have to write. A detailed explaination of those steps can be found in the next section, "How it works...". The last section, "There's more..." (what's with all the ellipses?), contains supplemental information on how to improve upon the recipe and broaden its scope. I like the way the author divided each recipe this way because it makes it easier to find what you're looking for. For example, if I'm in the midst of coding something and need to refresh my memory about a particular method call or class name, I can jump right to the "How to do it..." section and not have to wade through a lot of text to find what I'm looking for.

I would say that the only prerequisite of the book is that you know how to program in Java, though having basic knowledge of Play beforehand will help. The Forward contains an informative overview that describes where Play fits into wider the Java web application world and how it differentiates itself from other web frameworks. And the recipes in the first chapter contain a lot of the basics of the framework, like how to define routes and create controllers.

But all the information in the first chapter can be found in the official Play documentation online. Where the book really shines is in the subsequent chapters. They describe how to perform specific tasks such as creating an RSS feed, creating PDFs, and generating JSON data. They also describe how to integrate Play with a variety of systems like Apache, MongoDB, Spring, Google Charts, and Jenkins. Getting an application to "play nice" with the other parts of a system, no matter what technology you are using, can be a hairy ordeal, so I think these recipes are especially helpful.

All of the code samples can be downloaded from the book's website. Each sample corresponds to a single recipe and is a complete Play application, which is great because it means the samples are self-contained and can be run right off the bat. However, I had a little trouble getting some of them to run. I had to manually enable the in-memory database (the "db=mem" config property) because I was getting JPA errors on application startup. But otherwise, the samples seemed to work fine.

All in all, I would recommend this book to anyone interested in learning more about Play. It covers a wide variety of topics, so you're bound to find something useful no matter what kind of Play application you're developing (or want to develop) or what your skill level is.

Upcoming book review: "Play Framework Cookbook"

2011-09-19T18:48:00.001-04:00

I got an interesting email last Friday. A publisher, Packt Publishing, noticed my recent blog posts about the Play Framework and asked if I would review the book, Play Framework Cookbook by Alexander Reelsen. I've never been asked to write a book review before, so I'm really excited about it! I'll be posting a review of the book within the next 2-3 weeks.

In the mean time, feel free to check out a sample chapter from the book entitled "Using Controllers" where the author covers topics such as how to generate PDFs and implement HTTP digest authentication.

The Play Framework - Part 6

2011-09-17T11:37:00.000-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world.

Here are a couple miscellaneous features of the Play Framework:

In the "conf/routes" file, a regular expression can be used to match a URL to a controller action. The regular expression is enclosed within < > characters and comes before the variable name. For example, the following route definition only matches if the last part of the URL is a number:

GET  /posts/{<[0-9]+>id}  Application.show

In a template, when looping through a list of elements, it is often helpful to know if the current element is the first or last element in the list. To do this, append _isFirst or _isLast to the end of the variable name. For example, the code below will output the contents of a list with each element separated by a comma. A comma should always be printed after each element--except for the last element, which shouldn't print a comma:

#{list items:tags, as:tag}
  ${tag}${tag_isLast ? '' : ', '}
#{/list}

The Play Framework - Part 5 - CAPTCHAs

2011-09-15T21:16:00.000-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world.

Play comes packaged with a CAPTCHA generator, which is good for ensuring that form submissions are sent by a human being and not by an automated bot.

A CAPTCHA is an image that contains random letters and numbers that the user has to type in order to submit a form. The letters and numbers are rendered in a non-uniform way in order to make it as difficult as possible for an automated image recognition algorithm to read (but still clear enough for a human to read). For example, the characters may each be a different size and font. It stands for (take a deep breath) "Completely Automated Public Turing test to tell Computers and Humans Apart".

To generate a CAPTCHA in Play, use the Images.Captcha class:

public class Application extends Controller{
  public static void captcha(){
    Images.Captcha captcha = Images.captcha();
    String code = captcha.getText("#990000");
    captcha.addNoise("#CCCCCC");
    captcha.setBackground("#996633", "#FF9900");
    renderBinary(captcha);
  }
}

In this code, I'm generating a CAPTCHA image and returning it in the response. Calling the getText() method sets the color of the text and returns the random string that the CAPTCHA is rendering. As you can see, the notation used to define the color is conveniently the same notation that is used in HTML and CSS (however, you cannot use the three-character abbreviated form used by CSS).

I'm also adding noise to the image to make it even harder for an image recognition algorithm to read (it will add a gray squiggly line through the image). And then, I'm setting a background gradient to have it fade from a dark to a light orange.

Because the Captcha class implements InputStream, it can be passed into the renderBinary() method to send the image to the response. It generates a PNG image.

The Play Framework - Part 4

2011-09-13T20:45:00.000-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world.

A special template syntax is used to link to other pages in the application (the actual URL of the page is not used). This makes things more flexible, because you can change the page's URL in the "conf/routes" file without having to go through each template file and edit all the links that point to that page. Take this example:

Template file:

<a href="@{Application.index()}">Home</a>

"conf/routes" file:

GET  /home   Application.index

Generated HTML:

<a href="/home">Home</a>

Play sees that the Application.index method is assigned to the URL "/home" and generates the appropriate HTML when a user requests the page.

To access a static resource (like images, Javascript files, and CSS files), put the path to that resource in single quotes. With Play, all static resources go in the "public" directory.

<script src="@{'/public/javascripts/jquery-1.4.2.min.js'}"></script>

As with seemingly every type of file in a Play application, any changes made to the "conf/routes" file while the application is running are applied immediately. No restart is needed.

The "form" template tag can be used to create HTML forms:

#{form @Application.postComment(post.id)}
  Name: <input type="text" name="author" value="${params.author}" />
  <input type="submit" value="Post" />
#{/form}

This ends up adding two more attributes to the generated <form> tag ("accept-charset" and "enctype") and also adds a hidden parameter called "authenticityToken" (probably used for security purposes). You can also add attributes of your own, by supplying them as tag parameters after the action URL:

#{form @Application.postComment(post.id), onsubmit:'return validateForm()'}

The Play Framework - Part 3

2011-09-10T15:35:00.001-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world.

If you want to execute some code when the application first starts up, you can create a class that has the @OnApplicationStart annotation and that extends the Job class. Put the code that you want to run on startup in a method called doJob(). This class can just go in the default package for simplicity, but it doesn't matter what package the class is in.

@OnApplicationStart
public class Bootstrap extends Job {
  @Override
  public void doJob(){
    if (User.count() == 0){
      Fixtures.loadModels("initial-data.yml");
    }
  }
}

In DEV mode, this will be run when the application gets its first request. That way, if an error occurrs, it will appear in the browser. But in PROD mode, the startup code will be run as soon as the play run command is executed and if there are any errors, the server will fail to start up.

To pass variables to a template, simply pass them as arguments to the render() method in the controller. The variable will have the same name inside the template as it does inside the controller.

If you want to run some code every time a controller handles a request, use the @Before annotation. For example, the addDefaults() method below will add two variables to the template every time a request comes in to the Application controller (the Play.configuration variable allows you to access values from the 'application.conf" file):


public class Application extends Controller {
  @Before
  public static void addDefaults(){
    //called before every request is handled
    renderArgs.put(
      "blogTitle",
      Play.configuration.getProperty("blog.title")
    );
    renderArgs.put(
      "blogBaseline",
      Play.configuration.getProperty("blog.baseline")
    );
  }

  //...
}

Play adds some helpful utility methods to various datatypes inside template code. Calling pluralize() on a number will return the string "s" if the number is not 1. Calling format('MM/dd/yyyy') on a Date object will format the date according to the given format string (you don't need to create a SimpleDateFormat object). Calling nl2br() on a string will replace all newlines with <br/> tags.

Tags are snippets of template code that you can insert into other templates by calling them like functions. The tag parameter names all begin with underscores in the tag file. Tag files go in the "views/tags" directory. The tag's name is the same as its file name (minus the file extension).

Calling a tag from a template:

#{hello person:'George', timeOfDay:'morning' /}

Tag file (views/tags/hello.html):

Good ${_timeOfDay}, <b>${_person}</b>!

The Play Framework - Part 2 - Database

2011-09-05T14:02:00.000-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world.

To connect your application to a database, modify the "db.*" settings in the "conf/application.conf" file. To get up and running quickly, uncomment the "db=mem" line to create an in-memory database (all changes will be lost, of course, when the application ends).

To save data to the database, start by creating entity classes inside the "model" package. An entity class is a plain Java bean class that has JPA (Java Persistence Architecture) annotations. Each entity class represents a table in the database, each instance of the class represents a row in the table, and each field in the instance represents a column in the row (getter and setter methods aren't needed for the fields--Play generates them automatically). By default, the table name is the same as the class name. To change the table name, use the @Table annotation.

Each entity class can also optionally extend Play's Model class, which gives it access to utility methods for saving and retrieving data. For example, to insert a new row into the table, simply create a new instance of the class and call the save() method:

User user = new User("Bob", "bob@gmail.com");
user.save();

To retrieve data, call the static find() method:

List<User> users = User.find("byNameAndEmail", "Bob", "bob@gmail.com").fetch();

You don't need to write any SQL code to interact with the database. Play uses Hibernate for its persistence layer, so HQL (Hibernate Query Language) can be used for more complex queries.

The Model class also automatically generates an "id" primary key column for you (something that every table should have). So basically, you should always extend this class.

Here's an example of an entity class. It represents a blog post:

package models;
import java.util.*;
import javax.persistence.*;
import play.db.jpa.*;

@Entity
@Table(name="blog_posts")
public class Post extends Model {
  public String title;
  public Date postedAt;

  @Lob
  //large amount of text
  public String content;

  @ManyToOne
  //many Posts can belong to one User
  public User author;

  //a list of all comments that belong to this post
  //(the comments are in a separate table)
  //CascadeType.ALL = the post's comments will be automatically deleted when the post is deleted
  @OneToMany(mappedBy="post", cascade=CascadeType.ALL)
  public List<Comment> comments;

  public Post(User author, String title, String content) {
    comments = new ArrayList<Comment>();
    this.author = author;
    this.title = title;
    this.content = content;
    postedAt = new Date();
  }

  public Post addComment(String author, String content){
    Comment newComment = new Comment(this, author, content).save();
    comments.add(newComment);
    save();
    return this;
  }
}

The Fixtures class provides more utility methods for interacting with the database. For example, to delete the entire database, call the deleteDatabase() method. It can also execute SQL code contained in a file or persist the data in a specially-formatted YAML file (YAML is a new standard for representing structured data. Its aim is to provide a better alternative to XML).

The Play Framework

2011-09-04T12:55:00.000-04:00

The Play Framework is a MVC web application framework for Java that focuses on being light-weight and easy to use--two qualities that you don't see very often in the Java web application world. Writing a web application in Java has always been more complicated than, say, writing a web application in PHP.

The first thing you must do after downloading the Play framework is configure your path so that you can invoke the "play" executable:

PATH=$PATH:/home/user/play-1.2.2

Everything you do with Play is done through this play executable.

To create a new application, run this command:

play new appName

This creates a folder in the current directory with the same name as the application. The folder is populated with the basic scaffolding needed for a Play application.

You can immediately run this application too. To run the application, execute this command:

play run appName

This will start up a web server and run your application at "http://localhost:9000". You don't need any web servers of your own, everything is bundled within the framework (you don't even need a JavaEE web container like Tomcat). The only thing you need to run Play is a JRE.

Note that, if you get an error that says, "ERROR ~ Could not bind on port 9000", then something on your computer must be using port 9000. Port 9000 is the default port Play uses to run the application. This setting is defined in "conf/application.conf" under the "http.port" setting (be sure to uncomment the line too--remove the "#" at the beginning of the line to uncomment it). I changed the port to 9001 because 9000 didn't work for me.

Notice the line that says, "Listening for transport dt_socket at address: 8000", when the application is starting up. This means the web application has opened a port which allows you to debug the application as it is running using an IDE like Eclipse.

After it successfully starts up, open a web browser and navigate to "http://localhost:9001". When the web application gets its first request, it compiles all the Java code and template code in the application for you (it saves the .class files to the "tmp" directory). You don't have to manually compile any of the .java files yourself like in a normal JavaEE web application. If you change any Java code or template code while the application is running, the application will detect this and automatically recompile it for you. You don't have to restart the application. This is how the web application behaves in DEV (development) mode (the default). If started in PROD (production) mode, then this is not done in order to increase performance.

The application's unit tests (in the "test" folder) can be run from the web application itself. To do this, execute this command:

play test appName

This will start the web application and also enable the unit test functionality at "http://localhost:9001/@tests". The page displays all available tests and lets you select which tests to run.

Eclipse Integration

You can edit all of the files in a basic text editor of course, but for serious development work, this can be tedious. To configure the Play application for Eclipse, execute this command:

play eclipsify appName

This will create all the necessary configuration files (such as the ".project" file) needed to open the application as an Eclipse project. It also creates a directory named "eclipse", which contains launch configurations that can be run from Eclipse:

appName.launch - Running this has the same effect as running play run appName from the command line (it starts up the web application). To run it, right-click on it and select "Run As > appName".
Connect JPDA to appName.launch - This will connect Eclipse to the JPDA port (8000) while the application is running and allow you to debug the application in Eclipse. Note that the application must already be running for this to work. To run it, right-click on it and select "Debug As > Connect JPDA to appName". Go to the "Debug" perspective in Eclipse to see if it connected properly (you can see all the different web server threads in the upper-left hand corner).
Test appName.launch - Running this has the same effect as running play test appName from the command line. To run it, right-click on it and select "Run As > Test appName". Make sure that you first shut down the web application if you had already started it using "appName.launch".

Import the project into Eclipse by going to "File > Import..." and then selecting "Existing Projects into Workspace".

If you try to open some of the template files (the .html files in "app/views") in Eclipse, some files will display an error that says, "Character encoding "${_response_encoding}" is not supported by this platform." This is because Eclipse considers the file to be an HTML file and it looks at the <meta> tag to determine the character encoding of the file (even though the file has a ".html" extension and has HTML tags, it isn't really an HTML file--Play uses a special templating engine that has a special syntax). Well, the character encoding here is defined as a template variable, so Eclipse sees the variable name and gets confused. Click the "Set Encoding..." button and set the encoding to "UTF-8".

PHP - Relative paths and Phar Archives

2011-06-14T21:33:00.000-04:00

Working with file paths and Phar archives in PHP can be tricky. The PHP code inside of a Phar file will treat relative paths as being relative to the Phar archive, *not* relative to the current working directory. Here's a short example:

Say you have the following files:

phar/index.php
test.php
my.phar

The index.php file is located inside of the phar directory. It is the bootstrap file for the phar archive:

function does_it_exist($file){
  return file_exists($file) ? "true" : "false";
}

The bootstrap file is executed when the phar file is included from a PHP script. Our bootstrap file will simply cause the function "does_it_exist" to be declared.

The file my.phar is the packaged phar archive (see the phar-util project for an excellent phar-building tool). It contains the index.php file at its root.

Let's try running different code inside of test.php and see what the results are for each run:

//Run 1:
require_once 'phar/index.php';  //PHP file
$file = __DIR__ . "/index.php"; //absolute path
echo does_it_exist($file);      //prints "false"

//Run 2:
require_once 'phar/index.php';  //PHP file
$file = "index.php";            //relative path
echo does_it_exist($file);      //prints "false"

//Run 3:
require_once 'my.phar';         //phar file
$file = __DIR__ . "/index.php"; //absolute path
echo does_it_exist($file);      //prints "false"

//Run 4:
require_once 'my.phar';         //phar file
$file = "index.php";            //relative path
echo does_it_exist($file);      //prints "true"

Look at Run 4. This code includes the phar file and passes the function a relative path. Relative to the current working directory, index.php does not exist. But relative to the contents of the phar archive, it does exist, which is why it prints "true"!

Any file paths that you manipulate inside of a phar archive must be absolute. If they are relative, the phar will treat the path as being relative to the phar archive, not relative to the current working directory, which may not be what you intended!

WSDL SOAP bindings confusion - RPC vs document

2011-05-08T10:03:00.001-04:00

What is the difference between RPC and document styles in SOAP web services?

I was asked this question at a job interview and was embarrassed to discover that I didn't know the answer (considering that I listed "SOAP web services" on my resume). I've heard the terms before, but couldn't remember what they meant. The main difference lies in what the body of the SOAP message looks like.

RPC vs document styles

The body of an RPC (remote procedure call) style SOAP message is constructed in a specific way, which is defined in the SOAP standard. It is built around the assumption that you want to call the web service just like you would call a normal function or method that is part of your application code. The message body contains an XML element for each "parameter" of the method. These parameter elements are wrapped in an XML element which contains the name of the method that is being called. The response returns a single value (encoded in XML), just like a programmatic method. The WSDL code for a RPC-style web service is less complex than that of a document-style web service, but this isn't a big deal since WSDLs aren't meant to be handled by humans.

A RPC-style request:

<soap:envelope>
  <soap:body>
    <multiply>    <!-- web method name -->
      <a>2.0</a>  <!-- first parameter -->
      <b>7</b>    <!-- second parameter -->
    </multiply>
  </soap:body>
</soap:envelope>

A document style web service, on the other hand, contains no restrictions for how the SOAP body must be constructed. It allows you to include whatever XML data you want and also to include a schema for this XML. This means that the client's and server's application code must do the marshalling and unmarshalling work. This contrasts with RPC in which the marshalling/unmarshalling process is part of the standard, so presumably should be handled by whatever SOAP library you are using. The WSDL code for a document-style web service is much more complex than that of a RPC-style web service, but this isn't a big deal since WSDLs aren't meant to be handled by humans.

A document-style request:

<soap:envelope>
  <soap:body>
    <!-- arbitrary XML -->
    <movies xmlns="http://www.myfavoritemovies.com">
      <movie>
        <title>2001: A Space Odyssey</title>
        <released>1968</released>
      </movie>
      <movie>
        <title>Donnie Darko</title>
        <released>2001</released>
      </movie>
    </movies>
  </soap:body>
</soap:envelope>

The main downside of the RPC style is that it is tightly coupled to the application code (that is, if you decide you want to call these web methods like normal methods--this is not a requirement, but this is what the RPC style was designed for). This means that if you want to change the order of the parmeters or change the types of those parameters, this change will affect the definition of the web service itself (just as it would affect the definition of a normal function or method).

Document style services do not have this issue because they are loosely coupled with the application code--the application must handle the marshalling and unmarshalling of the XML data separately. For example, with a document style service, it doesn't matter if the programmer decides to use a "float" instead of an "int" to represent a particular parameter because it's all converted to XML text in the end.

The main downside of the document style is that there is no standard way of determining which method of the web service the request is for. It's easy to get around this limitation, but, however it's done, it must be done manually by the application code. (Note: The "document/literal wrapped" style removes this limitation; read on for more details.)

Another point to note about the document style is that there are no rules for how the SOAP body must be formatted. This can either be seen as a downside or a strength, depending on your perspective. It's a strength if you are looking for the freedom to handle the message the way you want, but a downside if you don't want to have to do the extra marshalling/unmarshalling work that it requires.

Encoded vs literal encodings

In addition to the RPC and document styles, there are two types of encodings: "encoded" and "literal".

Literal means that the SOAP body follows an XML schema, which is included in the web service's WSDL document. As long as the client has access to the WSDL, it knows exactly how each message is formatted.

Encoded, on the other hand, means that the SOAP body does not follow a schema, but still follows a specific format which the client is expected to already know. It is not endorsed by the WS-I standard because there can be slight differences in the way in which various programming languages and web service frameworks interpret these formatting rules, leading to incompatabilities.

This makes for 4 different style/encoding combinations:

RPC/encoded - RPC-style message that formats its body according to the rules defined in the SOAP standard (which are not always exact and can lead to incompatabilities).
RPC/literal - RPC-style message that formats its body according to a schema that reflects the rules defined in the SOAP standard. This schema is included in the WSDL.
document/encoded - Document-style message that does not include a schema (nobody uses this in practice).
document/literal - Document-style message that formats its body according to a schema. This schema is included in the WSDL.

There's also a 5th type. It isn't an official standard but it is used a lot in practice. It came into being to compensate for document/literal's main shortcoming of not having a standard way of specifying the web method name:

document/literal wrapped - The same as document/literal, but wraps the contents of the body in an element with the same name as the web service method (just like RPC-style messages). This is what web services implemented in Java use by default.

Is my understanding of all this accurate? Which approach do you think is the best? Let me know in the comments.

References:

Dropbox Tech Blog

2011-04-24T11:26:00.001-04:00

The folks over at Dropbox recently started a new blog, which is focused on the technology that the Dropbox application uses. Their first blog post describes how the Dropbox team translated Dropbox into multiple languages.

It starts by acknowledging the success that Facebook had using community translation to translate their site to French in 2008. Dropbox, however, chose not to use this technique because of the significant effort required to build a framework that not only accepts community translations, but quality-checks them and converts them to the multitude of i18n file formats that Dropbox uses.

Instead, they decided to go with a hybrid approach--professional translators would do the bulk of the translation, while users would have the ability to make corrections. The idea being that since users are the ones using the application, they have a much better idea of the context in which a particular word or phrase is being used. Professional translators on the other hand, have likely never even heard of Dropbox, so may produce translations which don't quite capture the true meaning of a word or phrase.

PHP-SQLUtils released

2011-04-02T16:41:00.002-04:00

Yesterday, I created a project on github called PHP-SQLUtils. This is a small PHP library that helps make building SQL queries easier and less error-prone. I've been using a variant of this library at work and it has definitely come in handy. One thing that it can do is build INSERT statements.

In your typical INSERT statement, the list of column names come first, followed by the list of column values.

INSERT INTO cars
(make, model, year, purchased) VALUES
('Ford', 'Focus', 2005, '20050520 00:00:00')

It's a little hard to tell what values go with what columns. It gets harder as the list of columns gets longer.

INSERT INTO cars
(make, model, year, purchased, owner, seats, mpg_city, mpg_highway, color, sun_roof) VALUES
('Ford', 'Focus', 2005, '20050520 00:00:00', 'George Washington', 5, 30.4, 35.1, 'gray', 0)

PHP-SQLUtils provides an interface for building INSERT statements. It puts the column names right next to their values, making it easy to figure out what values belong to what columns and reducing the chance of a bug appearing:

$utils = new MssqlUtils();
$insert = $utils->insert("cars");
$insert->s("make", "Ford");
$insert->s("model", "Focus");
$insert->n("year", 2005);
$insert->d("purchased", strtotime("2005-05-20"));
echo $insert->done();
//prints "INSERT INTO cars (make, model, year, purchased)
//VALUES ('Ford', 'Focus', 2005, '20050520 00:00:00')"

Check out PHP-SQLUtils here.

PHP Getter/Setter method generator

2011-03-22T20:16:00.000-04:00

Here's a utility I created that generates getter and setter methods for a PHP class. Copy and paste your PHP class into the textbox, then click generate. It uses a regular expression to identify the private and protected fields and generates a getter/setter pair for each. You can also customize the indentation and newline preference of the generated code:

http://www.mangst.com/projects/getter-setter-gen/

Running PHPUnit from Eclipse

2011-03-16T21:00:00.000-04:00

If you've ever written JUnit tests in Java with Eclipse, you'll know that JUnit is integrated right into the IDE. Eclipse doesn't give you built-in unit testing support with PHP, but it's still possible to run your tests without having to leave the IDE.

In this screen cast, I'll show you how to run PHPUnit from Eclipse.

PHP PEAR libraries and Eclipse

2011-03-14T20:52:00.003-04:00

If you develop PHP applications using Eclipse, here is a tip that may come in handy. PDT, the Eclipse plugin for PHP development, will, among other things, display the documentation for any method, function, or field that you hover your mouse over. And you can jump to the definition of that method, function, or field by control-clicking on it.

But if your application uses libraries installed with PEAR, then this extra functionality will not work for those libraries. However, this can be fixed by providing Eclipse with the locations of these libraries.

Note: Click on the screenshots below to see a larger image.

1. Right click on your PHP project and select "Properties".

2. Select "PHP Include Path" from the left-hand menu. Click the "Libraries" tab, then click "Add Library...".

3. Select "User Library" and click "Next".

4. Click "Configure...". Another new dialog will appear.

5. You want to add a new library, so click "New..." and type in a name for the library. I'm going to add PHPUnit.

6. Select the newly added Library and click "Add External folder...". This is where we point to the file system location of the library.

7. Select the path to the library. In Windows, PEAR installs its libraries inside the "PEAR" folder under the PHP installation directory. On Mac OSX, its location is "/usr/local/PEAR". On Ubuntu, it should be at "/usr/share/php/PEAR". If you can't figure out where your PEAR directory is, you can find it in the "include_path" setting in your php.ini file (when PEAR is installed, it automatically modifies the PHP include path to include itself). Click "OK".

8. Click "OK", then click "Finish"!

In my next post, I'll explain how to run PHPUnit unit tests from Eclipse without needing any plugins!

Do you have any tips for developing in PHP on Eclipse? Let me know in the comments section!

Freeing hard disk space in Ubuntu Linux

2011-03-13T13:11:00.001-04:00

Here are some helpful disk cleanup tips that I came across during my quest to free up space on my tiny 4GB netbook hard drive.

1. apt-get

This command is what you use in Ubuntu to install and update the applications on your computer. But apt-get can also be used to free up disk space:

sudo apt-get clean
Deletes package files that were downloaded when installing or updating software.

sudo apt-get autoclean
Like "clean", but only removes packages which are no longer in the Ubuntu repository or for which a newer version exists.

sudo apt-get autoremove
Removes dependencies that are no longer used by any applications. For example, a spell checker library may have been installed along with OpenOffice. However, when OpenOffice is uninstalled, the spell checker might not be.

2. df

This command won't itself free any space, but it can show you which files and directories are the biggest hogs (maybe there's a large video file buried somewhere that you don't know about). The following command will recursively list all files and directories in the given directory and sort the list by file size.

du -a folder/path | sort -n

It's going to list every single file, so the list might get quite long. It might be good to direct the output to a file and reverse the sort so the largest files appear at the top of the file:

du -a folder/path | sort -nr > disk-usage.txt

3. localepurge

If you can only speak one language like me, then check out localepurge. It will remove all help files from your system that are in languages you don't understand. It automatically runs every time you install any new software.

sudo apt-get install localepurge

As soon as it is installed, it will ask you for the languages you want to keep. For example, I selected "en", which is the two letter code for "all variants of the English language" (American English, British English, etc). I selected this instead of just "en_US" (American English) to err on the side of caution. Note: Press the spacebar to select a language from the list and then press enter when you are done.

Then, run the program:

sudo localepurge

It will go through your hard drive and delete various files such as man pages. It freed a good 53MB for me:

4. Language packs

You can also try removing language-related packages using the Synaptic Package Manager. Search for "language" and remove all languages that you have no need for. Sort by the left-most column so that the installed packages appear first in the list. This freed even more space for me--about 200MB.

Do you have any tips for freeing up hard drive space? Let me know!

Setting up a Virtual Host in Apache 2 on Linux (Ubuntu)

2011-03-12T12:13:00.004-05:00

When you install Apache on your Linux computer, it is configured to treat all files in "/var/www" as the web root by default. To run a website locally on your machine, you could copy all files into this directory. However, this is not the best way to go if you are doing serious development work, such as designing multiple websites on the same computer.

What you should do is create a virtual host. This allows you to store your website files anywhere you want:

1. Configure Apache
First, you have to add a virtual host to Apache. At least on my distro (Ubuntu), Apache splits its configuration settings into multiple files for organizational purposes. The "apache2.conf" file is the "root" file and imports all the other files. Virtual hosts are defined in the "/etc/conf/apache2/sites-enabled" directory, each inside their own file. The name of the file does not matter. Create a new file in this directory and enter the following text, replacing paths where necessary:

<Directory "/home/michael">
 Order Deny,Allow
 Allow from all
</Directory>

NameVirtualHost  127.0.0.1

<VirtualHost 127.0.0.1>
 DocumentRoot "/home/michael/testhost"
 ServerName testhost.local
</VirtualHost>

The "ServerName" will be the URL you use to access your website. ".local" is a naming convention used in the industry for running websites locally.

Apache must always be restarted whenever you change its conf settings:

sudo service apache2 restart

Tip: In order to edit these files, you have to open them as root. Using a command-line text editor can be cumbersome, especially if you need to have multiple files open at once. Instead, just open up a GUI text editor as the root user. For example:
sudo gedit &

2. Edit hosts file
Next, edit your host file. On Linux, this is located at "/etc/hosts". Add the following line at the end, replacing "testhost.local" with the "ServerName" parameter in the Apache conf file.:

127.0.0.1 testhost.local

This will tell your browser to access this URL from your local machine, instead of trying to find it out on the Internet.

Tip #2: In Linux, it can be hard to remember the locations of all these configuration files, especially because they can change depending on the distro. Any time you come across a new config file you haven't seen before, store its location in a centralized place, such as in a text file in your home folder. That way, you'll know exactly where to find it the next time you need it.

3. File permissions
Make sure the folder and all files in it are globally readable. Otherwise, Apache will not be able to see them and you will get "Forbidden" errors.

chmod -R 744 /home/michael/testhost

Google's Cr-48 netbook

2010-12-14T10:45:00.000-05:00

If you haven't heard, Google is releasing its own operating system, called Chrome OS (check out this marketing video). It's based on Linux, but is tailored solely for use with online, web-based applications, such as Gmail and Google Docs. They are also releasing a netbook, called Cr-48, that comes installed with Chrome OS. The whole idea is that all your data is stored in the (mysterious, but reliable) cloud, so if something bad happens to your computer, none of your data is lost. It's almost like we're returning to the dummy-terminal days of yore. A pre-production version of the netbook has been given to a number of people so that Google can get feedback on the device and improve upon it before releasing it to the public.

One thing that differentiates Cr-48 from other netbooks is the keyboard. Google modified it slightly to better integrate it with the OS. The "Caps Lock" key was replaced with a "Search" key, which opens a new tab in the browser. The function keys (F1, F2, etc) were replaced with keys that perform tasks such as controling the brightness of the screen, controlling the speaker volume, and going back and forward in the browser.

The touchpad works as follows: Tap it with one finger for a left mouse click, tap it with two fingers for a right mouse click, drag two fingers across its surface to scroll. People online have expressed frustration with this setup, but this is exactly how the touchpad of my Asus Eee netbook functions and I'm satisfied with it. Cr-48 comes with a single USB port with extremely limited functionality. It doesn't support anything other than a mouse or keyboard, not even a thumb drive. It comes with an SD card slot, although Chrome OS currently doesn't recognize it. A headphone jack is included and is fully functional. It has a 16GB solid state hard drive.

In terms of internet connectivity, Wi-Fi and 3G (which is on Verizon's network) are supported. You get a free 100MB per month of 3G bandwidth for the first two years. The netbook doesn't have an Ethernet jack, so you're limited to wireless connections. It comes with a VGA port, allowing you to plug in a larger monitor or a projector.

The idea of an OS which stores all of its applications and data in the cloud is an interesting concept, but I'm in no hurry to start using it. Even though Google provides very well designed online applications, I would imagine that they are not as powerful as their desktop equivalents. Cr-48 sounds more like something you'd take with you on the road or to the coffee shop to do casual work.

Java: The Next Generation

2010-12-03T16:40:00.000-05:00

A few weeks ago, plans for the next version of Java were released. It will be split into two different versions in order to more quickly get a new version out the door. Two key features that were originally slated to be released in Java 7, will be pushed back to Java 8.

One of these features is called Lambda, which adds closures to Java. From what I understand, a closure is like a function that can be assigned to a variable. This means that the closure can be passed as an argument to another function, and then called inside of the function's body. If the closure is declared inside of another function, it has access to the parent function's local variables. Many languages support this. Here is an example of how closures can be used in Javascript (taken from Wikipedia):

// Return a list of all books with at least 'threshold' copies sold.
function bestSellingBooks(threshold) {
  return bookList.filter(
      function (book) { return book.sales >= threshold; }
  );
}

The function filters the bookList array, returning only the books that have sold a mininum number of copies. It does this by passing a closure into the filter() method, which gets executed on every book in the array. If the closure returns true for a particular book, then that book is filtered out.

The other major feature that was pushed back to Java 8 is called Jigsaw. The goal of Jigsaw is to break the JDK into separate modules. This would help applications boot up more quickly. It also would help in situations where a JRE is packaged with the application. This JRE could be reduced in size by only including the modules that the application needs.

Java 7 is slated for release in mid-2011 and Java 8 is to be released sometime in 2012.

Linux patch increases multi-tasking performance

2010-11-19T14:32:00.000-05:00

A couple days ago, a programmer named Mike Galbraith submitted a patch for the Linux kernel which significantly increases the performance of multi-tasking on the Linux desktop. With this patch, you can run multiple, CPU-intensive applications at the same time more effectively. Phoronix has two videos demonstrating the effect of this patch. Each video shows a computer that has number of tasks running simultaneously, such as compiling the Linux kernel and watching a high definition video. On the computer without the patch, the video was very stuttery and looked like a still image most of the time. But on the computer with the patch, the video plays almost perfectly. The patch is only about 200 lines of code, which is small compared to the large improvements the patch brings. Linus Torvalds, the developer in charge of the Linux kernel, liked the patch very much, calling it "one of those 'real improvement' patches", meaning that it creates few, if any, negative side-effects.

The patch achieves these performance gains by tweaking the scheduler (the scheduler determines when each running program is allowed to get CPU time). It works by grouping all programs by the TTY (terminal) that they were started in. CPU time is then spread out evenly across each group, as opposed to being spread out evenly across each individual program. Slashdot user tinkerghost gives a good example:

As an example, we need to run extinguish_fire and evacuate_building at the same time. extinguish_fire spawns a thread for each bucket in the brigade, while evacuate_building only spawns a thread for each escape route. Now, if there are 96 buckets and 4 escape routes, extinguish_fire will consume 96% of the CPU and choke out the evacuate_building threads...By grouping all of the threads from a program, extinguish_fire and evacuate_building get equal footing regardless of the number of threads they spawn.

Because it groups according to the TTY, it seems that you will only notice a performance improvement if you are running command-line programs. From what I've read, GUI programs are TTY-less. With this patch, GUI programs would all be lumped into the same group, so there would be no performance gain. In order to take advantage of the new scheduler, you would have to run a program from a new terminal in order to place it into another group.

Today, a Red Hat developer named Lennart Poettering found out that you can achieve the same thing just by running a few commands and editing a configuration file. This alternative solution is implemented differently, grouping programs by session instead of TTY, but it achieves the same effect. Linus says that this is actually how the kernel patch was originally tested, but that it should still be included in the kernel because it's such a good improvement.

Internationalization in Java

2010-11-09T15:01:00.000-05:00

Internationalization means developing your application in such a way so that people from different countries who speak different languages can still use your program. For example, a French user, when presented with a "Yes/No" dialog, should instead be shown a "Oui/Non" dialog. The term "internationalization" is often abbreviated to "i18n". The "18" represents the eighteen letters between the first and last letters of the word.

Java provides a very nice way of doing this. It involves putting all of the text that your application uses into .properties files, which reside somewhere on the classpath. Each translation has its own .properties file and is named according to the language and (optionally) the country. For example, a German properties file would look like "messages_de.properties" ("de" being the standard, two-letter abbreviation for "German"). A British English properties file would look like "messages_en_UK.properties".

The properties files are organized in a hierarchy. This means that when searching for a particular string, the language/country file is looked for first (messages_en_UK.properties) followed by the language file (messages_en.properties) followed by the default file (messages.properties). One benefit to this is that if two translations are mostly the same (like US and UK translations), the parent file can contain most of the text, while the child files can define the differences. This helps to reduce duplication. It also allows for "fall-back" translations. For example, if there is no American English file, then it will use the English file.

To access these files in the Java code, the ResourceBundle class is used like so:

ResourceBundle messages = ResourceBundle.getBundle("com/example/messages");
System.out.println(messages.getString("hello.world"));

In this example, the .properties files are located in the "com.example" package and have names that begin with "message". It finds the property with a name of "hello.world" and prints its value to the console.

To determine which language to use, the Java application looks at the default Locale, which is automatically set in every Java program. ResourceBundle uses the default Locale to determine which properties file to use. For example, a French user using your application will automatically be shown the French translation because his or her default locale is set to France. No extra work needs to be done by the application.

Additionally, you can add arguments to your property values. This allows you to customize a message at runtime. For example, the following property contains two arguments:

hello.someone=Hello, {0}!  I'm {1} to see you!

To populate the arguments, you must use the MessageFormat class like so:

ResourceBundle messages = ResourceBundle.getBundle("com/example/messages");
String hello = messages.getString("hello.someone");
System.out.println(MessageFormat.format(hello, "Joe", "happy"));

The above code would print "Hello, Joe! I'm happy to see you!" to the console.

How HTTPS Works

2010-11-05T15:15:00.000-04:00

I'm currently reading the book Java Web Services: Up and Running by Martin Kalin. In Chapter 5, he discusses issues related to security. He starts out by giving a brief overview of HTTPS.

HTTPS is a secure version of HTTP, the protocol that web browsers use to access websites over the Internet. With HTTPS, all communication is encrypted so that it can't be intercepted or altered by malicious attackers. This is crucial to ensuring that, for example, nobody steals your credit card information when you purchase something from an online shopping site.

When your browser visits an HTTPS website, it first must initiate the connection in a process known as a handshake. The browser starts by requesting the server's digital certificate. The digital certificate contains the server's public key as well as a digital signature, which is said to sign the certificate. The digital signature is usually from a CA (certificate authority) such as VeriSign, but can also be self-signed. The browser checks its trust-store to see if it has either (a) a certificate matching the server's certificate or (b) a certificate corresponding to the digital signature. For example, the browser's trust store may not have a certificate for Amazon, but it probably does have a certificate for VeriSign, which is the CA that signed Amazon's certificate.

If the browser can't find an appropriate certificate in its trust-store, then it will show a scary security warning saying that it's dangerous to proceed with the connection. The danger is that a malicious attacker could create a certificate which tries to present itself as being from a reputable organization, like Amazon. He or she could create a fake website which looks like the Amazon website and fool you into buying something, thus giving him or her access to your credit card information.

If the certificate validates against the trust-store, the client generates a pre-master secret key, which is random string of 48 bits. It then encrypts it with the server's public key and sends it to the server. Since only the server has the private key, only the server can decrypt it, which means that the key can't be intercepted by a malicious attacker. Public/private key encryption is called asymmetric encryption. Then, the client and the server use the pre-master secret key to create a master secret key. Because they both used the same pre-master secret key, the master secret key will be identical on both the client and server. This master secret key is then used to encrypt and decrypt all subsequent communication between the client and server. This is called symmetric encryption because only one key is needed to both encrypt and decrypt the data. Symmetrical encryption is much faster than asymmetric encryption (about 1000 times faster).

Screen-scraping Wikipedia

2010-10-30T16:31:00.001-04:00

In order to screen-scrape a page on Wikipedia, there is one extra step that you must take in order to successfully download a page for processing. You must include a User-Agent header in your HTTP request. Wikipedia requires that this header be included or else it will return a 403 Forbidden error. I found this out thanks to a user on the #mediawiki IRC channel. They suggest that you set the User-Agent to something which uniquely identifies your program or application. They strongly discourage using the User-Agent string of a browser because this signals that you might be doing something malicious.

It is easy to set the User-Agent header in PHP. You can either edit your PHP installation's php.ini file or add the following line of code to your PHP script. The cURL library also supports setting HTTP headers, but this library is not included in the standard PHP installation.

//tell it what value to use for the User-Agent header
ini_set('user_agent', 'My Cool Screen-Scraper (+http://www.mangst.com)');

//includes the above User-Agent header in this request and all subsequent requests
$page = file_get_contents('http://en.wikipedia.org/wiki/Pumpkin');

Note that this is different from the header() function. The header() function is used to set the headers of the HTTP response that the PHP script itself is generating. This has nothing to do with any HTTP requests that the script makes in the process of generating its response.

Poor Man's FTP

2010-10-25T16:42:00.000-04:00

In the November issue of Linux Journal Magazine, Kyle Rankin wrote an article about his experience attending the DEF CON conference. One lesson he took away was the importance of knowing how to use the basic Linux commands, such as vi and sh. Being familiar with these commands means that you won't be dead in the water if you have to work on a computer with a minimal Linux install.

One of these commands is netcat (nc). Netcat allows you to open TCP and UDP connections with other computers as well as listen for connections. Kyle described many interesting ways that you can use this command. My favorite was using it to transfer files. I think that this technique would come in very handily if ssh or ftp is not installed.

It's very simple. The computer receiving the file runs this command:

nc -l 31337 > output_file

And then the computer sending the file runs this command:

nc hostname 31337 < input_file

This will send the file through port 31337 and automatically close the connection when the transfer is complete. It doesn't matter what port you use, so long as the port isn't being used by another program.

Starbucks Wi-Fi

2010-10-22T18:21:00.000-04:00

I ran into a small problem at a Starbucks store the other day. I had my netbook with me and wanted to connect to their Wi-Fi network. The service is free, but in order to access the Internet, you must first visit a Starbucks webpage that asks you to accept their terms and conditions. Any attempt to visit any other website will redirect you to this page.

I like to have my browser reopen all the tabs from my last browsing session when it starts up. My problem was that, because I have to accept the terms and conditions first, all my tabs would redirect to the Starbucks page. Clicking the back button after accepting the terms and conditions doesn't return me to my original page. I think that this is because it's a HTTP 3xx redirect response. So I basically lose all my tabs.

Getting around this wasn't too tricky. The terms and conditions page is just an HTML form with a bunch of hidden parameters and a checkbox for "I agree". I wrote a Java program to parse all the parameters out of the page and submit the form. So if I run this before opening my browser, my browser will reload all its tabs no problem. No annoying redirects to the Starbucks page.

You can download it here. I put all the classes in one file to make it simpler. I also wrote some JUnit tests to test the part that parses the HTML page. To run it, just compile the file and run java Starbucks -v. The -v (verbose) is optional and will cause it to print status messages as it's working. Run this program as soon as you connect to the Starbucks Wi-Fi network (and before you open your browser).

Peer-to-Peer (P2P) Systems

2010-10-18T15:18:00.002-04:00

In the current issue of the magazine Communications of the ACM, there is an article called Peer-to-Peer Systems by Rodrigo Rodrigues and Peter Druschel. Along with discussing the pros and cons of P2P networks and including examples of how they are used, the article goes into technical detail about how they work.

The article divides P2P systems into two types. One type is partly centralized. In these systems, there exists a single controller node, which keeps a list of all nodes that are connected to the network, along with the resources that each node is sharing. A good example of this kind of P2P network would be Napster (now non-existent). When you searched for a song on Napster, a request would be sent to a centralized server owned by the Napster folks themselves. This server would then search its database for all computers in the P2P network that had the song, and return this list to you. You then downloaded the song by directly connecting to the computer hosting the file. Without this server, there would be no way to get the song that you were looking for because you wouldn't know what computers, out of all the computers in the entire Internet, are both sharing their music collection and have the song you are looking for.

The other type of P2P network is decentralized. In a decentralized P2P network, there is no controller node that knows about all the computers in the network. Computers connect to the network through a bootstrap node, which is just one computer in the network that makes its IP address publicly known. This lack of centralization makes these types of P2P networks more robust, as the network is not dependent on a single server being functional. But because of this lack of centralization, there is no straight-forward way of knowing what computers are connected to the network or what resources they are sharing. This makes searching for particular resources trickier. The article describes two ways in which a decentralized P2P network can be structured in order to solve this problem of search.

One way of structuring a decentralized P2P network is by using an unstructured overlay (an overlay is a graph that describes how the nodes are connected with each other). Each computer in the network only knows about a few other computers that are also connected to the network. To search for a file, the computer will query its neighbors first. If none of its neighbors have the file, then it will ask its neighbor's neighbors. If none of these computers have the file, then it will ask its neighbor's neighbor's neighbors, and so on. This kind of decentralized network is fine if the resource you are looking for is replicated across many other nodes. However, if the resource is rare, then finding that resource could take a very long time. For example, imagine having to search for a file which exists on only one computer in a network of one million computers. The odds of that computer being within a short search distance to your computer is slim.

The other way a decentralized P2P network can be structured is by using a structured overlay. I didn't quite understand the specifics of this technique, but it involves the use of unique keys. Each computer in the network is assigned a unique key in such a way that all keys are evenly spread out in the key space. For example, if the key space is 0-999, the first node will be assigned a random key, say 432. Then, the second node will be assigned a key around 932 (999/2+432, on the opposite side of the "circle"). The third key will be assigned a key around either 682 or 182, and so on. Each node only directly knows about its two neighbors, so the overlay graph looks like a circle. The advantage to this type of overlay is that it makes searching much faster. It's able to use these unique keys to quickly find a computer hosting the resource. This is called key-based routing (KBR). Even if the resource is rare, it will still be able to find it quickly (unlike unstructured overlays, which must spend the time to ask each node directly). However, the downside is that there exists an overhead to maintain these keys. Extra work must be done every time a node enters or leaves the network, so if the number of computers that are connected to the network is constantly changing (called churn) this may not be the best solution.

It was a very good article, but I do have one criticism: it considers applications like SETI@home to be P2P. How is this P2P? You do not communicate with the other peers on the network. You only communicate with the centralized SETI server in order to download new data to process. I think that a better category for SETI@home would be "distributed computing", not P2P.

How to access your home computer over the Internet with VNC

2010-10-02T19:57:00.000-04:00

The VNC protocol gives you remote control of another computer's screen. You can see and interact with the computer as if you were sitting right in front of it.

In this blog post, I'm going to describe how to set up your computer so that you can connect to it anywhere in the world through the Internet. If your home computer is connected to a router (which it probably is), then the process is a little tricky, which is why I thought it would be helpful to write this blog post.

1. Install a VNC Server

First, you must install a VNC Server on the computer you want to control.

Mac OS X already comes with the necessary software, so you don't have to install anything. To enable Mac's VNC Server, do the following:

a. In System Preferences, click on "Sharing".
b. Check the "Screen Sharing" checkbox to enable it.
c. Click the "Computer Settings..." button.
d. Check the box that says "VNC viewers may control screen with password". Type in a password, then click OK. Remember that your computer will be visible to the world, so make sure the password is secure!

If you are running Windows, can you use TightVNC Server as a VNC Server.

2. Configure your router

Your home computer is probably connected to a router, either through a wired, ethernet connection or a wireless connection. A router connects all of your computers together to form a home network and acts as the gate keeper to the Internet. But if your computer is connected to a router, then it doesn't have its own IP address, which is what you need in order for VNC to connect to your computer.

To get around this, you must tell your router to forward VNC traffic to the computer you want to control. VNC communicates over port 5900, so you must tell your router to forward all data it receives from this port to the 5900 port on your computer. Here is how I did this with my Belkin router:

First, open the router's configuration web page by typing its private IP address in a web browser. My router's private IP is 192.168.2.1. (Private IP addresses are only visible within your home network--they are not visible from the Internet.)

Then, click on the "Virtual Servers" menu option under the "Firewall" category. It will ask you for a password. If you haven't configured the router with a password, then just click "Submit". This page lists all the data that the router will forward to other computers on the network. Pick an empty row and enter 5900 for the Inbound and Private port fields. Then, enter the private IP address of the computer you want to control. Finally, click on the "Enable" checkbox, then click "Apply Changes". As you can see in the screenshot, you can do this for other services too like SSH, FTP, or HTTP.

3. Test it out

You'll need a VNC Viewer in order to connect to the computer. Chicken of the VNC is a good VNC Viewer for Mac. For Windows, you can use TightVNC Viewer.

You'll also need the IP address of your router. An easy way to get the IP address of your router is to visit whatismyip.com from one of the computers that are connected to the router.

If the VNC Viewer asks for a display number, enter "0". Display "0" maps to port 5900, display "1" maps to port 5901, display "2" maps to port 5902, etc.

4. Get a free domain name (optional)

DynDNS is a free service that maps your IP address to a domain name like foobar.dyndns.com. Check to see if your router supports this service. My router will automatically update my DynDNS account whenever the router's IP address changes (which can happen often). Using DynDNS means you don't have to memorize your IP address or worry about it changing.

More Screen Scraping!

2010-09-22T17:26:00.001-04:00

The website CelebrityBookSigningsAndEvents.com contains a list of book signing appearances that different celebrities are making across the country. I like to go to this website occasionally to see if there is anybody I would be interested in seeing. However, it does not contain any search functionality that allows you to see what celebrities are visiting your area. I thought that it would be fun to screen scrape the webpage and make the data more searchable.

Screen Scraping Overview

Screen scraping means taking the HTML source code of a webpage and extracting data out of it. For example, one thing that I wanted to extract from CelebrityBookSigningsAndEvents.com was the titles of all the books. Looking at the source code, I could see a pattern to where the titles were located in the HTML. They all looked something like this:

<div class="modWrap">
  <p>
    <font size="4">
      <strong>
        <em>
          <u>
            <font color="#0000ff">Mysterious Galaxy</font>
          </u>
        </em>
     </strong>
    </font>
  ...
  </p>
  ...
  <p>
    <font size="4">
      <strong>
        <em>
          <u>
            <font color="#0000ff">Called To Coach</font>
          </u>
        </em>
     </strong>
    </font>
  ...
  </p>
  ... more tag hierarchies like these
</div>

To extract these book titles, I used XPath queries. This is a query language that allows you to pull data from specific parts of an XML document. To get all the book titles from my example, the XPath query would look like the following:

//div[@class="modWrap"]/p/font/strong/em/u/font

This will return a list all the font tags that are nested within this particular hierarchy of tags. The two slashes are the beginning mean that it doesn't matter what tags come before the div tag. The [@class='modWrap'] part returns only those div tags that have a class attribute with a value of "modWrap". The PHP code to run this query would look like the following:

//load the HTML source code into a DOM
$html = file_get_contents('http://www.celebritybooksigningsandevents.com/events');
$dom = new DOMDocument();
$dom->loadHTML($html);

//run the XPath query
$xpath = new DOMXPath($dom);
$bookTitleNodes = $xpath->query('//div[@class="modWrap"]/p/font/strong/em/u/font');
foreach ($bookTitleNodes as $node){
  echo $node->textContent; //the title is the text within the "font" tag
}

Problems

Aside from slight inconsistencies in the structure of the HTML, which I could account for by tweaking the XPath queries, the biggest problem I ran into was the fact that the source code was littered with these two strange characters--ASCII 160 and ASCII 194. Having ASCII values greater than 127, they were not part of the normal ASCII character set, which gave me problems in places where I needed to access individual characters in a string. They appeared as spaces in my web browser, but were not treated as spaces in my PHP code. Simply replacing all of these characters with spaces before creating the DOM fixed this issue.

The DOMDocument::loadHTML() function was throwing warnings that didn't affect the screen scraping results, but that I didn't want appearing on my webpage. You can silence the error messages that a function generates using PHP's @ operator: @$dom->loadHTML($html);

I also ran into a problem of my web server running an earlier version of PHP than my local computer. It took me forever to track down. FYI: DateTime::getTimestamp() is only supported in PHP versions 5.3 and above...

Implementation

After scraping the page, I save the data to an XML file and use it as a cache. If the file gets to be more than an hour old, it will refresh the cache by re-scraping the original webpage. This keeps the cache up to date with any changes that were made to the original webpage. And by using a cache, CelebrityBookSigningsAndEvents.com is not constantly harassed by requests from my website.

Bemuled!

2010-08-31T14:25:00.000-04:00

Yesterday, I finished designing a small game written in Java. It is called Bemuled! and it's based on the infamous game Bejeweled. The rules of the game differ slightly in that you are only given ten moves in which to make the most amount of points that you can. It uses Java's Web Start technology, so you can launch it simply by clicking on a link in your browser. It took about a week to make and I had a lot of fun making it. You can read more about the technical aspects of the application by reading the Technical Overview WHITEPAPER.

Does screen brightness significantly affect battery life?

2010-07-25T11:38:00.001-04:00

Many laptops will automatically dim the screen when running on battery power in order to conserve energy. But does this significantly increase the battery life? How much power is actually saved when the screen's brightness is turned down? My guess has always been that the energy savings are negligible, that a darkened screen might give the laptop 10 more minutes of battery life, nothing more.

To discover the answer, I ran a small experiment. I ran a script which polled the battery data on my netbook every 5 seconds. I monitored this data for 2 minutes with the screen on its brightest setting and 2 minutes with the screen on its darkest setting. I disabled my netbook's wireless network card and muted the speakers to avoid any interference in the data. The results were marginally better than I expected:

	Avg. mA	Bat. life (4200 mAh battery)
Brightest	1120.8	3h 42m
Darkest	1018.6	4h 7m

If kept on its darkest screen setting, my netbook battery would last about 25 minutes longer. If the brightness were to be set somewhere in between the two extremes to a level where the screen would actually be readable, the energy savings would probably be about half that--12.5 minutes.

The battery data was collected from "/proc/acpi/battery/BAT0/state". The script used to poll the data was written in PHP and is shown below:

<?php
while(true){
 $state = file_get_contents('/proc/acpi/battery/BAT0/state');

 preg_match('/present rate:\\s+(\\d+)/', $state, $matches);
 $presentRate = $matches[1];

 echo date('M d G:i:s'), ' ', $presentRate, "\n";

 sleep(5);
} 
?>

UNetbootin Troubles

2010-07-21T17:37:00.001-04:00

If you are having trouble running the UNetbootin Linux binary, you can try installing it using apt-get. This will automatically download and install any required dependencies.

1. Add the following lines to "/etc/apt/sources.list".

deb http://ppa.launchpad.net/gezakovacs/ppa/ubuntu [name] main
deb-src http://ppa.launchpad.net/gezakovacs/ppa/ubuntu [name] main

Where "[name]" is the name of your Ubuntu version:

Version	Name
10.04	lucid
9.04	jaunty
9.10	karmic
8.10	intrepid
8.04	hardy

2. Add the GPG key.

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 72D340A3

3. Get the list of packages from the newly added source.

sudo apt-get update

4. Install UNetbootin. It will appear in the "System Tools" menu.

sudo apt-get install unetbootin

Resources:
1. How to install UNetbootin in ubuntu

Flair Additions

2010-05-31T21:16:00.001-04:00

So I made some improvements to my developer forum flair!

Firstly, I added a Sun Forums flair, which displays your username, number of posts, and number of Dukes.

The background image to the left changes depending on how many "Dukes" you have, which are like these special points you can get for answering questions. It will also put a star next to your name if you are a moderator.

This one was a little more complicated because I couldn't pass the HTML directly into a DOMDocument object like with JavaRanch. I had to use tidy to clean up the HTML before passing it to DOMDocument:

//tidy the page
$tidy = new tidy("http://forums.sun.com/profile.jspa?userID=$id");
$tidy->cleanRepair();
$tidy = preg_replace('/<\/?nobr>/', '', $tidy); //<nobr> tags must be removed for DOMDocument

//load the page
$html = new DOMDocument();
$html->loadHTML($tidy);

I also had to pretty much do all development on the mangst.com server because the PHP installation on my iMac doesn't have tidy installed and it doesn't support all of the GD graphics functions.

Secondly, I added the ability to view the flair as a dynamically created image (nerdgasm!!!). Just change the "type" parameter to "png" or "jpeg":

<img src="http://www.mangst.com/flair/sun?id=1071155&type=png" />

I was actually surprised how easy this was. I had this fear that generating an image programmatically would be hugely complicated, but it wasn't really. The hardest part was keeping track of the pixels to make sure everything lined up right. I tried to make it look as close as possible to the Javascript and HTML versions, but for some reason the bold version of Verdana comes out looking a lot less bold in the image.

JavaRanch Flair

2010-05-25T18:14:00.000-04:00

Check out my JavaRanch flair!

I was so inspired by the flair over at Stackoverflow that I thought it would be fun to create my own! It basically works just like Stackoverflow's does. You have three options for how to include it in your webpage:

Javascript - You can include it using a <script> tag, which injects the flair into the DOM via Javascript.

<script
  src="http://www.mangst.com/flair/javaranch?id=209694&type=js"
  type="text/javascript">
</script>

HTML - You can include it using an <iframe>, which loads the flair as HTML into the frame.

<iframe
  src="http://www.mangst.com/flair/javaranch?id=209694&type=html"
  marginwidth="0"
  marginheight="0" 
  frameborder="0"
  scrolling="no"
  width="220"
  height="100">
</iframe>

JSON - You can get the raw data with JSON and handle the data however you wish with Javascript.
```
JSON URL: http://www.mangst.com/flair/javaranch?id=209694&type=json
```

On the backend, what it does is it screen scrapes your JavaRanch profile page, plucking out the information that it needs. It uses a PHP DOMDocument object to load the HTML, then uses XPath to get the data fields.

$html = new DOMDocument();
$html->loadHtmlFile("http://www.coderanch.com/forums/user/profile/$id");
$xpath = new DOMXPath($html);
$username = $xpath->query("//span[@id='profileUserName']")->item(0)->textContent;
//...

I think it's pretty elegant, though it will break if JavaRanch decides to do any site redesigns. I was afraid that it wouldn't be able to load the HTML into a DOM, since webpage HTML tends not to be well formed XML, which would have prevented me from using XPath. The SimpleXMLElement class wouldn't accept the HTML, but the DOMDocument class did.

SimpleDateFormat and Thread Safety

2010-05-07T16:10:00.000-04:00

SimpleDateFormat
The Java class SimpleDateFormat converts Dates to Strings and vice versa. For example, the following code converts the String "04/15/2010" to a Date object and back again:

DateFormat df = new SimpleDateFormat("MM/dd/yyyy");
Date d = df.parse("04/15/2010");
String s = df.format(d);
"04/15/2010".equals(s); //true

This class is documented as not being thread-safe, but I decided to see for myself if this was true.

Thread-safety proof
I created a program that proves that SimpleDateFormat is not thread-safe. It creates X threads which run concurrently. Each thread generates Y random Dates, then formats each Date in two ways: using a local instance of SimpleDateFormat that no other thread has access to, and using a static instance of SimpleDateFormat which all threads use. The Strings created by the local instance are added to one List and the Strings created by the static instance are added to another List.

If everything is synchronized properly, then these two lists should be identical. But because SimpleDateFormat is not thread safe and all threads use a shared static instance, the lists do not always come out identical (I've found that around ten threads and ten dates-per-thread consistently produce different lists).

If the call to "staticDf.format()" is wrapped in a synchronized block, then the lists always come out identical, which shows that SimpleDateFormat needs to be manually synchronized and is therefore not thread-safe.

Usage
The following command will create ten threads, each of which will generate twenty dates:
java SimpleDateFormatThreadSafe 10 20

Source code

import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Calendar;
import java.util.Date;
import java.util.List;

/**
 * Proves that SimpleDateFormat is indeed not thread safe (as documented in the
 * javadocs).
 * @author mangstadt
 */
public class SimpleDateFormatThreadSafe {
  private static final String format = "MM/dd/yy";
  private static final DateFormat staticDf = new SimpleDateFormat(format);
  private static int numThreads = 10;
  private static int numLoopsPerThread = 10;

  public static void main(String args[]) throws Exception {
    //get the arguments
    if (args.length > 0){
      numThreads = Integer.parseInt(args[0]);
      if (args.length > 1){
        numLoopsPerThread = Integer.parseInt(args[1]);
      }
    }

    //create the threads
    MyThread threads[] = new MyThread[numThreads];
    for (int i = 0; i < threads.length; ++i) {
      threads[i] = new MyThread();
    }

    //start the threads
    for (MyThread t : threads) {
      t.start();
    }

    //check the results
    boolean allIdentical = true;
    for (MyThread t : threads) {
      t.join();
      if (!t.localList.equals(t.staticList)){
        System.out.println(t.getName() + " lists are different:");
        System.out.println("local:  " + t.localList);
        System.out.println("static: " + t.staticList);
        System.out.println();
        allIdentical = false;
      }
    }
    if (allIdentical){
      System.out.println("All lists are identical.");
    }
  }

  private static class MyThread extends Thread {
    public final List localList = new ArrayList();
    public final List staticList = new ArrayList();

    @Override
    public void run() {
      DateFormat localDf = new SimpleDateFormat(format);
      for (int i = 0; i < numLoopsPerThread; ++i) {
        //create a random Date
        Calendar c = Calendar.getInstance();
        c.set(Calendar.MONTH, randInt(0, 12));
        c.set(Calendar.DATE, randInt(1, 21));
        c.set(Calendar.YEAR, randInt(1990, 2011));
        Date d = c.getTime();

        //add formatted dates to lists
        localList.add(localDf.format(d));
        //synchronized (staticDf){
        staticList.add(staticDf.format(d));
        //}
      }
    }
  }

  private static int randInt(int min, int max) {
    return (int) (Math.random() * (max - min) + min);
  }
}

The Data URI

2010-04-11T16:48:00.015-04:00

Typically, an image is included in a web page by referencing an image file:

<img src="take-this.jpg" />

But it's also possible to put the image directly inside the web page source. This is done using a data URI:

<img src="data:image/jpeg;base64,image-data-goes-here" />

The URI contains the content-type of the image ("image/jpeg" in this case, since this image is a JPEG) and the image data encoded in base64.

This is supported by most browsers. Internet Explorer support lags behind, with only version 8 supporting it (to a limited extent).

This technique shouldn't be used on a regular basis, since it bloats the size of the web page considerably and makes it take longer to load. Including images as separate files (the typical way) lets the user explore the page while the more bandwidth intensive images are loading.

Check out my Data URI Generator to generate a data URI from an image of your choosing.

Bad HealthVault Method Schema URLs

2010-04-08T20:23:00.003-04:00

One thing to note when working with the HealthVault XML method schemas is that the links to these schemas in the HealthVault Developer Center (http://developer.healthvault.com/methods/methods.aspx), as well as in the GetServiceDefinition response, are not always correct. Some versions of some methods refer to the version 1 schemas, when they actually have their own schemas.

For example, the URL to the GetThings3 request schema is listed as:

https://platform.healthvault-ppe.com/platform/XSD/method-getthings.xsd

This URL incorrectly points to the version 1 schema. The version number must be added to the end of the file name in order to get the correct schema:

https://platform.healthvault-ppe.com/platform/XSD/method-getthings3.xsd

This holds true for the following methods:


CreateConnectPackage2  request
GetServiceDefinition2  response
GetThings3             request, response
OverwriteThings2       request
PutThings2             request

Creating a Connect-Request in HealthVault

2010-03-27T21:26:00.009-04:00

I've been doing a lot of work with Microsoft HealthVault at my job lately. I did some experimental work with the platform when I first started working there and since then, I've sort of been the team's HealthVault expert. It's been fun learning all about it and challenging as well, since most of the HealthVault libraries, tutorials, etc are centered around .NET (we use Java). So HealthVault is something I think would be fun to blog about.

This blog post will be about how to use what are called connect-requests in HealthVault. It assumes that you already have some knowledge of HealthVault from a developer's perspective. It includes Java code samples that use the JAX-B classes from the HealthVault Java Library.

Connect-requests are used by non web-based applications to create a connection to a HealthVault record. The process must only be completed once--not every time the application wants to access the record. They work like this:

1 The application prompts the user for the following information:

Friendly name - This can be anything, but should be the name that's on the HealthVault record.

Question - A question of the user's choosing (such as "What high school did I go to?").

Answer - The answer to the above question.

2 The application uses the above information, along with an external-id, to create a connect-request. The external-id can be any value that uniquely identifies the connect-request (the current time in milliseconds works fine). This value must be saved somewhere, as the application will need to use it again later (see step 5). Using these four bits of information, the application sends the request to HealthVault:

//create the request
CreateConnectRequestRequest request = new CreateConnectRequestRequest();
request.setExternalId(System.currentTimeMillis() + "");
request.setQuestion("Question?");
request.setAnswer("Answer");
request.setFriendlyName("Joe Smith");

//send the request / get the response
SimpleRequestTemplate srt = new SimpleRequestTemplate(ConnectionFactory.getConnection());
CreateConnectRequestResponse response = (CreateConnectRequestResponse) srt.makeRequest(request);
String identityCode = response.getIdentityCode();

Note: Because we're using the JAX-B request classes, be sure to use
com.microsoft.hsg.methods.jaxb.SimpleRequestTemplate
and not
com.microsoft.hsg.request.SimpleRequestTemplate

3 HealthVault returns an identity-code, which is a sequence of 20 random letters separated into five groups of four:

DHLE-ELHP-AQLG-TLPZ-PQKD

The application must show this to the user. It should also show (what I call) the patient connect URL. The user will have to visit this URL in order to validate the connect-request:

https://account.healthvault-ppe.com/patientwelcome.aspx

Remove the "-ppe" when going live of course.

4 The user visits the patient connect URL in a web browser. This page will step the user through a process, requiring her to (1) login to her HealthVault account, (2) enter the identity-code, (3) enter the question/answer, and (4) choose which record in her account to grant the application access to.

5 Once the user has done this, the application is able to retrieve the person-id and record-id of the user's HealthVault record:

String externalId = //retrieve the external-id you saved in step 2
GetAuthorizedConnectRequestsRequest request = new GetAuthorizedConnectRequestsRequest();
SimpleRequestTemplate srt = new SimpleRequestTemplate(ConnectionFactory.getConnection());
GetAuthorizedConnectRequestsResponse response = (GetAuthorizedConnectRequestsResponse) srt.makeRequest(request);
for (ConnectRequest cr : response.getConnectRequest()) {
  if (cr.getExternalId().equals(externalId)) {
    String personId = cr.getPersonId();
    String recordId = cr.getRecordId();
    //save to persistent storage (like a database)
    break;
  }
}

However, the application has no way of knowing when the user will approve the application (when she completed step 4). So, the application must continually poll HealthVault for newly approved connect-requests (e.g. a separate thread which calls GetAuthorizedConnectRequests every 30 seconds or so).

Note:

The CreateConnectRequest method has a "call-back-url" parameter, which is a URL that HealthVault is supposed to call when the connect-request is validated by the user. However, at the time of this writing, it is not supported.

Mac OS X Mouse Acceleration

2010-03-22T19:52:00.005-04:00

If the mouse acceleration settings of Mac OS X ever frustrate you, check out the Mouse Acceleration Preference Pane. This is a utility created by Christian Zuckschwerdt which lets you adjust these settings to your liking.

Apple changed around OS X's mouse API when version 10.6 was released, which made existing mouse acceleration tools useless. However, Christian just recently released an updated version supporting 10.6! My hand feels less cramped already...

Working with Scala and Maven

2010-03-14T20:50:00.002-04:00

In order to get Maven to properly recognize Scala code, there are are a number of steps you must take. Included below are pom.xml samples, along with explanations.

1. Name your source directories properly:

<project><build>

  <sourceDirectory>src/main/scala</sourceDirectory>
  <testSourceDirectory>src/test/scala</testSourceDirectory>

</build></project>

2. Add the scala-tools.org repositories.

These are required in order to download the necessary Scala dependencies (see step #3):

<project>
  <repositories>
    <repository>
      <id>scala-tools.org</id>
      <name>Scala-tools Maven2 Repository</name>
      <url>http://scala-tools.org/repo-releases</url>
    </repository>
  </repositories>

  <pluginRepositories>
    <pluginRepository>
      <id>scala-tools.org</id>
      <name>Scala-tools Maven2 Repository</name>
      <url>http://scala-tools.org/repo-releases</url>
    </pluginRepository>
  </pluginRepositories>
</project>

3. Add the appropriate dependencies:

<project><dependencies>

  <!--
  This contains the scala compiler, which Maven will use to compile your code and run your unit tests.
  -->
  <dependency>
    <groupId>org.scala-lang</groupId>
    <artifactId>scala-library</artifactId>
    <version>2.7.7</version>
  </dependency>

  <!--
  This is Scala's unit testing framework.  It's also possible to use JUnit, but when in Rome, right?
  -->
  <dependency>
    <groupId>org.scalatest</groupId>
    <artifactId>scalatest</artifactId>
    <version>1.0</version>
    <scope>test</scope>
  </dependency>

  <!--
    If you want to use scalatest, unfortunately you also need to include JUnit as a dependency (see step #5).
  -->
  <dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.8.1</version>
    <scope>test</scope>
  </dependency>

<dependencies></project>

4. Add the scala-tools plugin:

<project><build><plugins>

  <plugin>
    <groupId>org.scala-tools</groupId>
    <artifactId>maven-scala-plugin</artifactId>
    <executions>
      <execution>
        <goals>
          <goal>compile</goal>
          <goal>testCompile</goal>
        </goals>
      </execution>
    </executions>
  </plugin>

</plugins></build></projects>

5. Add the @RunWith annotation to each unit test.

If you are using scalatest as your unit testing framework, you must trick Maven into thinking that your tests are JUnit tests. Otherwise, Maven will not run your tests:

import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalatest.FunSuite

@RunWith(classOf[JUnitRunner])
class MyScalaTest extends FunSuite{
  //...
}

Tip: If you're using Eclipse, you can right click on one of these unit tests and select "Run As > JUnit Test" to manually run the test.

XPath and Java

2010-03-10T20:45:00.017-05:00

XPath is a domain specific language which is used to extract data from an XML document. It's supported by many different general purpose languages like C#, PHP, and Java. Take the following XML document for example:


<library>
  <book>
    <language>en</language>
    <title>Solaris</title>
    <author>Stanislaw Lem</author>
  </book>
  <book>
    <language>fr</language>
    <title>Le Petit Prince</title>
    <author>Antoine de Saint-Exupéry</author>
  </book>
  <book>
    <language>en</language>
    <title>Dune</title>
    <author>Frank Herbert</author>
  </book>
</library>

The following XPath query retrieves the titles of all English-language books:


/library/book[language='en']/title

Java makes working with XPath (and XML in general) kind of complicated, since many different classes are involved. First, the XML document must be loaded into a DOM (Document Object Model).


StreamSource source = new StreamSource(new File("books.xml"));
DOMResult result = new DOMResult();
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.transform(source, result);
Node documentRoot = result.getNode();

This means that the XML text is read into memory and organized into a tree of nodes where each tag is an element node. The top element node would be the <library> element, which would have three child element nodes (<book>), and so on.

To demonstrate the power of XPath, this is what Java code might look like if XPath did not exist. The programmer would have to manually iterate through the entire DOM to get what she needed.


List<String> englishTitles = new ArrayList<String>();
Node books = documentRoot.getFirstChild();
if (books.getNodeName().equals("library")){
  for (int i = 0; i < books.getChildNodes().getLength(); ++i){
    Node book = books.getChildNodes().item(i);
    if (book.getNodeName().equals("book")){
      boolean english = false;
      String title = null;
      for (int j = 0; j < book.getChildNodes().getLength(); ++j){
        Node bookChild = book.getChildNodes().item(j);
        if (bookChild.getNodeName().equals("language") &&
            bookChild.getTextContent().equals("en")){
          english = true;
        }
        if (bookChild.getNodeName().equals("title")){
          title = bookChild.getTextContent();
        }
      }
      if (english && title != null){
        englishTitles.add(title);
      }
    }
  }
}
for (String title : englishTitles){
  System.out.println(title);
}

As you can see, this is very very tedious and error prone! XPath is the better solution by far:


XPath xpath = XPathFactory.newInstance().newXPath();
NodeList nodeList = (NodeList)xpath.evaluate("/library/book[language='en']/title",
documentRoot, XPathConstants.NODESET);
for (int i = 0; i < nodeList.getLength(); ++i){
  Node node = nodeList.item(i);
  System.out.println(node.getTextContent());
}

Namespaces

XML documents often use namespaces. These are sort of like Java packages--they group related elements together and prevent name collisions from occurring. Let's say that each <language> element belonged to a namespace:


<book>
  <language xmlns="http://translate.google.com">en</language>
  <title>Solaris</title>
  <author>Stanislaw Lem</author>
</book>

Note: While namespaces technically can be anything (like "abc123" for example), they should be globally unique. There's no way to enforce this, so the convention is to use a URI belonging to the person or company creating the namespace. For example, if Oracle wants to use a namespace, they can be fairly certain that no one else in the entire world is using one starting with "http://www.oracle.com".

To make Java aware of namespaces,a NamespaceContext object must be created and added to the XPath object.


XPath xpath = XPathFactory.newInstance().newXPath();
xpath.setNamespaceContext(new NamespaceContext() {
  public String getNamespaceURI(String prefix) {
    if ("tr".equals(prefix)){
      return "http://translate.google.com";
    }
    return null;
  }
  public Iterator getPrefixes(String uri) {
    return null;
  }
  public String getPrefix(String uri) {
    return null;
  }
});

This will assign the prefix "tr" to the namespace "http://translate.google.com". The prefix can be anything, but the namespace must match the one in the XML document.


NodeList nodeList = (NodeList)xpath.evaluate("/library/book[tr:language='en']/title",
documentRoot, XPathConstants.NODESET);
for (int i = 0; i < nodeList.getLength(); ++i){
  Node node = nodeList.item(i);
  System.out.println(node.getTextContent());
}

To learn more about XPath, you can visit w3schools.com.

Dropbox

2010-03-09T19:59:00.004-05:00

I just found out about this cool service called Dropbox, which I thought I'd write about. I read a blurb about it in the March issue of Linux Journal.

Dropbox is a free service that lets you store files online. That sounds pretty boring. But it's more than just a free FTP server. What happens is, you install a small app on your computer, which creates a special "dropbox" folder in your home directory (or in My Documents if you're using Windows). The app monitors this folder for any changes and syncs these changes with the Dropbox server. For example, when you copy a file (Word document, MP3, whatever) to this folder, it will immediately upload it to your Dropbox account. If you delete a file, it will delete it from your account.

But the really cool thing is that you can connect multiple computers to your account by installing the Dropbox application on each one (there are Windows, Mac, Linux, and iPhone versions). So if you add a file to your Dropbox folder on your desktop PC, it will immediately download to your MacBook laptop, Linux Netbook, and whatever else.

An Amazon Affiliate Tutorial

2009-10-22T21:19:00.010-04:00

Ok, so I'm a little behind the times. Amazon's Affiliate Program has been around for a while--since 1996 to be exact. Everyone and their grandma must have an account by now, making millions off sales of "The Secret" that they linked to on their blog (this is actually the secret that the book talks about). But incase you happen to be Amish, here's a short tutorial on the whole process of signing up for an account and building affiliate links.

1. Visit https://affiliate-program.amazon.com/ and click the "Join now for FREE!" button. It requires that you create an Amazon account first if you don't already have one.

2. It then asks you for your contact information and has some questions about the website/blog where you'll be putting your affiliate links.

3. And that's all the information they need to get your account set up. You don't even have to submit your payment information (although, you'll want to do this eventually...after all, making a few bucks is why you're here, right?). As you can see in the screenshot, Amazon assigns you a unique "Associates ID". This important identifier will be stuffed into all the affiliate URLs you create so that Amazon knows who give money to when a sale is made. To start making links, click the "Learn More" link in the "Product Links" box.

4. It's really easy to create affiliate links. It just takes a few clicks of the mouse and some copy and pasting. First, search for the product you want to link to. I chose a book I recently read.

5. Next, find the product in the search results and click the "Get Link" button.

6. This will take you to the page that lets you build your link. There's a lot of options here you can play with, so take some time to explore this page on your own. Once you're done crafting the link to your liking, copy the HTML code in the "step 3" box and paste it on your site. It's no secret--it's that simple!

Using Magpie to parse Blogger feeds

2009-10-06T22:46:00.028-04:00

I recently redesigned my website and one thing I changed was putting my blog on the front page instead of just linking to it. But what's cool (at least, according to my own geeky tastes) is that my blog is hosted on Blogger--not on my website! In this post, I'm going to walk through how I did this.

1. Blogger setup
First, you need to make sure your Blogger feeds are configured to include the full content of each blog post (as opposed to just the first paragraph or whatever). I think this is the default setting, but to make sure, to go the Settings page and click on the "Site Feed" tab. The "Allow Blog Feeds" option should be set to "Full".

2. Atom feed URL
Now, you must get the URL of your blog's Atom feed. Go to your blog's homepage and view the source (in Firefox, this is under "View > Page Source"). Look for a "link" tag that looks like the one below, and grab the value of its "href" attribute.

<link rel="alternate" type="application/atom+xml" title="mangstacular - Atom" href="http://mangstacular.blogspot.com/feeds/posts/default" />

Note: In this tutorial, I use the Atom feed. The RSS feed also has all the same information--it's just arranged differently.

3. Magpie setup
Download Magpie. This is a wonderfully easy-to-use RSS/Atom parser written in PHP which we'll use to parse the Atom feed.

You'll want to configure Magpie to cache the Atom feed so that it only downloads it when your blog changes in some way (i.e. when there's a new post or a new comment). This way, your website won't have to download the feed from Blogger every time someone visits your page. Open the "rss_fetch.inc" file and add these two lines somewhere at the top:

define('MAGPIE_CACHE_ON', true);
define('MAGPIE_CACHE_DIR', 'cache');

This will turn on caching and instruct Magpie to save the cached Atom file in the directory you specify.

Note: Be sure that the permissions of the cache directory allow your web server to write to it. The way you do this varies from server to server, based on the user that your PHP process runs under, but here are the commands that I had to run:

chmod 775 cache
chgrp web cache

What you do NOT want to do is set the folder to be globally writable. While this would work, it would also allow anyone on the Internet to write to that directory--not a good thing.

4. Parse the Feed
Now you're ready to write the code that fetches and parses the feed. Simply calling Magpie's "fetch_rss" function will parse the feed and return all the data in an associative array (despite "rss" being in the function name, it will also parse Atom feeds). No need to deal with any XML. Below is some sample code.

Note: There are a couple quirks, which may be Blogger-specific--be sure to read the code comments.

require_once('magpierss/rss_fetch.inc');

$atom = fetch_rss('http://mangstacular.blogspot.com/feeds/posts/default');
foreach ($atom->items as $item){
  //var_dump($item); //see all the stuff that's in each item

  $date = date('F j, Y', strtotime($item['published']));
  $title = $item['title'];
  $content = $item['atom_content']; //no need to run html_entity_decode() or anything
  $url = $item['link'];

  //because there are two <link> tags whose "rel" attributes are the same...
  //...it stuffs the "href" attributes from both tags into this one string...
  //...so you must extract the URL you want...
  //...which in my case is the URL to the comments page
  $commentsUrl = substr($item['link_replies'], strpos($item['link_replies'], 'https'));

  $numberOfComments = $item['thr']['total'];

  //generate HTML for the entry
  //...
}

And now it looks like you're running fancy Wordpress software! ;)

Google Maps and its Traffic overlay

2009-09-29T19:08:00.005-04:00

Ever since I started my new job, I've gotten into the habit of turning on the morning news while getting ready for work in the morning. They do reports on rush-hour traffic and this morning, there was one whopper of an accident. A tractor-trailer had slide-lined three cars, completely blocking the flow of traffic in that direction. It was on a road I take everyday to work, but I wasn't sure exactly where on the road it was, so didn't know if it would impact my commute.

Then, I remembered that Google Maps has a live traffic feature that overlays the amount of traffic congestion on the map. I could see where the congestion was and figure out if I needed to find an alternate route. Oddly enough, it reported all roads as being more or less clear, which I knew was not the case. The roadside webcams (another Google Maps overlay) also didn't show any signs of an accident.

It turns out that the data just isn't very "live" after all. Not only was the traffic overlay not reporting the current traffic situation, but the webcam images weren't current either--they were anywhere from 30-60 minutes old. Clicking on the image takes you to the website that operates the webcams, and only going there can you get an up-to-the-minute view of traffic.

So it turned out that my drive wasn't affected. And I learned that, while the traffic overlay on Google Maps is great for seeing general traffic trends, it's not so good for live traffic updates.

Using phpDocumentor with Eclipse

2009-08-16T18:04:00.004-04:00

I just finished making improvements to a PHP/MySQL-powered photo gallery website I had written during college (it kind of felt good to see how much my coding skills have improved since then). Besides moving all the SQL into a DAO, cleaning up the UI, and learning how to do file uploads in PHP, I went through all the code and formally documented all the functions and classes. I decided to use phpDocumentor because the format it requires the comments to be in is nearly identical to Java's Javadoc, which I was already familiar with.

I wanted to set up an Ant target that I could run that would generate the HTML documentation. It was pretty straight-forward--just create a target using the Exec task to run a console command. However, even though I could call the "phpdoc" command straight from the command line, Ant had problems with that. I think this is because "phpdoc" is not an executable, but a PHP script. At any rate, it worked fine once I found the script that the command was linked to and used the "php" command to run the script.

<target name="phpdocs" description="Generates phpDocumentor docs">
  <delete dir="${doc.dir}" />
  <exec executable="php">
    <arg value="/usr/local/PEAR/PhpDocumentor/phpDocumentor/phpdoc.inc" />
    <arg value="-f" />
    <arg value="*.php" />
    <arg value="-t" />
    <arg value="${doc.dir}" />
    <arg value="-ti" />
    <arg value="Photo Gallery Documentation" />
  </exec>
</target>

SCJP Book Errata

2009-07-15T18:42:00.005-04:00

I recently read the book A Programmer's Guide to Java SCJP
Certification: A Comprehensive Primer (Third Edition) by Khalid Mughal and Rolf Rasmussen in order to prepare for the SCJP exam (which I took and passed last week). I sent the authors of list of mistakes that I found and some have made it into the book's online errata!

How cool is that! :-P

OpenOffice and DOC

2009-07-14T18:43:00.008-04:00

Sorry, I need to rant a little. >:(

OpenOffice isn't as great as everyone says it is. Or at least as great as I though it was. I'm looking for a job and have been using the open source office suite to update my resume. Most recruiters want resumes in Microsoft Word's ".doc" format, but have no fear! OpenOffice supports that format...

...mostly...sorta kinda.

My resume is pretty basic in terms of features used. It uses two tables, a few bulleted lists, and three drawn lines, along with text of various sizes and fonts. But apparently that's too complex. It can never quite seem to get the bullets in the bulleted lists right:

One time, it used an icon of those things used in movies at the start of a take. Just completely random.

For throw-away documents, this is bearable. But for something important like a resume, you want your .doc to come out exactly right. And OpenOffice fails to do this.

NetBeans and Workspaces

2009-03-02T11:04:00.004-05:00

If you're familiar with both Netbeans and Eclipse, you'll probably know of one obvious difference which sets the two apart: Eclipse supports "workspaces", while Netbeans does not (or at least, doesn't appear to--read on). A workspace is sort of like the "state" of the IDE--it's a collection of open project folders and IDE settings. Workspaces are useful when you are working on two unrelated projects which use different code and require the IDE to be configured differently.

By default, Netbeans only allows you to have one workspace. That is, when you exit Netbeans, all your open projects and settings will be restored when it is launched again. This is different from Eclipse, which asks you to specify a workspace every time it is launched. There is a way to specify your workspace with Netbeans as well, though it's not through an easy-to-use UI. An extra parameter must be included when invoking the Netbeans executable:

--userdir C:\path\to\workspace

Here is how to do use this parameter in Windows:

Create a shortcut on your desktop to the Netbeans executable: C:\Program Files\NetBeans x.x\bin\netbeans.exe.

Right-click on the shortcut and click "Properties".

In the "Target" textbox, add the extra parameter to the very end:

--userdir C:\path\to\new_workspace

Click "OK" to exit the Properties window and double click the shortcut. Netbeans will launch and create/load the workspace at that location.

Tip: I like to use my default workspace as a "jumping-off point" for all my new workspaces. This saves me time I would otherwise have to spend resetting all my IDE settings, like SVN and code formatting preferences. Just make a copy of the default workspace folder and create a shortcut pointing to that folder, as shown above. The default workspace is located here:

~/.netbeans/<version>

Clever Clock UI

2009-02-27T15:43:00.004-05:00

There was a power outage in my apartment building the other day, so I went to reset the clock on my stove.

Instead of making you select the hour and minute in two separate steps like most appliances do, you do it in just one. Press the up/down buttons once, and the clock will increment/decrement by one minute. Hold the buttons down, and it moves in ten minute steps for as long as it's pressed.

I thought that was kind of clever. It keeps the task of "Setting the Time" consolidated into a single step, as opposed to breaking it up into as many as three (setting the hour, minute, and am/pm).

Creating websites WITHOUT using HTML

2009-01-18T13:24:00.004-05:00

Want to see something that will blow your mind? Something that will shoot your brain out the back of your skull, through the window, and onto your neighbor's freshly waxed Pinto?


<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="layout/frontpage.xsl"?>
<page lang="en_us">
  <frontpage/>
</page>

This is the source code for a website's entire home page (which is that of the upcoming computer game Starcraft 2 by Blizzard Entertainment).

I could be in the minority, but I've never seen XSLT used to transform XML documents into web pages...and I've never realized how powerful it can be. After exploring the source code of the website, here are some advantages and disadvantages of this technique that I've discovered:

Advantage - Separation of concerns

Using XSLT, you can more effectively separate the content of your website from the presentation. The Starcraft 2 website has *all* of its text separated out into XML files (pretty cool!). For example, you can find all the text on the Terran page, in a file devoted solely to storing Terran-related text (terran.xml).

Advantage - i18n

By separating the content from the presentation, another benefit is that it makes it easy to internationalize the website. This is what Blizzard has done. The XML files used to store text are located in their own directory, named according to the language of the text (such as "fr_fr" for French). The language of the guy viewing the website is stored in the "lang" attribute of "page" tag of each individual web page (see the above code snippet). This attribute is used to determine which directory to retrieve the text from.

Also, you can do things that you would normally think could only be done using server-side languages:

Advantage - Access to special functions

Since XSLT uses the XPath language, you get to use all the special functions that XPath comes with. Tasks like string manipulation, date/time formatting, rounding numbers--all things that one might typically associate only with server-side languages, can all be done with XPath. For example, the following code, taken from a shared file called includes.xsl, uses the "concat" function to build a URI that points to a language-specific version of an XML file:


<xsl:variable name="terran" select="document(concat('/strings/',$lang,'/terran.xml'))"/>

Advantage - Avoiding code duplication

Just as you might define a function in a server-side script that performs some task, you can define function-like constructs with XSLT that "output" HTML. If you look at movies.xsl (a file used to generate the Movies page) you can see that for each movie listed on the page, a template is called:


<xsl:call-template name="movies-entry">
 <xsl:with-param name="title" select="$loc/strs/str[@id='sc2.labels.movies.movie6.title']"/>
 <xsl:with-param name="movieid" select="'6'"/>
 ... more parameters ...
</xsl:call-template>

The template, defined in a shared file called includes.xsl, contains the HTML code used to display the movie:


<xsl:template name="movies-entry">
 <xsl:param name="title"/>
 <xsl:param name="movieid"/>
 ... more parameters ...
 ... HTML/XSL code used to display the movie ...
</xsl:template>

So instead of having to do a lot of copying and pasting, as you would have to do if using plain HTML, the code stays centralized in one place, making bug fixes and other maintenance tasks easier to handle.

Disadvantage - Backward-compatibility

XSLT didn't become a W3C recommendation until 1999, so any browser created before or around that time probably won't have strong support for the language (this includes IE 5 and earlier). So you have to consider the software your audience is likely to be using before building your site. Blizzard probably thought it safe to use this technology, since gamers tend to use the latest and greatest browsers and are therefore unlikely to run into any compatibility issues.

So in conclusion, I've found that using XSLT to build web sites gives you many advantages that you don't get when developing with vanilla HTML. Perhaps I will use it for my next web development project!

SudokuSolver released

2009-01-11T12:42:00.005-05:00

I've developed a bit of an addiction recently. Thankfully, it's not alcohol-related. But like drinking, it does take up time and money.

If you're not familiar with the puzzle game Sudoku, you've probably at least seen it either in your daily newspaper or somewhere online. The goal is: given a square board 9 cells high and 9 cells wide, fill each row, column, and 3 by 3 sub-square with all the numbers 1 through 9. No number can be repeated in any of those three places. For example, the number 5 can't appear more than once in the same row. The board starts with some cells already filled in to give you a starting point. The less completed cells you start with, the harder the puzzle is.

After solving enough of these, I thought it would be an interesting exercise to write a program which solves them automatically! As I got deep into creating the algorithm, one data structure stood out as being the keystone--sets.

In mathematics, a set is a collection of numbers where:

the order the numbers are in does not matter and

no number can be repeated.

For example, this is a set:
{ 3, 7, 5, 9 }

Since order doesn't matter, this is the same set as the one above:
{ 9, 3, 7, 5 }

But this is NOT a set because there are two 7s:
{ 9, 3, 7, 5, 7 }

In this algorithm, sets are useful when trying to figure out what number a cell should be. The algorithm can be summarized with this pseudo-code:


while board is not complete
  for each cell in board
    if cell is empty
      possibleValues = findPossibleValues(cell)
      if size of possibleValues == 1
        cell.empty = false
        cell.value = possibleValues[0];

function findPossibleValues(cell)
  possibleValues = new set{1,2,3,4,5,6,7,8,9}

  for each rowcell in cell's row
    if rowcell is not empty
      //if possibleValues doesn't contain rowcell.value, then nothing happens
      possibleValues -= rowcell.value

  for each colcell in cell's column
    if colcell is not empty
      possibleValues -= colcell.value

  for each squarecell in cell's square
    if squarecell is not empty
      possibleValues -= squarecell.value

  return possibleValues

Given a current board state, for each cell in the board, a set containing all values the cell could possibly be is created. If this set only contains one number, then that's the only number the cell can be, so we fill in the cell with that number. This keeps repeating until all cells are filled in. (There are some other tricks that have to be taken into account, but this is the most fundamental part of the algorithm.)

If you'd like to try it out or look at the source code, you can download it here:

http://www.mangst.com/projects/sudoku-solver

Right vs. Right

2008-05-11T18:05:00.004-04:00

Somewhere, deep in the bowels of a florescent-lit cubical farm, a battle wages on. A battle that has been fought since the dawn of computer programming. A battle of..."right vs. right"?

This sums up the approach that Michael Lopp, better known as "Rands" in the blogosphere, takes to software engineering. In his book Managing Humans: Biting and Humorous Tales of a Software Engineering Manager, the battle he describes is one of Incrementalists vs. Completionists--those who want to get things done quickly verses those who want to get things done correctly. Both mindsets are equally "right" in Lopp's eyes. It's the balanced combination of the two that makes for a solid software team.

Lopp takes this even-handed, "ying/yang" approach throughout his book, which is mostly made up of posts from his blog, Rands in Repose. As with the incrementalists and completionlists, his characterizations of office personality types are colorful, insightful, and even-handed. In the chapter entitled Meeting Creatures, Lopp takes you on a wild safari, describing the different kinds of people you'll encounter around the conference table at work. These meetings, he describes, are a rich ecosystem made up of unique, but interdependent personalities. You never have any idea what a "Curveball Kurt" is talking about, for example, which is why you need a "Translator Tim" to help out.

In regards to writing style, each one of the short, thirty-four chapters are written in a light, conversational tone that's easy to digest. Being a collection of blog posts, it's easy to pick up and put down without getting involved like you might with a novel.

So, the book is great, but it should be noted that people may find it hard to relate to some of Lopp's stories. Lopp works in the fast-paced, high-energy Silicon Valley, where startups come and go like contestants on American Idol. This can make for some quite stressful times. He talks, for example, about The Monday Freakout which describes the deluge of yelling and screaming that comes pouring out of a project lead after a weekend spent worrying about the project. I think it's safe to say that not everyone works in an atmosphere like this. Many readers may have a job that, while engaging, doesn't keep them stressing over the weekend.

But the bottom line is that no matter where you work, if your job is somehow related to software development, you can learn something from this book. Read on! It's the "right" thing to do.

ThumbThru released

2008-05-05T22:47:00.000-04:00

I just recently finished putting together a small utility I had written a while ago for some websites I had designed during college. I spent a few days cleaning it up and turning it into something generic that anyone could download and start using right away. I've called it...**drumroll**

ThumbThru

It's a simple photo gallery utility coded entirely in Javascript and written in such a way so that it takes as little time as possible to get up and running. It's a lot like the galleries you'll find on the In Pictures section of the BBC News website--in fact, you could say that's where I got my "inspiration".

Some things I learned when writing this:

How to create a Javascript class

The difference between the GPL and the LGPL (I've licensed it under the LGPL, btw)

The fact that Ant can create ZIP files (pretty cool!)

All future blog posts perhaps...?

Anyway, feel free to download ThumbThru and give it a try! Or at least check out the live demo!

I understand you

2008-05-02T22:16:00.003-04:00

So, I go to check my Gmail account like I do on a regular basis. But when I logged in, I noticed something different.

There was this red bar above my inbox:

It turns out that I did indeed have Firebug (a Firefox extension) enabled! And I wouldn't have known it otherwise. Google probably decided to include this feature after getting complaints from users who didn't realize that it was their own stupid fault their mail was loading slowly. Or maybe Google decided to include it out of the kindness of their hearts. Either way, it's pretty cool to see a huge corporation that certainly doesn't need do something like this, take the time to do it. They could have buried the issue somewhere deep inside a FAQ page and left it at that.

In the short-term, spending time to include something that only benefits a small minority of users a small minority of the time is a waste of money. Some developer might have spent a day designing this feature. Google's stock price certainly won't rise the next day because of it. He should have been working on Adsense or something!

But it's the longer term impact that is affected by small enhancements like this. What this is saying isn't "turn off Firebug". It's saying "I know you want to get to your email quickly and Firebug may interfere with that." Or more simply:

"I understand you."

When another product of equal or better quality comes around, if you've been regularly reminding your users that you understand them, they'll be more likely to stick with you during tough times.

Music from the Amazon

2008-04-27T22:16:00.007-04:00

For the longest time, the iTunes music store has pretty much dominated the music download market. Apple's service has a gigantic selection of music to choose from and they make it super easy to find that song you want to hear. But there's one downside to all this: most of the songs are protected by digital rights management or DRM. This puts limits on things like on what devices the songs will play and how many times a song can be copied. A song without these limitations is certainly a better deal for the customer, who can instead copy the song to as many devices as he desires and play the song on all of them.

Amazon, traditionally the book-selling behemoth of the interwebs, fairly recently started providing music for download. While it doesn't have the extensive selection iTunes is known for, all of Amazon's music is DRM free and encoded in the generic MP3 file format, giving you more freedom over how you can use the music you paid for.

Just today, I downloaded my first album off of this service. It was a pretty painless procedure, though there's a few things you should be aware of:

One - If you want to purchase an entire album, you've got to download this special "Amazon MP3 Downloader" software. It's always annoying when you have to install an entire application to do something as simple as downloading a couple of files. But one feature that takes the edge off of this inconvenience is that it automatically puts a copy of your songs in either your iTunes or Media Player (depending on your OS) library, saving you the small, but sometimes tedious, step of organizing your music by hand.

Two - They have no refund policy. Once your credit card is charged, there's no way to get your money back. Makes sense, since the product you're paying for can't get damaged the way it could if sent through the mail.

Three - You can only download your songs once. If your hard drive decides to stop working one day, that's it, they're gone. You can't download them again without paying a second time. While it doesn't take much effort on your part to make a backup, this still seems like a silly policy. Because the music is DRM-free and can be copied ad infinitum, it seems that what you're really paying for is permission to play the music, as opposed to the music itself.

This is what I like to think of Valve's Steam doing. Steam charges for downloadable content just like Amazon and iTunes does, only instead of charging for downloadable music, it charges for downloadable computer games. Since Steam lets you download the games you've paid for as many times as you want, it's charging you for permission to play the game--not necessarily the game itself. I don't see why Amazon couldn't implement a scheme like this as well.

In conclusion, despite the software it forces down your throat and the one time download limitation, I do indeed recommend Amazon's music download service. Its policy of providing DRM-free music for purchase is something that would be nice see amongst the larger music sellers. Always search Amazon for that song you want before looking anywhere else. DRM-free is the way to be!

Upgrading Drupal

2008-04-24T21:02:00.000-04:00

In light of the up and coming (at the time) Pennsylvania Democratic primaries, I was asked by my alma mater's student Democrat organization to create a website for their group. I thought this would be a great opportunity to give an open source CMS (content management system) a try. I ended up choosing Drupal and was able to get it up and running with relative ease. To my surprise (and frustration), only a week later, the Drupal developers released a new version, compelling me to upgrade. In this post, I'm going to talk a bit about the Drupal upgrade process and upgrade processes in general.

Upgrading Drupal, or any application for that matter, is often much harder than performing an installation from scratch. Two major hurtles come to mind.

(1) Compatibility. Will all your stuff still work with the new version?

When Microsoft releases a new version of their Office suite, for example, you want to know that *all* your Word documents created by previous versions can still be opened.

Drupal, unfortunately, doesn't make things that convenient. In the cosmic balance between the software pragmatists and the software idealists, the Drupal team, unlike Microsoft, fights for the idealists. Any time they release a new "major" version (that is, 5.x to 6.0, for example), they pretty much guarantee that all existing custom themes or modules will *not* work. This is because they are very much willing to throw away a large part of the codebase if they think there's a better way of doing something.

These compatibility issues, however, are not an issue with "minor" upgrades (such moving from 5.4 to 5.5). This is because minor upgrades usually deal with security fixes and implementation details that your custom modules and themes don't care much about. Luckily, the upgrade I wanted to perform, 6.1 to 6.2, was a minor upgrade, so I knew with a high degree of confidence that all my custom themes and modules would still work (and they do).

(2) Data preservation. Will any of your data be lost in the upgrade process?

Of course, you want this answer to be a resounding "NO WAY!". For example, if you upgrade to a new version of Firefox, you don't want to load up your new version, only to find that all your bookmarks are gone.

In Drupal's case, this means not loosing your custom themes, custom modules, site content, and files your users uploaded. What makes this a bit complicated when upgrading Drupal is that the Drupal team recommends you completely remove all files from your existing Drupal installation and then do a fresh copy of all the files from the new version (as opposed to just overwriting everything). This means that you must be aware of all the ways in which your Drupal installation differs from the "base" or "initial" installation. That is, you must know about your custom themes, custom modules, etc. so you can put them back after you've copied over the new version.

I find it helpful to maintain a list of all of these differences. That way, whenever I have to upgrade, I know exactly what files to put back into the new version to keep my site running smoothly.

Now that you've got an overview of what to expect from your Drupal upgrade, you might want to do some reading on the Drupal website for more of the technical details. Good luck!