John O'Conner's Blog

John O'Conner is a software architect, consultant, author, and speaker. After many years developing the internationalization features of the Java platform at Sun Microsystems, he is now actually trying to use those features in real world projects.

Another solution for non-UTF8 source files in NetBeans 6.1?

Recently I mentioned a potential problem when saving source files in a non-Unicode charset encoding. The potential data loss is significant for large projects. After thinking about the problem a little more, I have a potential solution, a solution that allows you to save to a non-Unicode encoding but also prevents data loss.

You are familiar with \u notation for non-ascii characters in property files? I think the same encoding can work for non-ascii characters in any Java source file. I'm not suggesting that should be the preferred representation. I think the \u notation is only tolerable, something to be avoided whenever possible. However, in this situation -- saving files that were once in UTF-8 -- this might be the only option for storing the files without data loss.

Here's how it would work. First, NetBeans 6.1 uses UTF-8 for a project's default source code and configuration file encoding, an excellent choice by the way. So, now imagine that your source code has the Euro currency symbol in it. That's Unicode code point U+20AC. And the character itself is this: €. If you can't see the actual character (maybe you don't have a capable font?), here's the image instead:

Now, let's imagine that you need to change your project encoding for some reason. So, maybe you choose ISO-8859-1, which doesn't contain the Euro symbol. You can still represent the Euro character, but you'll have to encode it with the \u. Wouldn't it be nice if NetBeans did this for you, creating \u20AC in your file instead of converting the character to a meaningless ? question mark. I think that would be better. And it's entirely possible. It doesn't prevent NetBeans from converting the file to the target encoding as requested by the user, and it allows NetBeans to prevent data loss by using the \u encoding for characters that are not in the target charset.

So, what do you think? Maybe the NetBeans team can get this into the 6.1 product before final release?

Have you seen NetBeans 6.1? Give it a try, and blog about it. Who knows, you might win $500!

Also posted to joconner.com.

Call for participation: Internationalization and Unicode Conference #32

The Internationalization & Unicode Conference is the technical conference for software and web internationalization engineers. If you have a product that implements the Unicode standard or an idea that will help others work with this standard, share your knowledge.

The "call for participation" is an invitation for Unicode gurus, clients, and vendors to submit papers for this upcoming conference. Learn more at the Unicode Conference web site.

Also posted to my software internationalization blog at joconner.com

NetBeans 6.1, UTF-8 encoded source files, and a tale of corruption

I'm always happy when a company or product adopts Unicode as its charset. I think it makes perfect sense to do so. There are lots of good reasons why standardizing on Unicode is the right thing:

• the data charset can represent all modern, useful -- and many not-so-useful -- scripts
• charset consistency helps prevent data loss across application boundaries, including those among application, business tiers, application servers, and databases
• a single, multilingual charset simplifies the localization process

I was pleased to see that NetBeans 6.0 and the 6.1 beta uses the UTF-8 encoding (a Unicode encoding) as its default for project configuration and source files. The following figure shows the default setting in the project's property sheet:

This makes it much easier to edit non-ASCII, non-English source and property files. You can type text in any supported Unicode script right into Java source code. A legitimate usage would be comments or even localizable text in ListResourceBundle files. You can do that because NetBeans will save your files using the UTF-8 encoding by default...and NetBeans even puts this setting into the project's Ant file settings so that it compiles the file properly using the correct javac arguments. For example, NetBeans would use the following for a UTF-8 source file:

javac -encoding UTF-8 YourSource.java

Despite the potential benefits of this, NetBeans 6.1 still doesn't support this correctly in my humble opinion. Why not? Well, the biggest reason is simple: file corruption and permanent data loss. Ouch!

Let's take a simple "Hello, world!" example in Japanese. This is simple for NetBeans because of the UTF-8 encoding. The NetBeans editor even displays it correctly as shown here:

Unfortunately, the joy of this discovery was short lived when I discovered how easy it is to corrupt this data. Feel like experimenting with the charset encoding? Surely someone will. I suspected what would happen, so I didn't do this with any substantial code base...but someone will. I sure hope they use version control software.

Reopen that project property sheet, select another encoding, say ISO-8859-1 or windows-1252 since so many American and Western-European programmers use that by default. What happens? It appears as if nothing has happened. In this NB 6.1 beta, I see the same editor screen as before. NetBeans apparently hasn't yet reloaded the file from disk, so I'll force the issue by closing and reloading this file. The result is here:

Some of you, the super careful, nit-picky ones will now argue with me, "But John, you haven't really lost anything yet. 8859-1 and CP 1252 don't have those characters, but the original byte values are still entact. You can get them back in this example." OK, I concede the point. But now I'll show you some serious data loss, no messing around this time. Instead of ISO-8859-1, pick US-ASCII as the target charset. Add a comment or another line of code to the file. Save. And there you have it as shown here:

Now that's just not good. Did NetBeans save the file correctly? Sure. However, NetBeans can do better than this. I would argue that if NetBeans knows that the target encoding does not support the source encoding, it should at least warn the user that the resulting file will contain garbage characters and that parts of the file will be lost--permanantly in many cases.

So, just in case anyone over there in the NetBeans developer group can hear me...you have to fix this. Yes, I know it's a silly mistake for someone to do this, but NetBeans can help them avoid the problem. Just provide a warning dialog, "Saving this file in the target encoding will cause data loss because the target encoding does not support all characters in this file or project." Keep the encoding feature, just perfect it by helping some users avoid this costly mistake. The fact is that most software developers still don't understand character sets and encodings, and this is just an accident waiting to happen.

On a personal note: I really love NetBeans. And I hope this blog qualifies me for the NetBeans 6.1 blogging contest! I could probably file this under the "suggestions on how to enhance NetBeans 6.1" category.

Will blog, write, or program for JavaOne pass

Although I was there last year -- writing, blogging, communicating to the community -- I don't have a pass to JavaOne 2008 yet. Unfortunately, I have no concrete plans. No corporate or community sponsor has agreed to help yet either. It's a sad, sad thing. Indeed, these are trying, desperate times.

I want to go to JavaOne...and I'm willing to take drastic action. Sure, I could just put up the money myself. The $1595 price tag for a 4-day pass is expensive but not entirely out of my budget. Still, there's no fun in that, especially if I can do something more interesting than just sending in the registration cash. So, I'm going to do something a little different in order to participate this year. I'm hiring myself out!

Yes, that's right. The first sponsor to offer a full 4-day Conference Pass and room will receive my professional consulting/writing/blogging/programming/go-fer services for the duration of the event!

Here's what you'll get:

• One full week of blogging for your products of choice at the JavaOne conference.
• Booth duty...you know you need a break from the pavilion booth at some point. Let me help.
• A feature article...or two...from me if you're the editor of a Java-related journal.
• Any other reasonable task that you might need.

Yes, it's a bit unusual. But the JavaOne conference is unusual too. I think this is going to work out just great.

You know how to reach me: john at joconner dot com.

Beans Binding and my Flex friend

I recently wrote a brief introduction to Beans Binding called Synchronizing Properties with Beans Binding. I was enthusiastic about Beans Binding, and then a coworker threw a wet towel on me. The conversation went something like this:

John: I just finished an article on java.net. This API might be useful in this project.

Coworker: Yeah, what's that?

John: Beans Binding. It's an API that will help us bind beans together, synchronizing properties.

Coworker: What do you mean?

John: You know...say you want to update a table with new data anytime a result set changes, or maybe you want a price field to display the current total of all purchased items in a shopping cart. You can use beans binding to synchronize the properties without having to write all the plumbing and event handling yourself.

Coworker: Hmm. Java hasn't always had that?

John: Well, yes. Java has always allowed you to synchronize properties, but we've never had an API to make it this easy.

Coworker: Really? We've had this in Flash and Flex forever.

John: You've had binding?

Coworker: Binding, or Beans Binding...whatever you call it. We can bind a variable or property to another variable or property so the updates trickle throughout the application. It's a great feature, a must-have, and I couldn't do without it. I use [Bindable] among other things.

John: Yes, it's great. I agree. So take a look at that article. Beans Binding, it's pretty cool.

Coworker: You're just getting binding in the Java language? Hmm... really? I just assumed Java would have had that all along.

John: Um, it's not exactly part of the language. It's a new API. Making changes to the language is difficult and usually takes a long time...

Coworker: Oh, I misunderstood. It's not part of the language? Just a separate API? Got it. Hmm...ok. Well, uh, gotta run.

John: Hey, wait. But Java FX Script has a bind keyword that is part of the language. Remember? I told you about Java FX Script earlier.

Coworker: Right. Java FX Script. It's about time Java got got a bind operator or keyword, something.

John: No, not Java. Java FX Script. That's the new language for rich client app development. Java FX Script has the bind operator. Java has the Beans Binding API.

Coworker: OK, right...well...hey...I have to get back to work on this Flex app. But I agree, binding is great, couldn't do without it, use it all the time.

Videos and the cubicle effect on content consumption

They do great work over there on the java.sun.com editorial team, and they've been improving the site with lots of videos lately. The last three featured content pieces have been videos. That's excellent work, but it's inaccessible for a couple reasons.

First, I like to hit java.sun.com first thing in the day from my workplace. That's been a cubicle environment recently. I can quietly read a text article, even print it and read it later during a lunch break. Unfortunately, I can't do the same with the videos. They're fun, sure, but unwatchable because video and audio are distracting to others in the office. Bandwidth issues aside...people just don't like to hear this from another cube. I understand that, so I don't intrude on their peace and quiet. Unfortunately, the video is inaccessible to me. And when I get home, well, there's a better than even chance that I won't watch a tech video then either.

Second, the videos are inaccessible to those with hearing or sight impairments. Seems obvious, right? Can't hear...the audio isn't usable. Can't see...can't watch the video either. I suppose you could listen to the video -- but not in my cube. So, the videos are inaccessible to others too. Granted, those who can't benefit from the videos are a small minority. Still, wouldn't it be nice to include them as far as possible?

I think a solution is possible. Video is a great idea for java.sun.com, and clearly the medium is popular. Just witness the popularity of YouTube, etc. I think the videos could be improved though with something very simple. Transcripts.

Providing a transcript with the video would allow cubicle dwellers access to the content. The same transcript makes the content accessible to hearing and site impaired developers too. Can't hear -- you can read the transcript content. Can't see? Text readers exist for that case. Either way, the videos become more accessible to a larger group of people.

Is this a complaint? No way. I enjoy practically all of the content provided by java.sun.com. This is just a friendly suggestion, hoping that it will bring the content to an even wider audience.

It has been a long, long time since I last worked in a cubical environment. I find the experience undesirable but tolerable. It makes me curious though...just how many of us work in a cube environment? Might be a great polling opportunity for the java.net editors...

Should we expect more db acquisitions?

Last month Sun announced it had acquired MySQL, which may very well be the most influential and widespread open source database in the world. Aside from a few announcements, there wasn't nearly as much comment on this as I expected.

Here are the databases that I see being supported at Sun:

• HSQLDB in Open Office
• Apache Derby
• MySQL

HSQLDB? You would agree that Sun is the largest supporter of Open Office, right? I mean even though OO is open source, that doesn't mean it doesn't have huge corporate sponsorship. And that huge corporate sponsor is Sun, which donates time, material, money, employees, and advertisiing to Open Office. Open Office has adopted HSQLDB as its internal database. Surely HSQLDB's project gets support from Sun too. After all, it is an integral and important part of the Open Office suite.And its a 100% Java implementation. Maybe this isn't an expensive investment compared to MySQL...but why bother with HSQLDB, especially when you have this next db?

Apache Derby. Sun's branding calls this Java DB, but underneath, it's still Apache Derby. Derby is the 100% Java database that supposedly can scale from embedded applications to larger, departmental, transactional apps too. Impressive for sure, but certainly duplicated by the HSQLDB product's abilities, no? Regardless of the overlap in abilities, these are two different products with different supporters and communities. Still, that's another db, and more money, time, and resources from Sun.

Then we have the heavier hitter, MySQL. And this is by far the biggest ticket item. What was the price? $1 billion? I put this db in a different league from HSQLDB and Derby. 100% Java? Not at all. However, it does have more enterprise credentials than either of the other databases. It's probably not fair to even put the others in the same arena as MySQL.

As I dig around, I see that Oracle purchased Innobase, the company that provides InnoDB within MySQL, a couple years ago. So, Oracle owns InnoDB, and Sun owns MySQL. Interesting, but what's that mean about InnoDB within MySQL now?

I see Sun developing more and more interest in database technologies, from small systems like HSQLDB and Java DB to the larger MySQL. It's a pattern, and I wonder what it means. Can we expect Sun to actively support and even purchase more db technologies? What might those be?

With limited time resources, I can't work on all the things I'd like to. I can't read all the books I would like to. I can't give to all the charities that deserve my attention -- or money. I think companies have similar constraints on their resources, money, employees, etc. As far as I can tell, Sun has now made committments to at least three different database systems. Which one will prevail at Sun, and which will get the resources to maintain itself?

Java applets not viable for this application?

I've been working with a company that creates a chat system that helps companies sell more of their products online. The backend is all Java, complete with Spring, servlets...all good Java stuff. The frontend is...well, it's Flash.

Preferring Java everywhere when possible, I asked about the Flash choice on the customer-facing app -- the actual chat client. The conversation went something like this:

John:Wow, this product uses Java everywhere except on the front, where the customer actually interacts with it. Why choose Flash there?

Dev mgr:Well, Flash was just easy. It was simple to use for such a simple UI too.

John: I see, but it's pretty easy to do that chat UI in Java too. Swing's layout managers are easy now, particularly if you use an IDE like NetBeans to layout the UI.

Dev mgr: That may be true, but the choice is based on more than just ease of creating the UI.

John: Yeah? So what's that?

Dev mgr: Flash is on every client browser. We don't have to install the runtime.

John: Java should be there too, right? Most PC manufacturers include it.

Dev mgr: Maybe, but we know Flash will be there. Flash is all over the web, and chances are that a customer has the runtime already. If they don't, the download and install is fast and easy.

John: OK, but Java will also install easily.

Dev mgr: But Java takes so long to install. It's too big.

John: I don't know if that's so true anymore. There's been a lot of work to make the runtime smaller and more compartmental. So you only download what you need.

Dev mgr: What we need is something very, very simple. A very simple UI with simple functionality to relay chat text. Flash does the job well.

The conversation continued...but I didn't get too far. Maybe Flash does do applets better than Java? Working with Sun for so long, I just naturally imagined that Java was everywhere and that everyone else loved it too. I've always been able to make things work...but apparently not everyone else has. The download size never bothered me...but it does bother some others.

Additionally, I find that perception is as important as reality. So what if Java is now faster, leaner, and great working within browsers? Somehow it doesn't matter. The old perception is still out there -- that it's slow, huge, and difficult to work with in a browser. It's amazing to find out that some just don't see Java as their solution in that space anymore.

I called a few friends who put together consumer-oriented websites. They use Flash too. What about Java? Sure...all on the back end business logic. But what about applications that the customer will use? If they're embedded in the browser, they're using Flash. Wow.

So, how would you approach this? What does the Java community need to do to fight back old perceptions? Why is Flash so much more popular in browser based applications? And how can we get Java there instead?

April 2008

Search this blog:

Categories

Archives

April 2008
March 2008
February 2008
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
March 2005
December 2004
November 2004
July 2004
April 2004

Recent Entries

Another solution for non-UTF8 source files in NetBeans 6.1?

Call for participation: Internationalization and Unicode Conference #32

NetBeans 6.1, UTF-8 encoded source files, and a tale of corruption

Articles

Synchronizing Properties with Beans Binding (JSR 295)
The idea of setting up listener relationships between your GUI models, views, and controllers is simple enough, but grinding the same "glue" code dozens or even hundreds of times is wasteful and error-prone. JSR-295, Beans Binding, offers relief from the drudgery. In this article, John O'Conner shows how it works and what it can do for you. Mar. 20, 2008

All articles by John O'Conner »

This work is licensed under a Creative Commons License.