Behind the Curtain at TCG

Race condition at my bank

I have a bank account that's set up so when I withdraw funds from that account by wire or check or whatever, if there's less in the account than the amount I'm spending, the money is automatically pulled from my line of credit. I keep a balance of $0.00 in the account to avoid confusion on my part.

Monday morning, I wired $5,000 from that account.

On the same day, that money was received by the recipient and then wired back. (It's a long story.)

The net result at the end of the day should have been $5,000 in the account, and a charge against my line of credit of $5,000.

But something went wrong. Yesterday I was at the bank and saw the account transaction screen for that account. It showed that $5,000 had been wired into the account, then $5,000 had been wired from the account, leading to a balance of $0.00. My line of credit was not tapped, so I won't be charged interest on the one-day loan.

What happened?

My theory is this: The bank does a reconciliation once a day. But apparently the order of operations isn't set according to the time the events happen. As a result, the two events went in with a random order - the bank recorded that $5,000 came INTO the account, and then was wired OUT of the account, rather than the truth (that $5,000 was wired OUT of the account, and then came back IN to the account).

In essence, according to the bank, $5,000 that did not yet exist appeared in the account, then was spent and went to a place where it never arrived. Time travel!

The bank was quite confused. I explained my theory to them, and told them it was likely what's called a "race condition" - a programming term for dealing incorrectly with apparently simultaneous events. This kind of programming error can be very expensive to a bank...if I had a $1 billion line of credit, and I wired the full amount and this had happened, the bank would have lost the interest they should have gotten for a one-day loan of $1 billion (at my 7.5% rate, about $205,479).

I've encountered this sort of behavior before with various banks (often involving my moving money into an account to make payroll, then the payroll charge coming in later that afternoon, and the bank bouncing payroll because funds weren't available) so my suspicion is that it's not an isolated problem. I've rarely seen it so clearly-defined before, and so much to the detriment of the bank!

One solution is for the bank to always record withdrawals before recording deposits. This would lead to a fair number of bounced checks, and that would be good for the bank and bad for customer service. Doing it the other way (deposits before withdrawals) would be bad for the bank in some cases and neutral in others, but never good for the bank.

The best solution is probably to write better code so either
(a) a whole day's activities aren't tallied all at once, or
(b) using a predicate (where it checks for "current" information before implementing the change).

Copying Text from Encrypted PDF Files

PDF encryption is sort of silly.  If you really wanted to grab the text, you can screen capture each page and then OCR it.  So clearly the true intention of the encryption is to deter the 99% of the users who wouldn't go to such lengths to try to copy text.

Recently, I received a PDF of a text document that I needed as plain text (it was a database DDL commands of a system I was analyzing).  For some reason the author had encryption turned on so that I couldn't copy and paste the text, and I was a bit impatient and didn't want to wait for the author to send me the original text file.  However, this PDF still allowed printing.

Since I had Acrobat, I tried to print the file into a PDF to remove the encryption.  Nope.  Acrobat is smart enough to keep you from doing that.  But then I installed the Windows "Generic Text" printer driver and set it to print to a file.  By printing my "encrypted" PDF to the Generic Text printer, the text of the whole document was nicely saved for me.  After removing the page breaks and the margin I had my original database text document.

The punchline is: If you don't want users to easily copy text from your encrypted PDF files, you not only need to turn off the text copy capability, but also the print capability.  Why?  Because it can easily be foiled using the Windows Generic Text printer driver.

The Promise of Ajax and DOM: Simple Event-Driven Widget-Based Web Applications

As a Web developer, I have a confession to make: I prefer writting desktop applications. 

In an enterprise-level Web application, just to get a usable display tier you need to perform gymnastics in JavaScript, CSS, HTML, JSP, Java, and Struts (a similar set of gymnastics is required with ASP).  To create a "widget" that appears to work across multiple page submissions you need to coordinate code segments distributed across multiple JSP and Servlet scripts.  It requires great energy, organization, and complex system architecture diagrams to make sense of the flow of responsibility between classes and pages in a Web application.

With desktop applications you can have a single class that represents a GUI widget through its complete lifecycle.  Instead of form submissions and page requests to drive activities, desktop applications use events that are associated with the widgets that generated them.  The interface does not need to be recreated with every request, but instead modifications to existing widgets can be used to report errors, create context-sensitive help, or update information in real time.

I dream of the ability to write a Web application using an event-driven widget-based program.  I long for the capability to write a single line of code that displays a modal window, waits, and returns with the user's response.  Instead of using page-scope or session-scope variables, I'd love to see the flow of control return to a sleeping thread that represents the running application.  In the end, it would be amazing to have a simple open standard GUI "library" against which you compile a desktop-like application, and *poof*, it works on the Web.  With the maturity of W3C's Document Object Model (DOM) and Ajax, I believe it is not only theoretically possible, but truly feasible.

To make an event-driven Web application environment you need three components:

• A DOM event-driven widget library
• An Ajax control framework that manipulates DOM widgets, and passes and interprets XML messages
• A server-side framework that exposes Ajax-generated events and messages as a GUI interface, and maintains Web application threads.

DOM is a browser-independent W3C standard that allows script-based manipulation of an HTML page.  It is so powerful that you can create whole window-based interactive widgets within a Web browser.  Take a look at JPWM (http://jpwm.sourceforge.net/demo/) or WinLike (http://www.winlike.net/).  Thanks to DOM, client-side scripts can create HTML elements and react to events.  With DOM, you can create event-driven widget-based client-side applications, but since browsers don't allow interaction with the file system (for security), DOM applications are extremely limited in what they can do by themselves.  In other words, while you can write a word processor in JavaScript with DOM, you can't save the resulting files anywhere!

Ajax is a programming approach of passing XML messages between client-side scripts and a web server without a full page refresh.  The code used to send and receive XML messages is currently rather browser-dependent (W3C, we need your help here!),  however, Ajax libraries have been developed to hide these browser issues.  An Ajax control framework can be developed to create and manipulate DOM widgets, and send data and events to and from a web server.

Finally, a strong server-side framework is required to link the DOM widgets and Ajax messages to a running Web application.  One important challenge is to take the random Ajax-generated events and data, populate objects and awake the appropriate sleeping Web application thread, gather new commands and data from the Web application, and the new information back to the Ajax control framework.

With the combination of these three frameworks, an event-driven application can link to a GUI library and magically work on the Web.

I should note that JavaServer Faces (http://java.sun.com/j2ee/javaserverfaces/download.html) works hard on the server side to make complex reusable GUI components and events, however, a "page submission" remains as a primary concept, leading to large collections of JSP and Servlets bound together by XML configurations.  In order to get rid of the concept of a "page submission" it needs the Ajax engine.

So, when can I stop mucking around with JSP or ASP scripts?

NCI warns users about using IE

According to an email apparently sent to users at the National Cancer Institute (part of the National Institutes of Health, which is part of the Department of Health and Human Services) on Friday, an IE bug could allow evil web sites to damage your computer just by going to the site. It goes on to say that there is currently no fix and that to reduce the risk users should avoid "unfamiliar" web sites or just use "Mozilla browser." I assume they mean Firefox but, again, who knows?

Not an outright rejection of IE, but certainly permission to run something else. This is new because over the last few years I've personally encountered several internal sites that function only when the user is running IE. They "fix" them when they are pointed out, but it's clear a lot of the development work at NCI is ignoring other browsers in their testing.

(I choose to believe it's bad testing rather than a concerted effort to push users towards IE.)

Microsoft and OpenDocument

I've been following the firestorm over Massachussetts' possible adoption of the OASIS OpenDocument XML format as their standard format for office documents with interest. The ability of governments (and indeed, all computer users) to have guaranteed access to their data beyond the lifecycle of a particular product or vendor is extremely important. Recently, Microsoft changed the licensing terms for their XML Office formats with the intent to allow others to use them in their solutions without fear of litigation. These changes have been described as "Microsoft's most significant olive branch to date". This could imply that Microsoft is (finally) willing to work seriously towards interoperability. But this could also suggest they're desperate to avoid widespread adoption of OpenDocument because they would no longer have absolute control over an important "standard" document format.

It strikes me as odd that I haven't seen any serious attempts by Microsoft to implement the OpenDocument standard. Brian Jones, an Office program manager, said that the reason for this is feature incompatibility:

The interoperability problems will start to come up if there are features that are present in one application but not present in the other application. You have to assume this will be the case since every application out there has a different set of customers that request different features. From the Microsoft point of view we have so many features we built over the years and it would be extremely unlikely that those features work exactly the same way in other applications. Believe me, there are *tons* of features in Word, Excel and PowerPoint, and we have a responsibility to our customers to continue to support them.

It's hard to fault him for that. But I find it difficult to believe that it's impossible to implement the common features and address at least some of the major incompatibilities, while still supporting the more esoteric features of Word with their own formats. This would be the "80% solution" that Microsoft has been so good at in the past. Once that baseline is supported, they could work with OASIS over time to move towards a comprehensive standard that includes all of their features. It won't happen right away, but incremental progress can be made with each new Office version. And with each small step, more and more customers win.

In his post, Jones does scope out the size of the formats involved -- 1600 distinct tags for Word 2003 alone -- and that gives some idea of the size of the job. But he does not give (and I have not been able to Google) a detailed technical discussion, or even a rough estimate, of specific incompatibilities between the OpenDocument and Office 2003 XML formats. Without that, it's very difficult to judge whether there's truly an insurmountable problem, or if it's just "Not Invented Here" syndrome.

foXpose - Tile your tabs!

I just installed Viamatic foXpose and my web browsing will never be the same again. With a single click, all of my open Firefox tabs are represented in thumbnails one one screen, enabling quick review and identification of interesting content.

Kim Nelson, EPA CIO, going to Microsoft

Nelson to step down as EPA CIO. Kim's been a vocal supporter of e-government and a strong manager. This is a great move for her...and a great catch for Microsoft.

Information sharing awards - Excellence.Gov 2006

Every year the eGov Shared Interest Group of the Industry Advisory Council (IAC), which is part of the American Council for Technology (ACT), organizes the Excellence.Gov Awards. Nominate your program now! The deadline has just been pushed back to December 9th. Here's the latest mail from IAC:

The eGovernment Shared Interest Group is now accepting nominations for its sixth annual Excellence.Gov Awards.  Building upon the charter to recognize and expand results-oriented electronic government, the theme for this year's awards is Information Sharing. 

Nominations of Federal government programs only, including Federally-led programs done in collaboration with State and Local government partners, will be accepted online at www.actgov.org/ExcellenceGovAwards through Friday December 9, 2005. 

Nominated programs or initiatives should exhibit active participation by multiple stakeholders, support strategic goals and objectives, leverage industry standards and best practices, and demonstrate a sound approach to the security and privacy of electronic information.  Emphasis will be placed on implemented projects with demonstrable benefits.  The Excellence.Gov Awards will be presented in a ceremony and reception taking place in February of 2006.  The panel of government and industry judges will consider the following evaluation criteria:

• Effectiveness in meeting the information sharing objective in terms of overall mission, business process and service improvement.
• Effectiveness of the approach and degree of collaboration across organizations in the implementation and governance of the information sharing solution.
• Incorporation of industry and government best practices and standards related to information sharing including access security and data privacy safeguards.
• Evidence of an engineered, sustainable, multi-constituent solution

For more information on the Excellence.Gov awards and/or to obtain a nomination form, please visit www.actgov.org/ExcellenceGovAwards.

Backing up in real-time

I just purchased some new software: NTI's Shadow 2. It's pretty neat - makes an instant copy (to any mounted drive or directory) of any file that changes on your windows box. It even keeps past revisions of files, so you can roll back changes. I found it invasive when using Eudora, so I set it to back up that directory only once an hour, which seems to work very well for me.

It'd be just "neat" and not worth purchasing, except that it's $0.99 through the end of the year. That's hard to beat...I don't know why they chose that price point, but in my opinion it's very smart. Their cost of creation of the software is sunk, and the $0.99 price is for a download of the software. Figure $0.29 for the credit card charges, maybe $0.01 for the bandwidth, and their incremental profit on every product sold is $0.69. Doesn't sound like much, except that it gets people to their website, where they can buy many other products. It's not a loss leader - they're still making money and getting customers they most likely wouldn't have gotten another way - and it's going to give them a killer mailing list.

Personally I'm doing the backing up to both a central TCG fileserver (on our VPN) and to a little box I have sitting under my desk: Netgear's Storage Central ($99.99 from Buy.com). It's a very cool product - basically a file server with no storage. You buy your own IDE drives (I had a couple lying around the office, and bought a 250 gig one from TigerDirect for $69.99 after rebate), plug them in, close the box, and poof! Instant personal SAN. I also got one of these things for my home network, so we can serve my 7000 legal MP3s (honestly - recorded every one myself, from my own purchased CDs) and the occasional wedding video or StrangerHood episode, and we can backup all the laptops and desktops sitting hither and thither in the house. Fun!

NASA ditches IE in favor of Firefox

A little birdie told me today that NASA has given up entirely on Internet Explorer. Now, if you are an employee of NASA, every time you go to a page using IE, you get up to three prompts telling you how risky it is to run scripts. The official line is that the newest IE vulnerability was the proverbial straw, and now NASA's standard internal browser is Firefox.

That NASA has taken this step isn't all that surprising - NASA has been odd man out for many years, encouraging innovation instead of pushing for standardization in their IT systems. But this is interesting because I don't know of another government agency that has recently standardized (internally) on any browser besides IE for...well, it's been a long time. Years. Time was Mozilla (Netscape) was standard at NCI, but even that holdout has given up the ghost. Maybe this will give them precedent so they can move to a good browser.

Update 9:24 AM 12/5/05: Perhaps this rumor is just that - a rumor. See one of the comments below, from someone with a .ru address but who says they're coming from NASA.

Update 10:24 AM 12/5/05: And another, below, from (apparently) NASA software contractor.

Update 4:27 PM 12/5/05: And a couple more, below, from more apparent NASA employees.

I think it's time to put this rumor to rest. There are three options:

1. It's completely false and my NASA employee (aka "little birdie") was flat-out wrong.
2. It's true for his part of NASA.
3. It's true for all of NASA but a bunch of people haven't heard about the policy shift yet.

The only way to resolve it is for someone to corroborate my birdie's story, or for the furor to die down...