Many kinds of computing are "append-only".  By that, I mean that observations about the world are recorded in perpetuity.  Derived results from those observations may be calculated on demand (or kept as a running tally) but you can't rewrite history. 

I spent much of my career working on the transaction plumbing for database systems.  In these systems, all changes are made to a transaction log.  The log describes both the before and after images of all record changes made by each transaction.  The log is appended to at a rapid clip (indeed, some of the fun performance optimizations in achieving the astonishing throughput of SQL Server and other database systems are about optimizing the writes to the tail of the log).  You NEVER modify the log other than by appending to the end of it.  You archive the log for posterity.  In the past, you might delete old log files after many months or years.  Nowadays, legislation makes that more problematic.

Well, if EVERYTHING in the history of the database is represented in the log, what is the actual database itself?  Well, it is a rollup of a view of the changes represented in the log.  The database is a cache of a subset of the log (the subset that represents the latest updates to all the records by the transactions that happened to commit).

So, accountants don't use erasers or they end up in jail.  When an entry is made in the accounting records, it will live forever.  There may be another entry added to ensure that a mistake is corrected but the mistake will not be erased.  The summary of the impact of all of these accounting entries will occasionally be tallied (e.g. your end-of-month balance for your checking account).  Now, the June statement may not reflect some outstanding (and uncashed) checks you've written but those are likely to show up in July's statement.   Quarterly results issued to corporate stockholders will usually include some small correction to earlier quarterly results because of small transactions that legally occurred in the earlier quarter but whose accounting information wasn't rolled up in time for that quarter.  There is uncertainty in the result and that is acknowledge by the mechanism of republishing corrected results.

The Append-Only View of Distributed Single-Master Computing

Distributed work is frequently represented as append-only artifacts on the interoperating systems.  When you deposit your neighbor's check in your account, an entry is made into your bank account showing a deposit and some linkage to the check and your neighbor's bank.  If your bank is smart, a hold is placed on the funds.  When the physical check is shipped to the clearing house to be sent to your neighbor's bank, his bank identifies the account and attempts to debit the account.  If the check clears, funds are moved from bank to bank and the hold is released on the funds (admittedly, sometimes these holds are timeout based rather than linked to the clearing of the check).  If the check bounces (it is, after all, from your neighbor), a debit is appended to your account (both for the amount of the check and for the fine associated with having an idiot for a neighbor).  None of this involved deleting or updating ANYTHING except the interim rollup balance of your checking account which could be calculated on demand.  Frequently, a new interim balance is just appended to the bank account.

Lots of today's computing follows this pattern.  Inventory balances are maintained by tracking incoming and outgoing changes to the inventory.   Sometimes, the inventory balance (in a centralized system) is characterized as "At this time on this date, the inventory for product-X is amount-Y".  In this fashion, the information about the balance is true in perpetuity.  It just becomes as artifact of history. 

Knowledge and Event-Horizons

In a replicated system (say there are N replicas with an opinion of the balance), it becomes more complex.  Imagine a world in which independent actions are taken by replicas which are making guesses (see "Memories, Guesses, and Apologies") and the knowledge of these guesses is gradually oozing back-and-forth across the replicas.  By all the replicas sharing their knowledge of the changes that they have made over time, you end up with two opinions chasing each other within each replica.

First, there is a point in history prior to which your replica-K has complete knowledge of what ALL the other replicas have done... "Gee, I haven't heard from replica-J since 9AM last Wednesday but prior to that, I know what all N replicas have done."  That means YOU have complete knowledge of what happened to the inventory of Product-X up until 9AM last Wednesday (assuming a tight control of the set of N replicas).  Since then, you have partial knowledge.

Now if I hear from Replica-K about his changes that have occurred since last Wednesday, I can advance my uncertainty window to Wednesday at 10PM which was the last time I heard from Replica-1.  This is an ongoing way to cope with independent decision making and the coping with uncertainty.   It is an extension (for multi-master decision making) on the way in which a running balance is managed in a bank account (which processes "the truth as of a point in time" in a centralized fashion).

Now, you say, I am MAD!  Stark raving MAD!  How can people cope with the chaos and confusion.   Well, as I pointed out when discussing my take on the CAP Conjecture, consistency is what people are frequently willing to sacrifice to ensure unfettered availability in the face of partitions.  

[As an aside, in my misspent youth, I was chief architect and lead programmer for Tandem's NonStop Transaction Monitoring Facility (TMF) which did the database logging and recovery for Tandem's NonStop SQL.  Think of this as the data-protecting and preserving part of the super-duper high-availability system of its day.  On the (really pretty rare) times when a system was stuck and needed help to get going, early in my tenure I worked hard to guarantee transactional consistency.  I wanted to ensure we did NOT bring the database back online unless we correctly processed all the committed and aborted transactions and recreated a pristine and correct system.  As I gained more experience, I noticed a large percentage of the customers (but certainly not all of them) just plain wanted the durn thing up and running... They would figure out the mess later.  As I matured (well... at least aged), I learned to ask at the beginning of the crisis whether the priority was availability or the priority was correctness.  I was surprising the number of times availability was more important. ]

So, two interweaving themes are being presented:

1) You don't really ever delete ANYTHING.  As a matter of fact, you don't update anything either... You just accrete new knowledge and distill new implications based upon your increasing knowledge.

2) People (and applications) will act on their partial knowledge and then, when learning new facts, take new actions.   Sometimes, you realize you shouldn't have taken the action but, gee, it was a great decision based on what you knew at the time.   Since you weren't omniscient, you could have perfect knowledge and could make perfect decisions... Shit happens and you deal with it.

Idempotence and the Lack of Erasers

One final observation...  If you never forget anything and never change it, providing idempotent processing takes a completely different twist.  You wrote everything down and, if you hear it again, it's just old news to be dropped on the floor.   That's cool. 

More to be written soon about:

1) Idempotence, and

2) Generating output with elevated abstractions to ensure commutativity

3) Dampening algorithms in redundantly executing multi-master systems...

I've added these to the list of blog entries to write... the list keeps getting longer as I think of things to rant about faster than I am ranting.   Sigh.

- Pat

This definitely falls into the category of TMI (Too Much Information) but it seems to be the way of the blogosphere so what the hell...

I had been obese basically all my life.   I remember being overweight at age 8 and enjoying overeating.  I was over 300 pounds since age 30 and it just was a lifestyle of consuming too much food and being good at cracking self-deprecating jokes (some of you may remember me describing my 4XL shirts as "scalable middlewear").  In late 2002 and through 2003, I packed on even more weight.  I remember a discussion with my doctor that I should consider one of those surgeries (e.g. gastric bypass) to help you lose the weight...  That didn't seem like an appealing idea.

Here I am with with my friend Gurpreet Pall in early January, 2004.

In mid-January, 2004, I went to "Bed, Bath, and Beyond" and bought a digital scale.  I brought it home and noticed it had a weight limit of 330 pounds.  When I stepped on it, the scale said "Err".   This didn't seem right so I took of my clothes and tried again... it still said "Err".   This wasn't good!   At the time, I was having difficulty getting my 4XL shirts and 48 inch pants on since I was too large for them to fit.  I made a decision to change this.

Starting sometime in January, 2004, I started eating less.  My rules were:
1) Absolutely no sweets,
2) Absolutely NOTHING between meals except water, coffee, and diet coke, and
3) Two small plates of food served (lunch and dinner) and NEVER return for seconds.

Later, I added a fourth rule: If you are still eating and you are not hungry, stop eating.

Now, as I got started, the size of the two meals was a bit larger...  It took a few weeks of adjusting to get down to the meals being close to a salad plate in size.  I did not (initially) exercise but I had always enjoyed walking.

It took a bit of time but I soon adapted to life with these rules.  I found small portions in the cafeteria.  I learned that I could order a chicken salad at a restaurant when out to dinner.  Soon, the shirt and pants were a bit looser and I didn't feel the need to buy 5XL shirts and 50 inch pants.  I kept this up and was determined to lose the weight.  There are MANY social situations that drive people to undermine this effort ("Oh, you've been so good, won't you just have a small piece of cake?").  Fortunately, after a month or two of NO sweets, a simple taste of the sugary stuff was awful.

A bit more on purchasing clothes...  America has these places called "Big and Tall Stores" (or what I affectionately referred to as "Seattle Tent and Awning").  You simply can't buy clothes that fit you when you are morbidly obese (which is the technical term for my status for most of my life) unless you drive yourself to these separate stores.  I would show up at TechEd and other events and they would have speaker shirts in any size you want up to 2XL... I would joke that two 2XLs still won't work... "We seem to have a problem with scalable middlewear".

As February, March, and April wore on, I gradually got smaller.  Every time I stepped on the scale, it said "Err".  By March, I was able to fit into 3XL shirts and 46 inch pants.  Still, the scale said "Err".  These smaller clothes started getting looser... Still, the scale said "Err".  I think it was sometime in May that I stepped on the scale and it said "329.8"!   I got really excited and leaned forward... the scale then said "Err".   I don't really know, but I think my peak weight was 360 pounds. 

It was right around this time that my mother who had been very ill passed away.  You have to make a decision that your health won't be driven by the emotional challenges in your life and I managed to follow my rules for dietary intake while going through these personal challenges.  A few days after the funeral, my wife, Michelle, started having so much nerve pain in her legs ("neuropathy") that she stopped being able to walk... Again, you have to separate the emotions from the reality of your health.  Overeating won't help...

As the Summer and Fall progressed, I was dropping 8 to 10 pounds a month simply by portion control (following my four rules).  We were all surprised when Michelle passed away at the beginning of September, 2004, and that was very hard on me.  Still, I knew in my heart that it was essential that I control my health and future.  I had lived a life where I ate for emotional uplifting and (big surprise!) it didn't make me happy.  It is a very human thing to look to food for emotional support but it doesn't end up doing anything but make you feel bad about yourself and perpetuate the cycle.  Even with major life stresses happening around me, there was no upside to eating for emotions rather than eating for nutrition.  In September, I was still right around 300 pounds and that clearly wasn't where I needed to be.  So, I kept going with the portion control.

It was around this time (I was down 70 pounds) when I asked a young (and petite) colleague at Microsoft named Maegen how much she weighed.  Maegen said "Oh my gosh!   I've gained SO much...  I weigh 96 pounds!!"  My response was that we needed to have a party when "Pat lost a Maegen".

One thing that was a big joy was the downsizing of the clothes... 4XL to 3XL.  3XL to 2XL.  48 to 46 to 44 to 42 inch pants...  Each time, I gave away ALL the old clothes to charity and bought new clothes.  Being a nerd, I didn't need to buy too many outfits.  By now, I knew that a size was only going to last 2 months or so and I didn't need to buy a lot of stuff.  Still, I was absolutely determined to give away ANYTHING that became too loose.  There was no going back to the old me.  I remember the excitement of being able to buy a 2XL leather jacket in October or so.  The 4XL leather jacket was swimming on me and I bought a smaller one.  This was still at the Big and Tall Store but getting noticeably smaller.

In October, I went with my friend Gurpreet on a business trip to India and we stopped by to see his family home and meet his family.  I vividly remember the astonishing food!  I was happy to enjoy modest amounts of it and still control what my intake was.  It was an important lesson to me to own my health as I was going through so many different emotional changes.  Still, I couldn't see how I was going to be happier by burying myself into some dessert!  Here I am in India in mid-October, 2004.

I remember Thanksgiving, 2004 at my daughter's house.   It was weird to serve all the normal turkey and its fixings but to serve a small single portion.  It felt right and it was a good gathering in spite of all the changes and difficulties of 2004.

By December I was under 270 pounds.   I remember being down 92 pounds and asking my friend Maegen when we can have a "Pat lost a Maegen" party.  She said "Oh, No!!  I've gained weight and am now 100 pounds!".   My response was "Work with me... Let's meet in the middle!"

By March of 2005, that same old scale that used to say "Err" read in the low 240s.  In April, the mid 230s.  These few months were when I had decided to try something new, had moved to downtown Seattle, changed jobs to try Amazon, and had just met Lisa.  Now, I needed to balance courting this lovely lady with not gaining too much weight.  Fortunately, I was living and working in downtown and walked EVERYWHERE.  Getting to work was a brisk 15 minute walk and then a ride on the bus in the downtown bus tunnel.  Everywhere we went was a walk.  Moving around allowed me to increase my food.  Honestly, I let myself slip back a bit and allowed the scale to pull back up to around 240.  I was able to hold it there.  Soon, I was walking the entire 2 miles each way to work at Amazon.

Here I am with Lisa in June of 2005:

By April of 2006, Lisa and I (now married since November, 2005) joined the Washington Athletic Club and had personal trainers.  After a few months, we were both seeing Shaine at the WAC (I meet him at 6AM and Lisa at 10 or 11AM).  For a long time, it was 45 minutes of weights 3 times a week, now I'm up to an hour.  I combine that with a killer (500 calorie) cardio workout and it makes a WORLD of difference in how I feel.  My favorite cardio torture is the StepMill (affectionately known at the WAC as the "Gauntlet").  This is a conveyor belt stair that doesn't friggin stop!   It is normal for me to do around 140 flights of steps (at 16 steps per flight) over a 30 minute workout.  Every single time, I swear I can't finish it but somehow I seem to and, after recovering for a few minutes and showering, I feel fantastic!

So now, let's get to the real TMI part of the story.  It turns out when you drop a lot of weight, the skin doesn't get any smaller.  You start to look like a shriveled up raisin with lots of skin and not enough inside.  As I would talk to my doctor (who was THRILLED with my weigh loss without surgery), he said it is wise to hold your weight for at least a year before cutting off the skin.  Simple statistics show that most of us that lose weight go back to old habits and then regain the weight (sometimes even more than originally).  It is REALLY bad to cut off the skin and then pack back the weight so you really need to show yourself you have changed your lifestyle.

Anyway, last Fall, Lisa was feeling she wanted to get a tummy-tuck (abdominoplasty) since she was unhappy with how her clothes fit her (and she is a beautiful women who cares about how she looks).  We were sitting in the plastic surgeon's office working on her plans and I said "What the heck!  I'm never gonna make these shirts fit without help, either!".   So we decided to give each other tummy-tucks and liposuction for Christmas.  In my case, in addition to my belly, my chest was still misshaped from carrying too much blubber.   So, December 5th, was Lisa's surgery day (and I remember pounding away in the waiting room writing the final version of "Life Beyond Distributed Transactions: an Apostate's Opinion" since I was right up against the deadline...)  Lisa had a tough time recovering and ended up staying in the hospital for one night but quickly was ecstatic with the results.  As we looked forward to my planned December 22nd surgery I asked her what the experience was like.  She responded that she counted backwards from 10, 9, 8... and someone had taken a chainsaw to her abdomen...  I gulped.

December 22nd, 2006 was my day for surgery.  We had pre-arranged that I would stay in the hospital since Lisa, three weeks out of surgery, was concerned (legitimately) about caring for someone significantly larger than she is.  Lisa is a registered nurse and knows that you need to be careful about healing.  I went into surgery at 7AM and remember awakening at around noon.  For me, it was different, though.  For some reason, I wasn't too annoyed by the pain in either my abdomen or my chest (which had received liposuction).  Indeed, I got quite high energy moving around and saying I didn't hurt.  I didn't find the need for any of the pain medications.  Apparently, I get this from my father who has experienced major surgery (aorta repair) and not wanted pain medication...  I could mostly feel pain when I was moving and pulling but had no problem when lying still.  Being a high-energy nutcase, I did a lot of moving (more than was good for me).   This opened up the wounds a bit and Lisa had to work hard to get me to calm my butt down and let the wounds close... She ordered me to stay FLAT for the first night to let the bleeding stop (which I reluctantly did).  Our friend, Charlie (a VP at Amazon and a dear friend), came and drove me home from the hospital (since Lisa was so recently out of surgery).

We live in downtown Seattle in a condo on top of a very nice hotel with a wonderful restaurant.  My aspirations for recovery included having the special Christmas dinner sent to our home (via room service) three days after surgery.  I then hoped to be downstairs for the special New Year's Eve dinner nine days after surgery.  I met both goals. 

After the December 22nd surgery, I was off the exercise program for almost a month.  They had tightened up the muscles in the abdominal wall and removed over 10 vertical inches of skin (relocating the belly button).  Before surgery, you could grab fists full of skin on my belly.  The incision went on the belt line from pelvic bone to pelvic bone.  Of my 38 inch waste, about 40% of it was horizontally cut.  The front of my abdomen was now straight up and down and that was AWESOME.   It was a huge and noticeable change.  My chest was smaller but still looked weird.  Also, by pulling so much skin straight down my sides (right above the belt line) stuck out in a goofy way.  While it was a LOT better (and well worth the difficulties of the process), it wasn't looking good enough for me to feel happy wearing a tight shirt.  I kept wearing the loose Extra-Large Tall shirts to hide the distortions.  Still, it was a LOT better.

So, after much discussion and healing, I decided to go for "Tummy-Tuck 2.0".  On May 18th, 2007, I went back under the knife.  This time the surgeon removed skin from my chest and reshaped it (also moving the nipple upwards).  The scars are shaped like a comma with the tail of the comma moving away from the midline of the body.  For my belly, the belt line incision was extended to be about 65% of the circumference of my 38 inch waist (maybe 24-25 inches) and wraps quite a ways around my back on both sides.

The second surgery was a lot easier.  Surgery was Friday morning and I went home at around 3PM that day with Lisa driving and me walking just fine (if gingerly).  Charlie called Saturday morning to see if I was in the hospital still and was surprised to find out I was home.  We agreed he would come by and have dinner with us.  When our friends Mike and Barb called to say they wanted to drop off the case of wine they scored for us, we asked them to stay for dinner.  30 hours after waking up from surgery, I was downstairs at our restaurant having dinner in a party of five.  I ate light but had a great time... we visited for about 3 hours and I was tired.  Again, I moved around too much...  Seems like I'm always wanting to move.  I was working at home starting Sunday, had the drains removed Monday, worked a half day at MSFT on Thursday and then had the staples (yup, they stapled my incision -- all 24-25 inches of it) removed on Thursday afternoon and was back at work all day Friday a week after surgery.  Now, my excessive moving did come back to bite me as I opened up about a two inch gap on my left side... Consultation with the surgeon revealed that this will just close up over a 1-2 week period and, indeed, it mostly has.

So, I am a bit more than 3 weeks after surgery and have resumed weight lifting and cardio... Shaine, our trainer, keeps telling me "NO!" as I want to try something that may put pressure on my abdomen.  He also rats me out to Lisa when I overdo...     ;-)    I climbed 115 flights of steps in 30 minutes last Friday (exactly 3 weeks after being under the knife) and the hole in my side is closed up and on its way to healing.  I wanted to push harder but think I can ramp up over the next week or so back to my normal 140 flights in 30 minutes.

Here's some photos from the last week.   I have been accused of joining the witness protection program as no one can recognize me!  It is a LOT of fun when dear friends can't recognize me...  I shock the crap out of them.

Turns out my office at Microsoft was moved on May 18th, as I am now working for Jason Zander on the .NET Frameworks (more on that soon).   I have an INSANE office with lots of paraphernalia which takes HOURS to set up.  It has been killing me to leave the stuff packed and not get settled in.   I will be working on that this weekend since I am now feeling strong enough.   More blog entries soon on the office.

Anyway, I am not  proud of what I had let myself become.  Neither for the health aspects nor for how I presented myself.  At 240 pounds, I am still at least 30 pounds overweight and want to gradually work that down (and build muscle) to be healthier.   I am not panicked about the remaining weight loss but rather determined to have a lifestyle which leads to more health and happiness.  While I am not proud of where I was most of my life, I am proud of the changes.

   Jan, 2004  --> June, 2007     

I remember spending over 30 years feeling like I couldn't do anything about my weight.  When people ask me how I accomplished it I have a number of quips I frequently respond with.  The most honest and direct answer is "I changed my mind".   When you realize you own your body and you own your decisions then you make things change.  For me, I had to decide that I needed to eat for nutrition and not emotion.  Now, I must say that I am enjoying myself now and eating like a normal person does... Sometimes too much and frequently with wonderful food.  I'm going to have to either cut back on food and/or increase the exercise to move down from that 240 pound plateau.  For me and at this stage in my life, I want to make that be an enjoyable journey but I want to move forward gradually and in a way that is a permanent lifestyle. 

Lisa and I gave away all my Extra-Large shirts this week.

All my best,

- Pat

My friend Harry Pierson points out that I am blogging-challenged and didn't include the links to my paper and talk.

This talk and paper were given at CIDR (Conference on Innovative Database Research) on Jan 8th, 2007.

Thanks, Harry!

- Pat

Wow!   It’s been over two years since I’ve blogged or had much of a public presence and I am VERY glad to return.   There is SO much to tell all of you.

First of all, I returned to Microsoft on March 5^th after being gone for two years (to the day… my last day was March 4^th, 2005 and my first day back was March 5^th, 2007… it was an accident but it makes it easier to calculate my seniority!).   I am working in the Developer Division and have not quite settled on which technology project will be my new home but I am determined that I will be taking an active role in driving forward new products.  While you will see some of me in public (and on my blog), I will concentrate on building products.   Being back at Microsoft feels like being at home!  While I’ve had a marvelous two years (more on that below), I really missed Microsoft and I really missed interacting with customers and our Microsoft folks working in the field.   Now, I can see my old friends much more often!   I have SO much to tell you about…

First of all, my personal life has been transformed.  Right as I was leaving Microsoft (and I mean our second date was the evening after my last day at MSFT in March of 2005), I started dating a wonderful lady named Lisa.  She’s a whole three years younger than I am and is a bundle of joy.   Lisa was a paramedic and firefighter in Ohio/West Virginia for 12 years (the firefighters told her women couldn’t do that job so she became the first female firefighter in her department).  Later, she put herself through school to become a registered nurse working in ER trauma.  After living in Florida for eight years, Lisa decided she was bored and started working as a traveling nurse going from city to city working 3 to 6 month contracts.  When I met her, she had just arrived in Seattle (with her two cats Jasmine and Samantha) to work the night shift at Harborview, the only level-one trauma center for five states.  Lisa moved into the same apartment building in Seattle that I had moved to and we met hanging out in the lobby.   After meeting in February, we were inseparable by March, engaged in July, and married in November of 2005.   It has been a joyous time for both of us.   We bought a condo in downtown Seattle and LOVE the downtown living.  We walk EVERYWHERE and, for two years, I walked to work at Amazon.   It was truly a transformation for a fellow brought up in the suburbs…

What about Amazon?  I worked there almost two years and found it fascinating to see the amazing technology and super smart people creating that scalable and reliable web site.  Amazon’s use of service oriented architecture is one of the world’s most advanced (if not THE most advanced).  Watching the application of process, discipline, and organization to a rapidly evolving disconnected and yet connected business problem gave me a much deeper understanding of the challenges faced by all enterprises.  It was my privilege to work with people across the entire development arm of the company and to focus in on the area of the product catalog, search, and buyability (i.e. who is allowed to buy a product from which merchant and at what price).  This is service oriented architecture at its best.  It was a great experience but I truly missed the opportunity to communicate more broadly through the industry and, frankly, decided that I missed Microsoft and its customers!

Most of you would not recognize me (I have walked past many former colleagues and even my brother and his family without being recognized!).  After working hard to lose 120 pounds, I am literally 2/3rds of my peak weight.  On top of that, I shaved off my beard for my going away from Microsoft in March 2005.   The final change is that Lisa convinced me that everyone in the world (except me) knew I was bald and the removal of those last few diehard hairs on the top of my heard would be an act of mercy (think Dr. Phil’s hairdo…).   The net effect is that there’s pretty much nothing left visually of the old me except the twinkle in my eye and the shitty grin on my face.  ;-)

I’ve grown to love being a pedestrian and a bus rider!   It is normal for me to go two to three weeks without driving (and, indeed, I am typing this on the 545 metro bus home to downtown Seattle from Redmond).  This is part of how I’ve managed to keep fit.  That combined with meeting the personal trainer 3 days a week for an hour of weights and (mostly) doing 30 minutes of cardio a day and I seem to be able to eat food (and alcohol, too).   Living in downtown means it is natural to walk and the Redmond campus is a nice place to walk.  Having your commute consist of 30 minutes of email on the bus and 20 minutes of walking is pretty nice!

I did manage to publish one paper during my tenure at Amazon.  It is called “Life Beyond Distributed Transactions: an Apostate’s Opinion”.  In this paper, I argue that real scalable systems simply do not attempt to do cross-system transactions.  If you presume that distributed transactions are too fragile for large deployments, how DO you think about really scalable applications?   I presented this at CIDR (Conference on Innovative Database Research) in January 2007.   While I was there, I was privileged to spend a lot of time with one of my dearest friends and my mentor of 25 years, Jim Gray.   We stayed up past midnight for two of the evenings, talking into the wee hours.  Jim had been encouraging me for years to spend more time writing and communicating publicly and, frequently, I would take the easy way out and not do the hard work it takes to make this happen.   After being out of the public eye (and not at any events) for over a year, being at CIDR had a deep impact on me.   By the end of January, I had decided to investigate returning to Microsoft because I wanted to be able to participate more in the industry as my friend had encouraged me to do for so many years.   I sent an email off to one of the executives at MSFT on Friday, January 26^th to begin the discussion.   It was two days later that Jim went missing at sea and is still missing.

It would be hard to describe the profound effect that Jim has had on my life (and on many others).  I had started reading Jim’s papers in 1980 and, of course, my thinking was deeply impacted by his writing.  In 1982, I was working at a small company (which was having challenges) when a departed colleague working at Tandem Computers was late to a going away party “because he was at a meeting with Jim Gray”.   I looked around the room and decided to move to Tandem.  I started 3 weeks later.   During my almost 9 years at Tandem, I watched Jim influence with kindness and insight, always encouraging and uplifting.  He ran a weekly talk series and put me on stage many times (sometimes with only a few hours notice!) and taught me the value of communicating.  His patient and consistent mentoring was, by far, the largest force influencing my career.  Jim tapped me in (as a young kid in my 20s) to conferences with some of the most influential people in the database and transaction processing industry.  Whenever Jim would present, everyone (including me as his most avid fan) listened with rapt attention to his way of explaining the most complex problems in the simplest ways.  Jim would always know who was doing what and was constantly connecting people together for their common good.  I literally have never met anyone that holds Jim in anything but the highest esteem.  Every job I’ve ever taken since 1982 was directly traceable to Jim’s influence on my life.  My return to Microsoft was prompted by Jim’s advice before he went missing.  

While I write this with great love, I don’t want to be maudlin’ and sad.  All these lessons I carry with me every day and try to work in that spirit.  While we still don’t have answers to where Jim is after his trip out to sea more than three months ago, I know he has changed many lives for the better and many people carry Jim’s impact with us every day.  I miss my friend.

It is great to be back at Microsoft and, now I feel like I should be blogging regularly and getting that crufty old website of mine (www.pathelland.com) dusted off and filled with the work I have so far.   I look forward to seeing more of all of you!  

-          Pat

Just an update on how things are going personally…

It’s been more than 3-1/2 months since my wife, Michelle, passed away and things are settling into a new normal.  It is very hard to be without her but I am very, very grateful for the wonderful family that I have.  I’ve been living with my youngest daughter, Jenny (age 28), her husband, Ray (age 28), and their daughter, Destiny.  It is so nice to have a family to come home to each night.  They couldn’t be more wonderful!  My biggest complaint is that it’s kind of funky to call home to tell your daughter you’ll be home late.  Still, you can’t have someone sitting at home worrying.  They have all been incredible and it is such a huge help.  One of many funny stories is that I had gone out with some friends for dinner a week or so ago and my cell phone rings.  Jenny asks how I’m doing and I tell her that we decided to keep visiting and had moved from the restaurant to a bar.  Jenny says to me: “Have you been drinking, Dad?  Are you alright to drive?  You know I’ll come and pick you up… <pause> Gee, a little bit of a role-reversal, huh?”  It is just wonderful to be so loved (and I stayed out late, remained sober, and drove home on my own).

Last week I was at Disneyland with 5 of my 11 grandkids.  The biggest treat was seeing my daughter, Christine (age 34), her husband, Greg, and their 4 kids.  Christine has always been a joy and a huge part of our lives.  The time since Michelle’s death has been no exception and she has flown from California to Washington a number of times to help with many things and to just plain be there for us all.  Still, with Michelle’s illness, the funeral, the chaos and difficulties, I had not really seen the kids since June (I’m not counting when I saw them at Michelle’s funeral since we were all zombies).  It was just plain too long!  Seeing Anastasia (age 3), Branden (age 8), Vince (age 11), and Nick (age 15) was SUCH a treat.  As I settle into defining a new normal for our lives, I know that this simply has to include serious visiting to the Bay Area in California at least every couple of months.  They are simply too important to me to see so rarely.  I’ll never forget sitting in the live theatrical production of “Snow White” at the Fantasyland Theater holding Anastasia while listening to the singing.  Live musicals have always gotten me mushy but holding onto her while watching and listening really touched my soul with both the loss of this last year and the joy of such a rich family.  I will be down seeing them for New Years.

Both my sons, Mike and Rich (identical twins age 32), have been a source of great happiness.  Both of them have grown and matured so much and have handled the difficult times amazingly well.  It has been a special treat that I have been growing so much closer to each of them.  While we used to just do our own thing and see each other as our lives revolved around mom, now they call me for no reason at all but to chat.  It is an incredible blessing.  For each of them, we try to see each other for dinner or lunch every week or two (if possible) and that is so cool.  I’ve been to my grandson, Alex’s Junior High football championship game (he played defense and they were champions of their league).  Alex is 15 now and I had the astounding experience of watching him legally drive away (the age for learner’s permits in Washington State is 15).  It really hits you to watch your grandson drive…  I think I’m the oldest 48 year old I’ve ever met…  The instant family at age 19 really changes you.

We all made it through Thanksgiving very, very well.  Christine’s gang couldn’t come from California for the holiday since Greg had to work.  Jenny and Ray took over the preparation of the feast and did a magnificent job.  Jenny was terrified the food wouldn’t be perfect but it was incredible.  We had 15 people (Jenny’s gang, Mike’s gang, Rich’s gang, and my recently widowed 84 year old father (Mom died last May)).  While we stopped to remember the loss of both these ladies and how hard it was, we absolutely all felt a true giving of thanks at the blessings we have still with us.  It was so hard to have the holiday with such a big hole in the family and yet the family is so very strong we all truly focused on the many blessings we have.

We have successfully emptied the old house that Michelle, her mother, and I had moved into only this last February.  It never became our home and the house is not something that I want moving forward.  My mother-in-law is staying with a brother-in-law and doing fine.  With lots of family help (and frankly me doing a little too much avoidance of the work myself), we have sent furniture, family heirlooms, and lots of stuff to different family members.  The job is still only partly done as we have two storage units full of boxes that need to be sorted through.  It is a difficult challenge to disassemble a household filled with 29 years of memories.  You shift all too quickly from laughter to tears.

As we prepare for Christmas, it is hard but we are moving through it.  Michelle was an enthusiastic Christmas zealot who simply loved to get lots of presents for everyone (and there are a lot of people in that list of everyone) and she loved the decoration, excitement, and family time of Christmas.  She is strong in all of us as we try to define a new normal.  Still, there is a lot of happiness in trying to select presents for everyone!  Being male, I had largely ignored this (with a bit of Grinch-ness) as it used to swirl around me and I would focus on work.  I am finding that participating in the shopping (with lots of family help) is more fun than I had thought.

On another note, last March or so, I had decided that I was simply too fat and too large.  I started eating more normally with nothing between meals, no sweets and snacks, and modest portions for meals.  Even with the stress of this terrible year, I somehow felt that eating too much was not the answer.  The weight has been consistently peeling off and I am now down over 80 pounds (from 350 to 269 this morning).  This is the least I have weighed in 20 years.  My goal is to drop another 60 or so to be within striking range of a healthy weight.  I hope that when I come to visit all of you in the upcoming year, I don't leave quite as deep an impression as I have in the past!

I miss Michelle very much but I am also relieved that she is not in the pain that she suffered for so many years.  It is hard to look and see all that is lost and yet there is so much still here to care about and embrace.  I am beginning to feel emotionally stronger after all the difficult changes and promise to regain a new vigor for posting technical content.  You will see less of these cathartic family discussions and more of the nerd in me cutting loose.  I have a long list of posts in mind and will do a couple of them very soon.

I wish all of you a wonderful holiday season and I look forward to interacting and learning from you all in the new year!

Love and kisses,
Pat

I always talk about "Long Running Work" and steadfastly avoid "Long Running Transactions"...  My preference is to use the word "transaction" to mean an atomic operation that occurs at a single service within a second or so.  I mean the ACID (Atomic Consistent Isolated Durable) transactions that usually involve holding locks and that we transaction theorists love.

Whenever I hear about a "Long Running Transaction", I wonder about where it ends.  Let's consider the following example:
1) I decide to take a trip to Europe so I book some airline and hotel reservations.
2) The hotel in London hits a threshold of occupancy and decides to increase staffing and food for the restaurant.
3) The hotel orders more food from the Green Grocer.
4) The Green Grocer hits a threshold and orders another delivery from its shipper.
5) The shipping company hits a thresold and orders more diesel fuel for its trucks...
6) Two weeks later, I cancel my trip.

So, if I believe in long-running-transactions, the shipping company doesn't need any more diesel fuel!  I don't think so!

The long-running-transaction approach assumes that there is a cohesive set of work across loosely-coupled and independent systems that is atomic.  Notice it is not ACID in that Isolation is clearly different and Consistency is clearly different.  Hence, you would consider these "long-running-transactions" to exhibit Atomicity and Durability.  I am arguing that Atomicity is not a property that flows across multiple services as the semantics of interaction across these services is not tightly associated.

My preference is to talk about "long-running-work" across two services.  I can examine the semantic interaction of two services across a set of messages.  This leads to the notions of reliable messaging (and multi-message dialogs) which is one of my favorite topics and something I will be posting more work to my website (www.pathelland.com) soon.  It also leads to the notion of a contract across the messages that flow between two services.  I love these concepts. 

Another concept that I am strongly in favor of is the concept of a service being in the midst of multiple related dialogs.  My attempt to book travel arrangements puts me in multiple interactions with different airlines and hotels.  These interactions involve multiple messages and continue at least until the date of my travel (unless I cancel).  When my incoming reservation causes the hotel to increase its food requirements, the hotel will use business logic to control the relationship between my incoming messages and the restaurant related messages.  This allows it to consider the wisdom of canceling the food order when I cancel my reservation... very different than mindlessly assuming some coordinator associates these into a long-running-transaction.

Work flows across services and businesses in ways that do not magically "undo" even if you do master the use of compensations rather than the classic "undo" of replacing the before values.  The behavior of our interconnected systems is more an integration of lots of disparate work into summaries.  When I through a rock into the ocean by San Francisco, that can and does impact the waves that hit Tokyo but only in conjunction with countless other stimuli.

This is why I like to talk about long-running-work inside a single service or across exactly two services using a multi-message dialog.  The phrase long-running-transaction is a chimera and leads to a false hope of simplicity.  The reality is much richer and more complex.

Love, Pat

Howdy, Everyone!

I am just back from an 8 day trip to Amsterdam where I gave 6 sessions (and helped with one more).  It was an exhausting trip with little sleep and I am happy to say I actually saw a bit of the beautiful city of Amsterdam on Saturday before coming home Sunday morning.  More on this in a minute!

First, I want to annouce the launching of www.PatHelland.com .  This will be an ongoing site at which I plan to put all of my new work and gradually dust off my older work and put it up for all of you to see.  One of the stresses that I have felt as I want to blog something is that I am not sure you all have seen my other work and so I frequently don't feel I have the context to refer to in my blog comments.  I will try to get better about this now that I have an ongoing reference site to link to.  I would like to thank Harry Pierson (on my team at Microsoft -- the Architecture Strategy Team), Ed Siegrist (of Dynamic Events), and Cliff Thomas (of Logo Repair) for helping me get the web site up and running.

On the web site you can see the powerpoints for all of my presentations for the last year.  Now, many of these are repeats with minor hacks and/or template changes.  Still, I am going to publish ALL non-confidential work that I do moving forward.  Also, there will be an ongoing effort to ressurect anything old and interesting that I have done before.  We found 34 videos I have presented archived in the Microsoft video archives and we will pick the best and get them out for you to see.  I will try to keep links to get best versions (in my opinion) of the different works available.  Of particular note to be seen are the 4 talks descibed below that were given in Amsterdam (GNLARC, ARC402, ARC302, and ARC303).  The 5th (PNL005) was a question and answer session, and the six, “The Nerd, the Suit, and the Fortune Teller” will be available soon on www.PatHelland.com .  I am at home today and have the video on my laptop to get up on the web site this week.

Amsterdam was exhausting.  I landed late Staurday night after flying through Copenhagen (with a 6 hour layover).  When I left home, I was very unprepared for all the presentations I had committed to giving.  Between business at home (my wife is sick in bed) and a wonderful visit from my oldest daughter and her 4 children, I was behind...  Of the 4 talks, GNLARC was absolutely complete, ARC402 needed rewriting as I was unhappy with its stucture (although it had been given before), ARC302 (Metropolis: Buildings and Apps) was about 75% written and I had given a dry-run to a group of customers the previous week... it had not been as easy to understand as I had hoped!!    Finally, ARC303 (Metropolis:Interchangeability and Operations) was about 25% written.  I was in deep shit!!

On the plane flying into Amsterdam, I wrote on ARC302 to pound it into submission.  I continued to struggle to get to the spirit and soul of the talk.  When I saw the lovely room at the Okura Hotel (yes, a Japanese hotel in Amsterdam... it was a nice place and it was walking distance to the RAI convention center), I noticed that there was not anything comfortable to sit on!  I had visions of hours upon hours of me and my laptop!  I was able to convince the hotel staff to steal a comfortable chair from the lobby and bring it to my room!! Thank Goodness!!!

I spent all Sunday morning locked into my room writing on ARC302.  Sunday afternoon, I walked over to the RAI and found the speaker lounge.  There, I actually looked at my schedule and realized that I had 3 talks on Tuesday!  Arrrggg!!!  And ARC302 wasn't due until Thursday!  At this point, I walked back to the hotel and grabbed dinner on the way.  I locked myself into the hotel room and into that comfortable chair for over 26 hours straight... not emerging from the room except for Monday morning I had breakfast downstairs and bumped into Clemens Vasters while eating.  On Monday evening we had a rehearsal for “Mr. CIO Guy“ in the Forum room with Don Box and David Chappell.  After the rehearsal,  I was able to have dinner with a new friend of mine, Craig, who happened to be in town on unrelated business and we had a great time chatting!  What a nice break!  Soon, I was back at the hotel typing.

I first pounded away on ARC303 finishing the history section on the development of manufacturing interchangeability and then on the implications of interchangeability on service oriented operations.  This is a cool talk and I am very happy with it!  Next, I pounded away at the rewrite of ARC402.  It wasn't done when I went to the RAI to prepare for the GNLARC talk and song that was at Noon on Tuesday.  After completing this, I went to the speaker lounge to finish ARC402.  I finished at 14:30 and the presentation was at 16:30.  ARC303 immediately followed at 18:15 until 19:30.  A busy day!!!  Dinner afterwards was at the speaker lounge and then back to slideware as I still needed to get ARC302 done.  I hadn't slept more than 5 hours a night since Thursday night at home near Seattle and I crashed for a solid 8 hours.

Wednesday started with me on stage accompanying Wojtek Kozaczynski for ARC307.  He was concerned he wouldn't give a dynamic enough presentation and asked me on stage.  Well, he did a magnificent job and simply didn't need me there!  Still, it is a bit of work to sit up on stage and watch someone present while 800 people are watching you watch him present.  Wojtek simply did a great job and didn't need anyone to help and, believe me, I didn't help.  The rest of Wednesday was more slide work and the panel session PLN005.  I thought we did fine.  The panel consisted of Maarten Mullender, David Hill, and Steve Cook, and me.  It was moderated by Arvindra Sehmi.  I was happy with the discussion.  Back to slide work and then in the evening I went to join a lovely group of reporters at a very interesting and old building converted into a tiny restaurant in downtown Amsterdam.  Unfortunately, I was feeling the crunch and had to leave early (re-inforced by the fact that the Football (soccer) game was almost over and it was Holland against Portugal in the semi-finals.  I was afraid I wouldn't be able to get across town to the hotel to finish.  With 4 hours sleep, I handed in my slides at 8:30 AM Thursday in time for a last minute copying for a 10:30AM presentation of ARC302 (Metropolis:Buildings and Apps).  I am not yet as happy with this as I am with the others but I will work more on it and I believe it will become a very good talk by the next time I present it on July 19th.  It needs a rewrite.

Thursday afternoon was all about preparing for the Friday session on “The Suit, the Nerd, and the Fortune Teller”.  Rafal Lukawiecki had suggested this session and Clemens Vasters and I formed the threesome.  We spent time on Thursday afternoon working on slides and talking about the plan.  By Thursday night, we were in the venue rehearsing and preparing for the next day.  All of us were so exhausted we could barely stand and we kept working until quite late.  I know that both Rafal and Clemens had talks in the morning and I was fortunate enough to start Friday at Noon.  I slept 8 hours as I was simply pooped.

Friday was fun!  We had the Suit, Nerd, Fortune Teller presentation and I think we must have had 1500 people.  That was a demanding session because we were always having to be on our feet thinking of the next comment!  Still, it was a lot of fun!!!  Friday night was a lot of fun at the speaker party and I thoroughly enjoyed the comedians and had a good time outside chatting with Clemens...

Satrurday was sightseeing and then home yesterday... Amsterdam is beautiful and I thoroughly enjoyed the Van Gogh museum.  Thanks to my new friend, Craig, for recommending the museum and the audio tour.

Love, Pat

Sorry for the long quiet... family medical stuff... lots of travel... lots of excuses...  I'll try to do better...

Looks like I got an MSDN front page for an article I wrote that was published in Journal a few weeks ago. 

I got asked to post the following ad for JOURNAL:

“The new homepage for JOURNAL - Microsoft’s Architects Journal is live on MSDN Architecture Center: https://msdn.microsoft.com/architecture/journal/. This page describes what JOURNAL is and has links to JOURNAL articles that are now in the MSDN library. Currently, we have all six articles from JOURNAL1 plus MY new Metropolis article from the hot-off-the-presses JOURNAL2. Over the next couple of months, we will be publishing the remainder of JOURNAL2 articles in MSDN. In the meantime, we are providing PDF versions of both issues (available in A4 format for convenient printing) to users who register.”

Today, the article ( https://msdn.microsoft.com/architecture/journal/default.aspx?pull=/library/en-us/dnmaj/html/aj2metrop.asp ) is on the front page of MSDN.  Cool!

I've seen some mail on this and I wanted to post a couple of comments that I had made in response to a couple of questions:

1) We have had major problems getting our customers to except standardized structures, processes, and products. It seems that most people believe, or at least want us to believe, that the way they do things is better than everyone else and we should conform to their requirements.  Also, they are reluctant to share technology with their competition.  This leads to very complex Software due to the numerous configuration switches required to meet our customers different beliefs. This seems very similar to the standardized clothing example.  To make matters worse, they all want it easy to implement, which it is not because of the numerous configuration switches. To make matters worse, “consultants” are used for implementation.  Once a consultant gets involved they usually generate more conflicting requirements, hence the problem proliferates and gets worse. Do you have any idea how this type of obstacle was overcome in your Metropolis Comparison?

2) Who plays the role of “City Government” in the IT world?