When people find out that my wife Amanda lives in New Orleans, while I live here, a predictable dialog ensues wherein I always end up saying that I’m here because of Katrina.

When we move to Saint Paul, MN, that will no longer be the case. Amanda wasn’t attracted to any of the family medicine residencies in New Orleans. If I had stayed in New Orleans until now, we would still probably be moving elsewhere now. Therefore, once I reach Saint Paul I will (a) no longer have conversations about living apart from my wife, because I won’t, and (b) never state that my reason for being there is related to Katrina.

Honestly, I’ll be happy to avoid the conversation. I resolved to myself that I would quit talking like a victim about the whole thing after a year had passed. For the most part, I’ve been able to hold to that resolution, but I occasionally either slip up and talk about it out of self-pity, or it comes up in conversation as mentioned above. In the latter case, I find myself slogging through the same conversation every time.

“Oh, you’re married?” they say. “Yes, ” I reply, holding up the adorned ring finger. “Why haven’t I met your wife?” they ask. “Well, she lives in New Orleans.” They usually produce some look of pity and or astonishment at this point. “Yeah, I ended up moving here as a result of Katrina fallout.” “Oh, how’s the city recovering, blah blah…” You all know conversations like this. They keep asking the obligatory questions, and I keep giving the obligatory answers, feeling like I’m talking too much about myself when I’m not inclined to talk at all (so I don’t really ask about them in return).

Looking at this, talking about Katrina is probably avoidable in these conversations, but it’s the truth. It is the simple, straightforward explanation of why I am here and my wife is there. This has come up more frequently of late, whenever I’ve needed to inform someone that I’m moving away soon, which is what made me think about it all now.

I’ll be glad to have all of that behind me, and behind us.

I’m hovering between excitement and panic.

In two months, I’ll visit for one last week, and then the trips will largely stop.

We hope to move back there when Amanda is done with residency, if we can swing it. There will probably be some visits for holidays, as my family’s home is still just outside the city. Nonetheless, it’s slowly dawning on me that my monthly, re-charging dose of New Orleans is coming to an end.

At this point, the legacy of Katrina in my life is diminishing. Assuming that all continues to go well, I’ll be graduating from Johns Hopkins rather than Tulane. All sorts of other things both in my head and in my life have changed as a result of the flood. However, I will once again be living with my wife, we’ll go on with our post-school and post-New-Orleans lives. I’ll finally start to get some closure on something that can never really be fixed or undone.

So very many things have happened since then. Here are the highlights:

• 2002: Started learning the ropes of the lab
• 2003: Continued to familiarize myself with the computers and code in use in the lab. The most powerful machine in our possession was an SGI with 8 processors (the Origin 300 listed here). There was almost always a wait to use those processors. Spent my summer vacation working in the lab. This was the first time I was paid to to research. Some time during this year (I think) I created the lab wiki using MoinMoin. By this time I was administering the lab computers and was sick of answering the same questions over and over. In desperation I created a wiki and started putting answers on it, referring people to the wiki when I was asked a question. The wiki is now (as of November 2007) huge, and contains basically all of the documentation of everything used in the lab, as well as gigabytes upon gigabytes of attached models, data, and images.
• 2004: Graduated from Tulane with my Bachelor of Science in Engineering (BSE) degree. Joined the lab as a graduate student. Sometime in 2004 (I think), Tulane acquired a Linux Networx cluster, and we owned 20 nodes in that cluster.
• 2005: Shortly after returning from my trip to Niger, Katrina struck New Orleans. The lab was scattered. Few people in the lab had access to their data. A few lab members actually snuck past armed guards to get our file servers and some workstations from our lab at Tulane. We took up residence in St.Louis, MO for two and a half months, aided by our colleagues in the labs of Drs. Yoram Rudy and Igor Efimov at Washington University. By the end of the year, we had returned to a slowly-recovering New Orleans.
• 2006: Dr. Trayanova accepted a position as a professor at Johns Hopkins University. Almost the entire lab transfered to JHU and moved to Baltimore, MD.
• 2007: In April, I began discussing a cluster purchase with High Performance Computing (HPC) companies. Around that time, the weather warmed up, the server room could no longer be adequately cooled, and we started limping by on 4 compute nodes. By the end of July, we had placed an order for a new cluster. We moved from Clark Hall into the newly-completed though poorly-named Computational Science and Engineering Building. In mid-November, most of our new cluster arrived, though FedEx dropped and destroyed one rack, and the cluster was not completely set up.

That brings us to the present day. Now, looking forward a little:

In the next two weeks, the cluster set-up will be completed. We will have free rein on 140 compute nodes (20 old, 120 new), all managed from one head node. The new nodes will be connected by the fastest Infiniband interconnects available on the market, and each node will have 8 GB of RAM available, with the potential to hold 64 GB each. There are four 3.0 GHz Opteron cores per node, yielding a total of 480 processors and 960 GB of RAM on the new nodes alone.

To give you some perspective on what that means, let me give you some details about the kinds of models we run. When I joined the lab, our two largest models consisted of a 4mm thick slice of the canine heart, and a very smooth, idealized model of the rabbit heart. These models are composed of 1.6 million and 0.82 million tetrahedral elements, respectively. It took something like an hour of wall clock time per millisecond of simulation time to run these models. (In other words, to get one millisecond worth of simulation data it was necessary to wait about an hour.) We could run one or two simulations at a time, at that speed.

My newest model, and currently the largest model in use in the lab, is composed of 28 million tetrahedral elements. On a cluster similar to our new one (Lonestar on TeraGrid), using 32 processors, it takes about 22 minutes of wall-clock time to simulate one millisecond in the model. Using a crude estimated unit of speed of (minutes real time / millisecond simulation time / tetrahedral element), and focusing only on the number of simulations we can run at once, not the number of CPUs required:

• Old way: 60 minutes / 1 ms / 0.82 million tets = 73 minutes / ms sim time / million tets
• New way: 22 minutes / 1ms / 28 million tets = .78 minutes / ms sim time / million tets

We have increased our simulation speed by almost 100 fold. We can run two to four simulations of that size at a time, vs one or two the old way. But that’s not all. We can now run bigger models. Much bigger models. We are now capable of running something the size of a dog heart (we have verified this). More importantly, we now have the technical capacity to run a model the size of the human heart, with a resolution near that of the size of a cardiac cell, and to model contraction in addition to electrical activity. It remains only to develop such models. We are prepared to store the results: the new cluster has a storage capacity of 28 TB online, with the ability to add something like 40 or 50 TB more simply by expanding the existing storage device.

In my time in the lab, I have watched our abilities expand from serial jobs with relatively small models to massively parallel jobs with the capacity to model electrical and mechanical activity in the human heart. We are just beginning a very exciting time in the lab and in the field, and what’s really killing me is that fact that there’s so much more to tell you.

But I can’t just yet.

(This post was partly inspired by a conversation with Maria and Amanda)

As we walk around post-Katrina new orleans, and see things that still aren’t back to normal, it’s always heartwarming to see one of these signs.

After Katrina, I wasn’t sure whether my printer was salvageable (it was), and anyway I didn’t have one with me. I had to scan or fax and send a bunch of stuff, so I bought an HP OfficeJet. Modern OfficeJets can scan in one of two ways. They have a classic flat-bed scanner for complicated things, books, photos, etc. However, they also have an ADF on top of the flat-bed scanner. When you use it, the scan head moves over to the far end of the printer and remains stationary as the feeder pulls documents past it. This makes it possible to scan stacks of notes or whatever with the press of a single button. Having such a feeder makes a big difference — I would never scan that many pages manually.

Combined with some sort of optical character recognition (OCR), and the temporary use of paper for taking notes (which are then scanned and recycled), I’m now able to keep just about everything digital. A major advantage of this is that I can just keep all of my documents archived on a remote storage service, and access them from anywhere.

I’ve thought about buying a used Tablet PC on eBay, but I really don’t want to have to maintain a Windows XP machine.

Apparently you might not see this in Internet Explorer 7. Use Firefox.

I have kept a Katrina category on the blog. You can look at all of my Katrina-related posts here.