It’s not easy to tell at this point where things went wrong. Partisan interests try to spin this to their political goals. Government contractors try to pass the blame. Media pundits don’t understand technology. I’ll try to sort it all out here, and present a few lessons we can learn that apply to all web development.

What Went Wrong #1: Bad Project Management. A number of bad decisions seem to have been made. The requirement to have users create accounts and login was a huge change that was added very close to launch. [1]

Clear measurement of goals and holding parties accountable seems to have been a problem. As launch approached no one seemed fully aware of how complete, or incomplete, the site was. Especially with multiple vendors involved, oversight is key but it seems there was very little.

What Went Wrong #2: Complexity. Supposedly there’s been 500 million lines of code written so far, with 5 million more needed to fix it [4]. There are problems with getting the site to perform under a high volume of traffic, many different state and federal databases to interact with, and keeping personal information secure. Creating healthcare.gov seems to require solving a big set complex problems.

Or, does it? Just this week there was a start-up team of three guys working in their free time who created thehealthsherpa.com. Also, what healthcare.gov does is not particularly unusual, just a bigger version of some typical web application tasks. Having users create accounts and enter personal information in a safe and secure way is done by thousands of web sites already. Populating complex databases with a large number of inputs from site visitors is also a daunting problem, but relatively easily solved when you have billions of dollars in resources. Same with interfacing with external data sources, there are tons of commercial systems that tie together external data sources and systems on a scale comparable to what healthcare.gov set out to do. [3]

I’ve had a very hard time turning up an actual written scope for the healthcare.gov project, or a description of what contractors involved were responsible for what parts of the system. What we can be certain of is that at least some of the fatal complexity in this project was introduced by a procurement process that brought too many contractors on board, and failed to define an effective way to measure progress and create accountability. [2] Would it have been better to have fewer than 50 contractors involved, and to have spent less? Almost certainly so.

What Went Wrong #3: Testing. Early last week I loaded up Healthcare.gov and attempted to make an account. The first issue I encountered was that I was greeted with a blank white screen after completing my sign-up. It looked to me like the server crashed while delivering the page, and I got a page with no content. So, I reloaded a few times and finally got a page. Same thing happened once I started to “apply for coverage”. I had seen enough, I gave up.

What were the coordinators of this project thinking? That people might not notice when pages don’t load? It has been publicly admitted that up to two weeks before launch no one had yet tested the entire system through [1]. It’s also been claimed that no load testing had been performed to mimic the actual server load of the number of expected users. In short, noone checked if the site worked, and no one thought about performance concerns until it was too late.

Lessons we can all learn from this and bring to our own web projects…

  • Clearly defining project goals and carefully scoping and scheduling at the outset is critical.
  • Setting expectations that are realistic, and re-scoping and re-setting those expectations when a requirement changes are necessary to avoid bad surprises at the end of the schedule.
  • Complexity is necessary sometimes, but when you create complexity by bringing in additional vendors, or by adding features mid-development, it’s important to realize you could be jeopardizing your project.
  • Testing is important! Technical testing, security testing, useability testing are all absolutely necessary and require time to be reserved at the end of the development cycle.

References:

1. http://www.pcworld.com/article/2061940/healthcaregovs-problems-what-we-know-so-far.html
2. http://blog.dobt.co/post/63655420372/how-healthcare-gov-went-wrong
3. http://www.slate.com/blogs/future_tense/2013/10/21/healthcare_gov_problems_why_5_million_lines_of_code_is_the_wrong_way_to.html
4. http://www.slate.com/articles/technology/bitwise/2013/10/healthcare_gov_problems_what_5_million_lines_of_code_really_means.html