Did the AHA Drop the Ball with the National Homebrewers Competition?
Tuesday had us homebrewers all geared up to register our beers for this year’s NHC, National Homebrewers Competition. There are only a limited number of spots for each region of the country and many of us wanted to get our registrations in early so that we could get our beers in to our local region of choice and avoid getting shut out, like many did for the National Hombrewers Convention. With thousands of homebrewers slamming the servers at 1pm on Tuesday, the AHA’s new registration site just couldn’t take the load, shutting most homebrewers out of the site for over an hour.
I managed to register 3 beers, so I’m in or am I? I really don’t know, nobody knows at this point what is going on. So a quick recap of what I experienced; first I couldn’t even get into the site. After an hour or so of trying I managed to get in. When the home screen finally loaded, I saw 244 of the 750 entries for the NYC region were already taken, fine by me; I should be able to get in. Once I registered myself and selected my region, I was greeted with a note that all 750 spots for my region were filled. I was stunned! How could 500 entries have been submitted in less than 5 minutes? I assumed at this point it was a mistake and the site was counting the total entries for all the regions and it was a bug. The “Add New Entry” link worked, I filled out the form and successfully submitted my entry. Great! Well, I thought so until I saw my entry number was 861, that was 111 over the max for my region, but the system did allowed my entry.
I was able to add 2 more entries and pay for them without issue and received my receipts for the 3 entries. I figured at this point, no matter what happens I have paid entries, I’ll be in.
After logging out and going back to the home screen, I saw there were only 550 entries of the 750 for the region. Good, I guessed at this point, but who knew what numbers were real or representative of the actual submissions on the site.
By this time I was seeing a whirlwind of tweets and emails about what a clusterfuck this registration was becoming, many people were pissed and annoyed. I wanted to check on my entries to make sure they were still there, so I went back to the site and there were 900+ entries of 750 entries for the NYC region. Luckily for me all my entries still accounted and paid for. But here is where the programmer came out in me; I decided to add a new entry to see what would happen. I created a bogus IPA entry submitted it and the system accepted it! This is where I got pissed.
What developer, development company, or client would allow such simple quality assurance test cases to fail! If I allowed this at my day job I would be fired. This is not a complicated registration system, technically speaking. I’ve been writing these types of systems for the better part of 20 years, this is a below sub par job. Simple quality assurance test cases were obviously ignored.
So did the AHA Drop the Ball with NHC? I believe they did and this is why. They hired a development company that could not deliver on the product and went ahead with NHC registration anyway. I hold no ill will toward the AHA, they do a lot of good things for homebrewers, but it seems they did not learn anything from the debacle last month with the Conference registration. While the AHA did drop the ball and it is their face with mud on it now, I lay the blame almost wholly on the development company, which appears to be Better Beer Scores LLC, the company listed on the footer of BeerCompetition.com web site. Why blame the development company, because they are supposed to be the experts. I’m sure they promised the moon at a bargain basement price for the AHA.
Here is what the development company failed to do,
- Either inform or convince the client they need a dedicated server or hosting company that can guarantee the load of 10000 concurrent users.
- QA, quality assurance. This means you test the application to ensure the system blocks users at the limit, numbers add up correctly, and the system functions as promised.
- Eliminate the logic errors, which blocked some users from registering new entries and other entering when they shouldn’t have. Once again leads back to QA!
The AHA has a monumental task ahead of them, how do they fix this years entries? Furthermore, what do they do next year and going forward?
Here are some my thoughts of what they can do for next year and the years ahead:
- New development company and a dedicated server with their hosting plan.
- Stagger the sign up of regions based on the time zone and push it back further in the day so it isn’t in the middle of peoples work day
- Reduce the max number of beers that can be entered to 8, after several days if there are still open regions remove the cap
- Have only AHA Members sign up the first day then open it up to non-members.
There are lots of things that can be done to eliminate the initial load that kills servers and pisses of customers. What ideas do you have to make the registration better?