Did the AHA Drop the Ball with the National Homebrewers Competition?
Tuesday had us homebrewers all geared up to register our beers for this year’s NHC, National Homebrewers Competition. There are only a limited number of spots for each region of the country and many of us wanted to get our registrations in early so that we could get our beers in to our local region of choice and avoid getting shut out, like many did for the National Hombrewers Convention. With thousands of homebrewers slamming the servers at 1pm on Tuesday, the AHA’s new registration site just couldn’t take the load, shutting most homebrewers out of the site for over an hour.
I managed to register 3 beers, so I’m in or am I? I really don’t know, nobody knows at this point what is going on. So a quick recap of what I experienced; first I couldn’t even get into the site. After an hour or so of trying I managed to get in. When the home screen finally loaded, I saw 244 of the 750 entries for the NYC region were already taken, fine by me; I should be able to get in. Once I registered myself and selected my region, I was greeted with a note that all 750 spots for my region were filled. I was stunned! How could 500 entries have been submitted in less than 5 minutes? I assumed at this point it was a mistake and the site was counting the total entries for all the regions and it was a bug. The “Add New Entry” link worked, I filled out the form and successfully submitted my entry. Great! Well, I thought so until I saw my entry number was 861, that was 111 over the max for my region, but the system did allowed my entry.
AHA’s initial Facebook Response to the problem
I was able to add 2 more entries and pay for them without issue and received my receipts for the 3 entries. I figured at this point, no matter what happens I have paid entries, I’ll be in.
After logging out and going back to the home screen, I saw there were only 550 entries of the 750 for the region. Good, I guessed at this point, but who knew what numbers were real or representative of the actual submissions on the site.
By this time I was seeing a whirlwind of tweets and emails about what a clusterfuck this registration was becoming, many people were pissed and annoyed. I wanted to check on my entries to make sure they were still there, so I went back to the site and there were 900+ entries of 750 entries for the NYC region. Luckily for me all my entries still accounted and paid for. But here is where the programmer came out in me; I decided to add a new entry to see what would happen. I created a bogus IPA entry submitted it and the system accepted it! This is where I got pissed.
What developer, development company, or client would allow such simple quality assurance test cases to fail! If I allowed this at my day job I would be fired. This is not a complicated registration system, technically speaking. I’ve been writing these types of systems for the better part of 20 years, this is a below sub par job. Simple quality assurance test cases were obviously ignored.
So did the AHA Drop the Ball with NHC? I believe they did and this is why. They hired a development company that could not deliver on the product and went ahead with NHC registration anyway. I hold no ill will toward the AHA, they do a lot of good things for homebrewers, but it seems they did not learn anything from the debacle last month with the Conference registration. While the AHA did drop the ball and it is their face with mud on it now, I lay the blame almost wholly on the development company, which appears to be Better Beer Scores LLC, the company listed on the footer of BeerCompetition.com web site. Why blame the development company, because they are supposed to be the experts. I’m sure they promised the moon at a bargain basement price for the AHA.
Here is what the development company failed to do,
- Either inform or convince the client they need a dedicated server or hosting company that can guarantee the load of 10000 concurrent users.
- QA, quality assurance. This means you test the application to ensure the system blocks users at the limit, numbers add up correctly, and the system functions as promised.
- Eliminate the logic errors, which blocked some users from registering new entries and other entering when they shouldn’t have. Once again leads back to QA!
The AHA has a monumental task ahead of them, how do they fix this years entries? Furthermore, what do they do next year and going forward?
Here are some my thoughts of what they can do for next year and the years ahead:
- New development company and a dedicated server with their hosting plan.
- Stagger the sign up of regions based on the time zone and push it back further in the day so it isn’t in the middle of peoples work day
- Reduce the max number of beers that can be entered to 8, after several days if there are still open regions remove the cap
- Have only AHA Members sign up the first day then open it up to non-members.
There are lots of things that can be done to eliminate the initial load that kills servers and pisses of customers. What ideas do you have to make the registration better?
I got one entry in early, sub # 30 and that after much refresh hammer to the server. After that it was downhill for an hour, until they fixed the MySQL connection limit in the config. After the fix, entries were flying in because so many people had mashed up waiting to try and enter their beers. I did get all three of mine in under the 750 mark, and paid later that night after they re-opened the payment system. Still I agree they really messed up with how it worked. Personally I think a lottery system would be much better, you input how many beers you would like to enter, and then the system allocated available slots (days or weeks later) to people as random as possible, one beer at a time. If they have slots left, then you might get to enter more. If you can’t put your best beer forward, why are you entering? People don’t need to enter 15 beers when there’s only 8,250 slots.
lottery is an interesting idea. No matter what is done, it is still going to piss off people, not much you can do about that, it is a popular event. This year it just seem like a perfect storm of errors that I hope open some people eyes at the AHA.
I think the biggest problem is just trying to pull off this large of a competition. 30K members and only 8250 entries, that math does not make sense. And to top it off, you are locked into a single drop off location with a max of 750 spots. Even if only 25% of the members wanted to enter a couple beers that’s 15,000 entries.
I really think they need to reevaluate the competition. Maybe limit it to just a couple of entries, maybe a lottery or even regional qualification round.
The only blame I put on the AHA is not paying enough attention. Any homebrewer on twitter or facebook knows how big this hobby has gotten. How many beer blogs, podcasts, clubs and new breweries have popped up in the last 4 years? It’s crazy (good) right now.
If the AHA does want to change the format, then they should at least think about a few things.
1. Only open to AHA members.
2. Do a “first wave” entry where you can only enter one beer. Then wait few days to make sure payments are processed then re-open registration.
3. Limit entries to a reasonable amount, say 4.
4. Raise the entry fee after your first entry, this would make brewers think twice about how good their beer really is.
HI Robert, Great point about the AHA not being aware of the scope of their membership. This “hobby” is huge now, when I started… ehhem 16 years ago it was hard to find ingredients, especially quality ingredients. Now you can get a homebrewing kit at Bed Bath and Beyond, internet is a huge wealth of knowledge and places to buy the craziest ingredients.
Another problem the AHA is going to have is judges, there are not enough bjcp judges around to fill the need for the number of entries per region. more regions? You have Philly, NY, NJ, and New England going to 1 site, that is crazy. Maybe increase the number of regions and decrease the number of entries per region, that may help spread out the judging, If I were a judge, i doubt I would travel 6hours to judge in these rounds…
There are lots of questions and tons of ideas out there to make this more efficient.
Peter, you and I are “Old School” (16+ years), but last year was the first time I ever entered the Nationals. I totally agree about the bjcp judges and locations. San Diego was my choice, look at the map it’s also crazy how big of an area that covers for entries.
It’s pretty damn frustrating, but I knew it would be crazy. Even in perfect conditions the competition would have sold out in quickly. For those great brewers who couldn’t be in front of their computer at the beginning would have been out of luck regardless.
I love what the AHA has done for the hobby and I have no ill feelings towards the organization, but if they want the competition to happen they need to get creative….FAST.
They suck…. I entered a few years ago, and had no issues 2011.
The AHA has grown a lot and they have not scaled to meet the demand. I called them at 3:30 Eastern and got a “dude” on the phone…. his take on the issue reminded me of CRUSH from Finding Nemo.
I agree the website company should take the blame but I also think the AHA should step up. While ‘Home Brewing’ is a hobby for most, the National Organization should be a bit more professional. I get the feeling it is a bunch of college kids at a kegger.
I also think the design of the regional site was horrible too!!!
I design websites for a living and I had a hard time figuring out what to do first?
Do I Contact?
Do I register?
The AHA web page was vague as what to click on to register…
The email they sent all members the morning of the registration was unclear as well….. I was looking for a link to register… turns out you had to click on the JUDGING CENTER link to register…. I found the entire process confusing and convoluted.
My suggestion for scaling the competition is like Peter said, add more regions… and Robert by adding a preliminary round.
Lets face it they charge $33 a year membership… plus $12 a beer….. they have plenty of money to scale the event to the size it needs to be. They just need to step up to the plate!
there seems to be a lot of pressure coming from many different homebrew groups to pick up the pieces and have AHA get their act together.