Pre-qualifications have started!

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

I've just sent an email to all teams with instructions on how to do your pre-qualification.

IF YOU DID NOT RECEIVE THIS EMAIL, EMAIL ME: mark@halibut.com OR TWITTER: @SmittyHalibut AS SOON AS YOU CAN. (After checking your Spam Filter, of course...)

Pre-qualification Due Date is 6pm Pacific (UTC-7) July 16th (basically, when I get home after work on the first day after a complete week/weekend). If you have any questions, or you cannot make it by this due-date, talk to me early and we'll work something out.

As much as I love the forums, DMs and posts are not the most timely mechanism to get a hold of me (as several of you may have figured out by now... ;-) Email and Twitter are probably your best bets.

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

Re: Pre-qualifications have started!

No cursing our names here? I know we've had at least one team to the pre-qual, but I haven't looked at the stats recently. Anyone else having any luck? Or problems?

I'll be "around" this weekend and checking email pretty often, if its your intent to do it this weekend.

Good luck, everyone. :-)

[Gigs Member Find all posts View Profile Close Gigs]

Re: Pre-qualifications have started!

Damn you Smitty! :)

It's hard to complain too much in public without giving parts of the problem away. Don't worry, I'm sure your name is being taken in vain privately.

[Fletch F Fletch Member Find all posts View Profile Close Fletch F Fletch]

Re: Pre-qualifications have started!

Damn you Smitty! :)

It's hard to complain too much in public without giving parts of the problem away. Don't worry, I'm sure your name is being taken in vain privately.

Ow. My ass. As soon as I saw the problem specification, I started singing "Moon River...."

Team Frink Rules! is glad that we don't actually have to qualify. I think our submission is close, but we didn't get a "success". I submitted another solution just after the wire. I'd be curious to know how it did.

In summary, the problem was difficult, and other complaints. I won't say more yet. Other than tangentially note, on an unrelated issue, one of the two people who ever scored a perfect score on the MAT said it was the "least culturally-fair test I've ever taken." :)

I'm curious to see what will happen if we don't get anyone to qualify in the time limit. I have suggestions about crafting subtests that test parts of the spec, but it's not my position to say. It was hard enough that if yo, Frink can't solve it, then no other languages have any hope. I don't want to say more, either, but I was driven to drink after the programming. Programming sober for C&C just seemed so wrong. I didn't have hot smart women distracting me, either. We're so going to lose if we don't step up our training.

I think we should grant badges based on which teams actually paid for their beer last year. *Cough.*

[pH_Boston Member Find all posts View Profile Close pH_Boston]

Re: Pre-qualifications have started!

Definitely interested in taking a look at the test input(s) after the challenge is over. Wasn't able to get a correct submission in time and haven't had the time yet to take a second look at the problem - but definitely want to track down where I went wrong

[Gigs Member Find all posts View Profile Close Gigs]

Re: Pre-qualifications have started!

one of the two people who ever scored a perfect score on the MAT said it was the "least culturally-fair test I've ever taken."

Hah, yeah. My partner is from the other side of the world and I had to spend a lot of time explaining things to him.

[Gunn4r Member Find all posts View Profile Close Gunn4r]

Re: Pre-qualifications have started!

Damn, we just got "finished" Kobayashi Maru anyone?

I thought about pulling a captain Kirk, but figured that would be considered being a "dick"

[Gunn4r Member Find all posts View Profile Close Gunn4r]

Re: Pre-qualifications have started!

Damn, we just got "finished" Kobayashi Maru anyone?

I thought about pulling a captain Kirk, but figured that would be considered being a "dick".

My favorite part was your example input and output. Very representative of the problem set ;)

Anyway, We're at your mercy Smitty (and namniart)

[pH_Boston Member Find all posts View Profile Close pH_Boston]

Re: Pre-qualifications have started!

I thought about pulling a captain Kirk, but figured that would be considered being a "dick".

My favorite part was your example input and output. Very representative of the problem set ;)

Yes it was super fun writing up tons and tons (and tons) of different test cases. Will discuss more after it's all over. Also, I had an idea that was also a "dick" move but decided not to try - would rather play than risk being DQ'd

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

Re: Pre-qualifications have started!

That's it, the pre-qualification round has ended. However, the results were not clear-cut and we've got some qualitative grading to do. We're working on precisely how that's going to happen tonight and hope to have a result for y'all in a day or two.

For the time being, please don't talk specifics about the problem; I hope to lift this ban by tomorrow (Tuesday) night, Wednesday at the latest.

That's all I can post for now. More in a couple days. Sorry.

[Gigs Member Find all posts View Profile Close Gigs]

Re: Pre-qualifications have started!

That's it, the pre-qualification round has ended. However, the results were not clear-cut and we've got some qualitative grading to do.

Mmmm, I hope I get some kind of "veteran who overpaid for beer at the Riv" bonus to my score.

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

Re: Pre-qualifications have started!

Mmmm, I hope I get some kind of "veteran who overpaid for beer at the Riv" bonus to my score.

You will always have that badge of dubious honor to carry. Or maybe bear is a better verb... ;-)

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

problems with the pre-qualification grader

Ok, so let's start here: We fucked up pre-qual in a few ways. The quick summary of the mea culpa list is:

- The pre-qual problem ended up being more complex than we intended. It's not that it was HARD, just that there were lots of little fiddly bits and edge cases that had to be handled. We intended to grade based on how long it took you to solve the problem. We ended up accepting all 3 (of 11) teams who succeeded, then picked the 4 teams who had the most number of individual test cases passed.

- The grader did not give proper feedback. We improved this a little bit early on, but have learned our lesson and will be improving the feedback from the automated grader before DefCon. Specifically, we will be splitting test cases into their own files and reporting how many files succeeded and how many failed, so you at least know whether you're going in the right direction or not.

All the other problems I can think of really stem back to those two root causes. If you've got any other complaints about how pre-registration went, especially if you have a suggestion for how to improve it for the actual event, I'd love to hear it.

Having said all that, I don't think these problems prevented us from selecting the correct teams for the event. We were still able to rank the teams and pick the best ones, and some teams were still able to solve the problem, even with the grader problems. Obviously, if you disagree, I want to hear it.

The observant reader will notice at this point that we have 9 teams, not 8 as stated as a limit. There are a couple reasons for this. First, our "8 team limit" assumed 2 people per team, for 16 players. Two of the teams that were accepted are Ronins, programming solo, so that left space for one more team. Second, there was a clear line between the teams we accepted and those we didn't. It felt weird to arbitrarily cut off one team who was SO CLOSE when we had the space.

No, I won't talk about individual teams' scores. Sorry.

Anyway. Let the flaming begin. :-)

[SmittyHalibut That guy, ya know? Find all posts View Profile Close SmittyHalibut]

Re: Pre-qualifications have started!

oh, a third mea culpa I forgot to post earlier:

- This problem was inherently biased toward American audiences. Baseball is an AMERICAN sport (like Apple Pie, right? Ignoring the fact that I'm pretty sure the Germans invented that, but anyway...) Several of our contestants were not American and spent an unfortunate amount of time just trying to understand the rules. This was NOT an intentional decision, it was just a (rather large) oversight. We didn't think about it. Let's face it, Americans have the Ugly American image for a reason, and this is it: sometimes, we don't realize there's a whole world out there, and they don't all play baseball. I try not to let it happen to me, but it totally just did.

Again, this one is a lesson learned.

A potential 4th problem, but it could be argued that it roots in the Too Complex and Too American roots, is that the problem wasn't exceptionally well defined. It made a lot of assumptions that you understood the rules of the game. In our defense, we did link to the official rules which should have been able to answer those questions for you (to address the Too American issue), but that just wasted a lot of time (confirming the Too Complex issue.)

So, yeah. It was a bit of a cluster fuck in a few ways. We will learn from these ways and improve the system, both for the event this year, and for future years.

[Fletch F Fletch Member Find all posts View Profile Close Fletch F Fletch]

Re: problems with the pre-qualification grader

We ended up accepting all 3 (of 11) teams who succeeded, then picked the 4 teams who had the most number of individual test cases passed.

Yikes. Three teams actually passed that? We're doomed. I mean DIE REBEL SCUM.

Team Frink Rules! is heretofore taking an alternate tactic: learning to drink as much as possible.

Any word on changing drinking rules?