Comments on DEF CON 24: Make DEF CON 25 better...

[TheCotMan]

Hello,

We get some of the best ideas from our attendees on what at DEF CON was great, and what to improve and more importantly, how to improve it.

We want your feedback!
* What did you like about DEF CON? : What should return?
* What did you dislike about DEF CON? : Assume you were in control: how would you fix it?
* What was fine, but could be made better, and how?


Thanks!

-Cot

(Just like last year, and as mentioned in another thread, I will be taking the items brought up here by you and items brought up in other sources to assemble a report of things from DEF CON 24, which is then sent to DEF CON Department Heads to help with planning for next year.)

Constructive criticism is great! We want suggestions for solutions to problems. Often, the people that encounter problems are very interested in solutions spend time to come up with ideas to solve them. Share your ideas with us to help avoid issues you have encountered.)

EDIT: Things are winding down. Get your ideas and thoughts added to this thread before Aug 19, 2016 . After that, I will be building the report.

You are welcome to suggest ideas after Aug 19, 2017, but they are less likely to be reported.

Why a cut-off date? Planning for the following DEF CON begins after DEF CON ends. People have already had discussions on use of space at Caesars and some department heads tweeted in public they have been on a tour of spaces at Casars. Issues, ideas, thoughts, what worked and what did not can then be discussed in the early stages of planning.

[R33t]

Make sure that those who paid for badges actually get a real badge whole issue with BlackHat is completely unacceptable...

[Jenshadus]

Considered digital or analog signs? There is a lot of screaming going on. Example have a whiteboard outside (and inside) for signing up for the workshop sign up room on the first day to apprise people of workshop availability. Mark presentation lines with stand up signs rather than yelling directions or putting them on the floors. Never been to one of these before and it seemed chaotic. Maybe that's the whole point. Just wouldn't hire you for ant wedding planning

[_multithreading]

Since it ends up needing to happen every year anyway, why not divide the hallways into 'lanes' of traffic from the get-go? Make some signs to point people in the correct direction, have 'exits' for the different tracks, etc.

A wish-list item would be some service to check to see if a track is full and standing in line is a waste of time, though I do love me some linecon.

[TheCotMan]

Since it ends up needing to happen every year anyway, why not divide the hallways into 'lanes' of traffic from the get-go? Make some signs to point people in the correct direction, have 'exits' for the different tracks, etc.

This is an idea that I think we tried at the Riv. Though the concept was good, and should help in theory, there were some issues that we could not easily resolve:
* When traffic mostly is looking to exit, they will exit more slowly if only using half the hallway capacity
* Hackers are hackers, so when traffic was heavy in one direction, but light in another, people will walk into oncoming traffic.
* Suggestions to move the divider based on traffic patterns was suggested, but was found to be too difficult to apply quickly enough to enough dividers to be effective.

It could be a good idea, but some of the other issues became unexpected consequences. Considering the extra information, do you have ideas on how to refine your suggestion and improve it?

A wish-list item would be some service to check to see if a track is full and standing in line is a waste of time, though I do love me some linecon.

I don't know if this has been suggested before; this may be the first for this idea. At the Riv, and maybe the last year at the AP, goons would estimate a cut-off point, after which people would probably not be able to get in the room even if the room was emptied. Reporting line data for aggregation and publishing might be a next logical step.

Thanks to you and everyone that contributes to this. All of the ideas, issues, and successes discussed here will go into a report for department heads to help with planning for DC25.

I've also sought out issues, ideas, and successes in other spaces, which will be included. Unlike the usual rules in the forums about "me too" being bad form, for this thread, feel free to reply to a post with "me too" if you agree. A "me too" is like an agreement and a vote in favor of the idea. You can also use the "like" link under the post, far-right.

I will also be including comments from these threads which will eventually be consolidated to one thread:
* https://forum.defcon.org/forum/defco...394-badge-less
* https://forum.defcon.org/forum/defco...jor-badge-fail

[_multithreading]

It could be a good idea, but some of the other issues became unexpected consequences. Considering the extra information, do you have ideas on how to refine your suggestion and improve it?

I doubt I can think of much that goons haven't already tossed around, but:

* When traffic mostly is looking to exit, they will exit more slowly if only using half the hallway capacity

I can't really think of any way to do this that would not annoy people trying to get into another track, or people needing to poop, or goons that have to keep control of the flow, but I would say that you might consider people wanting to 'exit' the same way that you consider people wanting to 'enter'. If you enforce the exit of all people from major tracks back to a more open area (e.g., entrance to the conference area), you could engineer the traffic flows more efficiently. For example:

* One option could be to start exit lines *inside* of the track; this would potentially be a problem depending on the capacity of the room and whether or not there are chairs all the way back (not to mention fire code/needing to poop/hotel rules), but controlling the traffic flow inside of the room would make controlling the traffic flow outside of the room more easy, I'd assume. Like a smaller version of the winding registration line concentrated at the back/sides of the track, with a bypass door/route open only during the talk when traffic is lighter.

* Hackers are hackers, so when traffic was heavy in one direction, but light in another, people will walk into oncoming traffic.
* Suggestions to move the divider based on traffic patterns was suggested, but was found to be too difficult to apply quickly enough to enough dividers to be effective.

It sounds like more of an engineering/data-gathering problem than anything, in terms of being able to change the traffic lane widths quickly. Finding a one-size-fits all solution to all talks in all tracks would not be very easy. However, to a certain extent, I'd imagine that the attendance of a particular talk can be estimated, and lanes potentially adjusted several minutes before the end of the talk or before people start coming in. Ideally, having the red tape posts on some sort of mechanical track that can be slid remotely would be nice, but that's easy for me to say if I'm not the one tasked with building such a contraption.

[Bill_in_SD]

Test DCTV thoroughly before con. The ONLY reason I stay at the venue is for this.

When it does work, (Saturday on) it is AWESOME

And, the response I got on Twitter from the guys working the issue was quick!

So overall a B- on the DCTV - it really makes the conference a different experience when you can soak up all the info while hacking the badge at the same time.........

[Tangential]

Now that registration has been solved (discounting the paper badge fiasco), why does Thursday suck, and what can be done about it? 101 is too crowded, workshops fill up in seconds, no villages are open yet, and there is not enough space to accommodate the hackers that linger in the halls. What am I missing?

[braydawg]

I have not attended a lot of cons and would appreciate some feedback on this as I do not know what is par for the course.

On an unrelated note; l am a first time defcon attendee. I attended alone and find myself at one point very lonely and mopey. I did find one gentleman who spoke to me for an extended period of time about his affiliation with the conference on the 26th floor while he worked a party... I think the 803 party. I don't remember his name but I would like to give him a shout out if he happens to be around. He helped me feel a bit better. As a 30 year old man no one else is really responsible for my happiness but he still hung around. I wore black/red shorts and a red t-shirt if he reads these forums and happens to read this post. He wore a leather jacket and was bald with long hair. Also thanks to the 803(?), 801(?) party for providing me with free alcohol. This was one of the few times I showed up before the mob and managed to get my drink on and not sit in a line.

The first concern I have is with the presentations. Presentations I attended had difficulties with the A/V system that ate into a lion's share of the presenters time. I would like to know if there are any proposed fixes or ideas in place to address this problem or if this is just something that happens and I was unlucky [variance etc]. I believe at least two of my attended presentations had this problem leading to an after-the-fact rate of about 15%-20% of my attended talks. It may have been more...

I attended the con without much planning as I cobbled the trip together at the last minute. In one instance I was in Bronze 3 waiting for what I believed to be a talk on bluetooth locks as written on the conference schedule website. That talk never happened. When I asked a goon about the proceedings and scheduling I found that his schedule and the schedule on the internet were not synchronized. I brought this up to the information booth wherein they said they would look into the issue. And that was it.

I didn't bring my laptop. That may or may not have been a mistake. I dunno... the wireless network had a bit of a reputation for being .... unsafe. If someone would tell me otherwise I'd probably take their word for it. As far as I found there was a certificate system providing a trust level but I didn't take advantage of it. I would have liked to attend the workshops but did not really have any information about them before hand. Nor did I know that registration for the workshops began so early in the day. That, I believe, goes to my own poor planning. A quick glace at the website alerted me to the information about where to find registration and the time when it opened. Had I brought a laptop I probably would have tried a bit harder to attend some workshops. I'll try again next year.

If I learned anything its that attended defcon is probably best done with a group. I do have an amateur radio license so I'll probably find a handheld rig for the simplex frequency (anyone have any recommendations?).

Maybe a bit scattered but I think this covers my experience. I will provide any additional details that come to mind should they arise.

Bdawg.

[TheCotMan]

I'll comment on this part:

[chop]
I attended the con without much planning as I cobbled the trip together at the last minute. In one instance I was in Bronze 3 waiting for what I believed to be a talk on bluetooth locks as written on the conference schedule website. That talk never happened. When I asked a goon about the proceedings and scheduling I found that his schedule and the schedule on the internet were not synchronized. I brought this up to the information booth wherein they said they would look into the issue. And that was it.

I didn't bring my laptop. That may or may not have been a mistake. I dunno... the wireless network had a bit of a reputation for being .... unsafe. If someone would tell me otherwise I'd probably take their word for it. As far as I found there was a certificate system providing a trust level but I didn't take advantage of it. I would have liked to attend the workshops but did not really have any information about them before hand. Nor did I know that registration for the workshops began so early in the day. That, I believe, goes to my own poor planning. A quick glace at the website alerted me to the information about where to find registration and the time when it opened. Had I brought a laptop I probably would have tried a bit harder to attend some workshops. I'll try again next year.
[chop]

The issue with changes to the schedule last-minute have been a problem for decades. When DEF CON was a single track, it was easy: go on stage, announce the change, and done.
When DEF CON shifted to multi-track, other options were needed. In early years, few that attended (by percent) could afford a laptop, and those that could afford them might not want to bring them, and even with laptops, there was no 802.11* -- it was all modem, or , if you were lucky/resourceful, ethernet, and smart phones were not really a thing. Yes, DEF CON could put changes online, but few could see/access them. Hotel-provided "Business Centers" for loan/rent with Internet access were not really a thing back then. I think they added one at the AP between some of the years we were there.

"Sandwich Boards" and "A-Frame" with paper were tried, but quickly defaced, altered with bad information, or went missing. Similar with dry-erase boards. As for availability, these were not effective.

One of the better ideas that was created to help with this was the Information Booth, but not a great way to announce changes to all ~22,000 people. DCIB is really only good when people go up and use it. It becomes better when people share information with each other. ("Hey! They moved it! Let's all go to the new location.")

The rest of the options leverage an assumption of access to a smartphone. However, those that are paranoid do not want to use their normal smartphone at DEF CON. Even though some of the same risks exists at DEF CON exist elsewhere, the frequency and level of advancement at DEF CON is probably higher, and people worried about StingRay and similar devices also want to avoid use while at DEF CON. An expensive option is burner phones, which can add ~$100 or more to DEF CON expenses if a modern enough smartphone is desired which is not immediately obsolete at time of purchase. (There is "Hacker Tracker" an app you run on your phone, if you trust apps you download and install, or the less exposed option of actual text, or simple HTML as served in https://forum.defcon.org/forum/defco...books-and-html this project. Both HackerTracker and this other project require some mobile-data device and use at DEF CON , and an assumption these volunteer-based projetcs remain maintained while at con.

There is also following several key account on twitter, but that also requires a mobile device capable of getting twitter feeds and accessing them while at con.

When at the AP we started seeing lines forming outside speaker tracks, goons started (on their own) announcing which talk which line was for, though this was not codified back then. Such things help people that are "close" to the place the change has occurred, but no help to those if in a place where the thing was cancelled and/or moved to a new location. Paper signs on doors announcing last-minute changes risk the same issues of A-Frame/Sandwich boards.

We have not yet found a really good solution to this issue that is:
* Easy
* Accessible to all
* Easily accessible to all
* Low risk for defacement
* Available to people without people enabling or using tech
* Cost effective
* Time-effective (in human/goon hours)

If you (or anyone) have ideas on how we can do this, please suggest them. It is possible someone among us has an excellent solution this this problem.

Even if you can't hit all of the bullet points above, hitting a majority of them may help.

Thanks for your thoughts, observations and information!

-Cot

[Undertaker]

Ref comment by R33t "Make sure that those who paid for badges actually get a real badge whole issue with BlackHat is completely unacceptable"

You pay for access to the 4 days of DEFCON......you are NOT paying for a particular type of badge. If you got into the Con with the badge you had on, then you received a "real" badge.....just not electronic.

Just because you go to BlackHat and prepay for DEFCON only means you stand in a shorter line......nothing else.

[seeess]

Obviously there were significant AV issues. I won't claim to know the source of the problem, but in nearly all instances it was fixed with a volunteer laptop. There should be a defcon provided laptop in each track, this would've saved many presentations.

Even if there weren't AV issues, some of the 30 minute presentations did not have nearly enough time to get through their talk. In a few other 1 hour talks, they finished after 25-30 min. There probably needs to be more planning by the presenters to keep them on time. It seems like the current system of a goon holding up their hands with the amount of minutes left works, but can end up with a presenter trying to cram half their presentation into the last 5 minutes. Let me offer a better but more complex solution:

Presenters should be doing dry runs hopefully, from these dry runs they should have a rough idea of what slide they should be on after 5/10/15/20/25... minutes. From this information presenters can tell if they are falling behind earlier in their presentation with a basic red/yellow/green light.

[Gigs]

First off, compliments to the SOC and other goons. The new "kinder, gentler" cattle prods are a welcome improvement.


I said this in last years recovery thread too, but I'd really like to see a return of a random unorganized outdoor hangout area, like our old smoking areas in previous hotels. I know this is more up to the hotel layout than anything else, but it's something I really miss. The closest thing we had at Bally's and Paris was the casino area near the elevators in Bally's, but it just wasn't the same, being that it was in a traffic flow area, and there were casino floor security cameras staring at us all over.

[autobahn]

Hello everyone!

First off: Thanks to all who work so hard to put this event on. Your efforts are greatly appreciated.

I remember after DC21 and the announcement on the move, I think DT was saying that he was concerned about filling up all the new space - hah! We chewed through a hotel in 2 short years.

I'm going to start off with my PROS and CONS and elaborate a little more on how we can make the cons (and the con) better!

PROS:
-Goons this year were much more...tolerable? I don't know if this was because of deliberate action or otherwise, but if deliberate, bravo to those efforts. They do important work and I'm happy they were able to do this valuable work in a much nicer and pleasant way this year.
-Utilization of the space that DC did have - excellent effort on this one.
-Insoc! Awesome! I still would do countless terrible things to have Kraftwerk show up at Defcon, however.
-DCTV was AWESOME when it worked. Seriously, the whole concept is great and makes waking up a "little too late" for that talk you wanted to see in the AM so much easier to catch!
-ARCADE machines. Yay!
-Hardware Hacking Village placement! Holy crap still super busy but felt like you could actually have a chance of sitting down or at least being in the area without death!
-Car hacking village! Amazing stuff.
-CGC - I unfortunately missed the main event, but awesome stuff.

CONS:
-Water stations. Where was the water?
-Food cost. I know this is a tough one.
-The layout for some reason felt more "tiring" this year.
-Once again, the perennial lack of content for the "noobs".
-Hacker Jeopardy/Douchebaggery.


PRO/CON: Badges...

OK so first, BADGES. Once again, kudos to all involved, I know a ton of work is involved with the badges and I totally love the work y'all do.

Some contstructive criticism: Next time we have electronic badges, can they have a little more "out of the box" functionality? Even some more blinkenlights or otherwise. I just felt like my badge was sorta useless unless I was doing the badge challenge or attempting to hack it. That's not my priorities at con, but I'd still love to enjoy the badge. Also, the struggles for those who WERE trying to hack the badge illustrate this a little more. I guess what I mean is that I personally would love to see a little more priority on making the badges cool for all attendees and not just for those who want to hack and do the badge challenge. I get it, we're a hacker con, but...


WATER - OK, seriously. We're in Vegas, it's dry. PLEASE, PLEASE make this a priority for next year to add more water stations in more locations. I know there were a few in certain places but they should be everywhere. Hydate, hydrate, hydrate. It's good for everyone.

FOOD - I feel like this actually has been a step back from the Rio. While I think the "inherent quality" of the food inside the con areas is potentially higher, it was always served rather lukewarm and at that point where I wonder if food safety rules were being violated. Paying $$$$ for food that's not fresh is a real bummer. I get that there's a lot of restaurants around, but sometimes you just want to grab a hotdog in the chill room between things and it sucks to have to take a personal line of credit out for that and be unsatisfied. Also, what's with the shorty-ass soda bottles? Suggestions: work with the hotel to ensure food quality/hotness is up to par, and try to get them to reduce prices a little bit.

LAYOUT- The move to Caesar's will probably solve a lot of this, and maybe it's just because I'm getting older, but damn if getting between the two venues didn't get a little old. I guess people are probably working on this, so awesome!

NOOB CONTENT - I'm not a noob, but SO MANY PEOPLE ARE. I don't know if there needs to be a focus on this, or if a new group or task force or something needs to be spun up, but I talked to a lot of people who felt "lost". Now, some may say that defcon's not for noobs, and if that's the official opinion of the con, I get it. But if Defcon wants to be accessible to new attendees and provide good things for them, I'd suggest as I have previously that DC101 actually be a 101 track or partially a 101 track. It's not a 101 track right now, it's almost an extra regular speaking track. I get that there's workshops and villages but these aren't always apparent for the noobs and their schedules often aren't really easy to follow. Do you need speakers willing to do 101-level talks that may not be cutting edge? Ask! Reach out! I'm sure some people would love to help educate others. I've said this before but I want to keep repeating it because I think it's important.

DOUCHEBAGGERY - OK. So this is going to make some people groan, but it's my opinion and I'm going to throw it out there: #1: Hacker Jeopardy needs to ditch the women stripping for the daily doubles. It's just a relic of the past and reflects really, really poorly on our community in general. Frankly, it paints defcon as a place for juvenile hornballs to get their jollies while women are on stage for decoration and service. Plus, regardless of the "you don't have to go if you don't want to see it" BS, it does indeed make for a hostile environment to women. There's enough of them saying it to prove that. So please, if you're male, you don't really get to dispute that. There are plenty of strip clubs in vegas that people can visit on their own time.

I get it, people are going to disagree with me, and even some women will disagree with me. HJ can still be fun and crazy and zany without the sexist undertones, please.

In general, I think defcon needs to make a real effort towards inclusion and diversity. I can already hear the kiddies yelling "SJW SJW SJW!!!" but seriously. Defcon's behind on this. I'd love to see DT/others work on creating a diversity/inclusion department or at least including that as a core function of one of the existing departments. Defcon needs a much more solid code of conduct, that code of conduct needs to be communicated WELL to all attendees, and it needs to be enforced. Also, attendees need to feel safe and comfortable and have a place to report things that violate said code of conduct. I've heard comments from many that they simply don't feel safe reporting things to the goons some times, or that goons have actually dismissed their concerns. Some thought should go into this.

Once again, thanks to all involved for a wonderful con and thanks for listening!

[fiveatoms]

Background: This was my third DEFCON, and I'll be coming back next year. I arrived Wednesday afternoon, stayed at Bally's, and departed Sunday evening. I did not attend any blackhat or bsides events. Thank you for taking this feedback.

>>> Awesome, keep doing these things <<<
+ registration was super-efficient, the addition of the arcade games was pretty cool
+ artwork was FANTASTIC - tshirts/swag design, floor decals, badge work - all amazing
+ addition of arcade games to lounge area outside villages was cool
+ hotel experience was good
+ DC-TV once the sound issues were fixed was excellent
+ interactions with goons were mostly positive; I was (wrongly) accused of taking photos in a party where apparently that wasn't allowed. I explained that I was actually trying to avoid the people who were taking photos/video and the goon apologized and left me alone.
+ I said it in the 23 feedback and I'll say it again: It still felt like hacker summer camp. You are my people. <3 Everyone I talked to was so accepting and friendly in spite of my social awkwardness.
+ the addition of @wisp (women in security and privacy) in the vendor area was lovely to see

>>> These things need improvement; suggestions included where relevant <<<
- LINE MANAGEMENT: I suggested this last year and it would prevent the yelling/confusion problem that people were having: Assign one goon per line to stand at the end of the line with a sign on a tall pole that says "Track X - End of Line". Maybe it's a whiteboard taped to the end of a broom stick. Probably it's something more creative than that.
- DC101: Another suggestion I made last year to help n00bs out: Could we get some mileage out of the previous recordings by having a room that plays the "101" level talks from youtube? No speaker commitment, but still a level of group experience by sitting together to watch.
- volume of arcade games in the chillout room was initially way too loud (defeats the purpose of the room as a quiet chill space)
/ suggestion: keep arcade games in the lounge outside the chillout space
- DEFCON 101 track - the line was so freaking long that I got in after an hour wait because so many other people gave up; also, once I got in the room I noticed there were seats available but goons were not letting more people in
/ suggestion: ? Not sure if this was a breakdown in communication or what
- lack of water stations in the chillout room and general areas - needed to go into tracks or contest area to find water
/ suggestion: include more water stations in general areas like the chillout rooms
- hotel food criticism - Napoleon's sold me a pulled pork sandwich that was a degree or two colder than room temp; of course I was starving and hungover and didn't want to wait in line again so I slammed it down, but that doesn't mean I wasn't full of ragrats doing it
- shortage of swag, early closure of the swag room
/ suggestion: not sure, this one is hard to fix due to the way attendance is estimated
- I wasn't sure where to find which talks would be broadcast on DC-TV, are they published somewhere?
/ suggestion: publish this somewhere and/or improve communication RE where it is published
- technical issues connecting laptops to projectors: twice I heard speakers/speaker goons requesting mini display port to HDMI adapters
/ suggestion: communicate what type of hookup is going to be available so speakers can get the right hardware in advance, or provide an array of adapters so each speaker can connect quickly and easily
- pool party bouncers were confiscating people's water bottles in order to charge $4 for a tiny bottle of water once outside - this doesn't sit well with me at all; they're encouraging dehydration incidents
/ suggestion: none, it's probably hotel policy and won't change just because I whined about it, but at least I feel better having done so
- Hacker jeopardy douchebaggery and reports of harassment in general - I've addressed some of this on twitter and will post up here when I have more ducks in a row on that as far as concrete suggestions; bottom line is this: If you think that people who are upset about being harassed should leave, you are the problem and should kindly show your own self out.

Random suggestions:
. Help Wanted board at the DCIB for last minute setup help? I was bored on Wednesday night and was available to help but had no idea where to go. I wandered but didn't see anything promising.
. I'd be happy to serve/consult with the department tasked with addressing diversity and inclusion; have them contact me if interested. I don't have all the answers, but I do have a willingness to get this problem sorted out so that all people feel included.