Past DEF CON Conference Content

[Dark Tangent]

OK, I fixed two the of RSS problems.. waiting to get the Linton Wells audio working again. That should be the last problem. I'll be making the big announcements to mailing lists this Wed night I think.

(And there was much rejoicing)
Huzzah!

Good deal with the torrents. I'll probably join that later next week to help serve.

On the new Forums box front, the base system is almost installed and configured, and then it will be on to the applications..

[Dark Tangent]

Need help testing some Black Hat rss entries.

Here are the links to the Black Hat USA '05 talks. I'm going to announce them real soon, and want a quick once over before they go out.

http://www.blackhat.com/podcast/bh-usa-05-video.xml
http://www.blackhat.com/podcast/bh-usa-05-audio.xml

Thanks!

[Dark Tangent]

Dan Kaminsky and some Random CDATA just got cleaned up..

[converge]

sweet! starting to pull the videos, then followup with the audio. it seems like very few folks are opting for the audio when they can obtain the full thing.. but that may change when more broadly announced.. we'll see.

heh.. it seems like each advancing xml file has modified comments that require me to bone up the regex I'm using to pull the feeds... you devil you.

[Dark Tangent]

Nightmare on my street

Wow, what a nightmare of fixes.

Little did I know I had a grip of problems with the feeds starting with wrong charset and ending with mal formed tags.

I've fixed up the audio feed, and I'm on to fixing the video feed.

--

OK, fixored them up, as well as fixing some problems with http://www.defcon.org/defconrss.xml

[converge]

finally done pulling video, starting audio and will reprocess video rss to make sure I got all the fixes you've made since I started. with my current log there are two hits for "\<404\ " .. but these may already have been addressed while I was waiting for the download...

--edit: I won't have specific RSS issues until I parse through the actual entries invidually... but:

Video:

404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp4
404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp4

[Dark Tangent]

I'll check them out..

finally done pulling video, starting audio and will reprocess video rss to make sure I got all the fixes you've made since I started. with my current log there are two hits for "\<404\ " .. but these may already have been addressed while I was waiting for the download...

--edit: I won't have specific RSS issues until I parse through the actual entries invidually... but:

Video:

404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp4
404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp4

[converge]

For audio:

404 . 2005_BlackHat_Vegas-V16-Sparks_and_Butler-Shadow_Walker.mp3
404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp3
404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp3
404 . 2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3

[Dark Tangent]

OK:

V16 fixed.
V39 fixed.
V7 Looks ok to me..
V0 Looks ok too..

I'll check the error logs on the server and find out whats up with these last two..

Oh, and I fixed the Todd Moore .mp4 it was named incorrectly on the server.

For audio:

404 . 2005_BlackHat_Vegas-V16-Sparks_and_Butler-Shadow_Walker.mp3
404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp3
404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp3
404 . 2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3

[converge]

After a little deeper inspection, here's the current tally per fresh feed pulls this morning:

--
Defcon RSS:

404 . 2004-Defcon_12-Todd_Moore-Cracking_Net2Phone.mp4
--
Video:

404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp4
404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp4
--
Audio:

404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp3
*404 . 2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3

* (2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3) seems to have made it into my collection, so maybe it was previously working? .. but at the moment request returns 404.
comparing the two URLs suggests that the RSS link refers to '/Vegas2005/' instead of current location of '/bh-usa-05/'. This solution does not carry over to V7 (Barrall) though.
--

-Also of note, Barnaby Jack presentation truncates credentials at the end of the RSS description for both audio and video feeds

This does not yet include content verification (ie, issues like the Robert Morris video, someone else's talk named as another video file, etc). That's a whole other queue of enjoyment and popcorn frenzy.

[Dark Tangent]

After a little deeper inspection, here's the current tally per fresh feed pulls this morning:

--
Defcon RSS:

404 . 2004-Defcon_12-Todd_Moore-Cracking_Net2Phone.mp4
--
Video:

I'm showing success on that one.

404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp4

Not sure where it went.. will have to re-encode.

404 . 2005_BlackHat_Vegas-V39-Mandia-Effective_Incident_Response.mp4
--
Audio:

404 . 2005_Blackhat_Vegas-V7-Barrall-Dewey-Plug_and_Root.mp3

The elusive V7..
I just copied and pasted the file name from the web log over itsself.. if that doesn't fix it, I don't know what will!

*404 . 2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3

OK, fixed that.. will upload the new RSS soon.

* (2005_BlackHat_Vegas-V0-J_Moss_G_Louie-Welcome_Keynote.mp3) seems to have made it into my collection, so maybe it was previously working? .. but at the moment request returns 404.
comparing the two URLs suggests that the RSS link refers to '/Vegas2005/' instead of current location of '/bh-usa-05/'. This solution does not carry over to V7 (Barrall) though.
--

-Also of note, Barnaby Jack presentation truncates credentials at the end of the RSS description for both audio and video feeds

This does not yet include content verification (ie, issues like the Robert Morris video, someone else's talk named as another video file, etc). That's a whole other queue of enjoyment and popcorn frenzy.

[converge]

I'm showing success on that one.

odd... well, I test good now too. shrug

The elusive V7..

the filenames appear verbatim.. maybe it was something in the path? at any rate, audio for V7 appears to be fine now.

[Deviant Ollam]

converge... are you simply testing all the feeds and links or are you actually also going through the process of downloading everything? you might be a candidate for helping us with the Traveling TeraByte project, since i'd love to have all the defcon content on there. i suppose we could also send the drives to nikita or someone similar in the DC inner circle where the content resides formally. either way, i'm looking forward to kicking things off soon. woot!

[converge]

converge... are you simply testing all the feeds and links or are you actually also going through

I'm downloading it all.

1) adding to my data server for internal pleasures
2) adding to my DVD archive for the future survival of knowledge after the fires die and radiation levels become tolerable ... or until a better optical storage comes along at workable price
3) torrenting to help distribution

I originally started by brute forcing the feeds manually one by one, performing all testing steps. This meant taking longer to actually have the files on hand. Then TheCotMan shared his brilliant idea of scripting the pull.. so I took it a step further.

The current testing process goes a little something like this.. 1 script tests a directory to see if content exists, if it doesn't then initial pull is assumed and all available files are downloaded from the feed. If previous files do exist, then the pulled RSS is diff'd to the existing list and previously non-existent files are pulled. The error logs generated from this compile my first line of issues, since a simple grep for 404's will start the ball rolling. This process is 99% automated.. so I run it and check back some other time of day.

Then when I get a 30-60 minute interval, I tear through the RSS feed manually to further test. For each entry, I copy/paste the speaker info into an open vi session for the torrent nfo (scanning for description truncation/glitches/etc), I click the link to make sure it starts download for the file, verify it starts downloading a file named similarly to the presenter, rename the cached file to conform the filenaming a little more, cycle. For missing video files, I create an empty shell named appropriately but similarly as a placeholder until resolved. Once done I close the nfo and copy it into the dir. Then I move on to the counterpart [audio|video] feed, but instead of manually renaming the filenames I simply copy/paste the previously worked names, keeping the appropriate extension. Once done, I proceed to annoy DT with another round of RSS gripes.

When enough of the issues are resolved, I create the torrent, begin seeding it, transfer the torrent to my webserver and post for others to hop on board.

[Deviant Ollam]

what does the overall size look to be for all content in total?