Kenshoto video torrent or tracklisting?

[sammo]

Re: Kenshoto video torrent or tracklisting?

I should have considered a different title for that thread in Drunken Banter.

The DC14 discussions forums will remain so long as they are being used. Once it is noticed that traffic in the DC14 space is nearly stopped, I'll probably do like last year, and announce a closure as "coming soon" to give people a few weeks. (Last time, I gave it a few weeks, and then extended it and extended it again because people kept using it. :-)

Let's post the .torrent... then you can close the forums ;)

[theprez98]

Re: Kenshoto video torrent or tracklisting?

still being worked on. too many folk with too many serious jobs is harshin' our buzz on this. solutions in the works, i assure you... both primary and alternate distribution models.

You let your real job interfere with the forums?

[TheCotMan]

Re: Kenshoto video torrent or tracklisting?

There may had been one other reason for this thread and others dieing off as I believe that for the past week or two the "last post" by TheCotMan indicating the drunken banter subforum was closed kind of indicated the DC14/Post DC14 forum was closed as that was what was showing up in the main listing since it was the last post for quite some time. Though I am certain the average forum user knew otherwise it may have prevented some people from posting. There was even a few hours (I thought but after checking it was under 30 minutes) that a new post made it up only to get moved to the contest/event lock picking thread which further made it seem that this section may be closed.

I should have considered a different title for that thread in Drunken Banter.

The DC14 discussions forums will remain so long as they are being used. Once it is noticed that traffic in the DC14 space is nearly stopped, I'll probably do like last year, and announce a closure as "coming soon" to give people a few weeks. (Last time, I gave it a few weeks, and then extended it and extended it again because people kept using it. :-)

[signcarver]

Re: Kenshoto video torrent or tracklisting?

Wow this thread died off fast..did I miss something?

Though I expected the answer to be what Deviant Ollam gave I was wondering about it myself but didn't want to bother the people putting it together as I know they have work to do.

As for the torrent, I have an extra pc here detached from my usual network that can help seed but for a 30GB file I think I would have to limit the upstream to probably around 30-50K initially and about half that after a week though it would be fully connected for weeks if necessary. Even at that rate my ISP might have a fit.

There may had been one other reason for this thread and others dieing off as I believe that for the past week or two the "last post" by TheCotMan indicating the drunken banter subforum was closed kind of indicated the DC14/Post DC14 forum was closed as that was what was showing up in the main listing since it was the last post for quite some time. Though I am certain the average forum user knew otherwise it may have prevented some people from posting. There was even a few hours (I thought but after checking it was under 30 minutes) that a new post made it up only to get moved to the contest/event lock picking thread which further made it seem that this section may be closed.

[Deviant Ollam]

Re: Kenshoto video torrent or tracklisting?

still being worked on. too many folk with too many serious jobs is harshin' our buzz on this. solutions in the works, i assure you... both primary and alternate distribution models.

[litsuka]

Re: Kenshoto video torrent or tracklisting?

Wow this thread died off fast..did I miss something?

[sammo]

Re: Kenshoto video torrent or tracklisting?

I think I said earlier in this thread that I was going to be really annoying and post all the time, asking where the torrent is.

[hevnsnt]

Re: Kenshoto video torrent or tracklisting?

so can we have the .torrent yet?

[Z^2]

Re: Kenshoto video torrent or tracklisting?

I would love to help with this but my provider blocks everything incoming. I have run network scans against my IP from external sources but nothing gets through to the IP address. Unfortunately there are no other high speed bandwidht providers serving my area that are "affordable."

I have two web sites hosted externally but neither of them come with enough disk space nor, in treading the terms and conditions, do they permit peer to peer.

Yeah, I'm in the same boat. Also, I've just discovered the joy of bittorrents in the past three weeks (am waay behind). Where I live, the people don't want me to get anywhere near the router/do anything with the connection, but I am trying to see if I can get a stable connection somewhere on campus (umich) from my office (scouting for one starts tomorrow).

[DaKahuna]

Re: Kenshoto video torrent or tracklisting?

I would love to help with this but my provider blocks everything incoming. I have run network scans against my IP from external sources but nothing gets through to the IP address. Unfortunately there are no other high speed bandwidht providers serving my area that are "affordable."

I have two web sites hosted externally but neither of them come with enough disk space nor, in treading the terms and conditions, do they permit peer to peer.

[Voltage Spike]

Re: Kenshoto video torrent or tracklisting?

Holy hell. I knew it was familiar, but a quick forum search didn't reveal it. Google, however, found the old thread. You can probably disregard the following as redundant until I re-read what we had to say on the subject last year.

Okay. Done reading. I can't believe how ignorant I was last year.


Regarding Bit Torrent and UDP

Bit Torrent uses the UDP port for torrentless trackers. It never uses UDP for file distribution.


Regarding TCP hole punching

UDP hole punching exists. It works, and it is fairly widely supported. "Automatic" TCP hole punching does not exist in any equipment I've ever seen. I think that maybe it makes sense for peer-to-peer applications (not just Bit Torrent) even if we don't traditionally require it for most TCP applications (which are server-based).


Regarding Pre-Seeding Versus Super-Seeding

If you think of it in terms of total copies, pre-seeding requires the distribution of n copies (where n is the number of initial seeders) before a copy is available "in the network". Super-seeding requires the distribution of little over 1 copy before a copy is available in the network. Regular seeding will require, on average, a distribution of 1.5 copies (although assymetric leechers can seriously impact this number.


Regarding Pre-Seeding Using Super-Seeding To Connected Peers

For best results, you must be able to organize and communicate with all of the connectable peers ahead of time. Since this isn't really possible (you can only reach a fraction of the ultimately connectable), the idea is good but I believe can be improved.

One benefit, however, is that you can almost completely eliminate leechers off of the initial seeder.


Regarding Super-Seeding Versus Unconnected Seeding

(Note that the two concepts are orthogonal.)

Connected super-seeding introduces a bunch of middle men (unconnected peers) into the equation and should seriously slow down the overall speed in the initial period (especially with the skewed numbers TheCotMan provided).

Unconnected super-seeding may be the way to go. Only those best able to share will get the first pieces and connectable leechers are limited by their upload speed (because they have to share to proceed). In other words, they are contributing to the overall speed and prevented from leeching. Since the best seeders are not in seeding mode[*], assymetric connections are also pushed out. Or am I missing something?

I also realize that being unconnected can be done by the super-seeder in the application for more control and may, for all I know, already be implemented by some clients.
[*] Remember that true seeders don't discourage assymetric connections since they attempt to distribute copies as wide as possible by giving priority to those best able to download.

If all users were connectable, then issues of "wasted" loads from the primary source can be ignored, as the waste can be marginalized. In such a case, users that only download what they need and leave, only impact the longevity of the torrent, and th speed at which peers can download content.

In case people don't see it, the waste is marginalized through super-seeding.

How can a protocol be buit in such a way that there is no more overhead, and the source of data does not need to, "trust the client."

I don't see one. If we sacrifice the "no overhead" requirement, then there are ways to have a challange/response between clients that allow for the request of chunks they don't have and don't know about unless they have other chuncks, can prove it, and show that they are connectable. What is even better, is the challange response, can use crypto, with multipart keys, based on the "randomness" of the content being shared. :-)

I like my solution of an unconnected super-seeder better. Peers show they are connectable by virtue of the fact that they are connected to the seeder. At this point, peers don't have to prove they have pieces by virtue of the fact that Bit Torrent's natural tit-for-tat algorithms mostly elminate leeching. (I say mostly because unconnected clients without data will still receive a trickle of pieces due to optimistic unchoking.)



PS: Off-topic question (don't I love those?): is it possible that a forum search doesn't reveal the original thread because it was a moved thread? I post the question publically because vBulletin isn't exactly uncommon.

[TheCotMan]

Re: Kenshoto video torrent or tracklisting?

Holy hell. I knew it was familiar, but a quick forum search didn't reveal it. Google, however, found the old thread. You can probably disregard the following as redundant until I re-read what we had to say on the subject last year.

Semi-technical discussions with you Bascule and others is so fun, we get to have delayed-instant replays! ;-)

It is an interesting discussion and topic, so I guess it makes sense that it might have repeat appearances.

If all users were connectable, then issues of "wasted" loads from the primary source can be ignored, as the waste can be marginalized. In such a case, users that only download what they need and leave, only impact the longevity of the torrent, and th speed at which peers can download content.

Given the above paragraph, pre-seeding would make no sense, reguarless of a lack of sharing among users. In this case, these are actually separate issues.

However, once "non-connectable" users are added to the picture, then the addition of these "self-serving greedy users" not only impacts longevity of the torrent, but also impact throughput through "waste" of any content sent to them (especially when viewed from the position of other "non-connectable" users and the content they take from connectable users.) The weak link in the chain becomes the number of "connectable" clients.

How can a protocol be buit in such a way that there is no more overhead, and the source of data does not need to, "trust the client."

I don't see one. If we sacrifice the "no overhead" requirement, then there are ways to have a challange/response between clients that allow for the request of chunks they don't have and don't know about unless they have other chuncks, can prove it, and show that they are connectable. What is even better, is the challange response, can use crypto, with multipart keys, based on the "randomness" of the content being shared. :-)

(In this case, the lack of entropy in the keygen, and risk for attacks based on predictability are not an issue, because access to the information is much cheaper than "intelligent" keyspace searches-- users only need to be connectable and have some data to get priority. :-)

This can be defeated, but from my view, the cost of defeating it (CPU time) is greater than the cost of just playing fair and downloading it. (Even if it was scripted by a single smart person and shared.) If this model works, then the next weakness becomes the implementation.

[Voltage Spike]

Re: Kenshoto video torrent or tracklisting?

Didn't we discuss this before? Deja vu! :-)

Holy hell. I knew it was familiar, but a quick forum search didn't reveal it. Google, however, found the old thread. You can probably disregard the following as redundant until I re-read what we had to say on the subject last year.

once we post it, the people seeding will be slammed. Unless you have a lot of upstream bandwidth, it will take forever for even a few seeds to be established.

I dont see how they will be "slammed". They will have a lot of requests and the data will be distributed wider in the same period of time ... limiting the impact of users dropping out, right?

If the few of us in possession of the DVD are ready to go, everyone else will benefit.

Oh, well if you are doing this through DVD then it makes perfect sense. I was responding more to TheCotMan's comment about pre-distribution using Bit Torrent.

Last year, it seemed about 4/5ths of the people participatings did not have "open" ports for people to connect to them-- most had incoming SYN traffic filtered away.

Hmm, that is an interesting point that I hadn't considered. During the early phases, it can be really unhealthy to have such 1-sided connectedness.

Perhaps if the seeders were not connected ... encourage connectedness in the early phases, ensure priority to those most able to share. I might have to think on that a bit more, but I really like that idea.

Note, super-seeding works kind of like this: upload a piece to a peer and choke the peer until someone else advertises that piece. It doesn't quite encourage connectedness of that peer, though, as they might merely become a middle-man between the seeder and another connected user.

Well over half (3/5?) of the users participating only stayed on until they had what they wanted, and then they (probably) switched off the video torrent, and switched over to torrents they did not complete.

Assymetric connections can be a bitch ... especially when you consider that seeders give priority to those who can download the fastest (i.e., sharing doesn't enter into it). Is it really going to be slower, though? The leechers are going to arrive anyway.

I see two situations. The first is that we wait until we have multiple seeders and then download at a faster speed. The second is that we download at a slower speed but we don't have to wait for the distribution period. Again, I'm not quite sure which is better.

[TheCotMan]

Re: Kenshoto video torrent or tracklisting?

Perhaps someone could explain this to me again: what is the point of creating a batch of early seeders?

Didn't we discuss this before? Deja vu! :-)

More...

Last year, it seemed about 4/5ths of the people participatings did not have "open" ports for people to connect to them-- most had incoming SYN traffic filtered away.

Well over half (3/5?) of the users participating only stayed on until they had what they wanted, and then they (probably) switched off the video torrent, and switched over to torrents they did not complete.

A lack of a pre-seeding, would have meant most people would still have been downloading their content from the primary source.

Pre-seeding to 4 or 5 sources which were (eventually) connectable allowed for more people to download torrent content more quickly-- and even then, there were people complaining that the torrent was too slow. (Why was it too slow? See above about number of users not allowing people to connect to them because of filtered SYN packets.)

This is a security-related conference, so again, we will have a much larger group of people who are not connectable, when compared to the average population.

If the initial seeders have "fat pipes" are "reliable to remain online and seed for the long haul" and "are openly connectable by anyone" then we don't "waste" the original source's total throughput on people who:
* Won't stick around to share after they have what they want
* Aren't connectable

It may be useful if the original source doesn't want to seed (i.e., upload exactly one copy and be done with it), and I understand that. If this were the case, then the source should use the frowned-upon method of super-seeding and ensure that the early connecters are reliable (in that they promise to stay online until all of their pieces have been redistributed ... an Availability of at least 2 may be a good indicator).

If the original source does want to seed, then shouldn't we open the whole thing to the public? He can limit his upstream to whatever he chooses and, similarly, choose to disconnect once the Availability is decent.

PS: In case someone gets the wrong idea, I am not volunteering to be a reliable "first seeder" ... I use a laptop as my primary computer. Plus, I've only got about 50 KBps upstream available to me.

I'm not offering myself for this either. My fast pipe systems have insufficient free space on their drives, and my slow connection with lots of space is way too slow.

[jur1st]

Re: Kenshoto video torrent or tracklisting?

VS: once we post it, the people seeding will be slammed. Unless you have a lot of upstream bandwidth, it will take forever for even a few seeds to be established. If the few of us in possession of the DVD are ready to go, everyone else will benefit.