Nextnet wireless Internet

[Jizzle]

My ISP here in this non-first world country uses a cellular radio system sold by nextnetwireless/. They have a number of towers around town and the customer's radio automatically attaches to the strongest tower (or channel). I have recently had the opportunity to attach to various towers in the city and discovered that, even with a perfect radio signal, all but one of the towers drop packets pretty severely.

The "technical director" tests connectivity to his customers as near as I can tell with a default windows ping of 32 bytes and 4 echo requests. If he gets all 4 back he considers the connection to be "perfect." I learned a long time ago when dealing with networks that if you're working with ping you want to do a lot of them and you want to increase the payload size. Doing 1000 default linux pings (64 bytes) to the next hop usually lost around 6% of the packets on all the other towers and would go up to 20%-30% (or higher on occasion) when I increased the data size to 1250 bytes. (by the way, ping -A is wonderful). I don't think this is due to oversubscribing the network as I ran this every 1/2 hour all night long and it stays about the same all through the night. On these towers the ping time was 2-3 times higher than on the one good tower. Sometimes the ping time could be as high as 500 ms just on the local link. This is not due to distance as at least one "bad" tower is actually closer than the good one.

In my limited experience, if increasing the packet size causes increased packet loss rather than just increased echo response times, this indicates a misconfiguration in the network perhaps a timing source problem between serial lines. Does anyone have any other experience?

I kicked up a fuss and had them lock me on to the one good tower where I consistently get 0%-1% packet loss (because my radio kept latching onto other towers), however I am still trying to find any generally accepted ISP standards on what is acceptable local link ping times and % success rate. I didn't find anything useful with google. Does anyone have any references for this?

I am also wondering if anyone has used the nextnet equipment and what kind of results they've had.

I realize I'm just beating my head against the wall here but I may as well educate myself a little in the process.

Thanks,

Jizzle

[Thorn]

Not to say that your tests aren't valid, or that there isn't something going on, but you should understand that some packet loss on any wireless system is normal.

I'm not familiar with the NextNet equipment, but you might want to check through some WISP lists.

[TheCotMan]

Doing 1000 default linux pings (64 bytes) to the next hop usually lost around 6% of the packets on all the other towers and would go up to 20%-30% (or higher on occasion) when I increased the data size to 1250 bytes.

I am not very skilled with wireless, but that won't stop me from pointing out some obvious items. ]:>

There are a number of things that can cause problems with dropped packets.
* Physical Layer Interference, antenna failure, failure of NIC to separate noise from signal (special case of interference), channel selection, channel/signal arbitration (may be layer2) ,
* Datalink Layer issues (MTU/MMS issues for the chosen layer 2 communication, firmware/driver failure in header manufacture/recognition
* Network Layer issues (MTU can also play a role here, depending on the lower protocols and the selected network layer protocol.)

Let us assume it is a simple interference issue. People in this industry will probably have better formula for this, but let's test some probability.

You have a game where:
* after 100 plays with packets of length 64, there are ~6 losses, and 94 wins.
* after 100 plays with packets of length 1250, there are ~26 losses and 74 wins.

For ICMP, we have a checksum field with 16 bits that includes the checksum field set to all zeros, the header, and the payload. Therefore, a failure in bits located in the header, or the payload is sufficient to cause the received ICMP packet to be discarded, if the host that would reply to the packet, or the host that is waiting for the reply has a TCP/IP stack built to drop ICMP packets with invalid checksums.

Let us assume that the original source of the ping, and the initial destination both drop ICMP packets that fail checksum checks.

In this case, corruption from original source to first destination, OR the reply from the first destination bac to the original source may cause failure for a ping to yield a response.

We have 6 failure out of 100 when considering a round trip, or assuming failure can be from either end, 3 failures out of 100 for one way.

Now, concatenate your packets together, and by making more assumptions, consider 100*64*8= 51200 bits transmitted. All that is needed to cause failure in ICMP checksum, is a single bit failure, and I'll make another assumption that for any packet failed, there are "clumps" of failed bits in groups of 4, but only 4 failed bits per packet checksum failure. 3 failures times 4 bits is 12 bits. 12 bits failed for every 51200 bits, in any one-way transmission, leads to 12bits/51200bits or a simple view of One 4-bit-clumped failure for every 17067 bits.

Now, let us consider the case where 100 packets of 1250 byte length are transmitted. If we only use a simple linear view for probability of failure, compare One interference failure for every 17067 bits to 100*1250*8 = 1000000 bits being transmitted one way. 1000000/17067 =~59 failures (with the assumption that some packets have multiple failures.)

We can make a further assumption that failures will be clumped together with multiple failures per packet for failed packets.

From the above simple view (with too many assumptions) interference seems to be a plausible reason.

See if you can adjust your MTU/MRU (and control arbitration for these) at layer 2. If you can. Create an experiment where you try different values from 100 bytes to 1000 bytes. You mentioned using Linux, so check out a tool called "bing". Create a script that dumps bing results (from your IP to the next IP hop from you) for various limits to packet sizes, and see if there is an optimum maximum size for throughput to the bad towers.

In my limited experience, if increasing the packet size causes increased packet loss rather than just increased echo response times, this indicates a misconfiguration in the network perhaps a timing source problem between serial lines. Does anyone have any other experience?

Don't cell coverage towers and similar wireless coverage with towers use different frequencies per tower to allow for overlapping coverage while minimizing interference between towers? If this is the case, see if you can borrow a good radio oscilliscope and antenna to sample frequencies being used by the "bad towers" to see if they have more background noise than the "good tower" ?

I kicked up a fuss and had them lock me on to the one good tower where I consistently get 0%-1% packet loss (because my radio kept latching onto other towers), however I am still trying to find any generally accepted ISP standards on what is acceptable local link ping times and % success rate. I didn't find anything useful with google. Does anyone have any references for this?

I have no reference. However, any latency over 200ms for a common wireless ISP user (from user to next layer3 hop) seems too high. Afterall, anything over 200ms is close to dialup where latency is concerned.

[Jizzle]

Keep in mind that I can connect to one of the towers and get all of my pings back all of the time (except for an occasional single loss). This tells me that there is nothing wrong with the equipment on my end.

All that is needed to cause failure in ICMP checksum, is a single bit failure, and I'll make another assumption that for any packet failed, there are "clumps" of failed bits in groups of 4, but only 4 failed bits per packet checksum failure. 3 failures times 4 bits is 12 bits. 12 bits failed for every 51200 bits, in any one-way transmission, leads to 12bits/51200bits or a simple view of One 4-bit-clumped failure for every 17067 bits.

I'm not sure what point you are trying to make, but if I continue with your example it would take only 300 failed bits for all 100 packets to be lost, or 100% failure.

From the above simple view (with too many assumptions) interference seems to be a plausible reason.

Although I agree interference is a likely option, but I don't see how that was inferred from your examples.

See if you can adjust your MTU/MRU (and control arbitration for these) at layer 2. If you can. Create an experiment where you try different values from 100 bytes to 1000 bytes. You mentioned using Linux, so check out a tool called "bing". Create a script that dumps bing results (from your IP to the next IP hop from you) for various limits to packet sizes, and see if there is an optimum maximum size for throughput to the bad towers.

I talk ethernet to the radio, but I have no idea what protocol the radio is talking.

Haven't heard of bing. I always like to learn a new tool. Thanks.

Don't cell coverage towers and similar wireless coverage with towers use different frequencies per tower to allow for overlapping coverage while minimizing interference between towers? If this is the case, see if you can borrow a good radio oscilliscope and antenna to sample frequencies being used by the "bad towers" to see if they have more background noise than the "good tower" ?

Unfortunately I have no idea what frequency the radios talk at. I have an ethernet cable going into a little power supply which (I'm pretty sure) passes my ethernet through to the radio, adding DC power on one of the unused pairs. This cable goes into a "little white box" and that's all I know. There is no external detachable antenna. The radio has a web based interface but they've put a password on it.

There are various towers around town and they are all on different channels which would correspond to different frequencies. The most I can do is run the customer software which talks to the radio via UDP broadcasts and shows me which towers it can see and what their signal strengths are (ranging zero to 100). I was dropping pings on other channels despite having 100% radio strength, though I guess that doesn't mean there isn't interference.

I have no reference. However, any latency over 200ms for a common wireless ISP user (from user to next layer3 hop) seems too high. Afterall, anything over 200ms is close to dialup where latency is concerned.

Agreed, and especially when it's getting up to 1/2 second or more, which the company says is acceptable. I wonder why they think they can determine for the customer what is acceptable. I can get across several continents and oceans in 200ms but when I add in the last (wireless) hop that jumps to almost 1 second. To me that says there is something wrong.

Are there no established baselines for ISPs or in general network people to try to follow?

[TheCotMan]

I'm not sure what point you are trying to make, but if I continue with your example it would take only 300 failed bits for all 100 packets to be lost, or 100% failure.

Ah. I was trying to offer an inferrence:
increasing packet size does not always mean better throughput where interference is concerned. (And increasing percentages for packet failure does not necessarily mean decreasing effectiveness for throughput, when packet size is changed and interference is concerned.)

Simple example:
Since bps can be assumed to be constant, any number of bits can be equated or computed to be a specific length of time. (Interference every X seconds can be mapped to interference every Y bits with averages.)
Let's assume there is a predictable failure upon transmission of every 10000th bit. (For testing, we're assuming an average for failure becomes a reliably predictable periodic event.)

If you have a packet size of 1000 bits (125 bytes) , where 80 bytes are headers, and 45 bytes are payload, when transmitting 10 packets, you may see 1 packet totally lost (10% lost packets) , (45 bytes of payload) but 45*9 bytes of throughput for every 10000 bits (or 405/1250 for a stream of 10000 bits with 1 packet lost out of every 10 transmitted packets)

If you have a packet size of 5000 bits (625 bytes) where 80 bytes are headers, and 545 are payload, then every other packet would fail (50% packet loss) (545 bytes of payload) with 1*545bytes (or 545/1250 for a stream of 10000 bits as a measure of efficiency.)

If you have a packet size of 1250 bytes (10000 bits) then you could have 100% packet failure. (Special case) And any packets with size greater than 1250 would also be 100% packet failure.

What I was trying to show is how your measurement of significantly increased packet loss with larger packets can be an expected outcome when interferrence is considered. (Risk of failure per packet for large packets is significantly higher than per-packet risk for failure with smaller packets.) Smaller packets also take less time to retransmit.

Even though the effective throughput for the 125 byte packets was similar to the 625 byte packets (405/1250 vs 545/1250), the 125 byte packet system can have 10% packet loss while the other can have 50% packet loss.
Then, when you consider interactive use, the smaller-packet system with ~10% packet loss would be beter quality with occasional delays (retransmission required less often coupled with faster resend times due to smaller packet size) while "effective throughput" would favor 625 byte packets for efficiency if interactive use was not an issue (even though retransmitted packets would take longer to send.)

Over unreliable media, a smaller packet size is often a better choice than a larger packet size.
Over reliable media, small packets are inefficient, as the header takes a larger percentage of available "bandwidth" to yeild a lower efective throughput of payload.

The above theory is covered in a few networking books. IIRC, Computer Networks by A. Tannenbaum (the Minix guy) covers some of this theory, and even includes equations to estimate effective throughput if you are given interference rates (chances), packet size, and header size.

It is also covered in some Cisco literature, but I do not remember any title-- only that it was available back in 2000, and it was not my copy, but something that someone else let me borrow.

I'm also over-simplifying some issues, making assumptions about other issues and completely ignoring other issues. I'm hoping to pass an idea without weighing down the delivery with many other exceptions. (It also allows the reply to be shorter.)

All of the above was created to show that the values you found with packet loss % based on packet size can be associated with similar results created with inteference.

Are there no established baselines for ISPs or in general network people to try to follow?

I have no clue. If nobody answers here, Thorn's suggestion of a WISP list sounds like a great idea.

[Thorn]

Keep in mind that I can connect to one of the towers and get all of my pings back all of the time (except for an occasional single loss). This tells me that there is nothing wrong with the equipment on my end.

That may or may not be valid. It only tells you that your equipment is working with one tower. The rest of the system may be fine, but your equipment and that particular tower are affected in the same way.

I talk ethernet to the radio, but I have no idea what protocol the radio is talking.

According to NetNetWireless' specifcations the modulation type is ODFM. Therefore the protocol is probably an 802.11 variant.

Unfortunately I have no idea what frequency the radios talk at.

Depending on the particular model:
2300-2500MHz (2.3-2.5GHz)
2496-2692MHz (2.5-2.7GHz)
3300-3400MHz (3.3-3.4GHz)
3400-3600MHz (3.4-3.6GHz)

I was dropping pings on other channels despite having 100% radio strength, though I guess that doesn't mean there isn't interference.

Correct. A common misconception by people who don't understand radios, and how they relate to wireless networks, is that that good signal equates to good packets. Usually, the better the RF signal, the better the network throughput will be, but it is not necessarily true.

Simply put: Don't confuse RF signal with network throughput. It is entirely possible to have 100% RF signal strength, and still have zero packet thoughput due to interference.

Any nearby RF source which is operating in the same band can be causing enough RF interference to cause all sorts of nasty problems with the packets. For example, cordless 2.4GHz telephones do this to wireless LAN 802.11b/g equipment all the time. Some Bluetooth devices are another example; One misfit Bluetooth mouse can knock a whole WLAN for a loop. A 100% RF signal will be seen by the various devices, but the modulation of that signal gets so badly "mangled" by the interference, that no packets get through.

You might want to ask your WISP to do an RF sweep at your location, using a spectrum analyser. This may reveal some issue in or around your home which could be causing a problem. Local things such cordless phones, Bluetooth equipment, or 802.11b/g WLAN devices may be causing interfernce. Things that aren't local, but are strong RF sources (such as cellular phone backhauls or nearby AM/FM commercial radio stations), may also be sources of interference. I know of once case where a cellular backhaul went off-frequency and knocked a WISP's tower off-line. The cellular backhaul was 15 miles from the WISP's tower.

Agreed, and especially when it's getting up to 1/2 second or more, which the company says is acceptable. I wonder why they think they can determine for the customer what is acceptable. I can get across several continents and oceans in 200ms but when I add in the last (wireless) hop that jumps to almost 1 second. To me that says there is something wrong.

Are there no established baselines for ISPs or in general network people to try to follow?

Again, your best bet may be to look into some of the WISP lists, boards and forums.

[Jizzle]

TheCotMan: Thank you for that, I understand now perfectly what you are saying. It kind of depends on the nature of the interference (if indeed that's what is causing the problem). I am building my own linux router now on an old laptop (built-in UPS -- the power goes out a lot here) and I would like to play around with setting the MTU for the WAN side. Having said that I haven't studied networking as much as I would like but I have a very dim memory about intermediate links with low MTUs possibly causing connectivity problems. Maybe that was just if someone is blocking ICMP along the way and interfering with PMTU-discovery (which I read a little about a long time ago but don't remember all the details) then the sender won't get the message that they need to fragment, or something along those lines. No worries, I'll hopefully get around to playing with it.

Thorn: Signal to Noise ratio is I suppose what I want. However since (1) I've already been told that my connection is above their standard for residential customers and (2) he uses 4 windows 32 byte pings to determine if a connection is good, I rather doubt they are going to come out here to try to measure stray signals from other sources, even if this would help improve their service in the long run.

Thanks for all the discussion.

Jizzle