Re: Need help with bandwidth management . . .

On Sun, 11 May 2008 10:42:47 -0500, "JM" <jake@xxxxxxxxx> wrote:

All the above, actually. I'd like to have a method of capping each
connection, but I'm sure the equipment to accomplish that is not "free or
low cost."

Search Google for "bandwidth manager" or "bandwidth management". There
are a variety of Linux bases solutions that will work. I've used
DummyNet:
<http://info.iet.unipi.it/~luigi/ip_dummynet/>
for bandwidth management. The big problem is optimizing the
configuration for the traffic mix. That's neither easy or cost
effective as it's impossible to predict the type of traffic and number
of users in your obviously transient user setup. One P2P user will
break the system if they know a few tricks. There are lots of
articles on the web on how to configure various QoS applications. What
you'll soon find is that few of them agree with each other. That's
because everyone's situation is different.

There's a point where all this network management will outgrow the
capabilities of the WRT54G and DD-WRT. You're already at a
disadvantage by using the v8 hardware, which is lacking in sufficient
RAM to do much. I suggest you get a GS version with enough RAM to add
some additional applications that might be useful (i.e. MRTG). It's
also possible that you might be maxed out already. If there's any
growth planned, you might consider a better router (i.e. Cisco) with
much better system management and monitoring features. This would
also be a good time to separate the wired from the wireless parts of
the puzzle and switching to brain dead wireless access points and
wired connections.

I've worked a couple of hours today with the v24 RC5 firware's
QoS lan port settings, and I cannot get anything consistent. Theoretically,
I should be able to connect each of the 3 APs into one of the router's
switch ports and limit the bandwidth per port (the settings are
256k/512k/1m/10m/100m). However, this does not provide me "per connection"
bandwidth limiting - only "per AP" - and, besides, the lan settings don't
seem to work by the numbers. It does have some effect, but not in any
precise way.

The author of DD-WRT decided to sell a commercial version of DD-WRT
and reserved the "per-connection QoS" feature for the commercial
version. I really don't know much about it other than Buffalo
licensing the firmware and supplying it with some of their products.

As for applications control, can that be accomplished to any significant
degree by port filtering?

No. Some of the P2P applications use common ports. If you throttle
them by port number, you clobber the common applications. The only
effective way is to throttle by content which requires sniffing. A
few P2P apps have well known ports, but they are becoming the
exception.

You might want to look at the Hughesnet FAP (fair access protocol),
which has the same problem. How does one share a limited satellite
backhaul, with an inordinately large bandwidth demand.

Is it realistic that I could sniff the network
over time and identify ports that typically are used for things like music
and video downloads and then block these ports?

No. That's because some apps and users change port numbers if they
suspect they're being throttled. For example, BearShare, Limewire,
Morpheus and ToadNode all can use any port number to communicate.

Are these ports consistent,
or do they differ according to the particular service, vendor, client
software, etc?

Some use static port numbers, but most cannot be blocked by port
number.

I broached the topic of more bandwidth the first day I got involved.

Backwards. Ask about active user count and customer expectations.
That will determine the required bandwidth. The problem with P2P is
that it will saturate ANY amount of bandwidth you supply. If you give
them an OC-192, they'll fill it up.

Old rule of thumb for how many users can share a T1:
100 light users
10 business users
1 file sharing user
Unfortunately, it's true.

The
LEC that provides the T1 can bring in "business class" ADSL circuits for
about $80/month (the T1 costs about $350/month). I think the DSL is 4mb/1mb
or so.

It's probably a 6Mbit/sec by 640Kbits/sec DSL line, which will yield
about 5Mbits/sec download, and 570Kbits/sec upload.

I like T1s, from a network admin standpoint, but I'm not sure it's
the best solution in this case. It's an easy sell for the LECs, because
it's a dynamic pipe that carries the voice and data. The LEC provides an
IAD (fancy channel bank) and breaks out two connections - one that
terminates on a RJ-21'ish block for the phone system and a 10/100 port for
the customer router.

That's NOT a T1. That's an HDSL line:
<http://en.wikipedia.org/wiki/HDSL>
Watch out for the 100VDC or so on the line.

It's a good product, and I've had good experiences
with it for other customers, especially those with bursty voice traffic.

Yep. Very low latency with committed bandwidth. No sharing on the
backhaul makes it great for VoIP.

But this RV park almost never has more than two voice lines going at one
time. It has occurred to me that we could get 3-4 copper lines (at ~35 per)
and ~3 DSL circuits for what they are paying for the T1. See, part of the
thought process for the T1 (they used to have 2 with a different provider)
was to provide the guests with phone lines. However, it just hasn't
materialized. Everyone has cell phones, and almost no one needs a dial up
or fax line. There is a fax in the main office for publick use.

Or, you can just get a fat pipe of some sorts and switch all the phone
lines to VoIP. If the line can do G.711, fax will work. If you
compress with G.729, the fax will screw up. There are specialized FAX
over IP services available. Or, just use eFax and be done with it.

Or music. I've got a Sonicwall SOHO3 that actually provides very good data
of this type. I can stick that in there and watch for a few days.

Streaming or downloading? I stream music almost continuously on my
connection. About 100kbit/sec continuous download is not even
noticeable on a 1.5 or 3Mbit/sec DSL line. If they're downloading
music, then it's just another aspect of P2P file sharing.

That's what I think, too. FWIW, the 30-50 estimate may be a little high,
but still the point remains if the actual use is 20-30 or similar.

Nope. If my coffee shop customers are any example, I see 30 laptops
online all the time. I have no problem sharing a 3Mbit/sec DSL line
with 30 connections. I can't do that with 30 active users, but most
of the machine are just idle and doing nothing most of the time. I
just checked one of the busier coffee shops. 38 leases assigned. 17
active users. Average bandwidth use over the last hour is about
200Kbits/sec. Peaks to about 600Kbits/sec. Hardly being used at all.
Incidentally, DD-WRT v24 RC6.2 has cute graphs of the traffic usage on
the status page.

Well, I lied. I just looked again and the incoming traffic is up to
1.4Mbits/sec. Looks like someone is furiously downloading something.
I expected to find one user doing a big download. Instead, I find 3
users watching what appears to be YouTube videos. Sigh.

That's
potentially way too much for a T1.

Not really. It depends on what the customers are expecting. If
they're paying for access, they'll complain. If it's "free" or part
of the hookup, then they'll take whatever they can get. The easiest
way to know for sure is to install it with a limited bandwidth
connection and see if there are any complaints. If not, leave it.

Something I've given thought to this
weekend is an AUP (acceptable usage policy) that is at least posted in the
office, if not made part of the guest contract.

I've written (actually plagiarized) 3 different AUP/TOS documents. I
promised myself I would never do another.

Is it realistic that we
whitelist the open ports?

No, because you can't. Unless you're planning to deliver (or alias)
routable IP's to all the users, you can't open ANY ports on the router
to the clients machines. That means no servers of any kind. It also
breaks a few applications. You can get blocks of 32 IP's from some
ISP's, but what a waste of money for transient users.

I simply don't know enough about the range of
services "needed" for such a population of users.

That's what your traffic analysis will show. If it's like the wild
wild web, 75% of the bytes will be to/from P2P applications.

Can one limit the
available internet traffic to "the basics?" Is there such thing?

Yep. The easiest and messiest way is to use a SOCKS5 proxy server.
Only those applications that are allowed will go through the proxy
server. Each application has it's own configuration line. That what
is not specifically allowed, is blocked. Your clients will hate you,
the phone will ring constantly with complaints, and you will spend
many a sleepless night fighting the configuration. It won't work
anyway because it's essentially white listing by port number, and many
P2P applications can effectively spoof common applications.

In my never humble opinion, you really only have two options:
1. Sniff traffic and either block or throttle by content. Maybe some
port blocking for obvious problems (i.e. port 25 to prevent users from
becoming spammers).
2. Throttle by user count to insure there's always some overhead left
for ACK's. If there's only one user on, they get the whole pipe. If
there are 10 users, each gets 1/10th. Fair share and all that.

1. Number of active users. I suspect that there may be 30-50
connections, but they are not all active at the same time.

Well, that's an interesting thing. While monitoring the connections it
appears that many of the connections stay alive constantly, but the internet
usage is "on and off."

Duh. That's normal. Right now, I have 5 wired and wireless
connections to my router. All (but one) show up on the MAC address
list. None of them are generating any traffic. Ooops, one of my
neighbors machines just came alive with what looks like a periodic
check for email.

In other words, I see some MAC addresses maintain a
wireless connection over a period of hours, but the behavior of the user
seems to be on-off, on-off, on-off.

Actually, if you have the DHCP leases saved to NVRAM, the MAC to IP
address mapping will be essentially permanent. I was wondering why I
was seeing 200 users connected, and eventually figured out that they
were long gone, but their DHCP leases were still in memory. Uncheck
the box "save DHCP leases in NVRAM" on DD-WRT or you'll rapidly run
out of DHCP assigned IP's.

I guess this is not so different that
most networks, but it seems like these residents keep the internet up all
the time, and periodically use it for something specific.

Yep. That's why it's called a "full time" connection. No dialing
required.

These kinds of
connections are the usual, and they don't seem to be problematic. It's the
users that obviously are downloading content that are the killers.

Nope. Users *DOWNLOADING* isn't as much a problem as *UPLOADING*. The
asymmetrical nature of the DSL line makes uploading bandwidth far more
important than the larger downloading bandwidth. If the upstream is
saturated with P2P (server) content, the ACK's will not be received by
the various internet servers and they will try to resend whatever the
users are looking at. Or worse, they will time out the connection
even if there's downstream bandwidth available. This is why you
always want to preserve some upstream bandwidth.

2. Is there a PC available to do monitoring?

Yes.

Use it to monitor the existing connections. MRTG, RRDTool, various
SNMP monitoring tools, traffic sniffers, security monitors, etc.

3. Is everyone connected via wireless or are there wired connections?

The original plan was for both. Conduit is available for the purpose, but
no further network wiring is to be done. There is coax at every "pad" for
TV. I'm relatively sure management is locked into wireless. I do no think
they will consider other options, as long as a solution to the immediate
challenge is within reach.

Wireless lucks for 30-50 full time users in a confined area. It can
be done but 802.11 was never designed for that application. I can
list a few failure scenarios if you want. The easiest is that one
leaky microwave oven will take down the entire system.

Coax cable is a good thing. If the park has conduit, run CAT5. If
not, share the coax with one of several available products:
<http://www.multilet.com>
<http://www.coaxsys.com> (TVNet/C)
Worst case, lease a bunch of cable modems and get a contract with the
local CATV provider. Rent them to the visitors (with a suitable
deposit to cover the $200 cost per box).

Are all the wireless connections authenticated or is it a free for
all?

The latter, which is regrettable, in my opinion.

No, it's fatal. You cannot efficiently run, manage, or otherwise
operate a wide open system. You need some sort of security for the
paying and authorized users. If that means a RADIUS server with
WPA-RADIUS encryption and authentication, then that should be high on
your priority list. Who know.... perhaps your traffic will drop when
you kick off the free loaders.

But management claims that
security measures would be confusing to this particular user population, and
they don't want to give any reason for these users to go elsewhere.

Since when is a user name and password on a splash page confusing? I
can't believe that this would inspire a camper/trailer to go
elsewhere. If nothing else, the lack of wireless encryption will
expose them to sniffing issues, which is far more serious than some
theoretical "confusion".

Look at the various hot spot software included in DD-WRT.
Services -> Hotspot
I kinda like ChiliSpot, although WiFiDog seems easier to setup. You'll
eventually need an external RADIUS server for authentication.

If open, are you sure that all your users are your RV park
residents and not the neighbors?

I am not sure. To the contrary, I'm sure that we've basically built a free
WISP.

Fine. If you don't want to go through the trouble of securing your
mess, then there's no reason to be optimizing the traffic. Leave it
wide open, and may the most persistent user win all the bandwidth.
Never mind that it will be effectively useless for any of the guests.
Either do it right (encryption, security, traffic management,
monitoring), or just let it free run.

FWIW, this park is relatively isolated, but as we know, it only takes
1-2 abusive users to wreck the whole thing. I'm starting to see some kind
of authentication as a necessity.

Yep. Are you aware that a good size 24dBi dish antenna can connect
effectively over a distance of a mile or more?

Do you have a RADIUS server?

Not at this time, but I could provide one.

You'll need it for authentication. You could use an online RADIUS
authentication service until you get one setup:
<http://radiuz.net>
I've got an internet connected RADIUS server running that I use for
testing at some of my customers hotspots. It's not really reliable
enough yet but shows possibilities.

5. Are you prepared to bill for excessive bandwidth use?

I'm sure I couldn't get this approved.

If you had a monitoring station, that will generate a per-computer
traffic report, you could bill for abuse and overuse. This would be
an alternative to traffic management. Just let them do whatever they
want and bill them when they screw up. It's not a popular method, but
it works well if applied diplomatically. A friends apartment building
wireless network works this way. He posts the monthly traffic
summaries so that everyone can see who's hogging the wireless.
Needless to say, that even the teenagers have begun cooperating.

Thank you for the discussion.

Good luck. I think you're about to make a few major mistakes. You
really have no clue as to the number of active users, their traffic
patterns, or their expectations. You've also failed to investigate
the alternatives to wireless. The really big problem you're missing
is "who's gonna jump when the phone rings"? Are you going to get the
customer complaints or the park management? Do you really want phone
calls at 1AM when their email doesn't work for some reason? Think of
it this way: "What can I do with this system to prevent the phone
from ringing"?

--
Jeff Liebermann jeffl@xxxxxxxxxx
150 Felker St #D http://www.LearnByDestroying.com
Santa Cruz CA 95060 http://802.11junk.com
Skype: JeffLiebermann AE6KS 831-336-2558
.

Relevant Pages

  • Re: Need help with bandwidth management . . .
    ... The bandwidth managements is much better in v24 than in v23: ... QoS lan port settings, and I cannot get anything consistent. ... it common practice on their home connections. ... first one in my opinion is bandwidth management. ...
    (alt.internet.wireless)
  • Re: Adding Linux machine to existing wireless WinXP network - Heeeelllllp!
    ... 1 notebook) connected in peer to peer via wireless NIC's ... > router providing DHCP services and gateway to cable Internet. ... I connected it via hard-wire network cable to my LAN port on the ... > that the router is limited to four connections and that if I ...
    (alt.linux)
  • Re: NFS through a wireless router
    ... >> I've been running a wireless network in a rather insecure mode for a ... I purchased a wireless router to replace my ... >> cannot get my NFS connections to load through the NAT of the new device. ... > If you're using NAT then it'll be translating the source port from ...
    (comp.os.linux.networking)
  • Re: Port Attack Question
    ... Mark - The problem is one of the processing on the firewall and available ... The port whether it is POP/SMTPor HTTPis irrelevant aside from the ... connections and may slow the firewall down if enough connections are ... If the attackerhad enough bandwidth or networks to bounce the ...
    (comp.security.firewalls)
  • Re: Iptables FTP question
    ... for secondary connections. ... Some ftp servers don't allow passive mode because it is less safe from ... algs that allow port mode for client machines. ...
    (comp.security.firewalls)