# Re: A question about capacity planning and scalability [2]...

*From*: aminer <aminer@xxxxxxxxxxxx>*Date*: Wed, 8 Sep 2010 15:50:52 -0700 (PDT)

Hello again,

I will continu...

I wrote before that:

--

And of course on Capacity Planning for Enterprise Datacenters

and Websites , you can mirror many computer servers and load

balance between them with a software... to make the system much

FASTER, and this will be modeled as a Jackson network like this:

(1)

A -> M/M/n Server Queue -> M/M/1 Network queue ->

-> M/M/1 Client queue -> A

A: the arrival rate to the system"

--

We know also from an operational law of queuing theory that:

(2) the rate of job leaving any stable node must equal

its arrival rate.

We know for an M/M/c queue that:

Note: try to add servers with almost the same hardware

configuration...

C(c, U) = Erlang formula = P(c) / (1 - Utilization)

note: c the number of servers..

P(c): means the probability that all the servers are busy

P(0): means the probability that there is no waiting time in the

queue, that means also: AT LEAST one server among the C servers

are not busy...

The average waiting time in the 'queue' =

C(c,U) / (service rate x c x (1 - Utilization)) (3)

It's approximatly equal to:

Utilization^C/(service rate x (1 - Utilization^C)

Note: ^ means power

This approximation is exact for the M/M/1 and M/M/2 models,

but 'slightly' lower than the result in (3) if c > 2

and

Utilization = Density of circulation / C (number of servers)

Note: ^ means power()

and C means the number of servers

Response time = The average waiting time in the 'queue' +

(1 / service rate)

average numbers of users in the system = service rate x response time

average number of users in queue = service rate x average waiting time

in the 'queue'

Now as i said before:

--

So the equation for

Ni: number of jobs in each queue

Ui: utilization of each queue

Ni = Ui / (1-Ui)

Adding all the Ni in each individual queue will give the

average number of jobs in the entire queuing network.

After that we apply the Little formula:

A: network arrival rate

T: average response time

N = A*T => T = N / A

And after that from the mathematical analytical equation

we can simulate the jackson queuing network that model our

webservers...

---

If we try to calculate the Ni = Ui / (1-Ui) in

the Jackson network, and from the operational law (2) above

this will give us:

Ns for the M/M/n Server queue is:

(DC / n) / (1 - (DC/n))

and DC = Ss / A => Ns = ((Ss/A)/n) / (1 -((Ss/A)/n))

Ss: service rate at the queuing server.

A: Arrival rate to the jackson network

DC: is the Density of circulation

n: number of servers

And Nn for the M/M/1 Network queue is:

and Utilization in the M/M/1 network queue = Sn / A

this imply that => Nn = (Sn/A) / (1 -(Sn/A))

Nn: number of jobs in the M/M/1 network queue node

Un: Utilization in the network queue node

Sn: service rate at the queuiNg server.

A: Arrival rate to the jackson network

And Nc for the M/M/1 Client queue node is:

and Uc= Sc / A

this imply that => Nc = (Sc/A) / (1 -(Sn/A))

Nc: number of jobs in the M/M/1 client queue node

Uc: Utilization in the M/M/1 client queue node

Sc: service rate at the queuiNg server.

A: Arrival rate to the Jackson network

Adding all the Ni in each individual queue will

give the average number of jobs in the entire

queuing network that is equal to:

Ni = Nn + Ns + Nc

= (((Ss/A)/n) / (1 -((Ss/A)/n)) + (Sn/A) / (1 -(Sn/A))

+ (Sc/A) / (1 -(Sn/A))

After that we apply the Little formula:

A: network arrival rate

T: average response time

N = A*T => T = N / A

this imply that the T(the average response time in the Jackson

network)

is:

T = Ni /A = (Nn + Ns + Nc) /A

= (((Ss/A)/n) / (1 -((Ss/A)/n)) + (Sn/A) / (1 -(Sn/A))

+ (Sc/A) / (1 -(Sn/A))) / A

Now finally we have our mathematical analytic equation

and we can begin to simulate our Enteprise webserver

and also validate the performance data with the fwptt stress

webserver simulation tool...

And don't forget what i have said about USL:

"Suppose we have obtained a small number of measured

load-points with Loadrunner or others tools, and we

calculated the USL equation to predict scalability of

a webserver , how the USL model can predict if the

scalability/performance is limited by the network bandwidth

and not the server ? I think USL can not predict this."

Also i wrote about USL that:

"As you have noticed , this mathematical model of

this jackson network does in fact take into account

the M/M/1 Network queue node , the USL model can not

do this... and with this performance data from the mathematical

analytical model simulation we can for example validate

the performance data of the fwptt stress webserver simulation.."

Regards,

Amine Moulay Ramdane.

http://pages.videotron.com/aminer/

On Sep 8, 9:11 am, aminer <ami...@xxxxxxxxxxxx> wrote:

On Sep 8, 7:48 am, aminer <ami...@xxxxxxxxxxxx> wrote:

Hello,

I have cleaned all my previous posts , please read again...

I didn't know where to ask this question, so i decided to ask here..

I just read yesterday the following page, look at the the USL

(Universal Law of Computational Scalability) of Dr. Gunther,

he wrote this: ( seehttp://en.wikipedia.org/wiki/Neil_J._Gunther)

--------------------------

The relative capacity C(N) of a computational platform is given by:

C(N) = N

-------------------

1 + α (N - 1) + β N (N - 1)

where N represents either the number of physical processors

in the hardware configuration or the number of users driving the

software application. The parameters α and β represent respectively

the levels of contention (e.g., queueing for shared resources) and

coherency delay (i.e., latency for data to become consistent) in the

system. The â parameter also quantifies the retrograde throughput

seen

in many stress tests but not accounted for in either Amdahl's law or

event-based simulations.

----------

His website:http://www.perfdynamics.com/

If you read carefully , you will see that Dr. Gunther is using this

model to predict scalability after he simulates a relatively small

number of vusers in LoadRunner ( because of licensing costs, it's

cost-effective) and after that he finds the coefficients of the

2nd-degree polynomial (quadratic equation) and then transform

those coefficients back to the USL parameters using the α = b - a

and β = a.

And then he is extrapolating with the USL model to higher loads

to predict scalability.

He is also applying the model to webservers with heterogeneous

workloads. like in the following page:

http://perfdynamics.blogspot.com/2009/04/assessing-usl-scalability-wi...

Now my question follows:

Suppose we have obtained a small number of measured load-points

with Loadrunner or others tools, and we calculated the USL equation

to predict scalability of a webserver , how the USL model can predict

if

the scalability/performance is limited by the network bandwidth and

not the server ? I think USL can not predict this.

When we are modeling webservers , we have to include

the network&tcp/ip in our network queuig model

(that comprises the queue of the computer server side) ,

and since the service in the computer server is comprised of

multiple services (when we are using htmls , databases etc.)

the network&tcp/ip queue will not be markovian in the service

side, and we have to model the network&tcpip queue as an M/G/1

and this will complicate the mathematical analytic modeling...

So, i think the best way is to use a webserver stress tool

like http://fwptt.sourceforge.net/

You can even test the webserver with an heterogeneous

workloads by starting multiple fwtpp processes, and

you should increase the number of threads to 5 and after

that to 10 threads, 15 ... and so on until the webserver

applications stops responding propperly(and this will inform

on the maximum number of users that you can have in the same time...)

and as you are stress testing you can even see (by testing/measuring

it) if the network bandwidth is not the bottleneck... and this can

not be done by the USL model.

We already know that to satisfy a Poisson process we must

have that N(t1)- N(t0), N(t2)- N(t1) etc. must be independant

that means the counting increments must be independant.

We have the following relation between the Poisson law

and Exponential law:

the expected value E(X exponential) = 1 / E(X poisson)

and if the arrival is poissonian then the interarrivals are

exponential..

Now what about a webserver ?

I have read the following paper:

http://docs.google.com/viewer?a=v&q=cache:JFYCp_dSPP4J:citeseerx.ist.....

And it says that a simple model with M/M/1/k with FCFS discipline

can predict webserver performance quite well..

Hence, i think we can model our webserver over internet

with 3 queues connected as a Jackson Network like this

A -> M/M/1 Server Queue -> M/M/1 Network queue -> M/M/1 Client queue -

A

A: is the arrival rate

and we have the following:

Ni: number of jobs in each queue

Ui: utilization of each queue

Ni = Ui / (1-Ui)

Adding all the Ni in each individual queue will give the

average number of jobs in the entire queuing network.

After that we apply the Little formula:

A: network arrival rate

T: average response time

N = A*T => T = N / A

And after that from the mathematical analytical equation

we can simulate the jackson queuing network that model our

webservers...

Now there is still an important question that i have:

Our analytical jackson network model can give us insight

on the webservers behavivior.. but the difficulty that i find in

practice is that: suppose we have found the right parametters

that we want to choose, like for example the service rate of

the M/M/1 Server Queue , how , from this service rate, can

we buy the right computer that satisfies the service rate

that we want ?

We say in OR that:

"Understanding the behavior of a system is what Queueing Theory

and Little’s Law is all about. But, managing a Queue requires not

just understanding the behavior of a system, but also in-depth

knowledge of how to improve a system — improving both aspects

of Queueing will mean a better, more efficient and cost-effective

system and, more importantly, a much better customer experience."

I wrote before that:

---

"It says that a simple model with M/M/1/k with FCFS discipline

can predict webserver performance quite well..

Hence, i think we can model our webserver over internet

with 3 queues connected as a Jackson Network like this

A -> M/M/1 Server Queue -> M/M/1 Network queue -> M/M/1 Client queue -> A

A: the arrival rate

and we have the following:

Ni: number of jobs in each queue

Ui: utilization of each queue

Ni = Ui / (1-Ui)

Adding all the Ni in each individual queue will give the

average number of jobs in the entire queuing network.

After that we apply the Little formula:

A: network arrival rate

T: average response time

N = A*T => T = N / A

And after that from the mathematical analytical equation

we can simulate the jackson queuing"

--

As you have noticed , this mathematical model of

this jackson network does in fact take into account

the M/M/1 Network queue node , the USL model can not

do this... and with this performance data from the mathematical

analytical model simulation we can for example validate

the performance data of the fwptt stress webserver simulation..

But you have to take into account worst cases and the

peak traffic loads...

Let for example we have a a webserver hosting html pages

and it is receiving 1000000 HTTP operations

per day with an average file size of 10 KB.

What would be the network bandwidth required for this website

considering peak traffic if the peak traffic load from past

observations was four times greater than average loads?

Required bandwidth is solved by the following equation:

HTTP op/sec x average file size or

1000000 HTTP ops per day =1000000/24 = 41,667 op/hour =

41,667/3600= 11.6 HTTP ops/sec

The needed bandwidth is

11.6 HTTP ops/sec X 10 KB/HTTP op = 116 KB/sec = 928 Kbps.

If we assume a protocol overhead of 20% then the actual throughput

required is 928 Kbps X 1.2 = 1,114 Kbps.

However if peak loads as we say before is as much as

i mean: as i said before...

4 times greater, the bandwidth required to handle spikes

would be 4 X 1,114 Kbps = 4.456 Mbps.

So you have to think also about the cost of this line...

I will add the following:

As you have noticed i said that:

"As you have noticed , this mathematical model of

this jackson network does in fact take into account

the M/M/1 Network queue node , the USL model can not

do this... and with this performance data from the mathematical

analytical model simulation we can for example validate

the performance data of the fwptt stress webserver simulation.."

and i said that:

"Hence, i think we can model our webserver over internet

with 3 queues connected as a Jackson Network like this

An M/M/1 Server Queue -> M/M/1 Network queue -> M/M/1 Client queue"

And of course on Capacity Planning for Enterprise Datacenters

and Websites , you can mirror many computer servers and load

balance between them with a software... to make the system much

FASTER, and this will be modeled as a jackson network like this:

A -> M/M/n Server Queue -> M/M/1 Network queue -> M/M/1 Client queue -

A

A: the arrival rate to the system"

But there is still an important thing , as i have showed before

on my calculations:

"However if peak loads as we say before is as much as

i mean: as i said before...

4 times greater, the bandwidth required to handle spikes

it would be 4 X 1,114 Kbps = 4.456 Mbps.

So you have to think also about the cost of this line..."

I think that you have also to take into account the knee utilisation

of your M/M/n Servers Queues, if for example the number of computer

servers is 8 and the Knee is 74% that means that in our previous

example the bandwidth must equal to:

126% X 4.456 Mbps = 5.614 Mbps.

Cause as you know, over this Knee of 74% for 8 servers

I mean: above this knee of 74%

the curve of the waiting time does grow quickly ..

And we have to take into account the cost of the line ...

So be smarter !

Regards,

Amine Moulay Ramdane.http://pages.videotron.com/aminer/- Hide quoted text -

- Show quoted text -- Hide quoted text -

- Show quoted text -- Hide quoted text -

- Show quoted text -

.

**Follow-Ups**:

**References**:

- Prev by Date:
**Re: Why no locks in, um, lock-free algorithms?** - Next by Date:
**Re: A question about capacity planning and scalability [2]...** - Previous by thread:
**Re: A question about capacity planning and scalability [2]...** - Next by thread:
**Re: A question about capacity planning and scalability [2]...** - Index(es):