The cost of a program's execution

(running time, memory, ...)

rather than

The cost of the program

(# of statements, development time)

[In this sense, less-complex programs require
*more*`
`development time.]

We focus primarily on

Consider a program with one natural number as input.

Two functions that can be associated with the program are:

(1)

The functioncomputed bythe program --f(n)(2)

The running time of the program --T(n)

However, it is customary to measure T based on the "size" of the input,

rather than the input value.

Other size measures are also used

e.g. for an array computation the *size of the array*`
`might be used,

ignoring the fact that the *numbers stored*`
`in the array have *different lengths.*

In a *finer-grained*`
`analysis, these lengths would need to be
taken into account.

Primitive Operations

These are operations which we don't further decompose.

They are considered the fundamental building blocks of the algorithm.

+ * - / if( )

Typically, the time they take is assumed to be
*constant.*

This assumption is not always valid.

Example

In doing arithmetic on arbitrarily-large numbers,
the *size of the
numerals*` `representing those numbers has a definite effect on the
time required.

2 x 2

vs.

263784915623 x 78643125890

For a

e.g. decimal encoding:

size(n) = #digits of n

= Ïlog

_{10}n¸

Ïx¸ = smallest *integer*` `__>__ x

(called the

ceilingof x)

The

since *log functions
differ only by constant factors* :

log

_{b}(x) = log_{c}(x) / log_{c}(b)(log

_{b}(x) and log_{c}(x) have two different bases, and log_{c}(b) is a constant)

e.g.

log

_{2}(x) = log_{10}(x) / log_{10}(2)log

_{10}(2) = 0.301031 / 0.30103 = 3.32193

When we write "log" it is usually log

Unless we state otherwise, we'll assume

+ , * , / , < , etc.

We will further approximate that all take the same time, which we'll call

1 "step"

The time of a step varies among computers.

But the *number*` `of steps will be the same for a given program.

Counting Steps

Straight-Line Code:

Loop-Code:

Steps are a function of # of iterations

Recursive Code:

Steps can be determined by solving

recurrenceRecurrence for run-time T:

The recurrence

T(0) => 1;

T(n+1) => 1 + T(n);

has the *solution*

T(n) = n + 1

One way to see this is by repeated substitution:

Recursive Code can sometimes be analyzed by understanding the number of recursive cells, then multiplying that number by an appropriate factor.

In both loops and recursion, the number of cells/iterations is not always obvious.

Example

List L; L = ... ; while ( !empty(L) ) { : : L = rest(L); }

The multiplying factor (# of iterations) is the length of the list.

"O" (order) notation

expresses *upper
bound*` `on
execution time to within a *constant
factor*

[ For other than natural number args, this definition is inadequate. ]

^{°}[ sometimes T=O(f) is used ]

Example

T(n) = 1.5 n

^{2}f(n) = n

^{2}T O(f)

since by taking c=1.5

we have

We could also use

We usually abbreviate this by dropping one or both n Æ :

T O(1) means T is upper bounded by a

We say "O(1)" when we mean some constant (not a function of input)

Linear-Addressing Principle

Any access to memory is O(1).

Typical algorithm growth rates

O(n)` `("linear")

finding the max in an array

O(n^{2})` `("quadratic")

Sorting an array by a naive method

Example

Binary SearchThere are O(log n) steps.

Each step is O(1), due to linear addressing principle.

O(log n)overall

log n

Doubling n only

adds 1to log n.Doubling n

doublesn.

log arises in a recurrence of form

T(1) 0;

T(2*n) T(n) + 1;

solution:

T(n) = Ïlog n¸

To see this, use repeated substitution:

O(n log n)

Sorting an array by an optimal method

n log n

is better than n^{2}:Divide each by the factor n

Scalability Considerations

O notation focuses on comoparing

"growth rate"of functions:T O( n

^{2 })means T(n) grows no faster than n

^{2}grows.This is useful in comparing algorithms; it gives us an idea about how well the algorithm performs over an

infinite rangeof inputs.

2 algorithms for some problem

Algorithm A has a preferable

growth rate(O(n)) compared to algorithm B (O(n^{2})),even though A is worse than B in some

finite range.

Polynomial-time algorithms are preferred over non-polynomial

Some problems are not known to possess

[Some are also not known *not*` `to possess a polynomial-time algorithm.]

Examples

Checking whether an arbitrary SOP form is a

tautology.Traveling salesperson problem

TSP (Traveling Salesperson Problem)

Given n cities with costs between each pair, find the "tour" which minimizes total cost.

Tour = permutation of citiesThere are n! possible tours

obvious method is O(n!)

Tautology-Checking

Could use Boole-Shannon expansion:

Tree of height n (= # vars)

2

^{n}leavesO(2

^{n}) is bound for this algorithm