Introduction and Algorithm Analysis

Introduction and Algorithm Analysis

Basic Definitions

• Type: a collection of values
  Boolean type: collection of true or false value
  Integer type: collection of 1,2,3… value
  Data type: a type and a collection of operations that manipulate the type
  Integer data type: Integer type and + - x ÷ operations

• Data Item: a piece of information of a record drawn from a data type

• Atomic Data Type: contain no subparts
  e.g. Integer, Boolean

• Structure Data Type: contain several pieces of information
  e.g. Array, String

• Abstract Data Type: a definition for a data type solely in terms of a set of values and a set of operations on that data type. ADT operation is defined by its inputs and outputs Hide implementation details (Encapsulation)

• Data Structure: the physical implementation of an ADT Operations associated with the ADT are implemented by subroutines (functions)
  Usually refers to an organization for data in main memory

• File Structure: an organization for data on peripheral storage
  e.g. Disk drive

Logical vs Physical Form

• Logical Form : Definition of the data item within an ADT
  e.g. Integers in mathematical sense: +, -

• Physical Form : Implementation of the data item within a data structure
  e.g. 16/32 bit integers: overflow

Problem, Algorithm and Program

• Problem: A task to be performed
• Algorithm: a method to solve a problem

Correct
No ambiguity
A series of concrete steps
A finite number of steps
Terminate

• Program: an instance for an algorithm in some programming languages

Algorithm Evaluation

• Running Time T(n) : Some function T on input size n

Examples :

sum=0

$T(n)=c$
c : the running time to assign a value to a variable

for i=1 to n 
	for j=1 to n 
		sum=sum+1
	end
end

$T(n)=c+c{n}^2$
The c not related to n is not so important

s1=1;
s2=1;
s3=1;

$T(n)=c$

for a=1 ro n/2
	s1=s1+s1;
end

$T(n)=cn/2$

for b=1 to n*n
	s2=s2*s2
end

$T(n)=c{n}^2$

for i=1 to n
	for j=1 ro log(n)
		s3=s1+s2+s3;
	end
end

$T(n)=cnlogn$

s1=1;
s2=1;
s3=1;

for a=1 ro n/2
	s1=s1+s1;
end

for b=1 to n*n
	s2=s2*s2
end

for i=1 to n
	for j=1 ro log(n)
		s3=s1+s2+s3;
	end
end

$T(n)=c+cn/2+c{n}^2+cnlogn$

• Best, Worst, Average Cases
• The Change of The Complexity
  • Big-Oh
    • Indicates the upper bound of a growth rate
    • For T(n) a non-negatively valued function, T(n) is in the set $O(f(n))$ if there exist two positive constants $c$ and $n_0$such that $T(n)\le cf(n)$ for all $n>n_0$
      • $n_0$ is the smallest value of n for which the claim of an upper bound holds true
      • Actually value of $c$ is irrelevant
    • Algorithm Analysis: Big-Oh

    $O(1):constant$
    $O(log{2}^n):logarithmic$
    $O($ ${\log }_2{2}^n$ $):$ $logsquared$
    $O(n):linear$
    $O(nlo{g}_{2}n):nlogn$
    $O(n^2):quadratic$
    $O(n^3):cubic$
    $O(2^n):exponential$

Examples

$T(n)=3n^2$
$T(n)$ is in $O(n^2)$

$T(n)=3n^2+n$
$T(n)$ is in $O(n^2)$

$T(n)=c+cn/2+c{n}^2+cnlogn$
ps. $n>logn,asn\to \infty$
$T(n)$ is in $O(n^2)$

• Big-Omega
  • Indicates the lower bound of a growth rate
  • For $T(n)$ a non-negatively valued function, $T(n)$ is in the set $\Omega (g(n))$ if there exist two positive constants $c$ and $n_0$ such that $T(n)\ge cg(n)$ for all $n>n_0$
    • $n_0$ is the smallest value of n for which the claim of an upper bound holds true
    • The actually value of c is irrelevant

Examples

$T(n)=3n^2$
$T(n)$ is in $\Omega (n^2)$

$T(n)=3n^2+n$
$T(n)$ is in $\Omega (n^2)$
(与之前的区别是系数不一样)

• Big-Theta
  • An algorithm is said to be $\Theta (h(n))$ if it is in $O(h(n))$ and $\Omega (h(n))$

Example

What is the Big-O, Big-Omega and Big-Theta of the following program?

sum1 = 0;
for(i=1;i<=n;i++) //first double loop
for(j=1;j<=n;j++) //do n times
sum1++;
sum2 = 0;
for(i=1;i<=n;i++) //second double loop
for(j=1;j<=i;j++) //do i times
sum2++;

$T(n)=c_1+c_2{n}^2+c_1+c_{2}n(n+1)/2$
(注意第二个循环)
$\Omega (n^2),O(n^2),\Theta (n^2)$

How many elements are examined in worst case?(二分查找)

// Return position of element in sorted
// array of size n with value K. 
int binary(int array[], int n, int K) {
     
     
int l = -1;
int r = n; // l, r are beyond array bounds
while (l+1 != r) {
     
      // Stop when l, r meet
int i = (l+r)/2; // Check middle
if (K < array[i]) r = i; // Left half
if (K == array[i]) return i; // Found it
if (K > array[i]) l = i; // Right half
}
return n; // Search value not in array
}

$T(n)=T(n/2)+1，wheren>1andT(1)=1$ (比少一半多了一次‘二分’过程)
Therefore, $T(n)=lo{g}_{2}n+1$
Cost is $\Theta (lo{g}_{2}n)$

Other Control Statements

• While loop : Analyze like a for loop
• If statement : Take greater complexity of then/else clauses
• Switch statement : Take complexity of most expensive case
• Subroutine call : Complexity of the subroutine
• Reduce time but sacrifice space
  • Encoding or packing information
  • Table lookup