Data Management Technology(1) -- Introduction

(1)Data & Information

Information

Is any kind of event that affects the state of a dynamic system
Is the message (utterance or expression) being conveyed
Is an ordered sequence of symbols that can be interpreted as a message

As sensory input: Often be viewed as a type of input to an organism or system

Can be recorded and transmitted

Data

Definition

values of a qualitative or quantitative variables, belonging to a set of items
Used to record information
Is the carrier of Information

Data type and Data value

Data Type: the way values of the data can be stored in computer system
Data Value: records the meaning of information

Data & Information

Data on its own carries no meaning

Data must be interpreted to take on a meaning, so that the information can be revealed

Data is the lower level of abstraction; is the carrier of information

Information is the interpretation of data

(2)Data Processing VS Data Management

Main categories of Data Manipulation

Data Management: Database

Data Processing: Computer program

Data Transmission: Computer Network

Tasks of Data Management

Data Storage: Organize the data and store them into the storage device such as hard disk；

Data Maintenance: Insert new value, delete invalid data or modify old data；

Data Query & Data statistic: Retrieve information from the data storage

Application Requirements

store the data for a long period of time

large amounts (100s of GB)
protect against crashes
protect against unauthorized use

allow users to query/update:

allow several (100s, 1000s) users to access the data simultaneously

allow administrators to change the schema

Trying Without a DBMS

Storing data: file system is limited

size less than 4 GB (on 32 bits machines)
when system crashes we may loose data
password-based authorization insufficient

Query/update:

need to write a new C++/Java program for every new query
need to worry about performance

Concurrency: limited protection

need to worry about interfering with other users
need to offer different views to different users (e.g. registrar, students, professors)

Schema change:

entails changing file formats
need to rewrite virtually all applications

Schema Versus Data

schema: describes how data is to be structured

defined at set-up time

rarely changes

also called “meta data”

data is actual “instance” of database, changes rapidly

vs. types and variables in programming languages

DBMS

Data Definition Language – DDL

Easy to define schema

Data Manipulation Language - DML

query language

Storage management

Retrieve data from disk automatically for you

Transaction Management

concurrency control
recovery

Automate a lot of boring/mundane operations on data

so that we don’t have to program over and over
so that we can write complex data manipulations in just a few lines, so that we can concentrate on app logics

Make execution very fast

so that it scales up to very large data sets

Make concurrent access/modification possible

so that many users can use the data at the same time

Building an Application with a DBMS

Requirements modeling (conceptual, pictures)

Decide what entities should be part of the application and how they should be linked.

Schema design and implementation

Decide on a set of tables, attributes.
Define the tables in the database system.
Populate database (insert tuples).

Write application programs using the DBMS

way easier now that the data management is taken care of.

Querying a Database（查询数据库）

Database Technology

Started from 1960’s

Is an important branch of CS

Programming language
OS
DB
Network

Is the main component of computer infrastructure

Main Functions of DBMS

Data Definition数据定义

Provides Data Definition Language to define schema of database

Data Manipulation数据操作

Provides Data Manipulation Language to manipulate data in database: RETRIEVE, INSERT, DELETE, MODIFY

Database operation

Security
Integrity
Concurrency
recovery

Toolsets

Data loader
Monitor
Performance tuning tools

(3)Database

Efficient, convenient, and safe multi-user storage of massive amounts of well organized persistent data

(4)Database Management System

A Software System that manages database
Buy, install, set up for particular application

(5)Database System

DBS, information systems that based on database

Consists of database, DBMS, application, and users

硬件<OS<DBMS<App development tools<Database system

2.Development of DB tech

Stages

(1)Manual processing

Data stored in punched-cards
Data managed by hand
No data sharing

(2)File system

Data stored in files
Use OS IO interface to access data
Measures taken to accelerate the data access
Primary data-program independence

Drawbacks of file systems

Program-Data Dependence

All programs maintain metadata for each file they use

Data Redundancy (Duplication of data)

Different systems/programs have separate copies of the same data
Multiple file formats, duplication of information in different files
- Requires space, effort and result in loss of data & metadata integrity

Limited Data Sharing

No centralized control of data
- Each application has its own private files & users has little chance to share data outside their own applications

Lengthy Development Times

For each new application programmers must design their own file formats & descriptions from scratch

Excessive Program Maintenance

80% of information systems budget

Difficulty in accessing data

Need to write a new program to carry out each new task

Integrity problems

Integrity constraints
Hard to add new constraints or change existing ones

(3)Database

Main advantages compared to file systems:

data sharing

Less data redundancy

Data- program Independence

Convenient program interface

Efficient data access

Data integrity and data security

Concurrency management

…

Database advantages

Structured data storage

Organize the data and link them together by their inner relations
Automatically manage the data relationship

Data sharing

One copy of data for many applications
Allow many users to access data simultaneously

Less data redundancy

共享数据（对应关系变成表格关系）

Data integrity

Automatically check the input value of certain data items, according to the data integrity rules

Concurrency并发性

Isolate the concurrent accesses
Prevent the dirty use of data
- The modification that may jeopardize the integrity of data

3.Database Architecture

(1)Database Architecture (physical architecture)

The architecture of a database systems is greatly influenced by the underlying computer system on which the database is running:

Centralized集中的
Client-server客户端-服务器
Parallel (multi-processor)并行（多处理器）
Distributed分布式

(2)Database Users

Users are differentiated by the way they expect to interact with the system

Application programmers – interact with system through DML calls
Sophisticated users – form requests in a database query language
Specialized users – write specialized database applications that do not fit into the traditional data processing framework
Naive users – invoke one of the permanent application programs that have been written previously. Examples, people accessing database over the web, bank tellers, clerical staff

(3)Database Administrator

Coordinates all the activities of the database system; the database administrator has a good understanding of the enterprise’s information resources and needs.

Database administrator’s duties include:

Schema definition
Storage structure and access method definition
Schema and physical organization modification
Granting user authority to access the database
Specifying integrity constraints
Acting as liaison with users
Monitoring performance and responding to changes in requirements