First, service-oriented architecture (SOA Service Oriented Architecture)
RPC stands for Remote Process Call, Remote Procedure Call.
No matter what type of data must be converted to binary transmitted on the network. The serialized object into binary called objects, objects will be referred to as binary recovery deserialization.
Hessian is much higher than the built-in java serialization efficiency.
Or converted into json xml
HttpClient Http request by sending
two main url link style, a style is RPC, one is REST style.
The RPC service interface name url indicated require a remote call parameter in the http request directly, the service parameters required;
the REST Http request by the corresponding POST, GET, PUT, DELETE crud corresponding to complete the operation.
POST creation, GET access to information, PUT update, DELETE delete
Distributed application architecture system for reuse business logic demand is very strong, top business wants to borrow an existing underlying services to quickly build more and more rich applications. Public traffic is spun off, a common form of service, security code and logic greatest degree of multiplexing, to avoid duplication created, this design is called SOA.
Request arrives, the request is evenly distributed to the backend server, a load balancing server from the address list corresponding to the service by the corresponding balancing algorithms and rules, select to access a server, this process is called load balancing services.
Common load balancing algorithm includes a polling method, a random method, the original address hashing, weighted round-robin, weighted random method, minimum connection method or the like.
Polling method (Round Robin): turn requests sequentially assigned to the backend server, the server balanced treated, and do not care about the actual number of connections the server load. (Need to save the polling location, you need to lock, affecting system throughput)
randomly (Random): Depending on the size of the value of the back-end server list a randomly selected, based on the theory of probability and statistics, as the number of, getting closer and closer average distribute traffic to the backend server.
Original Address Hash (Hash): Get ip address accessed by the client, a value obtained by a hash algorithm, the server list length modulo operation with the value,
Weighted round robin method (Weight Round Robin): a high-profile, low-load machine configuration higher weight, more requests allowed, low load configuration of the machine is reduced.
Weighted random method (Weight Random): Depending on the configuration and load of back-end server, configure different weights.
Minimum connection method (Least Connection): backend server requests fast or slow, depending on connection, select the minimum number of connections the server to process the request.
Services Gateway
Second, the distributed system infrastructure
Distributed session: the session unified storage in a distributed cache, can ensure high read and write performance.
Mysql expand
a business split
2, replication strategy
with the increase in traffic, increasing the pressure of a library, you can copy data to a database server, Mysql cluster by accessing any one server, can read the same data.
3, sub-library sub-table
when the number of records in a single database table level of reach tens of millions or even one hundred million and the database is facing an extremely high concurrent access needs to be done to expand the handling capacity of the table.
Reduce the number of records in a single table to reduce the time required for the query, the database provides throughput.
Sub-table can solve a single-table query data is too large, reduced efficiency problems, but can not resolve concurrent read and write access. Split the database to improve the write function of the database, which is the sub-libraries.
3, Internet security architecture
security algorithms include digest algorithms, symmetric encryption algorithms, asymmetric encryption algorithm, information coding.
Digest algorithm: MD5, SHA, Base64 (reversible, insecurity)
symmetric encryption algorithm (secret key and security related): DES, 3DES, AES
asymmetric encryption algorithms: requires two keys, a public keys, a private key . RSA algorithm for
digital signatures: MD5withRSA, SHA1withRSA
Digital certificate includes the name of an object, the certificate expiration date, the certificate authority, authority digital signature certificates, public-key signature algorithm object
4, the stability of the system
log analysis commonly used commands: 1) to view the file content ab.txt CAT;
2), the access.log page display file less;
. 3), displays the file tail -f file File This parameter is used to monitor growth; -n Number Number line positions read from the file specified -f -n2 the access.log tail;
. 4), the file header head -n20 access.log display;
5), sort the contents in accordance with the size of the sort -n arrayed numeric values -k -t -r parameter specifies the column separator in the reverse order to sort -k 2 -t Sort '' -n the access.log
. 6), the character counts the number of bytes -l -c statistics statistics statistics number of character rows -m -w statistics -l words the access.log WC
. 7), the number of repeated rows or rows -c -count repeated view is displayed next to each column; -u or -unique ranks show only once; -d or only -repeated display recurring uniq ranks of the Sort | uniq -c -u
8), string search -c: display the total number of rows to be matched; -n: display the line number; -color: the match to the content highlighted in color grep -c qq access.log show support for regular expressions
9), file finder find path -option [-print] [-exe Command -OK C] {};
path: find command to find the directory path; -print: find command output file matching to the standard output; -exec: find command shell command file matching the given parameter
-name filename # find the file called filename
. find -print prints all files in the current directory
the Find ./name/log -name access.log
10), archive -c: establish compressed file; -f: Specifies the name of the package; -v: displays all process
tar -cf aa File god .tar
tar -xf aa.tar
11), the tool url to access the
curl [option] [url]
-A / -user-agent provided to a user agent server
-b / -cookie <name = string / file> cookie string or file reading position
-c / -cookie-jar after the operation is written to the cookie file in
-C / -continue-at breakpoint revolutions
-D / -dump-header information of the header is written to the file
-e / -referer URL
-f / -fail http error is not displayed when the connection fails
-o / -output written to the output file
-O / -remote-name is written to the output file, the remote file name retained
-r / -range retrieved from the HTTP / 1.1 or FTP server byte ranges
-s / - silent silent mode. Does not output anything
-T / -upload-file upload files
-u / -user <user [: password ]> set the server's user and password
curl http://www.linux.com >> linux.html (use the linux redirection save)
curl -o linux.html http://www.linux.com (curl can use the built-in option: -o (lowercase) to save the page)
curl -o / dev / null -s {HTTP_CODE% -w } www.linux.com (page return value test)
12 is), Cut
-d custom separator, default tabs.
The -f and -d together, specify which areas show
cut: in some way dividing line of the file in accordance with the
cat song.txt | cut -f 1 -d " "
5, data analysis
chapter to introduce and solve the problem:
1), distributed system log collection system architecture;
2), real-time streaming data analysis by Storm;
3), offline data analysis by hadoop, by Hive data warehouse;
4), the data import relational database HDFS, has been introduced into the HDFS data into a relational database;
5), will analyze the data presented to the user in the form of a good pattern.
Before data analysis, it must first log on to the individual operating systems to collect. Will collect good data is sent to a unified system for analysis and processing, filter out valuable content, visualization show.
For log collection, the most common way is polling. By setting a time interval, continuously reads the file until the tail. Inotify API mechanism for detecting file system changes. Inotify be used to detect a single file, the entire directory can be detected. When the object detection is a directory, the directory itself and the contents of the directory will become the object of detection.
The purpose of this mechanism is to appear when certain events occur kernel space, you can immediately notify the user space. To help consumers make specific operation.
Eight stages of the development of architectural design patterns
first stage, is the traditional architectural design patterns early stage of development
The second stage is the architectural design mode of Internet products prevailed
third stage, cloud computing architecture model based on cloud computing product
The fourth phase, which is calculated based on architecture design patterns
Micro Services Architecture
Micro service advantages: a small team can be developed separately; can use a different language development; are loosely coupled; instant requires expanded; each service has its own micro-storage capacity, it can have its own database. There may also be consolidated database.
Disadvantage of micro-services architecture: When increasing the number of services, management complexity increases; tracking difficult
Nine, 15 universal Architecture Principles
N + 1 Design: development system when a failure occurs, there is at least one redundant instance
rollback design: Ensure that the system is backward compatible.
Disabling design: publishing feature can be turned off any
monitor Design: The design phase should consider monitoring, rather than after deployment.
Live data center design
using sophisticated technology
fault isolation: Avoid single business occupy all resources. Avoid interaction between business 2. room isolation to avoid single points of failure
horizontal expansion: With the development of business, when you need to expand the service capabilities of the platform, without having to reconstruct the software system to meet the needs of business growth by adding new equipment .
The purchase of non-core
use commodity hardware
fast iterative
asynchronous design
stateless design
prospective design
automation
A good design:
1) to address existing needs and problems
2) to control the reality of progress and risks
3) predict and plan for the future, do not over-design, from iteration evolution and improvement.
Architecture design needs analysis: The main purpose is clear architecture to solve the current problem, to study the demands of the demand side.
How to start designing an architecture: Business -> Features -> technology -> architecture overview diagram
business architecture: determining the overall architecture, core processes, is the top strategic framework including business planning, business modules, business processes, the whole system the business split, the design of the domain model, the reality of the business transformed into abstract objects. All systems design principles to solve business problems must be the ultimate goal, from the actual business of technology infrastructure feelings tend to give into the pit system, not based on any business done whimsical architecture bullying.
Application Architecture: determination subsystem range of functions and solutions division: This application is the so-called individual logic modules or subsystems.
Stratified two ways:
one is the horizontal component (horizontal), the processing order is divided according to function application, such as the front end of the web into the system / intermediate service / background task, which is divided for operational depth.
Another sub-vertical (longitudinal), is divided according to different types of service applications, such as invoicing system can be divided into three separate application, which is divided into business-oriented breadth.
Technical Architecture: technical research to determine the core technology of the system point
For example, we want to design a micro-order system services:
1. Business: Determine the business process: determining the order key function points and processes
2 application: determining the order top-level design, system module, which interfaces interface protocol in the form of external exposure
3, Technical: to determine which technology point of use: mysql, mongo, consider whether sub-table and warehouses which use middleware.
4, the data: how to design a table structure
5, detailed design:
There were common principles: the right principle, simple principle, evolution
appropriate principles: the right superior industry-leading
simple principle: Simple is better than complex
evolutionary principles: one step better than evolution
Premise to figure out all the problems of traffic we face today how much, what kind of trend growth, and high concurrent resolution process, it must be a gradual process step by step.
Hain law: accidents are the result of the accumulated amount.
Murphy's Law: Anything not surface looks so simple.
· The development of all the things you will be longer than the expected time.
· Something wrong will go wrong will go wrong.
If you are worried about a situation occurs, then it is more likely to occur.
This requires that we have the technical ability to research, to have faith any phenomenon following principles: why is this happening? It occurred in how to deal with? How to restore? How to avoid? To conduct a thorough investigation of the problem, not because of the phenomenon of ignoring the problem is not obvious
Typical layered architecture design:
Access Layer: The main entrance flow through the simple
application layer: Foreign direct offer product features, such as Web sites, API interface. The application layer does not contain the complex business logic, only rendering and conversion.
Service layer: The service field of a single service each subdomain, divide and conquer.
Data layer: the database and NoSQL, file storage and the like.
Let us list the current system which links, each link is weak if the Client Access server, through many links, any part of problem, can not access:
Access layer:
. 1, hijacked DNS: domain name is for https.
2, hacking: if there is a weak adhesion, the server authority, permission database
. 3, DDoS attacks: whether it is necessary to use a high anti-IP access traffic.
4, CC attack: separate free and paid version of the domain name, the gateway if current limiting and anti-brush measures.
Application Layer:
1, the application server downtime.
2, application service bug.
3, third-party service is unavailable.
Service Layer:
1, the service is not available or there is bug
2, third-party service is unavailable.
Data layer:
1, the database server database is unavailable disk damage caused by other
Highly available database schema
quality of service management:
1, standardized service management: CMDB project, services, servers for unified management.
2, automated publishing: Publishing does not affect the user, improve the release process, automates the release, in time rollback.
3, automated testing: Full automated testing on the line after completion.
4, performance pressure test: The pressure test of the service, the service can carry understand concurrency that allows for server operation and maintenance by the expansion of early warning.
5, code control: the test environment using the testing branch, beta environment release tag, use the tag line publication.
6, the publishing process: on-line publishing process specification.
7, gray Published: gray Publishing Service.
8, emergency response mechanism.
Sound alarm monitoring mechanisms:
1, network traffic monitoring.
2, system monitoring: server resources and network-related monitoring (CPU, memory, etc.)
3, log monitoring: a unified log collection (various services) monitoring, tracking (log2).
4, application monitoring: Port survival, the process takes the resource, and other applications FGC
5, service monitoring: service interface logic function is normal
6, three-dimensional surveillance monitoring data collected after, except for the system performance evaluation, clustering flexibility and scale to predict the ultimate goal is also the risk of early warning data based on real-time monitoring, and server failover, automatic load adjustment, maximize the use of resources of all cluster machines.
Architect Responsibilities:
1, high availability architecture design: including business processes, combination module division, frame design, process flaws, and finally architecture design, technology step. Systematic thinking, on balance, various factors, forward-looking design architecture.
2, and operation and maintenance coordination and communication, made efficient service management solutions to control service quality management.
3, coordination and communication: communication between developers, communication between the product, marketing communications, operation and maintenance of communication, documentation and design graphical output after communication.
4, regulate and co-ordinate: to ensure orderly system, a unified, standardized, stable and efficient operation.
R & D responsibilities:
1), architect of the architect involved in the design and realization of details according to the design.
2), a self-test function for the development, pressure measurement.
3), the development of the code, the use of tools or components conform architect norm. Including coding standards, documentation standards.
4) code deployed in line with the deployment operation and maintenance specifications.
Ten, API Gateway
API request reaches the gateway needs to undergo a rigorous identity authentication, certification authority, in order to reach the back-end services. Support signature algorithms, support for SSL encryption.
API may be controlled within the allowable number of times per unit time is called.
API gateway is a server, only the inlet system. From the perspective of object-oriented design, it is similar to the appearance model. API Gateway encapsulates the internal system architecture that provides a custom API for each client. It may also have other functions, such as authentication, monitoring, load balancing, caching, and fragment management request, the static response processing.
XI, the architect of 12 practice
1, become elegant and professional way:
1) focus on the relationship between who is right even in the dispute (the relationship is everything, which determines which projects or work can be carried out, people also decide for you the highest priority project);
2) learn to delegate (to allow others to participate in solving a problem)
3) recognize that life is a reflection of (to take life more active)
4) effective communication of life and death (published and listen to positive the impact of discourse)
5) integrity, honesty and not straightforward (to not to offend way to express what you mean)
6) Do not cover up the problem
7) provide professional services
8) forget the previous offense
2, the principles of communication
1) listen before talk
2) dedicated
3) Positive Thinking
4) as soon as possible to apologize (for others to do certain things inappropriate or incorrect, put down the self-esteem to the affected humanitarian apologize)
5) Do not the defect leading to angry
2.2, communication strategy
1) say that less than
2) to listen to proposals to improve cooperation
3) to understand other people and their communication needs
3, the principle of negotiation
1) Do not surprising
2) Do not equivocal
3) delegate authority but not the obligation (whether you decide to delegate a person to do, you still have to be responsible for the consequences of those consequences)
4) difficulties to seek help
5) do not cover up the problem
6) instant difficult, but also insist on doing the right thing
3.2 Negotiation Strategy
1) listen to inner voice
2) managed to agree
3) Do not look for differences
4) to find common ground
5) If agreement can not, let everyone a little dissatisfied with it
6) will be negotiated as an improvement measure
