URL: http://www.dbanotes.net/arch/google_app_engine-intro.html
Note: This is a guest blog post series. Contributor Wu Zhu Hua had engaged in related research and cloud computing at IBM China Research Institute is now working on cloud computing technology.
By the two previous reports, we should have some knowledge of Google's powerful infrastructure. Benpian began to introduce Google App Engine to build on this strong infrastructure.
Google App Engine Introduction
Since the release of these two outstanding S3 and EC2 cloud services, Amazon has taken the lead in making cloud computing market a firm footing, and as this wave of cloud computing Google is certainly one of the sponsors not to be outdone, and in April 2008 the launch of the Google App Engine PaaS service, although it can not be called a revolutionary product, but it is certainly the most mature market now, and the most comprehensive PaaS platform.
Google App Engine provides a complete set of development components to allow users to easily build and debug web applications locally, and then allows users to deploy and run web applications on Google's powerful infrastructure and automatically depending on the application of a load to be applied extended, eliminating the need for maintenance and user applications and servers work. At the same time provide a lot of free credit and flexible standard tariff. In the development of language, now supports both Java and Python languages, and provides essentially the same functionality and API for both languages.
Features
Functionally, there are six main areas:
- Dynamic Network Services, and provides support for common network technology, such as SSL and so on.
- Persistent storage space, and support simple queries and local affairs.
- It can automatically expand and load balancing for applications.
- A full-featured local development environment that allows users to App Engine development and debugging on the unit.
- Email support includes user authentication and other services.
- Scheduled Tasks can provide a specified time and on a regular basis to trigger events and enables background processing tasks in the queue.
manual
Throughout the process includes five steps:
- Download the SDK and IDE, and to set up a local development environment.
- Develop and debug the application locally.
- Use GAE comes upload tool to deploy applications to the platform.
- Start the application in the management interface.
- To monitor the operating status and tariff entire application using the management interface.
Since this series is to focus on achieving both GAE and design, so it will not use GAE has a very in-depth reports, if you want to use for GAE terms of a deeper understanding of specific GAE can see what the official documents .
The main part of the Google App Engine
It can be divided into five parts:
- Application Server: mainly used to receive from the external Web requests.
- Datastore: mainly used for persistent information, and based on the famous Google's BigTable technology.
- Services: In addition to the necessary application server and Datastore, GAE also comes with a lot of services to help developers, such as: Memcache, e-mail, web crawling, task queues, XMPP and so on.
- Management Interface: The main application used to manage and monitor the operation status of the application, for example, the amount of resources consumed, send e-mail and log how many applications are running and so on.
- Local development environment: help local users develop and debug applications based on GAE, secure debug sandbox, SDK and IDE plug-ins and other tools include.
application server
The application server support in accordance with their different languages and have different implementations.
Python implementation
Python basic version of the application server is an ordinary Python version 2.5.2 Runtime, and consider adding support for Python 3 in in future releases, but since Python 3 to Python, like in the Java2 Java1, very large span , it is very difficult to introduce Python3 of. In Web technologies, support for Python Web frameworks such as Django, CherryPy, Pylons and Web2py, etc., and comes with the name "WSGI" the CGI framework. Although the Python version of the application server is based on the standard Python Runtime, but for safety and to better adapt to the overall architecture of App Engine, to run code in the application server sets a limit in many ways, such as can not load modules written in C and Python You can not create Socket and so on.
Java implementation
In terms of realization, Java version of the application server and Python version of basically the same, but also standards-based Java Web container, the Jetty and selected lightweight technology, and run on Java 6. Through this Web container to run not only the common Java Web technologies, including Servlet, JSP, JSTL and GWT, etc., and can run most common Java API (App Engine has a The JRE Class White List to define who can in Java API used App Engine environment), and some JVM based scripting languages, such as such as JavaScript, Ruby or Scala, but the same can not be created Socket and Thread, or to read and write files, does not support some of the more high-end API and framework , including JDBC, JSF, Struts 2, RMI , JAX-RPC and Hibernate.
Datastore
Datastore provides a set of powerful distributed data storage and query service, and can scale horizontally to support the vast amounts of data. But Datastore is not a traditional relational database, it is mainly in the "Entity" of stored data, an Entity including a Kind (conceptually and Table database of more similar) and a series of attributes.
Datastore provides strong consistency and optimism (optimistic) synchronization control, but in matters, it supports local transactions, ie transactions can only be performed within the same Entity Group.
In the interface, Python version provides a very rich interface, but also includes a query language called GQL, while the Java version provides a standard JDO and JPA two sets of API.
And Google has announced SQL database service will be included in future standard of App Engine for Business Suite at this year's Google I / O conference, but now is not sure that the implementation of the SQL database, MySQL is based on open source technology, or based on its proprietary realize that this is a problem.
service
Memcache
Memcache is a medium-sized site backup service, mainly used in memory to store frequently used data, and also includes the App Engine service. Interestingly App Engine's Memcache is developed by the Brad Fitzpatrick.
URL fetch (Fetch)
App Engine applications can fetch this URL resource services to crawl the Internet, and this may serve to communicate with other hosts. This avoids the application can not be used Socket in Python and Java environments embarrassment.
App Engine applications use this service to take advantage of the infrastructure Gmail to send email.
Scheduled Tasks (Cron)
Planning services allows the application at a specified time or at specified intervals to perform its tasks set. These tasks are often called Cron job.
Graph
App Engine provides a dedicated service to the image data of the image operation functions. Image service can adjust the image size, rotate, flip and crop images. It is also possible to use a predefined algorithm to improve the quality of the picture.
User Authentication
App Engine applications can rely on the Google Accounts system to authenticate the user. App Engine will support OAuth.
XMPP
Programs running on App Engine can communicate using XMPP XMPP compatible service, and other IM services (such as Google Talk).
Task Queue (Task Queue)
App Engine applications can insert in a task queue (in the form of Web Hook) to implement background processing, and App Engine will be arranged inside the task execution queue scheduling according to the setting area.
Blobstore
Because the data object storage support up to 1MB in size Datastore, so App Engine Blobstore launched a service to store and recall those binary data objects larger than 1MB but less than the 2G.
Mapper
Mapper can think is "Map Reduce" in the Map, that is, through the Mapper API for large-scale data parallel processing, these data can be stored in Datastore or Blobstore, but this feature is still in the development stage inside.
Channel
In fact, we often say that Channel's "Comet", by Channel API allows applications to push content directly to the user's browser, without the need for frequent polling.
In addition to Java version of Memcache, Email and URL are crawled using a standard API, other services either Python or Java edition version of its API is private, but provides a rich and detailed documentation to help users.
Management Interface
Used to let users better manage applications, Google provides a set of comprehensive management interface, the address is http://appengine.google.com/, and only the user's Google account will be able to log in and use. The figure below for screenshots:
Figure 1. Management Interface (click to enlarge)
Using this interface, you can manage many operations, including the creation of new applications, application settings for this domain name, view and access data and error logs, observe the usage of major resources.
Local development environment
For security reasons, local development environment uses a sandbox (Sandbox) mode, basically, and to restrict the application server mentioned above is almost like not create Socket and Thread, also can not read and write files. Python App Engine SDK version in the form of regular application of the release, the local need to install the Python Runtime, start the Python version of the Sandbox through the command line, but can also start in the Eclipse PyDev plug-in installed. Java version of the App Engine SDK is Eclispe Plugin release form, as long as the user to install the Plugin on his Eclipse, users will be able to start a local Java sandbox to develop and debug applications.
Programming Model
Because App Engine primarily to support Web applications exist, so the Web tier programming model for App Engine is the most critical. Web App Engine model primarily using CGI, CGI called the "Common Gateway Interface", it means very simply that a request is received from a process or thread to handle the request, after the end of treatment when the automatic processes or threads Close, followed by constantly repeating this process. Because when CGI in this way each treatment, we must re-starting a new process or thread, it can be said in terms of resource consumption is still very powerful, although there is a thread pool (Thread Pool) such optimization techniques. However, due to the simplicity of CGI in the framework of GAE become the preferred programming model, and because no state support CGI mode, so scalability is also very advantageous. And two language versions of App Engine comes with a CGI framework: in Python platform for WSGI. In the Java platform, compared with the classic Servlet. Recently, App Engine the introduction of scheduled tasks and task queues these two characteristics, so App Engine already supports background processes and scheduled tasks two programming models.
And tariff restrictions
First of all, talk about the use restrictions App Engine, in particular see the following table:
| category | limit |
| Each writer has in the project | 10 |
| The number of files per project | 1000 |
| The size of each project code | 150MB |
| Each request to perform the most time | 30 seconds |
| Blobstore (binary storage) size | 1GB |
| HTTP size of Response | 10MB |
| The size of each object in the Datastore | 1MB |
Table 1. App Engine Limitations of
Although these restrictions on developers is an obstacle, but for such a multi-tenant environment App Engine and it is very important, because if too many resources consumed by a tenant's application, it will affect the normal use of applications in the near , and App Engine is above these limits is to run applications can run safely for the sake of its platform at the top, to avoid the situation of a significant resource application or malignant influence to close the application. After the addition of security considerations, as well as scalable reason, that is, when the space occupied by an application (footprint) at a low state, such as less than 1000 less than 150MB file size and so on, it can be very convenient telescopic be implemented by copying applications.
Then, talk about the tariff, the tariff situation App Engine has two main characteristics: One is the high amount of free, free of existing lines can support the operation of a medium-sized site, and does not pay any fees. The second project is a very fine-grained tariff, common IaaS service charges, mainly CPU, memory, disk and network bandwidth these four, and App Engine is in addition to the common CPU and network bandwidth these two, but also includes many application-level projects, such as: number of calls and other messages Datastore API and API. Specific tariff mechanism is this: If the user's application of resources consumed every day less than this amount, and that they at no cost to the user, but when the free quota is exceeded, the user needs to pay for the part in excess. Because App Engine package of standard tariff is more complex, so here mainly to introduce its free credit details, please read the following table:
| Types of | The number of (daily) |
| Mail API calls | 7000 |
| Outgoing (outbound) bandwidth, | 10G |
| Incoming (inbound) bandwidth, | 10G |
| CPU time | 46 hours |
| HTTP request | 1.3 million order |
| Datastore API | 10 million order |
| Stored data | 1G |
| URL fetch API | 657 thousand |
Table free credit of Table 2. App Engine
Free credit from the above point of view, in addition to the capacity to store data, the other is very strong.
Benpian end of next App Engine architecture will be introduced.
--EOF--
Reproduced in: https: //www.cnblogs.com/licheng/archive/2010/09/09/1821970.html