client/server architecture
Before looking at HTTP, we will briefly introduce the properties of the environment in which they operate. This is often called a client/server environment.
A simple client/server system has one or more client processes and a server process, as shown in the following figure:
Although this seems simple, there are several properties that make it more complex than expected. Clients are independent entities running on many different computer systems. this
This means we need to consider the following:
1. The client process must be compatible with the client operating system. Things like support for languages and data communication protocols must be available and compatible.
2. Security is an issue because remote clients must be authenticated before they are allowed to access server information.
3. Many different client computers means that eventually, one or more will break during some client/server communication. This requires the facility to recover from the situation.
4. If the server can be accessed by many clients, the server's availability and error recovery need to be supported, especially if the client/server relationship is a commercial customer/vendor relationship.
5. Response times for client/server interactions must be acceptable and manageable.
6. In the Internet scale of networks, we also have to consider how clients find servers among the millions of servers they can access.
We must also realize that the simple client/server architecture pictured above does not mean that all clients are on different machines. In a client/server architecture, it is perfectly allowed for one or more client and server processes to be on the same computer system.
Modern client/server systems are even more complex. The diagram below illustrates how we interact with any combination of clients and servers in a computer system. Multiple clients coexist with multiple servers on any one computer system. Additionally, client processes can access multiple servers to achieve their goals, and servers can act as clients to satisfy their own client needs.
Now you will notice that in both diagrams above, the term middleware under the arrow represents client/server interaction. Middleware is the software component required by a client/server system to achieve the above goals. It is the software between the client program and the operating system and the server program and the operating system. It isolates client and server programmers from the complexities related to compatibility, security, mutual positioning, etc. by providing a standardized interface.
The middleware we'll cover is Hypertext Transfer Protocol (HTTP) and the infrastructure used to support it. Due to the nature of this unit (mobile system) we will be mainly interested in the client side of the infrastructure. However, understanding what is happening on the server side is crucial to better understanding the limitations and advantages of this technology.
Network Server
A web server is the server side of an HTTP-based client/server system. They are now the dominant server technology on the Internet, mainly due to the popularity of the Web and developments in network software design. Although "hypertext" in HTTP was originally an HTML document, there are now many more data formats used in HTTP, some of which we will examine in this unit. In this section, we cover basic web server operations. In the following sections, we modify HTML into a document markup language and discuss how HTML links form the basic interaction with a Web server. We'll then look at the <form> tag because it provides a way for the client to send data to the Web server, allowing the Web server to construct a unique page to send back to the client. Then, we'll look at a simple CGI system that allows a Web server to execute programs that build pages for clients. Some other modern technologies that have replaced CGI for high-volume web servers, PHP, and ASP will be examined.
The web server is the server part of the client/server that interacts with HTML pages. In its simplest form, it simply accepts a request for an HTML page and sends a stored copy of the page to the requesting client. However, modern web servers are much more complex than this simple interaction suggests.
The steps involved in a typical web server interaction are as follows:
1. A web client (usually a web browser) sends a web page request over the Internet.
2. The server validates the request to ensure that the page exists, any required software is installed, and security constraints are met.
3. Any software required by the server to execute client requests.
4. The server collects the HTML page from the server file system or as output from the program.
5. The server sends the HTML document back to the client over the Internet.
Steps 1 and 2 use the HTTP protocol to send requests to the server and HTML pages to the client. HTTP is a very simple protocol that uses REQUEST and RESPONSE messages to interact between clients and servers. While we don't need to know the details of HTTP, it's useful to see the general format of these messages. The following table shows the general form of HTTP request messages and sample messages.
| General form of HTTP request | Sample request |
|
method resource_id HTTP_version
header:value
…
header:value
blank_line
message_body
|
GET /jdk1.3/docs/index.html HTTP/1.0 Accept: text/html User-Agent: MacWeb |
In this example, the method "GET" is one of the possible HTTP methods. The HTTP version is 1.0, in this case, there is no message body. Note that the resource identifier is simply the file name of the resource on the server. The server's computer name is not required because to send a message, the client must already find the server name to know where to send the message.
The following table shows the general form of an HTTP response and a real example response containing an HTML page.
| General form of HTTP response | Sample response |
| HTTP_version result_code result_comment header:value … header:value blank_line message_body |
HTTP/1.0 200 OK Server: Apache/1.3 Mime_version: 1.0 Content_type: text/html <html> … </html> |
This HTTP response uses protocol version 1.0 and returns result code 200. Its header/value pairs indicate that the server is an Apache version 1.3 server, and an HTML page is sent back in the message body.
You have now encountered Universal Resource Locators (URLs). For example, the following URL identifies a web page named spike.scu.edu.au located on a web server at /jdk1.3/docs/index.html, which is accessible via the HTTP protocol.
http://spike.scu.edu.au/jdk1.3/docs/index.html
The HTTP request message only requires the file location on the web server because the protocol (HTTP) is selected by the first part of the URL and the server has been found using the computer name in the URL, in this case spike.scu.edu.au. You may have seen other protocols such as HTTPS and FTP (https: and ftp:) used in URLs.
Server-side programming
Server-side programming refers to the process of developing or programming directly within the server.
The benefit of using server-side programming is that the underlying data is accessible locally; no network connection or external resources are required to access the data. Additionally, embedded code running on the server is often faster than client-side code, which requires the user to access the database to access the data. We will study database access using NodeJS in the Week 4 module.
Additionally, server-side programming allows code to be incorporated directly into the database. Code can be managed and restored using standard database backup tools, just like any other database object.
In this section, we will discuss NodeJS in the context of a server-side programming language and explore the concepts required to build our first server application using NodeJS. But before that, we'll revisit some important topics like HTTP, the API needed to build our first server application.
Hypertext Transfer Protocol (HTTP)
As part of this module, you will need to understand the Hypertext Transfer Protocol (HTTP), a protocol used by computer networks. There are other protocols such as SMTP (Simple Mail Transfer Protocol), FTP (File Transfer Protocol), POP3 (Post Office Protocol 3), etc.
Protocols provide well-defined message formats, rules, syntax, semantics, etc., allowing devices with different hardware and software to interact with each other. Any device that is compatible with a specific protocol can communicate with any other device on the network.
Hypertext Transfer Protocol (HTTP) is arguably one of the most widely adopted Internet application protocols in history. It is the method by which clients and servers communicate over the Internet. Aside from browsers, HTTP is the protocol of choice for almost every application that connects to the Internet.
The fact that most operating systems include network protocols such as HTTP by default means that we do not have to install any additional software to access the Internet.
The HTTP protocol that drives the Web is called a connectionless protocol. It is the protocol used by web browsers to communicate with web servers. Every time a link is clicked, a form is submitted, or a search is performed, a request is sent from the browser to the target server. After requesting images, videos, and text from the server, the browser and server are no longer connected. This is because it operates on a request/response model.
When submitting a request, you must include the URL that identifies the resource and the method that will be applied (such as retrieve, delete, or publish). Additional information can be included in the form of URL parameters (field-value pairs), POST data, or cookies.
The HTTP protocol sets clear guidelines for communication between clients and servers.
- Normally, HTTP requests are sent only by the client to the server. The server responds to HTTP requests from clients. By using a technology called server push, the server can also populate the client's cache with information before requesting it.
- When requesting a file via HTTP, the client must provide the URL of the file.
- The web server must respond to every HTTP request, at least with an error message.
HTML and form <form> tags
You've seen the HTML <form> tag in the previous unit. What we haven't discussed so far is how we should send data to a remote server and manage delays and errors that occur in data transfer. So as a revision, we'll revisit the normal HTML <form> tag here.
An example of the HTML <form> tag taken from the Java website (java.sun.com) is as follows:
<form action="/cgi-bin/coffee" method="post">
Would you care for a cup of coffee?<BR>
<INPUT type="radio" name="coffee" value="Y" CHECKED="checked"/>Yes
<INPUT type="radio" name="coffee" value="N"/>No<P>
If yes:<BR/>
What flavor?
<SELECT name="flavor" size="1">
<OPTION value="French Roast"/>French Roast
<OPTION value="Vienna Roast"/>Vienna Roast
<OPTION value="Almond Mocha"/>Almond Mocha
</SELECT><BR/>
<INPUT type="checkbox" name="cream" value="Y"/>Include Cream
<INPUT type="checkbox" name="sugar" value="Y"/>Include Sugar<br/>
<INPUT type="submit" value="Brew It"/>
</form>The action attribute of this <form> tag specifies a program to handle the data to be sent to the server (in this case, /cgi-bin/coffee). The web server uses this information to launch programs to process the data. The method attribute uses its post value to specify that the HTTP POST method should be used in HTTP requests to send data to the server.
This tag also constructs a set of visual controls that allow the client user to select values to be sent to the Web server by interpreting tags separated by <form> and </form>. The browser will display something similar to the image below.
In this example, the <input> tag is used to construct radio buttons, check boxes, and the "Brew It" button at the bottom of the form. The <select> tag is used to build a drop-down menu that allows the user to select the type of coffee.
Common Gateway Interface (CGI)
In the previous section, we looked at how to use the <form> tag to collect data from the user to send to the server for processing. The first server-side system to use this data was the Common Gateway Interface, always referred to simply as CGI.
CGI is a simple system in which the web server launches a program called a CGI program to process form data. The CGI program then generates an HTML web page, which the server sends back to the client. The following diagram shows the complete interaction between client and server.
This diagram shows how the form is first loaded from the Web server, possibly through a previous CGI request. The client user then operates the form and sends the form data to the server for processing. The server then starts processing the CGI program specified by the action attribute in the <form> tag, which reads the data from the client and generates an HTML page for the server to send back to the client. Finally, the client displays the generated web page.
The data is assembled by the web browser when the user clicks the submit button or overrides the submit interaction, such as executing an appropriate JavaScript function. The data is assembled into label/value pairs before being transmitted to the server. The pairs are arranged in the format tag=value, with individual pairs separated by the "&" character. The browser also replaces special characters with characters that may be transmitted over HTTP, for example, spaces are replaced with "+".
The CGI program fetches client data in the same tag/value pairs from the Web server. For example, let's say we selected the controls for the coffee example from the previous section, as shown below.
The browser will construct the following string to send back to the CGI program and use the HTTP POST method to send the data back to the CGI program:
flavor=Vienna+Roast&cream=Y&coffee=YNow you should carefully compare it with the HTML code given above. Note that the order of the tags is different from the order declared in the HTML <form> tag. Generally speaking, this is the case, CGI programs cannot rely on any ordering of tags. Also, notice how the checkbox labeled sugar is not represented in this string. Typically, this is also true when the checkbox is unchecked, i.e. only checked checkboxes return label/value pairs.
The next step in the client/server interaction is for the Web server to receive the HTTP POST request and pass the HTTP message details to the designated CGI program using environment variables. The CGI program then retrieves the environment variables to determine the pattern of the HTTP request. Note that you may have seen environment variables before, such as PATH or CLASSPATH. CGI systems use different variables, which can be retrieved by CGI programs.
In this example, the CGI program will discover that the CGI request is using POST mode so that it can read the label/value pairs from the program's standard input. The program must interpret the passed string to remove various inserted characters ("=", "?", "+", etc.). The program will build an internal data structure to contain the tags and their associated values.
CGI programs may then process the data by accessing databases, files, and other programs. Finally, the CGI program writes the HTML page directly to the program's standard output stream. This can be done in a Java CGI program, for example, by using the System.out.println() function.
When a CGI program closes its standard output or exits, the Web server sends the HTML page back to the client. If the client is a web browser, it displays the web page.
CGI has several issues that make it unsuitable for high-volume web servers:
1. The CGI system is slow. This is because the Web server must start a new program for each CGI request. Starting a CGI program requires pulling together many system resources and loading the program from the file system. Better systems avoid these problems by using programs that remain running in the computer's memory.
2. CGI is stateless. Stateless means that the CGI program does not remember which client sent the last CGI request, so it does not remember what the client did on the last request. This means that CGI interactions are typically one-screen-one-response applications, and cookies and other state-saving devices have been introduced to solve this problem.
3.CGI programming is difficult. The solution to problem 2 makes CGI programming quite complex. Better systems have been developed to make programming easier.
Representational State Transfer (REST)
Representational State Transfer (REST) has become a common method for organizing applications based on the HTTP protocol. It uses HHTP message headers and protocol attributes in a natural and highly relevant way. In this section, we will outline the main features implemented by RESTful (as it is commonly known) web services.
There are four main aspects of REST web services:
1. With HTTP, message types (GET, POST, PUT, etc.) have clear meanings;
2. Communication is stateless;
3. The directory structure of the data is exposed to the client and server; and
4. The data is transferred in XML or other MIME type specified in the HTTP header.
Use NodeJS to create a web server and load HTML files
- Make sure Node.js is installed on your computer
- You've learned the basics required to create a Web server.
- Create a new folder called "My Server"
- Create a new file named "web-server.js" in the newly created folder my-server
- Add the following code to the file "web-server.js".
const http = require("http");Next, we will define two constants, the host and port that will be assigned to the server:
const host = 'localhost';
const port = 8080Web servers handle requests from browsers and other clients, as we discussed in the previous section. By using domain names, you can interact with Web servers, which are translated into IP addresses by a DNS server. localhost is the name of the address given when the computer connects to the network, also known as the loopback address. Also known as "Internal IP address 127.0.0.1", it is only available on the local computer and not on any network or the Internet that we are a part of.
All network connected devices have standardized ports, each with a unique number. There are many ports reserved for specific protocols. For example, port 80 is the default port assigned for Hypertext Transfer Protocol (HTTP) messages.
An IP address allows the transmission of messages to and from a specific device. A port number, on the other hand, enables a specific service or application to target a specific device and act as a gateway. Here we use 8080 as our port number. If the server has been successfully set up, we should be able to access it via http at http://localhost:8080 .
After accessing the server, the next step will be to display the message. To do this we need to create a function.
const requestLis = function (req, res) {
res.end("Hello world, this is my first web server using NodeJS");
};In Node.js, all request listener functions accept two parameters, req and res. The user's HTTP request is captured in the request object, which corresponds to the first parameter req. The response object in the second parameter res (res) is used to generate the HTTP response returned to the user. The res.end() function returns the HTTP response to the client.
const server = http.createServer(requestLis);
server.listen(port, host, () => {
console.log(`Server is up and running on http://${host}:${port}`);
});Now, save the file and go to Powershell or Terminal and execute the following command
node web-server.jsYou should now be able to view the following information on PowerShell/Terminal.
Server is up and running on http://localhost:8080Load HTML files using NodeJS:
Using the fs module we discussed in the previous section, we can load HTML files and use their data when writing HTTP responses.
Create an HTML document that will be returned by the web server.
index.htmlAdd some code to the html file to make the web page attractive.
Import fs-module.js by writing the following code to your web server to use the readFile() function required to load HTML files.
const fs = require('fs').promises;To comply with modern Javascript best practices, we use a variant of promises, hence .promises.
Finally add the following code to webserver.js.
Create an HTTP server using Express (Node.js framework)
Various frameworks such as Express.js and Koa.js are available for Node.js. These frameworks provide a variety of useful middleware capabilities, as well as many other useful features that developers can leverage instead of developing themselves. In this course, we'll explore the Express.js framework and create an HTTP server.
In order to use the Express framework, first, you need to install express. To do this, you have to go to Powershell/Terminal and enter the following command.
npm install expressThen use the require() function to import the schema named express. In order to use the Express framework, you must first import a schema named express using the require() function. Here is an example of how to import a module using the Express framework.
const express = require('express');
const app = express();The require() function is called initially specifying the name of the module as a string ('express') and then the returned object is called to create the Express application. You can then access the application object's properties and functions.
Create a new file called app.js and use the following code.
const express = require('express');
const app = express();
const port = 8000;
app.get('/', function(req, res) {
res.send('Hello World!')
});
app.listen(port, function() {
console.log(`Example app listening on port ${port}!`)
});Go to Powershell or Terminal and run the code
node ./app.js