altitude above sea level @ -g install latest
- [Instructor] Differently than with other web languages, like for instance PHP that runs as a module in Apache, Node.js comes with a web server built in. So Express is actually a hypertext transfer protocol, in short HTTP server. As such I think it doesn't harm if we quickly cover how HTTP actually works. HTTP is a quite simple text-based protocol. I use Telnet to show what's going on behind the scenes when you request a file in the browser. Don't be intimidated by what you see on the screen right now.
I'll break it down for you. First I connect to the server, in this case Google.com on port 80.That's the standard HTTP port. If the request should be encrypted, you would use port 443,which you know as HTTPS in your browser. Your browser knows this and selects the right port depending on if you use HTTP or HTTPS. My local system will now make a call to the domain name service, which resolves the domain name to an IP address. Once it knows it, it will try to connect to this IP address on port 80, and the system will hopefully reply that we are now connected.
Now the actual HTTP protocol starts. I type GET /, which means give me the index page of this web server. The web server will first tell me if the request went through successfully. It communicates this by HTTP status code, in this case 200, which means okay. Then the server replies with a bunch of headers that instruct the browser to do specific things, like setting a cookie and the html, which is quite a lot when it comes to Google.
The command GET as we used it here is a so called HTTP verb. It is usually used to retrieve data. The other word that is used often in browsers is POST. It is used to send the data with a request like when you submit a form. And there are a few other verbs that are not used when you browse the web, but are relevant when you work with so called REST APIs. These are application programming interfaces that use regular HTTP and its verbs to programmatically communicate between systems.
In this course, as we are creating a regular website, GET and POST are important because we will later create routes for it. What are routes? You see that we have a path here on every HTTP verb. A path is a part of the unique resource locator or URL.
That is everything in the browser address bar including: protocol, domain name, path, and query string. In my Google example this was the route path, which is identified by a slash.
But of course if you look into the address bar of your browser path can be much longer. They are similar to how a folder structure works actually to identify a document on the web. On today's web applications that serve dynamic data, those URLs usually don't point to a specific document, but to a handler that knows what to return for a given path. These handlers are often referred to as routes. In my example from before we also saw a status code, in this case it was 200. Status codes are used to tell the browser the outcome of a request, and there are plenty of it.
We will only cover the most important here. Everything from status code 200 to 299 indicates a successful request. 200 is used most commonly, and it simply means OK. Code 300 to 399 tell the browser to redirect to another page. For instance 301 means Moved Permanently, and you often use it to tell the crawl of a search engine to replace an indexed page by a new one.
Anything from 400 to 499 indicates a problem on the client. Like some kind of bad request indicated with 400. Insufficient privileges like with 403: Forbidden or a nonexisting URL indicated by 404: File Not Found. And then there are the errors from 500 to 599. They indicate a server problem. Most commonly you will see 500, which means Internal Server Error.
Equipped with this knowledge we are now ready to dive right into Express.
404
const http = require('http');
const url = require('url');
function handler(request, response) {
const parsedUrl = url.parse(request.url, true); // true: also parse the query string
// console.log(request.url);
// console.log(parsedUrl);
if(parsedUrl.pathname === "/") {
response.writeHead(200, {'Content-type':'text/plain'});
response.write('Hello, I am a webserver!');
response.end();
} else {
response.writeHead(404, {'Content-type':'text/plain'});
response.end();
}
}
const server = http.createServer(handler);
server.listen(2000);
If I see some cascade of ifs, like here. If I see some cascade of ifs, like here. Let's make sure that in the end of each if we really do a return to emphasize that we really do a return to emphasize that the execution of this function really ends here,
const http = require('http');
const url = require('url');
function handler(req, res) {
const parsedUrl = url.parse(req.url, true);
res.setHeader('x-server-date', new Date());
if(parsedUrl.pathname === '/') {
res.writeHead(200, {'Content-type':'text/plain'});
res.write('Hello, I am a webserver!');
return res.end();
} else if(parsedUrl.pathname === '/time') {
res.writeHead(200, {'Content-type': 'text/plain'});
res.write(new Date().toString());
return res.end();
} else if(parsedUrl.pathname === '/hello') {
// http://localhost:3000/hello?name=chen
const name = parsedUrl.query.name;
if(!name) {
res.writeHead(400, {'Content-text':'text/plain'});
return res.end();
}
res.writeHead(200, {'Content-type': 'text/plain'});
res.write(`Your name is ${name}`);
return res.end();
} else if([parsedUrl.pathname.startsWith('/user/')]) {
// http://localhost:3000/user/chen
const regex = /\/user\/(.+)/;
const matches = regex.exec(parsedUrl.pathname);
if(!matches || !matches[1]) {
res.writeHead(400, {'Content-text':'text/plain'});
return res.end();
}
res.writeHead(200, {'Content-type': 'text/plain'});
res.write(`This user profile belongs to ${matches[1]}`);
return res.end();
} else {
res.writeHead(404, {'Content-type':'text/plain'});
return res.end();
}
}
const server = http.createServer(handler);
server.listen(3000);
npm init -y
npm install --save express
const express = require('express');
const app = express();
app.use((req, res, next) => {
res.setHeader('x-server-date', new Date());
return next();
})
app.get('/', (req, res, next) => {
return res.send('Hello, I am home page');
})
app.get('/time', (req, res, next) => {
return res.send(new Date().toString());
})
app.get('/name', (req, res, next) => {
if(!req.query.name) {
return res.status(400).end();
}
return res.send(`Your name is ${req.query.name}`);
})
app.get('/user/:user', (req, res, next) => {
return res.send(`Userprofile of ${req.params.user}`);
})
app.listen(3000);
- [Instructor] In Express everything except initialization of the application in the beginning is the reaction to an incoming request. Express uses so called middlewares for that. Very much like the handle function of the purenode server a middleware gets to request and responds object to operate on. To pass control further a middleware can call next and there are a few methods under responsive check like send or render that will enter request and send the response to the client.
Let's look at an example. In the beginning the application initializes. It will execute all JavaScript code which will reduce all middlewares and routes. Then it will start the HTTP server and listen on a given port. At some point a request for some path comes in. Express will first pass its rule to chain of generic middlewares that will call next to call the next one. If you don't call next here the request will end.
This is a very common mistake. When no middleware is left it will try to match the route against re choosed routing middlewares. If a pass matches it will call the handle for it. In this case now the /feedback handler. Usually, this route will end the request for calling res.send explicitly or implicitly by functions like res.render. Let's look into middlewares in more detail.
General middleware is pretty simple. It will be called for each request getting the request and response object. There it can basically do anything like adding a header or incrementing a counter. And it will then pass on control by calling next in the end. Routing middleware is a bit more complex. First, it has a verb it should req to then a path and then a callback.
Let's look into the first two parameters in more detail. So the verbs are the ones that we already learned about when we talked about the HTTP protocol. So this would be get, post, put or delete. The second parameter would contain the path. And there are a variety of ways how such paths can be expressed. For instance this path here for /feedback will match exactly /feedback.
There are also parameter routes that would contain a param that is marked by the column here as you see it here with the column username. And this would then match /user/james, /user/john and it will then provide the parameter in request .params. Of course this path can have any length and contain more than one param as you see here.
So this is also valid. We can also use regular expressions. For instance this regular expression you see here users with the question mark in the end would match user/james but also users/james and provides the username in request .params. And here you could basically use any regular expression. So you can do a lot with those kinds of routes.
We also have optional params. Those are marked with first a column in the beginning of the param and with a question mark in the end which means that this would match against /usersand optionally also against users/james and provide again the username in request .params if provided. One important note here is that you that there are various ways to define a route in the paths can match multiple routes actually.
Express will choose the first it matches so you have to be careful that you don't define a routethat is very broad right on top because this would then most probably be a catchall and your actual route handler would never be called. Understanding middlewares is key to understanding Express. Revisit the video when you feel uncertain.
const express = require('express');
const app = express();
app.use((req, res, next) => {
res.setHeader('x-server-date', new Date());
return next();
});
app.get('/throw', (req, res, next) => {
throw new Error('Throw a error');
})
app.get('/next', (req, res, next) => {
setTimeout(() => {
next(new Error('Throw a error'));
}, 3000);
})
app.get('/', (req, res, next) => {
return res.send('Hello, I am a webserver');
});
app.get('/time', (req, res, next) => {
return res.send(new Date().toString());
});
app.get('/hello', (req, res, next) => {
if(!req.query.name) {
return res.status(400).end();
}
return res.send(`Hello ${req.query.name}`);
});
app.get('/user/:name', (req, res, next) => {
return res.send(`Userprofile of ${req.params.name}`);
});
app.listen(3000);
- [Instructor] Throughout it's development cycle an application needs to run in different environments. This is often something like development, testing and production and often the configuration and the way we expect the application to run differs. Like in development, you will most probably want to log everything and don't do any caching. While in testing, you'll only want to log fatal errors because you don't want to clutter your testing output. When you're running in production, you want to log all basic information and of course errors, as well.
Of course there you want to cache because it gives you a better application performance.Node.js environments are set by environment variable NODE_ENV. As you see here, I'm exporting this variable with the value production and when I then run my application, it will run in production mode. You can also do this as one liners by simply setting this variable and then calling NODE_ENV. Of course, this also works if you use npm to start your application.
Express is aware of this environment variable and it will automatically read it and will provide it to you in the application variable env. You get it if you call app.get('env') and this will return in the currently set environment. It defaults to development. So, if nothing is set, it will always be development. We can use this information to change the application's behavior, like loading different configuration depending on the environment.
And there are caveats to that. As a set, development is the default environment. So, if you don't explicitly set NODE_ENV, the application will run in development mode. Some while back I did some tests by running an application first in development mode and in production mode and then doing some CPU profiling. I used these three to create a so called sunburst chart, which shows what's going on on the CPU at a given time slice.
Here you see what's on a CPU, often express application running in development mode. On the other hand, that's an application now running in production mode. You very clearly see that there is much less going on here. Right? The reason for that is, if you're running in development mode express will rerender all templates every time a request is made. Which makes sense because while you're in development, you don't want to restart the app every time you change something in some HTML template.
But on the other hand, this means that if you're in development mode, your application may run up to 70% slower and that's okay when you're running in development. The trouble starts if you forget to set your application explicitly into production mode when you're running production because then you have a huge performance penalty without even knowing that.So, being aware in which environment you're currently running and making sure that your application runs in production once you deploy it, it's really key to optimal performance.
Start Project:
- [Instructor] Now is a good time to create the structure and the setup for our main project. For that I've created a new empty folder called Work and copied the provided HTML folder from the exercise files into it. Next, let's create the folder our project should live in, and I will call it conference. So I right click and select new folder, and I create a folder called conference.
Now I'm opening the integrate terminal and there I move into this folder already. In there we now want to create a package json which will manage all our dependencies for that, I will run npm init dash y, in integrate terminal, and we see that it created the basic package json file for us. We will edit and adapt it a bit later.
Next, obviously we want to install express, so we run npm install dash dash save express. so we run npm install dash dash save express. This will pull down express and all its dependencies and add an entry to package json. We see it here. You should also see the package lock json file, this is a rather new feature in npm, package log json contains not only the root dependency as package json does, it contains the whole dependency tree with the exact links to the package that was actually installed.
Additionally it also contains checksums to make sure that the package hasn't been tampered with. What does this give us? It's best practice to not commit the node modules directory into a version control or even add it to zip file that you pass to someone else because it contains trust third party dependencies and sometimes even compiled binaries that are specific to your operating system. There is also no point to version control code that will never be changed manually. By using the package log file npm makes sure that someone else setting up the project by running npm install will have the exactly same package as with the exactly same versions like you have.
Next, let us create some folders. The way I'm structuring the app worked well for me in many projects. Express itself isn't opinionated when it comes to how you set up your application. In the root directory of academy, I created the folder server so I make a right click here on the conference folder, select new folder, and I call it server. This will later contain all the server-side code of our application. Similarly, I will create folder public, Which will later contain all the client-side code of our application.
Now in server, I create our main application file. So on server, I do right click, select new file,and I call the file index.js. Now there I add const express equals require express Now there I add const express equals require express as we did before already, const app equals express with parentheses const app equals express with parentheses which gives us an instance of the expressed object.
And in the end let's add the app listen part, and I want to listen port 3000. Lastly, it's a good practice to always export from any module, so down here I will add module.export equals app. so down here I will add module.export equals app. So if someone requires this app file,they will get application instance back.
So now let's try that, and by convention, in node chest, if you select in any way any folder, it will always default to index chest file inside the folder. So when I now type node server, it will actually run my index chest file. Now let's head over to the browser and see if that worked. I'm opening localhost 3000, and we see that it works, but we haven't created any routes so far,so we get a generic message by express, that it cannot get the page referred to with the slash.
While we are at it, let's also change our package json a bit. It's good practice to define a start script in package json. By doing so someone else or your future self does not have to look into the code to find out how this application actually needs to be started. So we add in package json here
... right in the script section one new line, don't forget the comma, and there I add start colon node server, and there I add start colon node server, which means we've created a start script which will subsequently run node server.
So let's stop the server again and now let's run npm start and we see that the server runs again, let's verify that everything works. And it worked. We now have the basic application structure in place and we can even start the application already. What's missing now is the routes.
-------------------------------------------------------------- ref
module.exports is the object that's actually returned as the result of a require call.
The exports variable is initially set to that same object (i.e. it's a shorthand "alias"), so in the module code you would usually write something like this:
var myFunc1 = function() { ... };
var myFunc2 = function() { ... };
exports.myFunc1 = myFunc1;
exports.myFunc2 = myFunc2;to export (or "expose") the internally scoped functions myFunc1 and myFunc2.
And in the calling code you would use:
var m = require('./mymodule');
m.myFunc1();where the last line shows how the result of require is (usually) just a plain object whose properties may be accessed.
NB: if you overwrite exports then it will no longer refer to module.exports. So if you wish to assign a new object (or a function reference) to exports then you should also assign that new object to module.exports
----------------------------------
----------------------------------------- ref
var express = require('express')
var router = express.Router()
// middleware that is specific to this router
router.use(function timeLog (req, res, next) {
console.log('Time: ', Date.now())
next()
})
// define the home page route
router.get('/', function (req, res) {
res.send('Birds home page')
})
// define the about route
router.get('/about', function (req, res) {
res.send('About birds')
})
module.exports = router
Then, load the router module in the app:
var birds = require('./birds')
// ...
app.use('/birds', birds)
The app will now be able to handle requests to /birds and /birds/about, as well as call the timeLog middleware function that is specific to the route.
-------------------------------------------------
static files
- [Instructor] We now have all basic routes in place. Of course, later, we want to serve HTML from it, but a website usually consists of more than HTML. If we look at the provided folder,we see that there are also directories and files for Javascript, CSS, and images. These kinds of assets are called static files. Let's add them to a project now, so for that, I will simply copy CSS, images, and JS, and paste it into the public folder of our project.
There is also a data folder in the provided folder; you can leave it where it is right now, we will deal with that later. So how can we serve CSS style from CSS? The naive approach would be to create a route for it. In there, we would create a file from the file system and then use the Express method respond send file to send the file to the browser. We could then create a similar route for all images and Javascript files. Of course, this is a lot of work and does not scale, and soon we would look for a way to solve the problem more generically.
Luckily, Express already comes with this kind of functionality through its built-in static middleware. So let's add that to our main application file, so in server index chest, I will now simply add, before I even define the routes: app dot use and as middleware, app dot use and as middleware, we use built-in middleware of Express, and it's called Express dot static, and this middleware simply gets the folder where our public assets are in, and this folder is always relative to the application route folder and the assets here on the top level of the application,we can simply call it public here.
Now let's start application and let's open the browser, and then let's try to get the CSS file so I look for CSS style dot CSS, and we see that this works, so this is now served directly from the file system. Let's try if images work as well. Let's try slash images, slash misc, Let's try slash images, slash misc, slash background dot JPEG, and thus we see, this works as well.
While we are here, let's deal with something else as well. You might know that there is a file called fav icon dot ICO. This file contains the icon that is shown in the tab or address bar of your browser for a given web page. The browser automatically tries to request this fileoccasionally, which means that Express will try to find the route for it and send a HTTP 404 error then. As we don't have a fav icon file in our project, let's add a handler that lets the browser know that and prevents Express from for looking for a match.
So in our application now, right after defining the static route, I add a route for app, it's a get trick first, forward slash fav icon dot ICO, it's a get trick first, forward slash fav icon dot ICO,and in there I have again request response and next and in there I have again request response and next and there I will do a return response dot send status and I will use 204 here which means this is an empty response so we more or less tell the browser nothing to see here.
Let's try this out real quick. So in the browser, I will now open the developer tools, tools, developer tools, and now let's try to fetch fav icon dot ICO, and you see that we now get the error 204 back, so this fails early and does not run to all of our application. For static files from now on, Express will, for each route, look into a public folder and try to find the requested file and send it if it finds it.
If nothing is found it will call next to pass the control to the next route handler. The static middleware does have a few more things like setting specific headers for caching and cache calculation, but this is not too important. All you need to know by now is that Express static is the way to send assets via Express and if your project isn't the purest APR with no HTML served at all, you will need it.
- [Lecturer] Year back when the first dynamic websites appeared it was usual to make html code and program logic very often in one huge file. So a web application basically consisted of functions that concatenated bits and pieces of html code until it was ready to be dumped to the browser. This was of course extremely hard to maintain. The change the design of a website one would basically need to rewrite all of it and it was impossible for a non-programmer like PRO Designer to change the user interface in any way.
So it comes that today templates are used to separate presentation from logic. This means a template is the container for the contents filled by the application. Everything concerning the way a page looks, lives in the template, everything concerning the program logic lives in the application. This principal is also referred to as separation of concerns. A template engine provides an interface to fill templates with data and render them, and very often comes with its own syntax for creating templates.
Express is very agnostic when it comes to template engines. Any engine that provides a specific interface can be plugged into the framework. Looking at a screen shots from Express wiki, we see that around 20 engines are known to work with Express. Choosing the right one is mostly a matter of preference. Most importantly you should feel comfortable with the syntax.Today the most popular template engines are, Pug, EJS and Handlebars.
To understand the difference between them let's compare the syntax of Handlebars and Pug.With Handlebars you see that the syntax is very similar to regular html, it just uses double curly brackets to create placeholders for dynamic data. Using it is easy because there is no new markups syntax to learn. Pug on the other hand has a completely different syntax. It uses white spaces in the form of indentation to describe the structure of the document.
This is shorter but we have to learn a new syntax. Of course the end result rendered to the browser would be the same for each template engine. For this course we will use Pug as it's very commonly used and also the default in Express.
. For plain text, if written below a tag, it has to be preceded with a pipeso that Pug knows that this is not some HTML markup.
npm install --save pug
- [Instructor] Express is prepared to work with template engine and it's easy to set them up. To add Pug we obviously first have to install the module. For that I'll run npm install--save pug.Next I will open server dot index js, interestingly for the template engine, we don't have to require it here, we will simply, after we instantiate the Express at app.set view engine with a blank in between pug.
When we do that, Express will do the rest. It will simply require Pug then when it needs it. Next we have to tell Express where to look for the templates and I will create a dedicated directory for that in server, I create new folder, and I call it views. Now after setting the view engine, I add app.set views and then I want to set the paths in for that.
I will use the paths module which gives me a few functionalities to tell to create proper absolute paths into the file system, so I add const path is require paths, so that's a core module of no js, nothing to install here. Back in line seven, here, I now add app.set views and then path.join, and I want to create an absolute URL from the directory name to ./views as we see it here in the file system.
Let's add our first template to the route, let's make an index route. So in views, I now create a new file, index.pug, so that's the extension. And there, I now write doctype html, then I start the html block. In there I have a head that is empty and a body, and this body should contain a headline.
I call it Index Page, and then it should contain a paragraph that says, This is the index page.Next, we of course have to teach the router to render this page. For that, I open routes, index.js, and for the index router I can now replace this res.send with res.render so that's built in function from Express that will now use the configuration we made before when we set the view engine and also the view directory and tell it to render the index template here, index.pug.
So here you can also omit the extension pug. Let's start the application now and let's try looking at our index file. And we see that we get an index page back. Let's look into the html source real quick. So I right-click here and select View Page Source. And we see that Pug and Express created proper html for us.
We also see that the html code is super optimized to save bandwidth, all white spaces, indents, et cetera are gone. This is great in production scenarios, but for de-bugging, it's sometimes better to have a well-structured html document. To get this, we only need to switch one config variable in Express. So for that, I will simply server, index.js again, and there, right after I defined the view engine, I will now add app.locals.pretty = true.
And as I said, I only want to do that in development, so I will add a condition here that says if app.get env = development only then I need this pretty printing, so I add it here. Let's restart the app and I can simply reload the source code here, and as you see now, I have a prettyformatted html, which is much easier to read for humans.
npm install --save http-errors
So this will trigger 404 error, but we still have to implement the real error handler now. And Express has the convention that the error handler is the only middleware that takes four arguments. So we'll do that now, app.use. And in here, I'll now add error, so that's the error of check that's new compared to other middleware. Then request, response, and next. So every time an error is created, this middleware will be called now.
And I want to show the actual error inside the template and there are several ways to achieve that. And storing it as a property of res.local is one of them. We will cover all of that in more detail later. But for now, let's simply use res.locals.message equals err.message. This makes the error message available in the template.
Then we also want to create an error status, const status. And this should be either the statusthat is set on the error object. For instance, when it comes from the error handler here on line 22 with the 404. And if there is no status provided, it should be 500, internal server error,means something happened, but we actually don't really know what, so it has to be severe.
And we also will set this status, res.locals.status, to have it in the template. And if we are in development mode, very similar to how Express does it as well, we will also want to show the whole stack of the errors. So I add res.locals.error equals and if req.app.get, environment, env, is development, we want it to contain the error or in any other case, it should be an empty object.
Now, let's quickly also set the status on the response, so that we really send the proper error status back with our response. And all that is left to do now is going res.render, error. And let's also return from here. Okay, let's run the application and let's try that out.
So here we see the difference. In line 14 we are using app dot locals to set a variable that will be set every time the application starts. Now in line 17 we use the locals property of the response object to set the variable that will be evaluated for each request. And also, very important, we now have to return next from here. If we don't do that the application will just hang here and get stuck and will never return a response.
What we want to do next is we want to create the template variable that lets us create the speakers list here on the navigation. This means that would be a global template variable, and now the question is do we want to have it application global so that it is evaluated every time the application starts, or should it be evaluated for each request? And I would say we want to evaluate for each request because the speakers list could change at any time.
So if a new speaker is added I don't want to restart the application. So we want to create a middleware here and we will create it a little bit down below, right before the routes are declared, but after the express static middleware is called because we don't of course want to run this middleware for each static file like CSS or images. So now add app.use and it will be an asynchronous middleware.
npm install --save body-parser
Now of course we also have to add this middle here, so let's add this right before the route start and after this, adding it where, because we don't want the body parser to run for each image for CSS.
So add here now in line 25 app.use bodyParser.urlencoded app.use bodyParser.urlencodedbecause that's to format the form we send with and as argument I set extended to true. and as argument I set extended to true. The extended option tells the parser to also parse more complex data structures. We most probably won't need it for our example, but it doesn't harm to set it anyway.
So this middle well will now look into the raw HTTP body data, and if it finds form data it will parse it and store it into request.body. Now let's open the route for our feedback form. I open route, feedback index chest, and here, for the post route, I will now simply console.log req body. I will now simply console.log req body. I will now simply console.log req body. Let's restart our application and I can now simply reload the previously submitted form, and we see that we now get the form data via request body and in the next videos we will look intosanitizing, storing, and displaying it.
i don't understand:
If I change action='/feedback' to action='/feedback/index.pug' or action='/feedback/index', then it doesn't work???????
btw. you need to restart to make the above changes (npm start)
same weird thing happens when:
but this seems to be ok:
And it works. We now make sure that the data is complete. Of course there are more ways to sanitize the data. Like making sure that no JavaScript is submitted, and then executed by a user by opening the page later. There are various modules for that just look for Express sanitize. In our case no harm can happen because Pug will automatically escape potentially dangerous data if we don't proceed it with an exclamation mark to explicitly switch escaping off.