This series of books is like a friend, you can click the link to join the exchange QQ group (994,761,602) [C #] path-breaking territory of
convenient when you have doubt I can give timely feedback. At the same time, it can be considered to like-minded friends to provide a platform for the exchange.
In the first environment, we mainly understand some basic principles of reptiles that principle also, say the basics worth mentioning, the result is a small reptile already know how the birth of the ~ So now, please quietly Recall that in the first territory, you have mastered what? What is still relatively vague? If there is anything unclear or unstable surface, it was suggested to go back and see it again, after all look much faster than what I wrote -
If you have mastered almost, OK, the spirit of the book "read lose" philosophy, would not have to think about it, then congratulations you have read the book first Habitat friends, may have been the first official break territory, into the realm of reptiles throughout the second, then do not slack off, let's move on a new journey!
We are the first in the territory, it has been in use WebRequest and WebResponse two abstract classes. They are two of the core areas of Web crawlers, no exaggeration to say, master them, the equivalent of a major success :)
Note that here, I was very cautious wording in the "Web crawlers field" rather than reptiles field, this book abstraction (from the point of view to explain the division level), the reptiles can be divided into the following areas level, currently only going to start with the Windows platform for cross-platform friends, you may be disappointed, but I am, after all, limited personal energy, also like him to forgive up.
| Distributed expansion level | The main task distribution plan includes a multi-node crawler, load and so on. |
| ⇪ | |
| Application level | Configuration mainly used to control the acquisition, planning, management, and the subsequent processing of the data collected. It can be considered as a preparation to expand after it distributed system. |
| ⇪ | |
| Web tier | WebRequest and WebResponse to Web-based data collection. |
| ⇪ | |
| Socket-level | Socket connection to establish a direct data collection protocol and can support most of the custom protocol data acquisition. |
| ⇪ | |
| Drive level | Starting from the drive system, data acquisition mainly by filtering network packets. |
As can be seen from the above table, Web tier, in the middle position, so why start from the middle of it, because then down ChenDian the Socket level and drive level, using the scene is still relatively small, and the processing Socket, C # Fortunately, the process drives, C # on the powerless. We have from the beginning focused attention most of the students, but also on the market has to adapt to it. Web-level and application level, in the example, in fact, difficult to separate, so, we explain the process, but also together. Having Web tier + application level, mainly to talk about how to expand the system distributed crawler, this point is a lot of people are concerned about; then Socket another level as a supplement to a framework; as for the drive level, you can use C ++ to achieve, but if on the book, and the title seemed a little conflict, I'll think about it, if a separate one specifically written a book drive it: P
Well, congratulations again we completed the first reading of the territory, in the second environment, the main is to understand the level and application-level Web content, combining a number of cases, to practical operation is based, it will be on reptiles in practical work routines have more understanding. Let's go ahead and open a second border journey!
This series of books is like a friend, you can click the link to join the exchange QQ group (994,761,602) [C #] path-breaking territory of
convenient when you have doubt I can give timely feedback. At the same time, it can be considered to like-minded friends to provide a platform for the exchange.