For a long time, crawlers seem to be implemented by writing code. Today, I would like to introduce how Jmeter implements a web crawler!
The principle of Jmeter's crawler is actually very simple. It is to submit a request to a web page, then extract all the returned hrefs, and use the ForEach controller to implement url traversal. Is this explanation clear? The following is a brief introduction to how to operate.
First of all, we need to submit a request to the web page, take Tencent News as an example! We initiate a request like Tencent News. Observe the return value and find that there are many urls with href tags + text titles in the middle
We now need to extract these urls using powerful regular expressions! Remember to fill in the matching number with -1, which means to take out all suitable urls
a target="_blank" class="linkto" href="http:// *(.*l)"
Add a debug to see if it has really taken out the Longyuan Pavilion test development home 317765580
Or we can directly use regular matching in the result, we can see that many web links have been taken out
Next, we need to use the ForEach controller, and use this controller to traverse and trigger all the fetched urls. Remember to fill in the variable name in the controller, which is the variable name in the regular expression just now
Next, add another http request under the ForEach controller and use it to execute the request trigger
Now we can observe the results, and it is time to witness the miracle. Observing the results we found that all matching urls were fired!
Is not it simple? Let's do it in action! Longyuan Pavilion test development home 317765580