Tools and information
- QQ Group-Javascript advanced crawler https://jq.qq.com/?_wv=1027&k=5Bcu3YU-The author's self-built group, welcome to join!
- China trademark network encryption interface https://github.com/rockswang/wsjs.saic.mmewmd-for demonstration only
- awesome-jave-crawler https://github.com/rockswang/awesome-java-crawler-crawler related tools and materials collected by the author
Preface
I have been watching puppeteer in the past two days and found that it can also be detected by a certain number of front-end js!?
I searched the issue area of github. It turns out that there is navigator.webdriver attribute in the chrome started by puppeteer. What's the matter? The foreigner really needs to set up the torii...
plan 1
The solutions seen in the issue area:
await this.page.evaluateOnNewDocument(() => {
Object.defineProperty(navigator, 'webdriver', {
get: () => undefined,
});
}
But to be honest, this is still a bit problematic, because it can still be detected with "webdriver" in navigator.
Scenario 2
I want to find out which link to add the "webdriver" attribute, but text search found that there is no puppeteer source code...
Later, I looked at the command line startup parameters with chrome:version in chrome, and found this: "--enable-automation" ......
There is a problem at first glance. In order to confirm, I searched for the description of this command line parameter:
--enable-automation: Inform users that their browser is being controlled by an automated test.
After confirming that it can be killed, add an ignore default parameter when starting chrome:
const browser = await puppeteer.launch({ignoreDefaultArgs: ["--enable-automation"]});