Selenium3 Python WebDriver API source code analysis (20) WebDriver protocol overview

WebDriver protocol

In the early days Selenium, they WebDriverwere two independent projects. In the Seleniumearly days, they mainly relied on converting operations into JavaScript and then injecting them into the browser to automate WebDriverthe browser, encapsulating the browser's native API into a set of more object-oriented APIs, but due to the differences in the cores of different browsers, Therefore, in order to adapt, different APIs must be implemented for different browsers. In the Selenium2following two items can be combined, we often say Seleniumit is mainly refers to the WebDriverAPI. SeleniumThe main version of the current stable version is 3, and the beta version is Selenium4already on the way.

SeleniumOr WebDriverthe basis of interaction with the browser is the WebDriverprotocol (W3C recommended standard). WebDriverThe protocol is a RESTful Web service that uses JSON as the transmission format through the HTTP protocol and has nothing to do with the operating system and programming language.

WebDriverThere are currently two versions of the agreement:

• WebDriver 1：https://www.w3.org/TR/webdriver1/
• WebDriver 2：https://www.w3.org/TR/webdriver/

WebDriverThere is an abandoned agreement before the JsonWireProtocolagreement, seehttps://github.com/SeleniumHQ/selenium/wiki/JsonWireProtocol

WebDriverThree objects are involved in the agreement model:

• Client (Client or local end): It can be simply regarded as the program or machine that calls the WebDriver API. Our commonly used Python WebDriver API is a Python language binding of the client.
• Server (Server or remote end): the running RemoteWebDrivermachine, or WebDriverthe browser that supports the protocol , such as the supported geckodriverFirefox browser and the supported chromedriverChrome browser. WebDriverThe protocol actually only defines the behavior of the server.
• WebDriver: The various browsers WebDriverprotocol, such as geckodriver, chromedriver, WebDriveris equivalent to the client and server intermediary, providing WebDriverprotocol RESTful services. The client WebDrivercontrols the server (browser). For example, Firefox relies on webdriver.xpiextensions to control the browser.

After the client runs, a session is started. In the session, the WebDriverclient requests the server through the protocol RESTful service endpoint (endpoint, which can be simply understood as url), and the server returns a response to the client after accepting the request.
!

WebDriverThe protocol maps endpoints to commands. The relationship between endpoints and commands is as follows ( https://www.w3.org/TR/webdriver/#endpoints ):

Method	URI Template	Command
POST	/session	New Session
DELETE	/session/{session id}	Delete Session
GET	/status	Status
GET	/session/{session id}/timeouts	Get Timeouts
POST	/session/{session id}/timeouts	Set Timeouts
POST	/session/{session id}/url	Navigate To
GET	/session/{session id}/url	Get Current URL
POST	/session/{session id}/back	Back
POST	/session/{session id}/forward	Forward
POST	/session/{session id}/refresh	Refresh
GET	/session/{session id}/title	Get Title
GET	/session/{session id}/window	Get Window Handle
DELETE	/session/{session id}/window	Close Window
POST	/session/{session id}/window	Switch To Window
GET	/session/{session id}/window/handles	Get Window Handles
POST	/session/{session id}/window/new	New Window
POST	/session/{session id}/frame	Switch To Frame
POST	/session/{session id}/frame/parent	Switch To Parent Frame
GET	/session/{session id}/window/rect	Get Window Rect
POST	/session/{session id}/window/rect	Set Window Rect
POST	/session/{session id}/window/maximize	Maximize Window
POST	/session/{session id}/window/minimize	Minimize Window
POST	/session/{session id}/window/fullscreen	Fullscreen Window
GET	/session/{session id}/element/active	Get Active Element
POST	/session/{session id}/element	Find Element
POST	/session/{session id}/elements	Find Elements
POST	/session/{session id}/element/{element id}/element	Find Element From Element
POST	/session/{session id}/element/{element id}/elements	Find Elements From Element
GET	/session/{session id}/element/{element id}/selected	Is Element Selected
GET	/session/{session id}/element/{element id}/attribute/{name}	Get Element Attribute
GET	/session/{session id}/element/{element id}/property/{name}	Get Element Property
GET	/session/{session id}/element/{element id}/css/{property name}	Get Element CSS Value
GET	/session/{session id}/element/{element id}/text	Get Element Text
GET	/session/{session id}/element/{element id}/name	Get Element Tag Name
GET	/session/{session id}/element/{element id}/rect	Get Element Rect
GET	/session/{session id}/element/{element id}/enabled	Is Element Enabled
GET	/session/{session id}/element/{element id}/computedrole	Get Computed Role
GET	/session/{session id}/element/{element id}/computedlabel	Get Computed Label
POST	/session/{session id}/element/{element id}/click	Element Click
POST	/session/{session id}/element/{element id}/clear	Element Clear
POST	/session/{session id}/element/{element id}/value	Element Send Keys
GET	/session/{session id}/source	Get Page Source
POST	/session/{session id}/execute/sync	Execute Script
POST	/session/{session id}/execute/async	Execute Async Script
GET	/session/{session id}/cookie	Get All Cookies
GET	/session/{session id}/cookie/{name}	Get Named Cookie
POST	/session/{session id}/cookie	Add Cookie
DELETE	/session/{session id}/cookie/{name}	Delete Cookie
DELETE	/session/{session id}/cookie	Delete All Cookies
POST	/session/{session id}/actions	Perform Actions
DELETE	/session/{session id}/actions	Release Actions
POST	/session/{session id}/alert/dismiss	Dismiss Alert
POST	/session/{session id}/alert/accept	Accept Alert
GET	/session/{session id}/alert/text	Get Alert Text
POST	/session/{session id}/alert/text	Send Alert Text
GET	/session/{session id}/screenshot	Take Screenshot
GET	/session/{session id}/element/{element id}/screenshot	Take Element Screenshot
POST	/session/{session id}/print	Print Page

WebDriver Python API执行流程（反推）

WebDriver客户端API其实就是实现属性或方法到命令，再到WebDriver协议端点的变换过程。

For example: WebDriverthe current_urlcharacteristics of the class

1. current_urlThe characteristic is actually Command.GET_CURRENT_URLthe result of the execution of the command .

   @property
   def current_url(self):
        return self.execute(Command.GET_CURRENT_URL)['value']
   

2. WebDriverThe executemethod of the class is command_executor.execute(driver_command, params)executed by the command.
   command_executorIs a string (server url) or remote_connection.RemoteConnectionobject.

3. remote_connection.RemoteConnectionThe class is located selenium\webdriver\remote\remote_connection.py.
   remote_connection.RemoteConnectionThe _commandsattributes of the class define the relationship between the command and the endpoint.
   Command.GET_CURRENT_URL: ('GET', '/session/$sessionId/url')
remote_connection.RemoteConnectionThe executemethod of the class controls the server to execute commands.

4. WebDriverWhen the class is instantiated command_executor, command_executorparameters are passed in , and the parameters are the server address.

WebDriver Python API execution process (in progress)

from selenium import webdriver

driver = webdriver.Firefox()
driver.get("http://www.baidu.com")
# 服务端地址
url = driver.command_executor._url  
print(url)

The execution flow of the above code is roughly as follows:

1. driver = webdriver.Firefox()That is selenium\webdriver\firefox\webdriver.py, the WebDriverclass in the pair is instantiated, and the WebDriverclass inherits from selenium\webdriver\remote\webdriver.pythe WebDriverclass in. WebDriverThe command_executorparameter in the class construction method is the server address.
   At this point, WebDriverstart the browser, load the webdriver.xpiextension, and listen for WebDriverprotocol-based requests.
   A geckodriver.logsimilar 1116855012551 geckodriver INFO Listening on 127.0.0.1:54499record can be seen in the log file .

2. driver.get("http://www.baidu.com")It is equivalent to executing a Command.GETcommand. WebDriverObjects rely on remote_connection.RemoteConnectionobjects to command_executorexecute commands. The mapping remote_connection.RemoteConnectionfrom command to WebDriverprotocol is implemented in the class Command.GET: ('POST', '/session/$sessionId/url'). Executing the command is equivalent to sending a corresponding request to the browser. WebDriverSend the request to the browser.

    def get(self, url):
        self.execute(Command.GET, {
         
         'url': url})
   

3. After accepting the request, the browser performs the operation (opens the Baidu page) and returns a response (server address). WebDriverPass the response result to the code.