In this article, I will demonstrate how to set user agent for browser and how to read user agent in Python Selenium WebDriver. Many scenarios in the tests require manipulating the user agent.
What is a user agent?
The User-Agent request header contains a characteristic string that allows a network protocol peer to identify the application type, operating system, software vendor, or software version requesting the software user agent. It appears in HTTP request headers and does not apply to response headers. All browsers support it.
In short, a user agent is the identity of the client (user).
User-agent passed format:
User-Agent: Mozilla/<version> (<system-information>) <platform> (<platform-details>) <extensions>
Example:
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:47.0) Gecko/20100101 Firefox/47.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:63.0) Gecko/20100101 Firefox/63.0
Selenium does not implement any direct method to read request or response headers.
Note: To run the following examples, you need to set the browser driver path details in the PATH environment variable, or you must pass the browser driver path to the executable_path variable when creating the browser instance.
Get user agent value
Selenium doesn't have any direct way to query user agent from WebDriver instance. We need to use the execute javascript built-in method to do this and pass the script that returns the user-agent.
Once the browser starts, we can get the user agent by executing the following line of code
# Store it in a variable and print the value
agent = driver.execute_script("return navigator.userAgent")
print(agent)
# directly print the value
print(driver.execute_script("return navigator.userAgent"))
User agent settings in Firefox:
To change the user agent of the Firefox browser, set the variable "general.useragent.override" in the Firefox profile and use this profile when creating the Firefox WebDriver instance.
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference("general.useragent.override", "[user-agent string]")
# Below is tested line
# profile.set_preference("general.useragent.override", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:63.0) Gecko/20100101 Firefox/63.0")
driver = webdriver.Firefox(profile)
User agent settings in Chrome:
In Chrome, the Options object must be used to set the user-agent value.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
opts = Options()
opts.add_argument("user-agent=[user-agent string]")
# Below is tested line
# opts.add_argument("user-agent=Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36")
driver = webdriver.Chrome(chrome_options=opts)
Alternative way to pass driver path along with other details:
Optionally, when passing the driver path, pass other data at the same time, as follows
driver = webdriver.Firefox(profile, executable_path="path to geckodriver")
driver = webdriver.Chrome(chrome_options=opts, executable_path="path to chromedriver")
Note: There is no standard way of writing a user agent string; different web browsers use different formats (some quite differently), and many web browsers add a lot of information to their user agent data.