About this lesson
Selenium Python page_source: check what HTML the browser holds
Selenium Python AssignmentsAdvancedPage Data
How it works
The driver has a page_source attribute. It returns the current HTML of the page as one Python string. It is an attribute, so there are no parentheses, and calling it as a function raises TypeError: 'str' object is not callable. The text you get is the live document after scripts have run, not the raw file the server sent.
Because the result is an ordinary string, you can use every string tool Python has. The in operator asks whether a piece of text appears anywhere. The len function tells you how long the document is. count tells you how many times a piece of text occurs. The search is case sensitive, so a differently capitalised spelling is not found.
Source search tells you that something is in the markup. It does not tell you that a user can see it. This page has a field that is in the HTML but hidden from view, and the source still contains it. For visibility, find the element and ask whether it is displayed. Source checks are best for broad questions, such as whether an error banner or a tracking snippet was sent. Printing a boolean needs str, or the line fails with TypeError with the message can only concatenate str (not "bool") to str.
About the assignment
Open the Text Fields page and ask three yes or no questions about its HTML: does one id appear, does a second id appear, and is the source not empty. Print each answer as a labelled line.
What you will do
- Read the source once and keep it in a variable, so the browser is asked a single time.
- Use the
inoperator for each of the two ids named in the task. - Compare the length of the source with zero.
- Wrap each boolean in
strand join it to its label before printing.
Common mistakes
Calling page_source like a method
Writing parentheses after it applies them to a string and raises
TypeError: 'str' object is not callable. Java users do this by habit aftergetPageSource().Searching with different capitals
The check is exact. A search for an id in all lowercase is
Falseeven though the element is there.Treating hidden as absent
The hidden field is present in the source, so the check is
True, even though a visitor never sees it. A source search cannot show visibility.
In a real test suite
When a test fails in CI, teams often save page_source next to a screenshot. That file lets you read the page as the browser saw it, long after the container is gone. A pytest hook usually does this on failure.
The next lesson moves to structure: wrapping element reads in your own function.
Related lessons
- Add, Read, and Delete a CookieReads other page-level data from the driver, namely cookies.
- Check Whether an Element Is DisplayedChecks whether an element is displayed, which source search cannot tell you.
- Open Chrome and Read the Page TitleReads the title, another attribute that comes straight from the driver.
To try one of these calls on any page without a task, open the Online Selenium Python Compiler.
Starter code
The editor above opens with this code. Fill in the TODO lines, then run and submit it.
from selenium import webdriver
driver = webdriver.Chrome()
try:
driver.get("https://testkru.com/Elements/TextFields")
# Search the HTML of the page for the two ids
finally:
driver.quit()