Solvedselenium Shadow DOM traversal support

I've been gradually trying to push support of shadow DOM v1 into selenium (see #4230, #5762).

However, we are still missing one major piece: traversal.

There needs to be some ability to traverse shadow DOM when calling FindElement and what not.

So maybe we can discuss some possible implementations here?

What we can't/shouldn't do:

  • Alter how existing selectors work (CSS, XPath, etc. they should all behave as they do now)
  • Implement deprecated things like /deep/
  • Implement our own selector syntax/mechanism

The way I see this working is that we should treat the DOM as we would in the browser, meaning there will not be a way to select a deep element in one call:

element
  .FindElement(By.CssSelector("foo"))
  .ShadowRoot?
  .FindElement(By.CssSelector("bar"));

Which, in the browser, would be structured the same (no shortcuts):

element
  .querySelector('foo')
  .shadowRoot
  .querySelector('bar');

So maybe we just need to implement ShadowRoot on WebElement? Which can be null (just like in the browser).

48 Answers

✔️Accepted Answer

The idea is to conceptually treat the shadow DOM in the same way we treat frames.

I think a huge difference between ShadowRoots and frames is the sheer pervasiveness and number of them. Apps are likely to have 100s to 1000s of ShadowRoots, basically every Custom Element will.

And they're not nearly as special as iframes - they're in the same window and document, share the same globals, many style properties inherit through them, some events bubble up past them, light-DOM children are projected into them, and they're always attached to a host.

Many tests need to check on some interaction between elements in the ShadowRoot and outside of it - ie, set a property on the host and make sure it's reflected in the shadow; click on a button in the shadow and many sure that an event is fired on the host, or an attribute is added; add a child and make sure it's assigned to a slot, style a CSS ::part and check that it's applied, etc. The idea that a ShadowRoot is a different state just doesn't really hold. It's merely a different branch in the DOM tree (the tree of trees).

Iframes on the other hand have none of these APIs/interactions, and interactions between the iframe and it's host document are very limited, so these types of tests are probably non-existent and the pain that a switching-based API would cause haven't been felt.

ShadowRoots also very commonly nest, whereas with frames testing through arbitrary testing nesting levels is probably pretty rare. So we'll want the ability to traverse deeply into ShadowRoots, possibly multiple branches of the tree of trees in a single test to test cross-components interactions.

Consider a search form with three custom components - a text field, a button, and an output area - that all have shadow roots, all composed inside a shadow root.

One way the test looks like this:

const root = element.shadowRoot;
root.findElement(By.css('x-text-field')).shadowRoot.findElement(By.css('input')).sendKeys('hello');
root.findElement(By.css('x-button')).shadowRoot.findElement(By.css('button')).click();
assert.equals(root.findElement(By.css('x-output')).shadowRoot.getText(), 'hello');

Another way is like this presumably:

driver.switchToShadowRoot(element);
driver.switchToShadowRoot(driver.findElement(By.css('x-text-field')));
driver.findElement(By.css('input')).sendKeys('hello');
driver.switchToShadowRoot(element);
driver.switchToShadowRoot(driver.findElement(By.css('x-button')));
driver.findElement(By.css('button')).click();
driver.switchToShadowRoot(element);
driver.switchToShadowRoot(driver.findElement(By.css('x-output')));
// I'm not sure how to do this - how would we get non-Element Nodes from a Driver instance?
assert.equals(driver.getText(), 'hello');
// And make sure you switch back at the end of the test!
driver.switchToShadowRoot(element);

I personally find that much harder to follow because of the internal state of the driver, and there are certain things that are just not possible because you can't get a reference to a ShadowRoot, ie, you can't pass a root to a helper function, you're IDE won't help identify what root you're referencing on a line because it's not a reference...

Also, now that we're using WebDriver to interact with ShadowRoots, what do all the other APIs on WebDriver do? Why do I care about get(url), getCurrentUrl(), getTitle(), etc.? These can only return the window's state, but it's confusing to me at least to intermix global state like that and the state of an individual element. In contrast, these APIs do apply to Iframes.

Also if the shadowRoot is removed from that node

It's not possible to remove a ShadowRoot, even if you delete the shadowRoot reference. The only way a ShadowRoot is destroyed is if its host is - so the danger is exactly the same as if an Element was removed from the document.

Other Answers:

@justnpT Maybe it is not appropriate to mention it here, but for in our project, it was reason #1 to move to Playwright (https://playwright.dev). It has everything you could dream of for shadow dom support.

Related Issues:

31
selenium Selenium 3.4.0 Unable to find matching capabilities
@AutomatedTester actually for me it's exactly that driver = webdriver.Firefox() issue. ...
26
selenium selenium.common.exceptions.WebDriverException: Message: Can't load the profile.
this worked for me: Meta - OS: Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux Selenium Versio...
16
selenium Shadow DOM traversal support
The idea is to conceptually treat the shadow DOM in the same way we treat frames ...
14
selenium FluentWait changes WebDriverWait().until(ExpectedCondition...) fails with java.lang.NoSuchMethodError
Hi all Meta OS: Windows 7 Selenium Version: 3.2.0 Browser: Chrome (problem is not related to browser...
4
selenium No way to set maxSessions per browser type in Selenium Grid 4
Hello The convention followed is SE_component-name_option-name with underscores So for node max sess...
269
sdk dotnet build error - the project 'Web' must provide a value for Configuration
I had this infuriating issue when migrating a .NET Core 2.2 app to core 3.0 Steps to reproduce I run...
115
aspnetcore Discussion: ASP.NET Core 3.0 will only run on .NET Core
killing .NET Framework episode 2: same story same characters 🤣 Sadly This is a discussion item for ...
106
sdk dotnet new console, publish, makes a dll not an exe
Can we reduce the confusion somehow? I work on .NET Core and I was confused <OutputType>Exe</OutputT...
71
sdk VS2017 publish: project.assets.json doesn't have a target for .NETCoreApp,Version=v1.1
Adding <RuntimeIdentifier>win7-x64</RuntimeIdentifier> to the csproj beneath the <TargetFramework>ne...
67
MonoGame templates for visual studio 2019
I could not find templates for VS 2017 anywhere so that I had to install VS 2017 to get them ...
64
sdk Unable to find fallback package folder
If you are using Docker and getting this error make sure you have a .dockerignore and do not copy ov...
58
sdk Bad error message if Microsoft.NETCore.App isn't a "platform" in the project.json
Hopefully this helps someone I ran into this error message after upgrading some nuget packages in an...
53
core Cannot run dotnet command after installing 6.0 RC/preview using snap
I found a workaround It is possible to fix the snap package :) These are the commands I used to find...
53
react native windows react-native run-windows command is failing at build step
I was able to find a solution to the following error: × Failed to restore the NuGet packages: Error:...
52
core Unable to load DLL 'libgdiplus': The specified module could not be found.
You can try or https://github.com/mono/libgdiplus I can not use GDI+ related things on Fedora 30 I a...
52
sdk dotnet build results in error CS0579 duplicate attributes with AssemblyVersionAttribute (and others) specified
Setting GenerateAssemblyInfo property to false fixes for all attributes: Building a library project ...
52
mono System.TermInfoReader cannot handle new NCurses 6 TermInfo files
fix is in progress A simple workaround is to set TERM=xterm for example that is some terminfo file t...
51
core You must add a reference to assembly netstandard, Version=2.0.0.0
hi @rosieks I have a similar problem and I remove the RuntimeFrameworkVersion and the problem was fi...
50
core Install dotnet-sdk-3.1 on WSL with Ubuntu 20.04 tells me "Unable to correct problems, you have held broken packages"
After manually finding libicu here http://ftp.us.debian.org/debian/pool/main/i/icu/ ...
49
aspnetcore Rename Razor Components back to server-side Blazor
I think this is a good move There has been no end of confusion about the relationship between Razor ...
49
core .NET Core SDK 2.2 on Linux Ubuntu 18.04: Unable to locate package
It seems that the packages-microsoft-prod.deb tool doesn't properly handle third-party Bionic-based ...
46
sdk dotNet Core SDK 3.1.100 failing to restore because of bad versions
I used the following to get my dotnet nuget sources setup Then it worked Then the restore worked Tha...
42
sdk Version conflicts in test project depending on a Microsoft.AspNetCore.App project
@giggio actually there is a new workaround when using the 2.2 tooling (even for 2.1 apps): Add a ver...
41
sdk Installing and Running on OS X 10.11 SSL Fails to Link Using Brew
I have the same problem about that. Steps to reproduce Following steps here: https://www.microsoft.c...
38
sdk Failed to initialize CoreCLR, HRESULT: 0x80131500
You can find out what is missing by running: Replace /opt/dotnet with the directory you un-tarred th...
37
IdentityServer4 Unauthorized (401) during websocket handshake when authorizing SignalR client with JWT bearer token
I managed to replace the whole authentication handling mechanism and figured out what was happening ...
37
docs Please document how to change default console template.
There is no direct way to do this in .NET 6 There are two approaches: We want feedback Voting on thi...
35
core Unable to install .NET Core SDK 2.1 on WSL Linux Ubuntu 18.04
I had the same problem on a fresh installed Ubuntu 18.04 LTS When I run the last step of the procedu...
35
sdk "dotnet build" failed with error NETSDK1073 for the .NET core 3.0 app with reference to a .NET Standard 2.1 class lib
As a workaround you can put the following in your .NET Core project which references .NET Standard 2...
33
sdk "error MSB3552: Resource file "**/*.resx" cannot be found" only in particular environments
Getting similar error when trying to build on Docker microsoft/aspnetcore-build:2.0 /usr/share/dotne...
32
aspnetcore Blazor Authorization Should Redirect to Challenge When Default Challenge Scheme is Set
This scenario can be accomplished by first defining a RedirectToLogin component like this: and then ...
32
core Failed to bind to address https://127.0.0.1:5001: address already in use
If people are still commenting on a closed issue Trying to run a freshly created project I get the f...
28
core Port localhost:5000 not released causing error System.IO.IOException: Failed to bind to address http://127.0.0.1:5000: address already in use.
@wfurt: I try with lsof -i:5000 it shows occupied by dotnet so I kill it manually and then continue....
28
sdk [Arch Linux] MSBUILD : error MSB1025
Per @lewurm on the corefx issue: another workaround to this issue when using a 6.1 ncurses is to inv...
28
sdk In process hosting is not supported for AspNetCoreModule. Change the AspNetCoreModule to atleast AspNetCoreModuleV2.
I was getting the same error after upgrading from 2.1 to .Net Core 3.0 I have created a Asp.net core...
27
aspnetcore Enable web.config transforms for AspNetCore projects in VS
Totally agree with the first post It's by design pretty flexible but at the end it's turned out to b...
26
core This software needs to be updated. Contact the developer for more information.
There's a workaround and more info posted here: #3685 (comment) Show the file in the Finder and open...
24
sdk dotnet restore (OSX)
Looks like this fixes the issue: Credit goes to @davidfowl error: The type initializer for 'System.N...
24
lando Unhandled rejection Error: problem parsing null.tooling.cache. Ensure it is valid JSON!
Hi I had the same issue just now and apparently I only missed adding the name on my .lando.yml file ...
24
xamarin macios [Meta] Xcode 12.0 Support
Hey everyone - I know there was a lot of interest in WidgetKit This is a meta issue tracking the sta...
23
aspnetcore No CORS headers sent in case of error 500
From @Jungers42 on Thursday November 30 2017 1:15:53 PM Ugh .. From @iwbo on Wednesday October 19 ...
23
aspnetcore ModelState cannot be added to TempData (cannot be serialized)
Laravel (PHP) has redirect function in controller action which can be fluently chained to include er...
23
sdk Build error "An item with the same key has already been added."
I found another solution that seems to fix this problem do a: dotnet build throws an error when proj...
23
Paket Paket.bootstrapper.exe unable to communicate with GitHub (5.138.5)
Workaround Add registry keys to force NET framework to use strong encryption For example PowerShell ...
22
aspnetcore After upgrading to ASP.NET Core 2.1, get error that project will use 2.1.0-rc1. How to fix?
Adding <TargetLatestRuntimePatch>true</TargetLatestRuntimePatch> in my Project.csproj ...
22
core .NET Core 2.1 Roslyn cannot find System.Decimal in assemblies
I believe the absolutely right way would be to make use of compilation context preservation In your ...
22
mono Unable to update ca-certificates-mono package
I need to investigate more on how to prevent this from happening but for now please just mv /etc/mon...
21
dependabot core Dependabot triggered Actions cant access secrets or use a writable token
Hey folks Dependabot PM here First off apologies for the quick change and continued brokenness 😞 Wh...
21
aspnetcore Problems installing Dot Net Core 1.0.0 VS 2015 Tools Preview 2
Step 1 Open Command Prompt Step 2 Goto the folder path in command prompt where DorNetCore exe is loc...
21
aspnetcore Jwt Authorization .NET Core 2.0 always return Unauthorized (HTTP 401)
Order matters try this: Hello there I'm trying to do a JWT authentication in my web api application ...