Files
Andrei Solntsev c5bb3c9752 16612 download large files (#16627)
* #16612 support downloading large files from Grid

I added a new Grid endpoint "/se/files/:name" which allows downloading the file directly, without encoding it to Base64 and adding to Json. This transformation kills the performance and causes OutOfMemory errors for large files (e.g. 256+ MB).

NB! Be sure that `toString()` method of objects (HttpRequest, HttpResponse, Contents.Supplier) never returns too long string - it spam debug logs and can cause OOM during debugging.

* #16612 extract anonymous implementations of `Contents.Supplier` to separate classes

It makes debugging easier. You can easily see what instances they are and where they come from.

* #16612 optimize method `RemoteWebDriver.downloadFile()`

Instead of reading the whole file to a byte array, just save given InputStream directly to the file.

Now it can download large files (I tried 4GB) while consuming very low memory.

* #16612 just in case, return `Contents.fromStream` only when downloading files.

For json responses, still return `Contents.bytes` which allows re-reading its content multiple times. Just in case.

* #16612 fix flaky test: wait until the downloads folder gets deleted

After stopping a Grid node, the folder is deleted asynchronously (by cache removal listener). So we need to wait for it in test.

* #16612 fix flaky test: wait until the grid node is fully stopped

At least on my machine, stopping the node takes some time, and any checks right after `node.stop(sessionId)` often can fail.

* #16612 fix flaky test LocalNewSessionQueueTest

Gr... This is extremely hard to debug test.

After hours of debugging, I came to a conclusion that we just need to increase the timeout. On my machine, `latch` gets decreased after ~1.2 seconds. So 1 second was not enough.

* #16612 fix flaky test JdkHttpClientTest

I don't know why, but sometimes we receive `HttpTimeoutException` instead of `InterruptedException`. Seems reasonable to consider execution as interrupted in both cases. (?)

* #16612 remove unneeded code

None of `is.readNBytes` implementations returns -1. It always returns 0 or positive number.

* #16612 upload logs in case of test failure

* ignore few more IDEA files

* #16612 slightly improve logging in `W3CHttpResponseCodec.decode`

Don't log the entire response body - just content type and length.
2025-11-28 09:55:04 +01:00

150 lines
2.7 KiB
Plaintext

projectFilesBackup/
*.iws
.DS_Store
.svn
.credentials.dat
.ijwb
mockpiframe.log
mockpiframe.log.lck
junitvmwatcher*.properties
test-output/
.*.swp
*~
.aspect/
common/build
/build/
cpp/iedriver/IEReturnTypes.h
dotnet/.idea/
dotnet/packages/
java/client/src/org/openqa/selenium/ie/IeReturnTypes.java
java/server/test/org/openqa/selenium/example
javascript/deps.js
javascript/selenium-webdriver/node_modules/
javascript/selenium-webdriver/lib/atoms/find-elements.js
javascript/selenium-webdriver/lib/atoms/get-attribute.js
javascript/selenium-webdriver/lib/atoms/is-displayed.js
javascript/selenium-webdriver/lib/atoms/mutation-listener.js
javascript/safari-driver/node_modules/
javascript/webdriver/devtools/types/
.idea/bazelVersionCache.xml
.idea/GitLink.xml
.idea/vcs.xml
.idea/workspace.xml
.idea/dictionaries/
.idea/runConfigurations.xml
.idea/scopes/
.idea/shelf
.idea/sonarlint/
.idea/qaplug_profiles.xml
out
cpp/IEDriver/sizzle.h
third_party/gecko-*
cpp/iedriver/Generated/atoms.h
cpp/iedriver/sizzle.h
chromedriver.log
junit*.properties
*.rbc
.rbx
.yardoc
rb/.yardoc
.tm_properties
maven/*/src
maven/target
maven/*/target
maven/.idea
maven/*.iml
maven/*/*.iml
maven-metadata-.xml
maven-metadata-.xml.sha1
node_modules/
resolver-status.properties
java/build/
java/client/build/
java/client/log.txt
java/client/iedriver.log
java/server/build/
__pycache__
.tox
*.pyc
dist/
py/selenium/webdriver/common/devtools/**/*
!py/selenium/webdriver/common/devtools/util.py
py/selenium/webdriver/common/linux/
py/selenium/webdriver/common/macos/
py/selenium/webdriver/common/mutation-listener.js
py/selenium/webdriver/common/windows/
py/selenium/webdriver/firefox/webdriver_prefs.json
py/selenium/webdriver/remote/findElements.js
py/selenium/webdriver/remote/getAttribute.js
py/selenium/webdriver/remote/isDisplayed.js
py/docs/build/
py/docs/source/**/*
!py/docs/source/conf.py
!py/docs/source/*.rst
py/build/
py/pytestdebug.log
py/python.iml
selenium.egg-info/
third_party/java/jetty/jetty-repacked.jar
*.user
*.cache
dotnet/**/bin/
obj/
*.opensdf
*.suo
*.sdf
*.vs
*.opendb
*.VC.db
*.aps
launchSettings.json
Generated/
ipch/
/iedriver.log
/phantomjsdriver.log
geckodriver.log
ghostdriver.log
.tags
GPATH
GRTAGS
GSYMS
GTAGS
/rb/.bundle
/rb/.idea
rb/bin/
rb/lib/selenium/devtools/v1*
rb/.gem_rbs_collection/
.sonar/
.idea/sonarIssues.xml
.idea/inspectionProfiles/profiles_settings.xml
.idea/inspectionProfiles/Project_Default.xml
.idea/misc.xml
third_party/py/googlestorage/client_secrets.json
.venv
venv
venv3
py/.idea
rust/target
rust/.idea
# Bazel stuff
bazel-bin
bazel-genfiles
bazel-out
bazel-selenium
bazel-testlogs
MODULE.bazel.lock
/.vscode/
/.scannerwork/
/.bazelrc.local
# Engflow
*.crt
*.key
javascript/selenium-webdriver/.vscode/settings.json
dotnet-bin
.metadata/
.npmrc