New issue
Advanced search Search tips
Note: Color blocks (like or ) mean that a user may not be available. Tooltip shows the reason.
Starred by 64 users

Issue metadata

Status: Available
Owner:
Cc:
Components:
EstimatedDays: ----
NextAction: ----
OS: ----
Pri: ----
Type: ----



Sign in to add a comment

Headless mode doesn't save file downloads

Reported by i...@taapo.com, Feb 27 2017 Back to list

Issue description

Chrome Version       : Chromium 58.0.3023.0

What steps will reproduce the problem?
(1) Set headless mode (--headless) on command-line
(2) Point URL to downloadable file
(3) Nothing happens

What is the expected result?

When launching in headless mode and pointing to an URL with a downloadable file, file should be downloaded and saved in "Downloads" folder.

What happens instead?

Nothing happens, file doesn't get downloaded.


 
Cc: mkwst@chromium.org eseckler@chromium.org altimin@chromium.org
Status: Available
We most likely want to this to be something you need to either control or enable via DevTools to avoid letting random websites drop files on your disk just because you're running in headless mode.

Strawperson:
- Add a DevTools command for registering for download-related events.
- Modify DownloadManager so that downloads can be streamed to DevTools rather than going to a specific directory.

Comment 2 by bar...@gopromo.pl, Mar 6 2017

do we really need it off by default? Even when running in normal mode as far as i can tell 'download.prompt_for_download' defaults to false. I have managed to implement basic download in headless creating HeadlessDownloadManagerDelegate that's very similar to ShellDownloadManagerDelegate. If that kind of solution is acceptable i would like to  contribute a fix for this issue.
If we want, we could implement a default DevTools handler in Headless Shell that allows downloads by default (or with a command line switch).

The reason for preferring DevTools is that anything else will be hard to customize to all the different use cases folks have.
@bar...@gopromo.pl, can you publish your implementation as temporary workaround? It's a really critical feature

Comment 5 by bar...@gopromo.pl, Apr 13 2017

of course, i'll try to do that over the weekend

Comment 6 by bar...@gopromo.pl, Apr 15 2017

here's a patch with my change
https://gist.github.com/bartoszalksnin/a7652bedad431ea2badf6418d811d7be 

headless_download.diff
12.5 KB Download
hi bar...,

is there any chance of getting a binary for headless-with-downloads [for windows]?  i don't have the machinery set up to take advantage of a diff file.

thanks in advance [no guarantees expected, of course].

-dh

Labels: HeadlessUpForGrabs
hi chrome developers,

just a plea to let downloads function with headless.  i intend to use selenium/chromedriver to run a python program to download PDFs from the medical literature.  about 75% of the articles can be gotten with the requests module (python), but about 25% of the journal sites have dynamic DOMs which require clicking, i.e. selenium.  a prototype works fine with regular chrome, but i'd also like to do multiple documents at once (in separate processes).  to do this, i can't have multiple chrome instances (a) flashing on the screen and (b) contending for clicks; i.e. i need chrome to run in headless mode.  [my prototype runs fine with the headless version in chrome v 60, but no documents get downloaded].

thanks.

-dh
Owner: sushkov@chromium.org
I'll take a look at this.
So this is a simple outline of my proposed solution:

1. create a HeadlessDownloadManagerDelegate
2. By default this delegate rejects all download requests.
3. Create a new DevTools domain and download_handler owned by the HDMD from the browser context.
4. Add DevTools commands to accept downloads by default to a default path
5. Add DevTools events to get callbacks when downloads are requested so that the devtools client can explicitly allow/reject downloads.

Does this sound ok and like it would address the issue? The default behaviour would still be safe in that downloads are rejected, but the devtools client can change the default behaviour or explicitly handle individual requests.
It sounds good.  I assume the user could change the download default path.
Would there be a way to know the download is complete? Or would there be an
implicit wait ( I use chromedriver with selenium)?  Thanks.
Could probably raise an event on DevTools when the DownloadItem completes.
Cc: skyos...@chromium.org
One issue of simply saving downloads to a directory is that it might not be compatible with some cloud/sandboxed/remote environments, for example. As Sami mentioned in #1/#3, we'll want a way to send/stream the downloaded file's contents to the DevTools client (DevTools has a streaming API under the IO domain that might be useful for this).

That said, we could consider two modes for downloads, either (a) to a local directory as you suggest or (b) streamed via DevTools. Maybe we can start with (a) since it seems much easier to realize and add (b) later. Sami, WDYT?
one issue for me with a streaming interface (which i assume looks to python
like an iterator) is that it can be difficult to tell the end  of the file
from a simple delay  in the middle of transmission.  that said, an iterator
would otherwise be OK.
#14: My thought was that since (a) can be implemented in terms of (b) then we should probably just do (b) :) (a) could then be implemented as a simple utility in Headless Shell.
C#14 sounds fine to me.  as an additional bonus -- not a deal breaker -- it
would be good if the user could also control the name of the file after
downloading (the default is the "host" name of the file, which is rarely
what you want).  i am using chrome to download files and would like to
change filenames from something like "1-s2.0-S0002929710005987-main.pdf", a
typical filename on the host, to something like
"Smith_2016_File_downloading.pdf".  but this is only a desideraturm, not
essential.
#16: sure. my concern is that (b) requires a more invasive change to the download manager / delegate interface (that's all about files, rather than streams, at the moment). So it may be nice if we can get a download-to-file mode out quickly, before tackling that problem? Btw, the headless_shell utility also wouldn't won't work if --remote-debugging-port is used (two client problem), right?

#15: there'd still have to be an event for "download complete" in the streaming interface too :)
It would be nice for this not to have to use the Debug Protocol. We're looking into using downloads to get canvas snapshots onto disk with createObjectURL because they take too long to ship across with Debug Protocol (about 20 times slower, in fact).
Components: Internals>Headless
#19: Interesting, that's 20 times slower to do a readback over DevTools in headless mode compared to downloading in regular Chrome? That overhead seems a lot larger than I would expect.
#21: Yes, we can do a 1920x1080 canvas snap (using Blob) and download (a.download or Filesystem API) in 200ms. Sending the same snap as base64 over the dev protocol takes 4-5 seconds (and to be clear it's not the base64 taking the time). Page.createScreenshot has the same issue, which leads me to believe it's simple serialization/shipping/deserialization doing it.
I have an initial CL that will introduce the minimal functionality needed to download files in Headless mode. Later I intend to add the ability to get events through devtools when a download is requested and allow/deny the downloads individually.

CL is here: https://codereview.chromium.org/2886693002
hi sushkov,

hopefully, this is the right bug section for headless file downloading.

i have the dev channel version of M60 (3107.4), but (at least with
chromedriver and selenium) file downloading doesn't work, although
everything else seems to work fine.  i can navigate to the button with the
URL for the PDF i want in headless mode, but the selenium command
driver.get(button.get_attr('href') fails.  the same code works perfectly in
normal (visible) mode.

i assume this is related to the issue you address in C#23.  any updates?

thanks in advance.

-dh
#23: will this minimal functionality be part of a release? The would fulfill my urgent needs.
Or will the release contain the full functionality (including events)?

Any plans on when the download feature in headless mode will be part of a release?
#25: The minimal functionality will likely be in Chrome M61. More flexible events have yet to be implemented.
Cc: sushkov@chromium.org
Owner: ----

Comment 28 by py...@summitps.org, Jun 29 2017

Hey y'all,

I was just wondering if anyone is going to pick this up. It doesn't seem to be working in the dev release (61.0.3141.7) on Windows x64, but then again, it doesn't seem like the code review was ever finished on that minimal functionality.

I just wanted to let you know I'm looking forward to this feature release!
Owner: dvallet@chromium.org
I'll look into patching Oleg's fix and see how difficult is to land

Comment 30 by mbpa...@gmail.com, Jul 18 2017

Hi everyone, 

Is there any update on this issue? Doesn't seem to be in the latest build (61.0.3161.0).
Is there a quick way for me to test/use the download feature in headless mode?
Or would I have to build everything locally? 
This gets my vote too. Having this feature would allow us to replace our current use of phantomjs and slimerjs with a more future-proof (well, -resistant, anyway) solution. 
Please fix it!! I need to donwload using headless
I may have some cycles to do into this. I'm patching https://codereview.chromium.org/2886693002 and finalizing the cl...
I've tried to apply this diff (https://codereview.chromium.org/download/issue2886693002_100001.diff) to 2 refs/tags (60.0.3112.78 and 60.0.3095.0) and in each case I get a broken binary i.e. if I run out/Headless/headless_shell it crashes with `Received signal 11 SEGV_MAPERR 000000000000`.

I did have to modify the .diff for 60.0.3095.0, however it applied without requiring modification for 60.0.3112.78.

I am building on an AWS c4.4xlarge machine using AMI Linux (amzn-ami-hvm-2017.03.1.20170623-x86_64-gp2 (ami-a4c7edb2)).

NB I doubt it's an environment/build problem - building without the patch applied results in runnable binaries.

Do I need to start with a different refs/tags?
I came up with similar problems, there's some fixing needed. My plan is to
submit a different patch, but it will depend on how difficult is to fix
things
I've discovered one fix which is simple, not sure whether you found others? In `headless/lib/browser/headless_browser_impl.cc` the patch is adding code after:

browser_contexts_[browser_context->Id()] = std::move(browser_context);
// new code which uses browser_context - which is now null

If you move the new code after the line which std::move(browser_context) then it runs. I haven't tested it very thoroughly 
I mean if you move the new to ABOVE (not below) the line - then it builds and runs without seg faulting 
But alas it doesn't seem to work. I run the binary then using another un-headless chrome go to chrome://inspect and drive the headless_shell to a test page. I click my "download" test button (which downloads a .pdf in non-headless chrome) and unfortunately no file is downloaded by the headless_shell. I get the following error in the shell where headless_shell is running:

[0727/001432.328335:VERBOSE1:navigator_impl.cc(241)] Failed Provisional Load: http://192.168.8.100:3000/download/, error_code: -3, error_description: , showing_repost_interstitial: 0, frame_id: 6
Here's my working cl if you are willing to try: https://chromium-review.googlesource.com/c/590913

Note that Mac tests are failing, so it may not work on Mac yet. 

Here's an example on how to run test (also attached):

const CDP = require("chrome-remote-interface");

async function setDownload () {
  const client = await CDP({tab: 'ws://localhost:9222/devtools/browser'});
  const info =  await client.send('Browser.setDownloadBehavior', {behavior : "allow", downloadPath: "/tmp/"});
  await client.close();
}

async function getDownload () {
 const protocol = await CDP({port:9222});
  try {
    const {Page, Runtime} = protocol;
    await Page.enable();
    await Page.navigate({url:"http://ipv4.download.thinkbroadband.com/5MB.zip"});
  } catch (err) {
    console.error(err);
  } finally {
    protocol.close();
  }
}

setDownload().then(getDownload()).catch((err) => console.error(err));

download.js
676 bytes View Download
Awesome thanks! I'll check it out tomorrow 
It's working for me (on AMI linux). Huge thanks!
But what about Windows?
Should work on Windows. 
I'm having problem passing the try bots tests (they time out for some reason) so I'll need to figure how to fix that to submit.
Great to see this progressing!
Any plans on getting it working on Mac ?
Howdy - I don't have a windows build machine atm. I may set one up but probably only towards the end of the week,
What Chrome version are we potentially looking at for release at this point?
The first CL will enable the basic functionally for Windows, Mac, and Linux

This will be released in 62
have we found the ability to use devtools to allow for downloads in the headless browser? 
Project Member

Comment 49 by bugdroid1@chromium.org, Aug 23 2017

The following revision refers to this bug:
  https://chromium.googlesource.com/chromium/src.git/+/1168352b7fc90f83f71ae014f0d97f6ce7402786

commit 1168352b7fc90f83f71ae014f0d97f6ce7402786
Author: David Vallet <dvallet@chromium.org>
Date: Wed Aug 23 03:20:46 2017

initial version of the headless download manager delegate

This cl implementes setDownloadBehavior devtools command, for both headles/non-headless


BUG=696481

Change-Id: Ib4385e478e89219062241983a6ceb523bf476d09
Reviewed-on: https://chromium-review.googlesource.com/590913
Commit-Queue: David Vallet <dvallet@chromium.org>
Reviewed-by: Pavel Feldman <pfeldman@chromium.org>
Reviewed-by: Sami Kyöstilä <skyostil@chromium.org>
Reviewed-by: Eric Seckler <eseckler@chromium.org>
Cr-Commit-Position: refs/heads/master@{#496577}
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/BUILD.gn
[add] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/devtools_download_manager_delegate.cc
[add] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/devtools_download_manager_delegate.h
[add] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/devtools_download_manager_helper.cc
[add] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/devtools_download_manager_helper.h
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/devtools_protocol_browsertest.cc
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/page_handler.cc
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol/page_handler.h
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/browser/devtools/protocol_config.json
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/content/shell/browser/shell_download_manager_delegate.cc
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/third_party/WebKit/Source/core/inspector/browser_protocol.json
[modify] https://crrev.com/1168352b7fc90f83f71ae014f0d97f6ce7402786/third_party/WebKit/Source/core/inspector/inspector_protocol_config.json

Is this fix available in current build of #62 for Windows? If yes, is the file download enabled by default or do we need to pass any switch to turn it on?
It should be available in current build, see the devtools, it needs to be enable with Page.setDownloadBehavior: https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-setDownloadBehavior
@dvallet.. is there a way to enable it through Selenium-ChromeDriver? Thanks in advance!
That would be a feature request for ChromeDriver, feel free to raise it!
@dvallet Is it possible to provide custom functionality for the download? For example, I want to intercept the downloads and have it uploaded to another place instead of downloading the file locally.
I played a bit around with it and it looks like if I download a file twice with the same name the behavior is a bit different from chrome as it dows not save the second file with an appended number like "file (1).pdf".
To be a bit more previse, this is not only in headless mode, this is now always when using the devtools protocol for a download. With and withoud the --headless option.

In fact it overwrites the existing file with the same name.
@dvallet which version contains this feature? I tried 62.0.3202.9, but I got error " TypeError: Page.setDownloadBehavior is not a function"

I am glad to see this being implemented.  For the uninformed, can someone describe how to enable downloads?  E.g., how to change Page.setDownloadBehavior
@djbore

https://bugs.chromium.org/p/chromium/issues/detail?id=696481#c39

Has an example of node js usage. There are similar libraries for python and ruby.
This should be available since: 62.0.3196.0
More information on how to enable can be found in the devtools command: https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-setDownloadBehavior

Regarding repeated download: yes the behavior it's a bit different, Chrome's default behavior is to rename, while headless will overwrite by default. We'll adding a bit more control to handle downloads in the future


Note that the behavior has changed and setDownloadBehavior, so https://bugs.chromium.org/p/chromium/issues/detail?id=696481#c39 would not work.: It is now associated to the Page, rather than the browser domain. Here's some node js code that shows its usage

-----------------------

const CDP = require("chrome-remote-interface");

async function getDownload () {
 const protocol = await CDP({port:9222});
  try {
    const {Page, Runtime} = protocol;
    await Page.enable();
    await protocol.send('Page.setDownloadBehavior', {behavior : "allow", downloadPath: "/tmp/download1"});
    await protocol.send('Page.setDownloadBehavior', {behavior : "default"});
    Page.navigate({url:"<<File-to-download>>"});
    await Page.loadEventFired();
  } catch (err) {
    console.error(err);
  } finally {
    protocol.close();
  }
}

getDownload().catch((err) => console.error(err));

@dvallet How about keeping the same behavior for headless and just provide an option to change to overwrite? That seems to be a bit more constant.
@dvallet Also it looks like if you set the download behavior in non headless mode the browser crashes when you download with a seg fault.
@dvallet When I try to download in headless mode (for example https://education.github.com/git-cheat-sheet-education.pdf), file is saved in download directory, but function OnLoadEventFired is not called.
How can I know download is completed?
#62: We are working on giving more fine grained instrumentation, so we'll keep this in mind!

#63: That sounds like a bug, would you mind reporting it: crbug.com/new

#64: We are working on adding Download events notifications, coming soon!

Comment 66 Deleted


headless mode downlod work by using puppeteer,is there any way to enable the Download behavior in java or command line on windows?
@iceleeku...  The best way to do download via the Windows command line is to use Node JS.  You can then create a small .js script as in Comment #61 that will start Headless Chrome and do the download. Use the "chrome-launcher" Node package to start Chrome, and the chrome-remote-interface package to go to the website and start the download.
So, how could i proceed on headless mode using selenium in C#? I tried wraping nodejs and still doesn't work 
How can this be implemented in python/selenium/windows? I think there's quite a few people who could really use this but don't have the js or chrome knowledge to continue.
I'm using:

Google Chrome	61.0.3163.100 (Official Build) (64-bit)
Revision	57c9d07b416b5a2ea23d28247300e4af36329bdc-refs/branch-heads/3163@{#1250}

ChromeDriver 2.30 (undefined)

How can we set this experimental feature to allow for downloads in ruby Selenium::WebDriver? We have the following in place:

# Register the driver
Capybara.register_driver :chrome do |app|
   opts = { args: %w[headless window-size=1280,1024] }
   prefs = { prefs: { 'download.default_directory': DownloadHelpers::PATH.to_s,
                      'plugins.plugins_disabled': ["Chrome PDF Viewer"],
                      'browser.set_download_behavior': { behavior: "allow" } } }
  caps = Selenium::WebDriver::Remote::Capabilities.chrome( chromeOptions: opts.merge(prefs) )
  Capybara::Selenium::Driver.new(app, browser: :chrome, desired_capabilities: caps)
end

Comment 72 Deleted

I am looking for a way to transmit the same arguments at the python selenium module. 
Anyone having done this ?
Many tries but not succeeded for now...

Comment 74 Deleted

Is there a way download files can be enabled with the  C++ API? 
Thank you
You can create a DevToolsClient and send the commands through there. 
For instance, that's what we do to capture screenshots in headless_shell: https://cs.chromium.org/chromium/src/headless/app/headless_shell.cc?dr=CSs&l=467

I'm driving chromium using golang / agouti, which runs chromium-driver. How can I insert this command? IIUC, connecting to the port and sending the command won't affect the session controlled by chromium-driver?

It would sure help if chromium had a command-line flag for this. Or, better yet, if headless mode worked the same as normal mode, so I can do apples-to-apples testing without excessive gyrations.

Any help would be appreciated!
I was able to trace down how to do this through chromedriver by looking at the source code and noticing the /session/:session_id/chromium/send_command path. I'm using an older version of the selenium-ruby gem and working code looks like this:

  def enable_chrome_headless_downloads(driver, directory)
    bridge = driver.send(:bridge)
    path = '/session/:session_id/chromium/send_command'
    path[':session_id'] = bridge.session_id
    bridge.http.call(:post, path, {
      "cmd" => "Page.setDownloadBehavior",
      "params" => {
        "behavior" => "allow",
        "downloadPath" => directory,
      }
    })
  end

Comment 78:

Thank you!!! Can you post your version of selenium-ruby gem for posterity? 

Cheers
Here is a python implementation, built upon Comment 78 above (Thanks!).

I had to add the command to the chromedriver commands. I will try to submit a PR so it is included in the library in the future. 

    def enable_download_in_headless_chrome(self, browser, download_dir):
        #add missing support for chrome "send_command"  to selenium webdriver
        browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')

        params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
        browser.execute("send_command", params)

Shawn - thank you so much for posting your code. I tried your suggested method, but without success I'm afraid. In my case I can download the file correctly when not headless. However, I am unable to do so when headless, even after adding support for 'send_command', as you suggested. I'm using Chrome 62.0.3202.94, ChromeDriver 2.33.506120 and Python 3.6.0 on 64-Bit Windows 7. Can you think of a reason why your code would not work for me? Is your browser parameter an instance of Selenium's Chrome WebDriver?
Stewart,

Yes the "browser" is an instance of selenium.webdriver.chrome.webdriver.WebDriver 

I've stripped down the class I use to instantiate the driver and posted it with a test to github: https://github.com/shawnbutton/PythonHeadlessChrome

I am able to run this test to download a file from an internet site.

Hopefully this will help.

Shawn - any solution on Java?
shawn solution works for me!
Hi,

How do I add the command to the chromedriver commands like shawn said? I can't get file downloaded in Headless Chrome.

POST: "I had to add the command to the chromedriver commands. I will try to submit a PR so it is included in the library in the future. 

    def enable_download_in_headless_chrome(self, browser, download_dir):
        #add missing support for chrome "send_command"  to selenium webdriver
        browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')

        params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
        browser.execute("send_command", params)"


My sample download code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

if __name__ == '__main__':
    chrome_options = Options()
    chrome_options.add_argument("--headless")
    browser = webdriver.Chrome(chrome_options=chrome_options,
                               executable_path=r'C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe')
    browser.set_window_size(1024, 768)
    browser.get('https://sisweb.tesouro.gov.br/apex/f?p=2031:2:0::::')
    browser.get_screenshot_as_file('initial-page.png')
    browser.find_elements_by_link_text('NTN-B')[0].click()
    browser.get_screenshot_as_file('initial-page-after-click.png')
    browser.quit()


Thanks in advance.
Hi gilsonmn...@gmail.com,

Below is an example of how you might use the method. It works for me.

A couple of notes:
- make sure you put the download directory into the method call. It will not use the normal default download location.
- don't close the browser until the download is done, or it will abort the download. 

-------------
from time import sleep

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

def enable_download_in_headless_chrome(browser, download_dir):
    #add missing support for chrome "send_command"  to selenium webdriver
    browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')

    params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
    browser.execute("send_command", params)

chrome_options = Options()
chrome_options.add_argument("--headless")
browser = webdriver.Chrome(chrome_options=chrome_options,
                           executable_path=r'C:\Program Files (x86)\Google\Chrome\Application\chromedriver.exe')
browser.set_window_size(1024, 768)

enable_download_in_headless_chrome(browser, "c:\temp")

browser.get('https://sisweb.tesouro.gov.br/apex/f?p=2031:2:0::::')
browser.get_screenshot_as_file('initial-page.png')
browser.find_elements_by_link_text('NTN-B')[0].click()
browser.get_screenshot_as_file('initial-page-after-click.png')
sleep(10)
browser.quit()
Thanks, Shawn. Works perfectly with the following corretion:

enable_download_in_headless_chrome(browser, r"c:\temp")


Guys, I spent a lot of time but didn't figured out how I can do it for C# + Chrome Webdriver (headless)
I managed to get it working using Ruby/Rails/RSpec/Capybara.

Here, built upon Comment #78 (thanks mate!) as well:

Capybara.register_driver :selenium do |app|
  options = Selenium::WebDriver::Chrome::Options.new

  options.add_argument('--headless')
  options.add_argument('--no-sandbox')
  options.add_argument('--disable-gpu')
  options.add_argument('--disable-popup-blocking')
  options.add_argument('--window-size=1366,768')

  options.add_preference(:download, directory_upgrade: true,
                                    prompt_for_download: false,
                                    default_directory: '/User/paulo/projects/app/tmp')

  options.add_preference(:browser, set_download_behavior: { behavior: 'allow' })

  driver = Capybara::Selenium::Driver.new(app, browser: :chrome, options: options)

  bridge = driver.browser.send(:bridge)

  path = '/session/:session_id/chromium/send_command'
  path[':session_id'] = bridge.session_id

  bridge.http.call(:post, path, cmd: 'Page.setDownloadBehavior',
                                params: {
                                  behavior: 'allow',
                                  downloadPath: '/User/paulo/projects/app/tmp'
                                })

  driver
end


// VERSIONS:

- capybara (2.16.1)
- selenium-webdriver (3.7.0)
- ChromeDriver 2.33.506106 (8a06c39c4582fbfbab6966dbb1c38a9173bfb1a2)

Comment 90 Deleted

Can someone please provide code on how to do it in Selenium java that would help us do this?

we have tried the following code but is not working : 

Code 1 ::
	Map<String, Object> commandMap = new HashMap<>();
	String downloadLocation = “<download-location>”;
	commandMap.put("behavior", "allow");
	commandMap.put("downloadPath", downloadLocation);
	Command cmd = new Command(((ChromeDriver) driver).getSessionId(), "Browser.setDownloadBehavior",
									commandMap);
	((ChromeDriver) driver).getCommandExecutor().execute(cmd);

Exception Occurred : 
	Exception:
		org.openqa.selenium.UnsupportedCommandException: Browser.setDownloadBehavior
		Build info: version: '3.7.1', revision: '8a0099a', time: '2017-11-06T21:01:39.354Z'
		System info: host: 'INIDC-KULKAO11', ip: '192.168.56.1', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
		Driver info: driver.version: unknown

Code 1 ::
	Map<String, Object> commandMap = new HashMap<>();
	String downloadLocation = “<download-location>”;
	commandMap.put("behavior", "allow");
	commandMap.put("downloadPath", downloadLocation);
	Command cmd = new Command(((ChromeDriver) driver).getSessionId(), "Page.setDownloadBehavior",
									commandMap);
	((ChromeDriver) driver).getCommandExecutor().execute(cmd);

Exception Occurred : 
	Exception:
		org.openqa.selenium.UnsupportedCommandException: Page.setDownloadBehavior
		Build info: version: '3.7.1', revision: '8a0099a', time: '2017-11-06T21:01:39.354Z'
		System info: host: 'INIDC-KULKAO11', ip: '192.168.56.1', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_131'
		Driver info: driver.version: unknown


Also, when tried with the 'send_command' that too was not working.

While going through the ChromeDriverCommandExecutor code, we couldn't find any way to add support for new commands. And currently the only custom command support provided in it is LAUNCH_APP.

Can someone please help us on how best we can perform this? Or the change would have to come in the Chrome's Command Executor only.

Thanks In Adv. 
Need help urgently.
Anyone got a sample for C# ?

Thanks in advance
Hi Team

Please find below a hack that i have used in Java to achieve this functionality. 

------------------------------------------------------------------------
static void sendCommandForDownloadChromeHeadLess(HttpCommandExecutor driverCommandExecutor,SessionId sessionId,String downloadPath) {
		Json json = new Json();
		Map<String, Object> paramsMap = new HashMap<>();
		paramsMap.put("cmd", "Page.setDownloadBehavior");
		Map<String,String> cmdParamsMap = new HashMap<>();
		cmdParamsMap.put("behavior", "allow");
		cmdParamsMap.put("downloadPath", downloadPath);
		paramsMap.put("params", cmdParamsMap);
		String content = json.toJson(paramsMap);
		log.debug("The request content is :: {}" ,content);
		URL remoteServerUri = null;
		try {
			Field field = HttpCommandExecutor.class.getDeclaredField("remoteServer");
			field.setAccessible(true);
			remoteServerUri = (URL) field.get(driverCommandExecutor);
		}catch (Exception e) {
			log.debug("The HttpCommandExecutor has been modified please check with the framework team",e);
			log.error("This will cause all the file validations to fail");
			return;
		}
		CloseableHttpClient httpclient = null;
		try {
			httpclient = HttpClients.createDefault();
			URIBuilder builder = new URIBuilder(remoteServerUri.toURI());
			builder.setPath("session/"+sessionId.toString()+"/chromium/send_command");
			HttpPost sendCommandPost = new HttpPost(builder.build());
			sendCommandPost.setHeader("Content-Type", ContentType.APPLICATION_JSON.getMimeType());
			StringEntity entity = new StringEntity(content, ContentType.APPLICATION_JSON);
			sendCommandPost.setEntity(entity);
			CloseableHttpResponse response = httpclient.execute(sendCommandPost);
			int statusCode = response.getStatusLine().getStatusCode();
			log.debug("The Response Status code is {}",statusCode);
			if(statusCode <= 200 && statusCode >= 300) {
				log.debug("Un-Successfull status code received");
			}
		}catch (IOException e) {
			log.error("Error Occured while enabling download file setting for chrome headless mode");
			log.error("This will cause all the file validations to fail",e);
		} catch (URISyntaxException e) {
			log.debug("this should never ever occur");
		}finally {
			if(httpclient != null) {
				try {
					httpclient.close();
				} catch (IOException e) {
					log.warn("Error Occured while closing the http client",e);
				}
			}
		}
		
	}
-----------------------------------------------------------------------------------------------------------------------------

Please note, that this is not the best of the ways to do this. We can definitely do this in a better manner. Once I have figured out that, will try to post that code over here.
Disclaimer : I think This will only work while using chrome headless in standalone mode and will not work with Selenium Grid. 

Thanks to everyone who have provided information for this in other programming languages. Used their code for ref.



For people who are searching for a Solution for C#. A very elegant solution has been provided by Cezary Piątek in his Tellurium framework. 
GitHub link to framework : https://github.com/cezarypiatek/Tellurium

Please refer to the class : Tellurium.MvcPages.SeleniumUtils.ChromeRemoteInterface.ChromeRemoteInterface

https://github.com/cezarypiatek/Tellurium/blob/master/Src/MvcPages/SeleniumUtils/ChromeRemoteInterface/ChromeRemoteInterface.cs

This class has the required implementation.

Comment 95 Deleted

we can use the last solution(using Tellurium) only for Core projects ? 
How we can use it .NET Framework 4.6 ? 
Anyone got a demo for C# ?
Thanks.
Hi. How can we download file from headless chrome using selenium java. Is there any capabilities added recently to chrome to attain this functionality? Do we have JSON string to add those capabilities to selenium Webdriver. Kindly advise.
How can Page.setDownloadBehavior be set in java selenium? Thank you.
I am running into an issue with the JAVA implementation. No matter what, if I attempt to download a file headless no other code will execute after that step as it closes the driver immediately. The error "chrome not reachable." In some cases the driver is closed before the file has even finished downloading. Any help or suggestions? 

Hi Nick,

Did you try adding wait time until download is complete? Can you share me the code you are trying out, I can try.

Thanks 
Hi Ankit,

What jar file is needed for Json json = new Json();code? I am not able to find an object with Json. Can you please advise. It is very critical.

Thanks a lot
We have used the following class :

org.openqa.selenium.json.Json
Hi Ankit,

I have imported selenium-java-2.41.0 jar but not able to find org.openqa.selenium.json.Json. What is the jar that was used for this import?
I have added a wait. The odd thing is, I can download files from other sites. However, what I am attempting to do is click a button which uses gets/post to generate a pdf, which is then downloaded.
I had to import selenium-java.3.7.1 to locate the org.openqa.selenium.json.Json class. 
Also, I just noticed that the content type I am trying to retrieve is Application/pdf and  Application/csv. Not sure if that helps.
i'm having the same issue as you nick. i'm using python implementation recommended in Comment 86. 

In my case the desired behavior is that clicking the download button opens a new tab and then the file downloads in that tab and then the tab closes. In headless mode I click download but nothing happens. Regular mode works fine. Here's my config:

            chrome_options = Options()
            if HEADLESS:
                chrome_options.add_argument("--headless")
                chrome_options.add_argument('--no-sandbox')
                chrome_options.add_argument('--disable-gpu')
                chrome_options.add_argument('--disable-popup-blocking')
                chrome_options.add_argument('--window-size=1440,900')
            else:
                chrome_options.add_argument("--kiosk")
            prefs = {
                'download.default_directory': DOWNLOAD_PATH,
                'download.prompt_for_download': False,
                'download.directory_upgrade': True,
                'safebrowsing.enabled': False,
                'safebrowsing.disable_download_protection': True}
            chrome_options.add_experimental_option('prefs', prefs)
            driver = webdriver.Chrome(
                chrome_options=chrome_options, executable_path=DRIVER_PATH)
            if HEADLESS:
                driver.set_window_size(1440, 900)
                enable_download_in_headless_chrome(driver, DOWNLOAD_PATH) # from comment 86 above
Hi all. How can Page.setDownloadBehavior be set in webdriver io? Thank you.
Status update: the download behavior flag lets you save downloads in headless mode, but we're still looking at adding download notification at the DevTools level.
Hi All. How to configure download behavior flag in web driver java?

Thanks
so the setDownloadBehavior works, but when its turned on, I don't get any network requests/responses for the url chrome headless downloads. Is anybody seeing the same behaviour?
#112: Right, currently downloads aren't seen by the network interception layer. We discussed adding them there as one potential solution here.

Comment 114 Deleted

Hi,

Can anybody help with setDownloadBehavior ? what is the exact code line in java to attain this functionality. Can this be added in chromeOptions under args and prefs?

Thanks
How can I apply it using robot framework please?

i have this code:

Create Chrome Browser
    [Arguments]    ${link_to_open}
    ${chrome_options}=    Evaluate    sys.modules['selenium.webdriver'].ChromeOptions()    sys, selenium.webdriver
    ${prefs}=    Create Dictionary    download.default_directory=${DOWNLOADS_DIR}
   Call Method    ${chrome options}    add_argument    headless 
   Call Method    ${chrome options}    add_argument    disable-gpu
  Selenium2Library.Go To    ${link_to_open}
Shawn and Stewart, in case you and/or other people need to know, that python workaround and possibly all the other workarounds need to be run on a per tab/window basis.

Comment 119 Deleted

Can someone post workaround code for java ?
Yes I tried that but I get the below error.How to fix this ?
[1517577658.639][SEVERE]: Unable to receive message from renderer

ChromeDriverService driverService = ChromeDriverService.createDefaultService();

	       ChromeDriver driver = new ChromeDriver(driverService, options);

	       Map<String, Object> commandParams = new HashMap<>();
	       commandParams.put("cmd", "Page.setDownloadBehavior");

	       Map<String, String> params = new HashMap<>();
	       params.put("behavior", "allow");
	       params.put("downloadPath", downloadFilepath);
	       commandParams.put("params", params);

	       ObjectMapper objectMapper = new ObjectMapper();
	       HttpClient httpClient = HttpClientBuilder.create().build();

	       String command = objectMapper.writeValueAsString(commandParams);

	       String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";

	       HttpPost request = new HttpPost(u);
	       request.addHeader("content-type", "application/json");
	       request.setEntity(new StringEntity(command));
	       httpClient.execute(request);

I tried this code but I'm not able to save that file to a directory 

Comment 124 Deleted

How we can use it .NET Framework 4.5 ? 
Anyone got a demo for C# ?

Comment 126 Deleted

Comment 127 Deleted

 Download in headless mode works fine for me in windows. However, it does not work in Linux. Did anybody made it work in Linux?
The code which works in windows is as shown below: 

Launch Headless Chrome
    Create Download Directory
    ${chrome options}=    Evaluate    sys.modules['selenium.webdriver'].ChromeOptions()    sys, selenium.webdriver
    Call Method    ${chrome options}    add_argument    start-maximized
    Call Method    ${chrome_options}    add_argument    --headless
    Call Method    ${chrome_options}    add_argument    --disable-gpu
    Call Method    ${chrome_options}    add_argument    --window-size\=1920,1080
    Call Method    ${chrome_options}    add_argument    --lang\=en-us
    Call Method    ${chrome_options}    add_argument    --no-sandbox
    ${prefs}       Create Dictionary    download.default_directory=${download directory}    download.prompt_for_download=false   
    Call Method    ${chrome options}    add_experimental_option    prefs    ${prefs}
    Create Webdriver    Chrome    chrome_options=${chrome options}
    Enable Download In Headless Chrome    ${download directory}
    Go To   about:blank

Below is the code used to enable download behavior and setting the download directory for chrome headless.

def enable_download_in_headless_chrome(download_dir):
logger.info('Getting SeleniumLibrary Instance')
instance = BuiltIn().get_library_instance('SeleniumLibrary')
driver = instance.driver
# add missing support for chrome "send_command"  to selenium webdriver
driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')

params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
command_result = driver.execute("send_command", params)
logger.info("response from browser:")
for key in command_result:
    logger.info("result:" + key + ":" + str(command_result[key]))

Comment 129 Deleted

There are Python and Java samples here
Anyone got a sample for C# ?
Thanks in advance
The python samples given here does not work on Linux for me. My code works fine in windows. I have no clue on why it's not working in Linux.
Anyone able to figure how can we achieve this in java ? Thanks in advance
Anyone got an example for C# ?
Thanks in advance
Hi guys, Any kind of c# dev to handle with this issue??

I´m facing this issue after chrome update. I can´t download files in chrome headless mode.

Thanks in advance
 
Hi guys! Does anyone have sample of this workaround on JavaScript? How could it be used with protractor?
Hi guys, who can help me

I have an automation process created with java selenium-webdriver, chromedriver and work with eclipse. The point is that by doing the process with headless and disable gpu I get that the download is correct but it does not appear in the directory and instead if the headless option and gpu if it appears in the directory. Attached the code below and if there is someone who knows how to do it and can help me, I will be grateful, thanks.

PROCESS:

System.setProperty ("webdriver.chrome.driver", "C: \\ Users \\ Dani \\ Desktop \\ driver \\ chromedriver.exe");
    String downloadFilepath = "C: \\ Users \\ Dani \\ Documents \\ BILLS"; // directory download
    HashMap <String, Object> prefs = new HashMap <String, Object> ();
        //prefs.put("profile.default_content_setting_values.automatic_downloads ", 1);
        prefs.put ("profile.default_content_settings.popups", 0);
        prefs.put ("download.prompt_for_download", false);
        //prefs.put("download.directory_upgrade ", true);
    prefs.put ("download.default_directory", downloadFilepath);
    ChromeOptions options = new ChromeOptions ();
        options.addArguments ("- headless");
        options.addArguments ("--disable-gpu");
        options.addArguments ("- window-size = 1920,1200");
        options.addArguments ("--disable-infobars");
        options.addArguments ("--disable-notifications");
        options.setExperimentalOption ("prefs", prefs);
    DesiredCapabilities cap = new DesiredCapabilities ();
        cap.setCapability (CapabilityType.ACCEPT_SSL_CERTS, true);
        cap.setCapability (ChromeOptions.CAPABILITY, options);
    WebDriver driver = new ChromeDriver (options);
    driver.manage (). timeouts (). implicitlyWait (20, TimeUnit.SECONDS); // indicate seconds between actions
Sobre el Traductor de GoogleComunitatMòbil
Quant a GooglePrivadesa i condicionsAjudaEnvia suggeri
I would also love to know if anyone has got this working with protractor
Whats the problem to download file in headless mode and what solution is current run in?
Currently, if a download is being started in a new window via target _blank link or window.open, it's not being saved. Any solution for this?
Our app uses a new tab for downloads and the standard solutions does not work for us either since it only effects the current page.  My theoretical approach is to register for the Target.targetCreated event and apply the Page.setDownloadBehavior command to the new page.  The only problem is I have no idea how to do this via chromedriver.
Javascript solution currently working with:

webdriverio 4.12.0
selenium-standalone 6.13.0
chromedriver 2.36
chrome 65.0.3325.162

on Ubuntu server 16.04.4. It allowes headless chrome session to download file:

const unirest = require('unirest');

let session = browser.session();
// key sessionId is webdriverio implementation, debug your session object
let sessionId = session['sessionId'];
let params = {
    'cmd': 'Page.setDownloadBehavior', 
    'params': {'behavior': 'allow', 'downloadPath': 'absolutePath' }};

unirest
        .post('http://localhost:4444/wd/hub/session/' + sessionId + '/chromium/send_command')
        .send(JSON.stringify(params))
        .end();

It is based on the answer https://stackoverflow.com/questions/48831273/protractor-file-download-test-fails-when-headless-chrome.
For those who were asking for a C# .net solution:

var workingDir = "C:\\WorkingFolder";
var opts = new ChromeOptions();

opts.AddArguments("--headless");

var driverService = ChromeDriverService.CreateDefaultService(workingDir);
var driver = new ChromeDriver(driverService, opts);


// Allow download in headless mode
var param = new Dictionary<string, string>();
param.Add("behavior", "allow");
param.Add("downloadPath", workingDir);

var cmdParam = new Dictionary<string, object>();
cmdParam.Add("cmd", "Page.setDownloadBehavior");
cmdParam.Add("params", param);

var content = new StringContent(JsonConvert.SerializeObject(cmdParam), Encoding.UTF8, "application/json");

var url = driverService.ServiceUrl + "session/" + driver.SessionId + "/chromium/send_command";
var httpClient = new HttpClient();
await httpClient.PostAsync(url, content);

I am facing a similar issue and my code base is Java. When I try to download a file in chrome using headless mode, the download does not happen. But the download works when the chrome is open. Please help me with a code snippet to assist on this problem
    In Java use following code :

System.setProperty("webdriver.chrome.driver", "/usr/local/bin/chromedriver");
             ChromeOptions options = new ChromeOptions();
                    options.addArguments("--test-type");
                    options.addArguments("--headless");
                    options.addArguments("--disable-extensions"); //to disable browser extension popup
    
                    ChromeDriverService driverService = ChromeDriverService.createDefaultService();
                    ChromeDriver driver = new ChromeDriver(driverService, options);
    
                    Map<String, Object> commandParams = new HashMap<>();
                    commandParams.put("cmd", "Page.setDownloadBehavior");
                    Map<String, String> params = new HashMap<>();
                    params.put("behavior", "allow");
                    params.put("downloadPath", "//home//vaibhav//Desktop");
                    commandParams.put("params", params);
                    ObjectMapper objectMapper = new ObjectMapper();
                    HttpClient httpClient = HttpClientBuilder.create().build();
                    String command = objectMapper.writeValueAsString(commandParams);
                    String u = driverService.getUrl().toString() + "/session/" + driver.getSessionId() + "/chromium/send_command";
                    HttpPost request = new HttpPost(u);
                    request.addHeader("content-type", "application/json");
                    request.setEntity(new StringEntity(command));
                    httpClient.execute(request);
            driver.get("http://www.seleniumhq.org/download/");
            driver.findElement(By.linkText("32 bit Windows IE")).click();
Can someone tell me if he could download with headless in java with chromedriver?
Thanks

Comment 146 by a...@view26.com, Apr 18 (3 days ago)

It doesnt work for me in headless mode. 
@Daniel does it work for you ?

Comment 147 by badboss...@gmail.com, Apr 18 (3 days ago)

Guys, help me please. I still get response from browser:
result:status:0
result:sessionId:6d06f404b019eedcf01d430a0515e94e
result:value:None
with chrome 66 and chromedriver 2.37. This is my Python code:

                def enable_download_in_headless_mode(self, browser, download_directory):
                    browser.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
                    params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_directory}}
                    command_result = browser.execute("send_command", params)
                    print("response from browser:")
                    for key in command_result:
                        print("result:" + key + ":" + str(command_result[key]))

                opts.add_argument("--disable-popup-blocking")
                opts.add_argument("--ignore-certificate-errors")
                opts.add_argument("--no-sandbox")
                opts.add_argument("--headless")
                opts.add_argument('--disable-gpu')
                opts.add_argument('--disable-popup-blocking')
                self.logger.info("WebDriverChrome")
                self.download_location = os.path.join(Config.Environment().path, Config.Environment().get_attr(
                    "component") + '/reports')
                prefs = {'download.default_directory': self.download_location,
                         'download.prompt_for_download': False,
                         'download.directory_upgrade': True,
                         'safebrowsing.enabled': False,
                         'safebrowsing.disable_download_protection': True}
                opts.add_experimental_option("prefs", prefs)

                self.driver = webdriver.Chrome(chrome_options=opts, desired_capabilities=d)
                self.enable_download_in_headless_mode(browser=self.driver, download_directory=self.download_location)
                self.driver.set_window_size(width=1920, height=1295)
                self.logger.info("WebDriverChrome - Done")

Sign in to add a comment