All the cookies are deleted automatically when the browser loads the first page

Issue #2890 resolved
Salvador Diaz Fau created an issue
  1. What steps will reproduce the problem?
    - Run cefclient.exe --cache-path=cache
    - Load http://magpcss.org/ceforum/
    - Login with a valid username and password
    - Close cefclient
    - Run cefclient.exe --cache-path=cache
    - The cookies were deleted and the user is no longer logged in.
  2. What is the expected output? What do you see instead?
    The cookies shouldn’t be deleted and cefclient should keep the session open but it deletes some cookies as soon as the browser loads a web page.
    I only used the forum as an example. This issue affects many other web pages.
    I opened the “cookies” file with “SQLite Expert Personal” after closing cefclient the first time and the cookies are still there. The browser deletes them as soon as it loads google.com the second time.
  3. What version of the product are you using? On what operating system?
    CEF 80.0.4+g74f7b0c+chromium-80.0.3987.122 on Windows 7 64 bits
    http://opensource.spotify.com/cefbuilds/cef_binary_80.0.4%2Bg74f7b0c%2Bchromium-80.0.3987.122_windows64_client.tar.bz2
  4. Does the problem reproduce with the cefclient or cefsimple sample application at the same version? How about with a newer or older version?
    The previous cefclient (CEF 79.1.38) didn’t have this issue. That version of cefclient keeps the forum session open.
  5. Does the problem reproduce with Google Chrome at the same version? How about with a newer or older version?
    I only could test Chromium 80.0.3987.87 on Linux 64 bits and it doesn’t have this issue.

Comments (18)

  1. Salvador Diaz Fau reporter

    I did more tests with the latest cefclient (CEF 80.0.8) :
    http://opensource.spotify.com/cefbuilds/cef_binary_80.0.8%2Bgf96cd1d%2Bchromium-80.0.3987.132_windows64_client.tar.bz2

    I used “SQLite Expert Personal” to check what cookies were deleted and I tried this command line to run cefclient :
    cefclient.exe --cache-path=cache --url=wikipedia.org
    Then load several other random websites except google and close the browser. Open the cookies database and you'll see all the cookies.

    Then run it again with this command line :
    cefclient.exe --cache-path=cache

    Close it when google is loaded and the database will only have google’s cookies.

  2. Salvador Diaz Fau reporter

    Just one more test with this command line :
    cefclient.exe --cache-path=cache --url=about:blank

    Close cefclient as soon as “about:blank” is loaded and the database still has the cookies from the previous session.

  3. Mitch Capper

    Here is some more testing on the issue:

    V80 has no problem reading former version cookies and presenting them to sites.

    Upon a cookie being set V80 however writes an invalid encrypted_value in the sql database for the cookie that cannot be decrypted (well technically upon browser close).

    When CEF goes to load the cookies for a site if one can’t be loaded then none of the cookies are shown so one corrupt cookie makes the rest seem like they ‘disappeared’ but they are still in the database itself.

    To test go to with a V79 browser:

    http://mitchcapper.com/cookie.html

    Set the 3 cookies (1 2 and 3). Save a copy of the Cookie database to look at later.

    Then close and open the same cookie store in a V80 browser.

    All 3 cookies will display.

    Set cookies 2 and 3. Close the browser. Look at the database you will notice:

    With the correct database:

    Properly encrypted values are 230 bytes long (no matter is the value is “CookVal” or the shorter value “1”.

    With the incorrect database:

    Cookie 2 will be 38 bytes long and the short cookie will be 32 bytes long.

    Once even one invalid value is written to the database all the cookies for the site will not load (in either V79 or V80) despite the rest being OK in the DB.

  4. Mitch Capper

    Once the invalid cookie is set on a site and cef restarted the cookies are erased by:

    https://cs.chromium.org/chromium/src/net/extras/sqlite/sqlite_persistent_cookie_store.cc specifically it says:

    There were some cookies that were in database but could not be loaded and handed over to CookieMonster. ... For data consistency, we drop the entire eTLD group.

    The fact the bytes are shorter may not be an actual bug. The encryption was recently changed to a new AES mode option using DPAPI:

    https://chromium.googlesource.com/chromium/src/+/265b39473af0faac989b44afb6d4eb5cb2fd2e24^!/

    It may be this commit that is causing problems or it may be more likely one of the recent commits to:

    https://cs.chromium.org/chromium/src/components/os_crypt/os_crypt_win.cc

    The most recent commit to os_crypt_win.cc dealing with corrupt keys and regenerating them is present in v80 so it is not the lack of this.

    I am guessing this bug is also Windows only. The issue may be the persistent key is not getting reloaded properly?

  5. Marshall Greenblatt

    Thanks for debugging this problem. I also suspect that 265b39473 is the culprit, and we need to bring some of those changes into CEF.

  6. Marshall Greenblatt

    The problem is that OSCrypt is storing the encryption key in g_browser_process->local_state()the contents of which CEF does not currently persist to disk across application restarts by default. As a result OSCrypt generates a new encryption key on each restart and cookies encrypted with the old (now unknown) encryption key are discarded.

    Setting CefSettings.persist_user_preferences = true should fix this based on the code in ChromeBrowserProcessStub::local_state. Unfortunately there are currently multiple PrefService objects overwriting the same “UserPrefs.json” file in the cache directory.

  7. Marshall Greenblatt

    Always persist local_state by default (fixes issue #2890)

    If a cache_path is specified local_state will now be persisted to a LocalPrefs.json file. This is necessary because local_state is used to store the cookie encryption key on Windows.

    → <<cset 892739653a99>>

  8. Marshall Greenblatt

    Always persist local_state by default (fixes issue #2890)

    If a cache_path is specified local_state will now be persisted to a LocalPrefs.json file. This is necessary because local_state is used to store the cookie encryption key on Windows.

    → <<cset 1606b8e4568f>>

  9. Marshall Greenblatt

    Always persist local_state by default (fixes issue #2890)

    If a cache_path is specified local_state will now be persisted to a LocalPrefs.json file. This is necessary because local_state is used to store the cookie encryption key on Windows.

    → <<cset 3d87a6856131>>

  10. Mitch Capper

    In testing the new code the following behavior was observed.

    The AppPrefs.json with the encryption key is always stored in the CefSettings.cache_path, if that is not set it is not stored and all data encrypted with it will be lost. No other setting seems to affect it. The first case below is a behavior change.
    Cases:

    CefSettings.root_cache_path: RootPath, CefSettings.cache_path: null, RequestContext.cache_path RequestSubPath

    Cookie database created in RequestSubPath, LocalPrefs.json (encryption key) not saved, cookies lost (non-obvious?).

    CefSettings.root_cache_path: RootPath, CefSettings.cache_path: InitSubPath, RequestContext.cache_path RequestSubPath

    Cookie database created in RequestSubPath, LocalPrefs.json created in InitialPath cookies saved

    CefSettings.root_cache_path: RootPath, CefSettings.cache_path: InitSubPath, RequestContext.cache_path null

    Cookie database not created, LocalPrefs.json created in InitialPath cookies not saved (expected)

    Storing the encryption key here would seem to make sense (as it is not per instance and only read once per master process) but may be non obvious by default.

  11. Marshall Greenblatt

    @Mitch Capper Good point about this not working well in combination with CefRequestContextSettings.cache_path. The problem is that OSCrypt and ChromeBrowserProcessStub::local_state are both global to the application. We should probably save the LocalPrefs.json file in CefSettings.root_cache_path instead of CefSettings.cache_path if the root_cache_path value is specified (a value for which is required when using CefRequestContextSettings.cache_path, and will default to CefSettings.cache_path if empty).

  12. Mitch Capper

    Actually I think the current behavior might be desired despite being non-obvious. The problem with saving in root_cache_path is if you need to have cache_paths in multiple locations in one instance of an application without a true shared root.

    For example if you wanted to have two cache paths of:

    c:\temp\test_cache

    c:\AppData\other_cache

    you can do so by setting the root_cache to c:\. Generally one wants to avoid writing data to the root, so for the new global prefs file you can simply set the initial cache_path to something like c:\AppData\GlobalSettings.

    I have no idea if anyone actually uses this use case, but if things start being written to the root_cache_path it might not be expected. Clearly from a sandboxing behavior it is also not desired so maybe should be discouraged anyway.

    Adding a new setting gives another more obvious solution but introducing new setting likely not desired. As OSCrypt is global for other platforms where does it store global data (or is windows the only platform the encrypted key needs to be stored)?

  13. Marshall Greenblatt

    I have no idea if anyone actually uses this use case, but if things start being written to the root_cache_path it might not be expected. Clearly from a sandboxing behavior it is also not desired so maybe should be discouraged anyway.

    Agreed, this should be discouraged.

    Adding a new setting gives another more obvious solution but introducing new setting likely not desired.

    Also agreed.

    As OSCrypt is global for other platforms where does it store global data (or is windows the only platform the encrypted key needs to be stored)?

    On Mac the encryption key is stored in the user’s keychain (also global, but managed by the OS). I’m not sure about Linux.

  14. Marshall Greenblatt

    If anyone thinks that storing LocalPrefs.json in CefSettings.root_cache_path instead of CefSettings.cache_path will be a problem for them please speak up in the next few days. Otherwise, I’ll plan to make the change.

  15. Log in to comment