[FIXED] Excuse me, why my playwright can't get the cookie?

Issue

Firstly, sry for my poor English

I want to use playwright to get the cookie, but I can't. I tried 3 ways I've found, and got nothing.

1. Using page.on

page.on('request',get_cookie)
page.on('response',get_cookie)

def get_cookie(request):
    allheaders = request.all_headers()
    print(allheaders)


>>>
{'accept-ranges': 'bytes', 'age': '9576', 'cache-control': 'max-age=600', 'content-length': '6745', 'content-type': 'image/png', 'date': 'Thu, 30 Jun 2022 01:09:20 GMT', 'etag': '"206578bcab2ad71:0"', 'expires': 'Thu, 30 Jun 2022 01:19:20 GMT', 'last-modified': 'Tue, 06 Apr 2021 06:11:52 GMT', 'server': 'NWS_SPMid', 'x-cache-lookup': 'Cache Hit', 'x-daa-tunnel': 'hop_count=1', 'x-nws-log-uuid': '16892018456232999193', 'x-powered-by': 'ASP.NET'}
{'accept-ranges': 'bytes', 'age': '9576', 'cache-control': 'max-age=600', 'content-length': '6745', 'content-type': 'image/png', 'date': 'Thu, 30 Jun 2022 01:09:20 GMT', 'etag': '"206578bcab2ad71:0"', 'expires': 'Thu, 30 Jun 2022 01:19:20 GMT', 'last-modified': 'Tue, 06 Apr 2021 06:11:52 GMT', 'server': 'NWS_SPMid', 'x-cache-lookup': 'Cache Hit', 'x-daa-tunnel': 'hop_count=1', 'x-nws-log-uuid': '16892018456232999193', 'x-powered-by': 'ASP.NET'}
...(and more like this)

returned somthing, but no cookie here

2. Using browser_context.cookies Resolved! Thx for @Charchit

context = browser.new_context();
page = context.new_page()
page.goto(url)
cookies = context.cookies
print(cookies)

>>>
<bound method BrowserContext.cookies of <BrowserContext browser=<Browser type=<BrowserType name=chromium executable_path=/Users/swong/Library/Caches/ms-playwright/chromium-1005/chrome-mac/Chromium.app/Contents/MacOS/Chromium> version=102.0.5005.40>>>

3. Using JS

cookie = page.evaluate('console.log(document.cookie)')
print(cookie)

>>>
None

I opened the network tab from the Chromium page,there was the cookie I want in Requests' header.

please help me, Thank you all!

Here's my code example. The site is in Chinese language, and hope you will not mind it. It's just a simple login page.

from playwright.sync_api import sync_playwright

url = 'https://so.gushiwen.cn/user/login.aspx'

def get_cookie(request_or_reqponse):
    headersArray = request_or_reqponse.headers_array()
    print('「headersArray」：', headersArray)


with sync_playwright() as p:
    browser = p.chromium.launch(headless=False)
    context = browser.new_context();
    page = context.new_page()

    page.goto(url)
    page.fill('#email','[email protected]')
    page.fill('#pwd', '[email protected]')

    page.wait_for_timeout(5000) # imput the captcha code manually

    page.on('request',get_cookie)
    page.on('response',get_cookie)

    print('loging in...')
    page.click('#denglu')

    page.wait_for_timeout(50000) # wait for nothing

    browser.close()

Solution

In your second method, change cookies = context.cookies to cookies = context.cookies(). It's a method, you need to call it. Check the documentation:

context = browser.new_context();
page = context.new_page()
page.goto(url)
cookies = context.cookies()
print(cookies)

Also, doing it like your first method is not advisable. This is because even if you get the Cookie header from the response, you can't really store and use it else where unless you use a factory function or a global variable. Besides, why do that when BrowserContext specifically has a method for it :)

Edit

The reason the first method seemingly does not work is because it returns the headers of the request and responses made. Cookies can also be created through javascript on the page itself, these may not show up in the headers at all.

Secondly, from the headers you have printed out for the first method in your question, it seems like it was only for a single request. After running your code, there were a lot more requests and responses received, which in place printed out a lot more headers. From the responses in particular, you can retrieve the cookies set by the server by searching for the header 'set-cookie'.

Answered By - Charchit