[FIXED] Scrapy find element by part of id and split attribute value

Issue

I'm new to Scrapy and am trying to create a CSS selector that finds an element by part of the id, and split its value attribute to get the nth item. So far I have been working with simple selectors, like these:

item['url'] = response.url # get url        
item['address'] = response.css('span.address::text').get().strip() # get address

However, now I want to select this element (all that have id start with mger) and split the value in its value attribute:

<input type="hidden" id="mger21110564343" name="mger21110564343" value="21110564343~1~50.37396877,5.90523487~2222GH">

I've been studying this page: https://docs.scrapy.org/en/latest/topics/selectors.html#id1 and Google but am still not sure how to do it.

Normally with regular CSS selectors I would use [id^=mger].

Pseudocode of what I want, to get value 21110564343 from value attribute:

response.css("[id^=mger]").attr('value').get().split("~")(0)

Solution

Your css should work, but the selector should look like this:

selector.css('[id^=mger]::attr(value)').get()

with scrapy you can also use xpath (in fact, css selectors get translated to xpath), so you can also use:

response.xpath('//*[starts-with(@id, "mger")]/@value').get()

If you are not getting information, maybe it means that the elements don't actually exist in the response body, you'll have to confirm that first

Answered By - eLRuLL