[FIXED] How to scrape only texts from specific HTML elements?

Issue

I have a problem with selecting the appropriate items from the list.

For example - I want to omit "1." then the first "5" (as in the example) Additionally, I would like to write a condition that the letter "W" should be changed to "WIN".

import re
from selenium import webdriver
from bs4 import BeautifulSoup as BS2
from time import sleep

driver = webdriver.Chrome()
driver.get("https://www.flashscore.pl/druzyna/ajax/8UOvIwnb/tabela/")
sleep(10)
page = driver.page_source
soup = BS2(page,'html.parser')
content = soup.find('div',{'class':'ui-table__body'})
content_list = content.find_all('span',{"table__cell table__cell--value"})

res = []
for i in content:
    line = i.text.split()[0]
    if re.search('Ajax', line):
        res.append(line)
print(res)

results

['1.Ajax550016:315?WWWWW']

I need

Ajax;5;5;0;16;3;W;W;W;W;W

Solution

I would recommend to select your elements more specific:

for e in soup.select('.ui-table__row'):

Iterate the ResultSet and decompose() unwanted tag:

e.select_one('.wld--tbd').decompose()

Extract texts with stripped_strings and join() them to your expected string:

data.append(';'.join(e.stripped_strings))

Example

Also making some replacements, based on dict just to demonstrate how this would work, not knowing R or P.

...
soup = BS2(page,'html.parser')
data = []

for e in soup.select('.ui-table__row'):
    e.select_one('.wld--tbd').decompose()
    e.select_one('.tableCellRank').decompose()
    e.select_one('.table__cell--points').decompose()
    e.select_one('.table__cell--score').string = ';'.join(e.select_one('.table__cell--score').text.split(':'))
    pattern = {'W':'WIN','R':'RRR','P':'PPP'}
    data.append(';'.join([pattern.get(i,i) for i in e.stripped_strings]))
data

To only get result for Ajax:

data = []

for e in soup.select('.ui-table__row:-soup-contains("Ajax")'):
    e.select_one('.wld--tbd').decompose()
    e.select_one('.tableCellRank').decompose()
    e.select_one('.table__cell--points').decompose()
    e.select_one('.table__cell--score').string = ';'.join(e.select_one('.table__cell--score').text.split(':'))
    pattern = {'W':'WIN','R':'RRR','P':'PPP'}
    data.append(';'.join([pattern.get(i,i) for i in e.stripped_strings]))
data

Output

Based on actually data it may differ from questions example.

['Ajax;6;6;0;0;21;3;WIN;WIN;WIN;WIN;WIN']

Answered By - HedgeHog