[FIXED] Python script to pull a table from url and output it in a csv

Issue

I am trying to get the table from this website, https://caniwin.com/poker/omahahilopreALL.php

What I want to do is write a python script that will take that data, and put it in a csv so I can sort by WinHi %

The script I currently have so far does this

import requests
import csv
from bs4 import BeautifulSoup

# Fetch the HTML content from the website
url = 'https://caniwin.com/poker/omahahilopreALL.php'
response = requests.get(url)
html_content = response.text

# Parse the HTML
soup = BeautifulSoup(html_content, 'html.parser')

# Find the table
table = soup.find('table')

print(table)

Which prints the table fine. The issue is, since I am using libre office, when I have tried to parse and put this into a comma seprated file it looks at janky or does not work.

For example, this script does not output it in a manner in which I can sort by value I want

import requests
import csv
from bs4 import BeautifulSoup

# Fetch the HTML content from the website
url = 'https://caniwin.com/poker/omahahilopreALL.php'
response = requests.get(url)
html_content = response.text

# Parse the HTML
soup = BeautifulSoup(html_content, 'html.parser')

# Find the table
table = soup.find('table')

# Extract table data
table_data = []
for row in table.find_all('tr'):
    row_data = []
    for cell in row.find_all(['td']):
        row_data.append(cell.text.strip())
    table_data.append(row_data)

# Output table to CSV file
filename = 'output.csv'
with open(filename, 'w', newline='') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerows(table_data)

print(f"Table data has been saved to {filename}")

Solution

You can use pandas to create a dataframe and save it to CSV:

import requests
import pandas as pd
from bs4 import BeautifulSoup


url = 'https://caniwin.com/poker/omahahilopreALL.php'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for td in soup.table.tr.find_all('td'):
    td.name = 'th'
df = pd.read_html(str(soup.table))[0]

print(df.head())
df.to_csv('data.csv', index=False)

Prints:

   Rank   Hole Cards  Overall  WinHi %  TieHi %  WinLo %  TieLo %  Occur %  Cumul %
0     1  Ax Ay 3x 2y  30.7361  18.9909   0.3723  28.0416  14.9648   0.0044   0.0044
1     2  Ax Ay 4x 2y  29.0354  19.1242   0.5388  25.2638  13.4865   0.0044   0.0088
2     3  Ax A- 3x 2-  27.7786  14.6640   0.3909  28.0942  15.0578   0.0088   0.0177
3     4  Ax A- 3- 2x  27.7640  14.6603   0.3902  28.0779  15.0054   0.0088   0.0266
4     5  Ax Ay 5x 2y  27.3725  19.2939   0.6131  22.5001  11.7472   0.0044   0.0310

and saves data.csv (screenshot from LibreOffice):

Answered By - Andrej Kesely