[FIXED] python beautifulsoup to download specific .zip files from a website

Issue

I am new to web scraping. I am trying to download only the weekly .zip files from the below website. I could able to parse the label and couldn't go beyond that to download the weekly zip files.

https://download.cms.gov/nppes/NPI_Files.html

html code for 'li' tag

import requests
from bs4 import BeautifulSoup
URL = "https://download.cms.gov/nppes/NPI_Files.html"
r = requests.get(URL)
#print(r.content)
soup = BeautifulSoup(r.content, 'html.parser')
#print(soup.prettify())
links=[]

ref = soup.select('li')
#print(ref)

for i in ref:

Solution

Try the next example

import requests
from bs4 import BeautifulSoup

URL = "https://download.cms.gov/nppes/NPI_Files.html"
r = requests.get(URL)
#print(r.content)
soup = BeautifulSoup(r.content, 'html.parser')
#print(soup.prettify())
links=[]

for ref in soup.select('div.bulletlistleft > ul > li')[2:]:
    zip ='https://download.cms.gov/nppes'+ref.a.get('href').replace('./','/')
    links.append(zip)
print(links)

Output:

['https://download.cms.gov/nppes/NPPES_Data_Dissemination_090522_091122_Weekly.zip', 'https://download.cms.gov/nppes/NPPES_Data_Dissemination_091222_091822_Weekly.zip', 'https://download.cms.gov/nppes/NPPES_Data_Dissemination_091922_092522_Weekly.zip']

Answered By - Fazlul