从0开始学爬虫7之BeautifulSoup模块的简单介绍

参考文档:

https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/

# 安装 beautifulsoup4

(pytools) D:\python\pytools>pip install beautifulsoup4

 

# coding=utf-8

from bs4 import BeautifulSoup as bs
import re


html_doc = """
<html><head><title>The Dormouse's story</title></head>

<p class="title"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>

<p class="story">...</p>
"""
soup = bs(html_doc, "html.parser")

# print(soup.prettify())

# print(soup.title.string)
# print(soup.a)
# print(soup.find(id='link2'))

# print(soup.find(id='link2').string)

# print(soup.find(id='link2').get_text())

# 获取所有a标签的内容
# for link in soup.find_all('a'):
#     print(link.get_text())

# 使用string获取不到p标签的内容
# print(soup.find("p", {"class": "story"}).get_text())

# 查找所有以b开头的标签
# for tag in soup.find_all(re.compile("^b")):
#     print(tag.get_text())

# 查找所有的a标签中href以 http://example.com开头的数据,其中的.号代表任意字符,如果需要 表示点号本身可以用\转义   href=re.compile(r"^http://example\.com/")
data = soup.find_all("a", href=re.compile(r"^http://example.com/"))
print(data)

 

    原文作者:reblue520
    原文地址: https://www.cnblogs.com/reblue520/p/11200073.html
    本文转自网络文章,转载此文章仅为分享知识,如有侵权,请联系博主进行删除。
点赞