Web Scraping with Python Learning CHAPTER2 Advaced HTML Parsing

bs.find_all and bs.find

from bs4 import BeautifulSoup

bs.find_all(tag, attributes, recursive, text, limit, keywords)

bs.find(tag, attributes, recursive, text, keywords)

tag

bs,find_all(['h1', 'h2', 'h3'])  相互之间是“或”关系

attributes

bs.find_all('span', {'class': {'green', 'red'}})

recursive

是否递归遍历。如果为false，则只会匹配出现的第一个结果，默认true

text

bs,find_all(text='...') 完全匹配

limit

只对find_all有效，限制查询结果数量

keywords

bs.find_all('div', {'class':'green'})

其他的BS对象

BeautifulSoup库中有4种对象：

BS object

Tag object

bs.h1

NavigableString object

Comment object

<!--like this one-->.

Navigating Trees

children  and descendants

all children are descendants, but not all descendants are children

siblings

next_siblings()/next_sibling()

previous_siblings()/previous_sibling()

parents

.parent

,parents

Regular Expression