python模块之 HTMLParser: 解析html,获取url

redswallow posted @ Sun, 18 Jul 2010 00:29:51 +0800 in python , 3033 readers

HTMLParser是python用来解析html的模块。它可以分析出html里面的标签、数据等等,是一种处理html的简便途径。 HTMLParser采用的是一种事件驱动的模式,当HTMLParser找到一个特定的标记时,它会去调用一个用户定义的函数,以此来通知程序处理。它 主要的用户回调函数的命名都是以handler_开头的,都是HTMLParser的成员函数。当我们使用时,就从HTMLParser派生出新的类,然 后重新定义这几个以handler_开头的函数即可。这几个函数包括:


handle_startendtag  处理开始标签和结束标签
handle_starttag     处理开始标签,比如<xx>
handle_endtag       处理结束标签,比如</xx>
handle_charref      处理特殊字符串,就是以&#开头的,一般是内码表示的字符
handle_entityref    处理一些特殊字符,以&开头的,比如 &nbsp;
handle_data         处理数据,就是<xx>data</xx>中间的那些数据
handle_comment      处理注释
handle_decl         处理<!开头的,比如<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
handle_pi           处理形如<?instruction>的东西

 

本文主要简单讲一下HTMLParser的用法.

使用时需要定义一个从类HTMLParser继承的类,重定义函数:

  • handle_starttag( tag, attrs)
  • handle_startendtag( tag, attrs)
  • handle_endtag( tag)

来实现自己需要的功能。

tag是的html标签,attrs是 (属性,值)元组(tuple)的列表(list).
HTMLParser自动将tag和attrs都转为小写。

下面给出的例子抽取了html中的所有链接:

from HTMLParser import HTMLParser
 
class MyHTMLParser(HTMLParser):
    def __init__(self):
        HTMLParser.__init__(self)
        self.links = []
 
    def handle_starttag(self, tag, attrs):
        #print "Encountered the beginning of a %s tag" % tag
        if tag == "a":
            if len(attrs) == 0: pass
            else:
                for (variable, value)  in attrs:
                    if variable == "href":
                        self.links.append(value)
 
if __name__ == "__main__":
    html_code = """
<a href="
www.google.com"> google.com</a>
<A Href="
www.pythonclub.org"> PythonClub </a>
<A HREF = "
www.sina.com.cn"> Sina </a>
"
"" hp = MyHTMLParser() hp.feed(html_code) hp.close() print(hp.links)

输出为:

['www.google.com', 'www.pythonclub.org', 'www.sina.com.cn']

如果想抽取图形链接

<img src='http://www.google.com/intl/zh-CN_ALL/images/logo.gif' />

就要重定义 handle_startendtag( tag, attrs) 函数

Avatar_small
Rylan Landon said:
Sat, 29 Dec 2018 16:12:35 +0800

Fine Posting, We're an important believer around writing commentary for web pages so that you can allow the site freelancers realise that they’ve increased a little something valuable so that you can the ether! Crawley Accountants

Avatar_small
payday loan consolid said:
Thu, 17 Jan 2019 20:08:55 +0800

I'd prefer someone's area. It happens to be superior to monitor all people verbalize in the heart and soul combined with readability within this theme important spot tend to be quickly diagnosed.

Avatar_small
this website santacl said:
Sat, 13 Apr 2019 15:29:42 +0800

St. Nicholas and Santa Claus are historically the same man. But unlike the jolly figure who purportedly flies on a sleigh from the North Pole, the saint came originally from the balmy Mediterranean coast.

Avatar_small
this website santacl said:
Mon, 15 Apr 2019 16:40:28 +0800

St. Nicholas and Santa Claus are historically the same man. But unlike the jolly figure who purportedly flies on a sleigh from the North Pole, the saint came originally from the balmy Mediterranean coast.

Avatar_small
diamond necklace set said:
Wed, 26 Jun 2019 15:23:24 +0800

<!--td {border: 1px solid #ccc;}br {mso-data-placement:same-cell;}--> Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post . diamond necklace sets indian jewelry

Avatar_small
buy dissertation UK said:
Fri, 12 Jul 2019 23:06:10 +0800

Extremely intriguing web journal. Alot of sites I see nowadays don't generally give whatever I'm keen on, however I'm without a doubt inspired by this one. Recently believed that I would post and tell you.

Avatar_small
Crawley Accountants said:
Sat, 17 Aug 2019 20:23:57 +0800

I would like to say that this blog really convinced me to do it! Thanks, very good post.

Avatar_small
consolidate payday l said:
Tue, 20 Aug 2019 20:25:57 +0800

This is a wonderful article, Given so much info in it, These type of articles keeps the users interest in the website, and keep on sharing more ... good luck.

Avatar_small
freeware change mana said:
Fri, 11 Oct 2019 22:14:18 +0800

Hi to everybody, here everyone is sharing such knowledge, so it’s fastidious to see this site, and I used to visit this blog daily

Avatar_small
voyance par telephon said:
Sun, 24 Nov 2019 21:04:11 +0800

What a really awesome post this is. Truly, one of the best posts I've ever witnessed to see in my whole life. Wow, just keep it up.

Avatar_small
mails-guide.com said:
Wed, 27 Nov 2019 22:13:18 +0800

I recently came across your article and have been reading along. I want to express my admiration of your writing skill and ability to make readers read from the beginning to the end. I would like to read newer posts and to share my thoughts with you.

Avatar_small
trang điểm cô dâu said:
Sat, 30 Nov 2019 18:51:45 +0800

I will really appreciate the writer's choice for choosing this excellent article appropriate to my matter.Here is deep description about the article matter which helped me more.

Avatar_small
enclosed flare said:
Sat, 30 Nov 2019 19:34:32 +0800

Great Article it its really informative and innovative keep us posted with new updates. its was really valuable. thanks a lot.

Avatar_small
AMZON PRIME said:
Mon, 02 Dec 2019 00:46:29 +0800

I have been searching to find a comfort or effective procedure to complete this process and I think this is the most suitable way to do it effectively.

Avatar_small
best online product said:
Mon, 02 Dec 2019 19:24:54 +0800

We are really grateful for your blog post. You will find a lot of approaches after visiting your post. I was exactly searching for. Thanks for such post and please keep it up. Great work.

Avatar_small
Najlepsze Ubezpiecze said:
Mon, 02 Dec 2019 19:56:41 +0800

I was taking a gander at some of your posts on this site and I consider this site is truly informational! Keep setting up..

Avatar_small
Voyance Amour Eterne said:
Mon, 02 Dec 2019 20:15:27 +0800

I want to say thanks to you. I have bookmark your site for future updates.

Avatar_small
pure voyance said:
Wed, 04 Dec 2019 19:07:39 +0800

Here at this site really the fastidious material collection so that everybody can enjoy a lot.

Avatar_small
physiotherapy in sha said:
Wed, 04 Dec 2019 19:27:23 +0800

Thank you very much for sharing such a useful article. Will definitely saved and revisit your site

Avatar_small
Custom Closets in UA said:
Wed, 04 Dec 2019 23:04:05 +0800

I see the greatest contents on your blog and I extremely love reading them.

Avatar_small
Pagesoutlook said:
Sat, 07 Dec 2019 20:40:23 +0800

It is extremely nice to see the greatest details presented in an easy and understanding manner.

Avatar_small
timber veneer said:
Sun, 08 Dec 2019 01:54:53 +0800

I think it could be more general if you get a football sports activity

Avatar_small
Cisco Exam Dumps said:
Sun, 08 Dec 2019 22:56:39 +0800

It is extremely nice to see the greatest details presented in an easy and understanding manner.

Avatar_small
Mesh Armband said:
Mon, 09 Dec 2019 18:52:12 +0800

I think it could be more general if you get a football sports activity

Avatar_small
Discount Realtor Den said:
Thu, 12 Dec 2019 18:45:59 +0800

It proved to be Very helpful to me and I am sure to all the commentators here!

Avatar_small
funeral services sin said:
Sat, 14 Dec 2019 19:09:04 +0800

I think it could be more general if you get a football sports activity

Avatar_small
Rewriting history said:
Mon, 16 Dec 2019 00:19:00 +0800

I have read all the comments and suggestions posted by the visitors for this article are very fine,We will wait for your next article so only.Thanks!

Avatar_small
hotmail login said:
Tue, 17 Dec 2019 21:40:25 +0800

Here at this site really the fastidious material collection so that everybody can enjoy a lot.

Avatar_small
termite control said:
Mon, 23 Dec 2019 20:12:12 +0800

Hi buddies, it is great written piece entirely defined, continue the good work constantly.

Avatar_small
Muhasibatliq kurslar said:
Tue, 24 Dec 2019 20:19:33 +0800

I see the greatest contents on your blog and I extremely love reading them.

Avatar_small
print photobook said:
Tue, 24 Dec 2019 20:30:46 +0800

I would like to say that this blog really convinced me to do it! Thanks, very good post.

Avatar_small
photo retouching ser said:
Fri, 27 Dec 2019 20:46:24 +0800

It is extremely nice to see the greatest details presented in an easy and understanding manner.

Avatar_small
cool shirts for wome said:
Tue, 31 Dec 2019 20:44:53 +0800

Hi buddies, it is great written piece entirely defined, continue the good work constantly.

Avatar_small
Politics said:
Sun, 05 Jan 2020 20:47:20 +0800

It is extremely nice to see the greatest details presented in an easy and understanding manner.

Avatar_small
relatiegeschenken said:
Wed, 08 Jan 2020 21:26:31 +0800

I have a mission that I’m just now working on, and I have been at the look out for such information

Avatar_small
cbd oil for sale said:
Mon, 20 Jan 2020 17:08:21 +0800

Well we really like to visit this site, many useful information we can get here.

Avatar_small
pocono vacation home said:
Fri, 24 Jan 2020 19:06:19 +0800

I think this is an informative post and it is very useful and knowledgeable. therefore, I would like to thank you for the efforts you have made in writing this article.

Avatar_small
คาสิโนออนไลน์ said:
Sat, 25 Jan 2020 20:32:35 +0800

Here at this site really the fastidious material collection so that everybody can enjoy a lot.

Avatar_small
Contently said:
Mon, 27 Jan 2020 19:35:24 +0800

I have been searching to find a comfort or effective procedure to complete this process and I think this is the most suitable way to do it effectively.

Avatar_small
Keynote speaker said:
Wed, 29 Jan 2020 20:53:57 +0800

Super site! I am Loving it!! Will return once more, Im taking your food additionally, Thanks.

Avatar_small
movies123 said:
Mon, 03 Feb 2020 19:53:36 +0800

Great Article it its really informative and innovative keep us posted with new updates. its was really valuable. thanks a lot.

Avatar_small
https://Mauriceandre said:
Mon, 03 Feb 2020 23:55:23 +0800

Hi buddies, it is great written piece entirely defined, continue the good work constantly.

Avatar_small
random card generato said:
Tue, 04 Feb 2020 22:14:53 +0800

Admiring the time and effort you put into your blog and detailed information you offer!..

Avatar_small
Refurbished iPhone X said:
Thu, 06 Feb 2020 21:49:19 +0800

Great Article it its really informative and innovative keep us posted with new updates. its was really valuable. thanks a lot.

Avatar_small
파워볼정보 said:
Fri, 07 Feb 2020 20:19:48 +0800

Always so interesting to visit your site.What a great info, thank you for sharing. this will help me so much in my learning 파워볼

Avatar_small
Furry Discord Server said:
Fri, 07 Feb 2020 22:03:13 +0800

Thanks for posting this info. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts.

Avatar_small
Click here said:
Tue, 11 Feb 2020 22:24:49 +0800

They're produced by the very best degree developers who will be distinguished for your polo dress creating. You'll find polo Ron Lauren inside exclusive array which include particular classes for men, women.

Avatar_small
Clinique dentaire said:
Wed, 12 Feb 2020 20:07:53 +0800

I was taking a gander at some of your posts on this site and I consider this site is truly informational! Keep setting up..

Avatar_small
Recommended said:
Sat, 15 Feb 2020 22:31:58 +0800

I want to say thanks to you. I have bookmark your site for future updates.

Avatar_small
voyance sans cb said:
Mon, 17 Feb 2020 21:13:12 +0800

Excellent effort to make this blog more wonderful and attractive

Avatar_small
warner robins ga pai said:
Wed, 26 Feb 2020 20:50:18 +0800

Hi buddies, it is great written piece entirely defined, continue the good work constantly.

Avatar_small
kopar at newton cond said:
Fri, 28 Feb 2020 02:41:29 +0800

Super site! I am Loving it!! Will return once more, Im taking your food additionally, Thanks.

Avatar_small
chris jones said:
Sat, 29 Feb 2020 19:07:38 +0800

I am always searching online for articles that can help me. There is obviously a lot to know about this. I think you made some good points in Features also. Keep working, great job

Avatar_small
ブルーベリー おすすめ said:
Mon, 09 Mar 2020 23:41:54 +0800

I think it could be more general if you get a football sports activity

Avatar_small
Face Mask Controvers said:
Mon, 16 Mar 2020 18:53:04 +0800

I was taking a gander at some of your posts on this site and I consider this site is truly informational! Keep setting up..

Avatar_small
안전토토사이트 said:
Fri, 20 Mar 2020 20:04:19 +0800

현재 이용 중인 베팅사이트 가 있다고.

Avatar_small
link said:
Mon, 23 Mar 2020 19:33:20 +0800

They're produced by the very best degree developers who will be distinguished for your polo dress creating. You'll find polo Ron Lauren inside exclusive array which include particular classes for men, women.

Avatar_small
my reading manga said:
Thu, 26 Mar 2020 18:35:55 +0800

I have been searching to find a comfort or effective procedure to complete this process and I think this is the most suitable way to do it effectively.

Avatar_small
what to write about said:
Fri, 27 Mar 2020 19:32:18 +0800

I see the greatest contents on your blog and I extremely love reading them.

Avatar_small
Creampie Porn said:
Thu, 14 May 2020 04:07:33 +0800

I feel a lot more people need to read this, very good info! .

Avatar_small
123mvoies said:
Thu, 28 May 2020 16:55:30 +0800

good movie site https://123movies7.com

Avatar_small
free sexting online said:
Tue, 09 Jun 2020 20:07:55 +0800

Very awesome!!! When I seek for this I found this website at the top of all blogs in search engine.

Avatar_small
chat with bdsm rooms said:
Sun, 14 Jun 2020 23:55:58 +0800

I wish more writers of this sort of substance would take the time you did to explore and compose so well. I am exceptionally awed with your vision and knowledge.

Avatar_small
فوز العتيبي said:
Wed, 05 Aug 2020 02:32:38 +0800

Here at this site really the fastidious material collection so that everybody can enjoy a lot.

Avatar_small
Immobilienvermittlun said:
Thu, 27 Aug 2020 02:17:03 +0800

very good post, i definitely love this site, keep on it

Avatar_small
politics said:
Mon, 31 Aug 2020 03:19:52 +0800

Well we really like to visit this site, many useful information we can get here.

Avatar_small
Marine Fernglas kauf said:
Fri, 04 Sep 2020 01:59:18 +0800

i am for the first time here. I found this board and I in finding It truly helpful & it helped me out a lot. I hope to present something back and help others such as you helped me.

Avatar_small
Concrete Scanning Se said:
Sat, 05 Sep 2020 19:45:34 +0800

A debt of gratitude is in order for sharing the information, keep doing awesome... I truly delighted in investigating your site. great asset...

Avatar_small
gurgaon call girls said:
Mon, 07 Sep 2020 19:50:55 +0800

This time we are reviewing Blackbeards Adult Resort, we haven't been there in a few years and believe me it is better. I know, we have it real rough here at Resort Holiday Weeks, going around visiting resorts all over the world and having resort owners and staff treat us like royalty so that we give them a good review.

Avatar_small
mississippi home ins said:
Sun, 13 Sep 2020 18:59:31 +0800

We still cannot quite believe that I was able to often be any type of those staring at the important points located on your blog post. My loved ones and i are sincerely thankful for your special generosity too as for giving me possibility pursue our chosen profession path. Basically information I acquired within your web-site.

Avatar_small
investissement immob said:
Wed, 30 Sep 2020 19:55:28 +0800

Aw, this has been a very nice post. In thought I must devote writing this way additionally – taking time and actual effort to create a excellent article… but exactly what can I say… I procrastinate alot and by no indicates appear to go accomplished.

Avatar_small
pak24tv said:
Mon, 05 Oct 2020 20:20:20 +0800

I am happy to find your distinguished way of writing the post. Now you make it easy for me to understand and implement the concept. Thank you for the post. usa whatsapp group links 50 most popular women fb status

Avatar_small
Ryse Residences show said:
Tue, 06 Oct 2020 00:19:19 +0800

Another wonderful publish on running a blog! Thanks so much to take time to share a person information and wisdom along with other writers.

Avatar_small
RajaQQ said:
Wed, 21 Oct 2020 19:17:05 +0800

It is perfect time to make some plans for the future and it is time to be happy. I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it!

Avatar_small
escort said:
Fri, 23 Oct 2020 20:31:25 +0800

Wow! Such an amazing and helpful post this is. I really really love it. It's so good and so awesome. I am just amazed. I hope that you continue to do your work like this in the future also.


Login *


loading captcha image...
(type the code from the image)
or Ctrl+Enter