Comments on: Python HTML Parser Performance http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/ Wed, 09 Jul 2008 12:08:17 +0000 http://wordpress.org/?v=2.2.3 By: Ian Bicking http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20642 Ian Bicking Wed, 09 Jul 2008 03:08:49 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20642 Well, hopefully you aren't considering scraping email addresses...? You could look at python-spidermonkey, but the task isn't all that easy, especially getting document.write to work. Well, hopefully you aren’t considering scraping email addresses…? You could look at python-spidermonkey, but the task isn’t all that easy, especially getting document.write to work.

]]> By: Jerzy Orlowski http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20627 Jerzy Orlowski Tue, 08 Jul 2008 23:31:04 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20627 Sorry, but the website formatted example input into html. It was a code in javascript that concatenates some variables to generate an email address. Sorry, but the website formatted example input into html. It was a code in javascript that concatenates some variables to generate an email address.

]]> By: Jerzy Orlowski http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20626 Jerzy Orlowski Tue, 08 Jul 2008 23:27:57 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-20626 Hi Is there any way to interpret javascript in Python? example input: document.write('<a href="mailto:biuro'); document.write('@inprofi.pl">biuro'); document.write('@inprofi.pl</a>'); Hi

Is there any way to interpret javascript in Python?

example input:

document.write(’<a href=”mailto:biuro’); document.write(’@inprofi.pl”>biuro’); document.write(’@inprofi.pl</a>’);

]]> By: Eric LEBIGOT http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-18336 Eric LEBIGOT Fri, 13 Jun 2008 13:15:37 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-18336 Great overview of HTML parsers & friends! Very useful indeed. Just one point about Mac OS X (10.4): I had no problem installing lxml! Just did a simple "sudo easy_install lxml". libxml2 and Python were both installed in a quite standard way and without any pain (through Fink). Great overview of HTML parsers & friends! Very useful indeed.

Just one point about Mac OS X (10.4): I had no problem installing lxml! Just did a simple “sudo easy_install lxml”. libxml2 and Python were both installed in a quite standard way and without any pain (through Fink).

]]> By: Subeen http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-18115 Subeen Mon, 09 Jun 2008 18:25:56 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-18115 Really nice article. Thanks. Really nice article. Thanks.

]]> By: Cesar Ortiz http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16743 Cesar Ortiz Tue, 15 Apr 2008 23:04:29 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16743 ... and I wasn´t able to find an article like this. This post can save time to a lot of people. … and I wasn´t able to find an article like this. This post can save time to a lot of people.

]]> By: Cesar Ortiz http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16742 Cesar Ortiz Tue, 15 Apr 2008 23:00:14 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16742 Hi, Some time ago I was looking for a fast html parse that could handle 'bad html' and the winner was libxml2. I just wanted a SAX parsing, because instead of a generic tree, we use other kind or data structures. Excelent article, Cesar Hi,

Some time ago I was looking for a fast html parse that could handle ‘bad html’ and the winner was libxml2. I just wanted a SAX parsing, because instead of a generic tree, we use other kind or data structures.

Excelent article,

Cesar

]]> By: Ian Bicking http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16474 Ian Bicking Wed, 02 Apr 2008 15:22:19 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16474 Dan: no, no qualification, I would recommend lxml for just about any HTML task. Dan: no, no qualification, I would recommend lxml for just about any HTML task.

]]> By: Dan http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16448 Dan Wed, 02 Apr 2008 09:29:49 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16448 "I would recommend lxml for just about any HTML task." You're missing a crucial qualification here - I'd say "for just about any HTML task <i>where performance matters</i>". 90% of the time when I'm parsing HTML, differences in speed and memory usage are totally insignificant. I'd certainly trade off an order of magnitude speed difference for even a slight benefit in ease-of-use. Still, it's good to see numbers on how big the performance difference is, and I'll probably look into using lxml with BeautifulSoup some time in the future. “I would recommend lxml for just about any HTML task.”

You’re missing a crucial qualification here - I’d say “for just about any HTML task where performance matters“. 90% of the time when I’m parsing HTML, differences in speed and memory usage are totally insignificant. I’d certainly trade off an order of magnitude speed difference for even a slight benefit in ease-of-use.

Still, it’s good to see numbers on how big the performance difference is, and I’ll probably look into using lxml with BeautifulSoup some time in the future.

]]> By: jgraham http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16437 jgraham Tue, 01 Apr 2008 22:26:44 +0000 http://blog.ianbicking.org/2008/03/30/python-html-parser-performance/#comment-16437 > There seems to be an incomplete C parser in the html5lib project too? Indeed, there is the very beginning of a pure-C HTML5 parser in the html5lib project. This has stalled recently as I have had other things to keep me busy and my total inexperience as a C developer makes it a tough project to dip into intermittently. If anyone fancies picking it up and giving it a push, they are most welcome to; you should be able to catch me in #whatwg on irc.freenode.net if you have any questions. > There seems to be an incomplete C parser in the html5lib project too?

Indeed, there is the very beginning of a pure-C HTML5 parser in the html5lib project. This has stalled recently as I have had other things to keep me busy and my total inexperience as a C developer makes it a tough project to dip into intermittently. If anyone fancies picking it up and giving it a push, they are most welcome to; you should be able to catch me in #whatwg on irc.freenode.net if you have any questions.

]]>