U
|
¡ý°dG: ã @ s d Z dZdgZddlmZ ddlZddlZddlmZm Z m
|
Z
|
m Z m Z ddl mZmZ ddlmZmZmZmZmZ d ZG d
|
d d eeZG d d deZdS ) zCUse the HTMLParser library to parse HTML files that aren't too bad.ÚMITÚHTMLParserTreeBuilderé )Ú
|
HTMLParserN)ÚCDataÚCommentÚ DeclarationÚDoctypeÚProcessingInstruction)ÚEntitySubstitutionÚ UnicodeDammit)ÚDetectsXMLParsedAsHTMLÚParserRejectedMarkupÚHTMLÚHTMLTreeBuilderÚSTRICTz html.parserc @ s| e Zd ZdZdZdZdd Zdd Zdd Zd d d Z d d dZ
|
dd Z dd Z dd Z dd Zdd Zdd Zdd Zd S ) ÚBeautifulSoupHTMLParserz¸A subclass of the Python standard library's HTMLParser class, which
|
listens for HTMLParser events and translates them into calls
|
to Beautiful Soup's tree construction API.
|
ÚignoreÚreplacec O s4 | d| j¡| _tj| f|| g | _| ¡ dS )a Constructor.
|
|
:param on_duplicate_attribute: A strategy for what to do if a
|
tag includes the same attribute more than once. Accepted
|
values are: REPLACE (replace earlier values with later
|
ones, the default), IGNORE (keep the earliest value
|
encountered), or a callable. A callable must take three
|
arguments: the dictionary of attributes already processed,
|
the name of the duplicate attribute, and the most recent value
|
encountered.
|
Úon_duplicate_attributeN)ÚpopÚREPLACEr r Ú__init__Ú already_closed_empty_elementZ_initialize_xml_detector)ÚselfÚargsÚkwargs© r úNd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\bs4/builder/_htmlparser.pyr . s ÿ z BeautifulSoupHTMLParser.__init__c C s t |d S )N)r )r Úmessager r r ÚerrorJ s z BeautifulSoupHTMLParser.errorc C s | j ||dd}| |¡ dS )zÏHandle an incoming empty-element tag.
|
|
This is only called when the markup looks like <tag/>.
|
|
:param name: Name of the tag.
|
:param attrs: Dictionary of the tag's attributes.
|
F)Úhandle_empty_elementN)Úhandle_starttagÚ handle_endtag)r ÚnameÚattrsÚtagr r r Úhandle_startendtagZ s z*BeautifulSoupHTMLParser.handle_startendtagTc C sÎ i }|D ]`\}}|dkr d}||kr\| j }|| jkr6qd|d| jfkrN|||< qd|||| n|||< d}q| ¡ \} }
|
| jj|dd|| |
|
d} | r¶| jr¶|r¶| j|dd | j |¡ | j
|
dkrÊ| |¡ dS )a3 Handle an opening tag, e.g. '<tag>'
|
|
:param name: Name of the tag.
|
:param attrs: Dictionary of the tag's attributes.
|
:param handle_empty_element: True if this tag is known to be
|
an empty-element tag (i.e. there is not expected to be any
|
closing tag).
|
NÚ z"")Ú
|
sourcelineÚ sourceposF)Úcheck_already_closed) r ÚIGNOREr ÚgetposÚsoupr! Zis_empty_elementr" r ÚappendZ _root_tagZ_root_tag_encountered) r r# r$ r Z attr_dictÚkeyÚvalueZon_dupeÚ attrvaluer( r) r% r r r r! i s6
|
|
|
þ
|
|
z'BeautifulSoupHTMLParser.handle_starttagc C s, |r || j kr | j |¡ n | j |¡ dS )zõHandle a closing tag, e.g. '</tag>'
|
|
:param name: A tag name.
|
:param check_already_closed: True if this tag is expected to
|
be the closing portion of an empty-element tag,
|
e.g. '<tag></tag>'.
|
N)r Úremover- r" )r r# r* r r r r" s z%BeautifulSoupHTMLParser.handle_endtagc C s | j |¡ dS )z4Handle some textual data that shows up between tags.N)r- Ú handle_data©r Údatar r r r3 ² s z#BeautifulSoupHTMLParser.handle_datac C sê | d¡r t| d¡d}n$| d¡r8t| d¡d}nt|}d}|dk r| jjdfD ]B}|sbqXzt|g |¡}W qX tk
|
r } zW 5 d}~X Y qXX qX|sÔz t|}W n& t t
|
fk
|
rÒ } zW 5 d}~X Y nX |pÚd}| |¡ dS )z×Handle a numeric character reference by converting it to the
|
corresponding Unicode character and treating it as textual
|
data.
|
|
:param name: Character number, possibly in hexadecimal.
|
Úxé ÚXNé z windows-1252u �) Ú
|
startswithÚintÚlstripr- Úoriginal_encodingÚ bytearrayÚdecodeÚUnicodeDecodeErrorÚchrÚ
|
ValueErrorÚ OverflowErrorr3 )r r# Z real_namer5 ÚencodingÚer r r Úhandle_charref¶ s*
|
|
z&BeautifulSoupHTMLParser.handle_charrefc C s0 t j |¡}|dk r|}nd| }| |¡ dS )zÈHandle a named entity reference by converting it to the
|
corresponding Unicode character(s) and treating it as textual
|
data.
|
|
:param name: Name of the entity reference.
|
Nz&%s)r
|
ZHTML_ENTITY_TO_CHARACTERÚgetr3 )r r# Ú characterr5 r r r Úhandle_entityrefÞ s
|
z(BeautifulSoupHTMLParser.handle_entityrefc C s& | j ¡ | j |¡ | j t¡ dS )zOHandle an HTML comment.
|
|
:param data: The text of the comment.
|
N)r- ÚendDatar3 r r4 r r r Úhandle_commentñ s
|
z&BeautifulSoupHTMLParser.handle_commentc C s6 | j ¡ |tdd
}| j |¡ | j t¡ dS )zYHandle a DOCTYPE declaration.
|
|
:param data: The text of the declaration.
|
zDOCTYPE N)r- rJ Úlenr3 r r4 r r r Ú handle_declú s
|
z#BeautifulSoupHTMLParser.handle_declc C sN | ¡ d¡r$t}|tdd
}nt}| j ¡ | j |¡ | j |¡ dS )z{Handle a declaration of unknown type -- probably a CDATA block.
|
|
:param data: The text of the declaration.
|
zCDATA[N)Úupperr: r rL r r- rJ r3 )r r5 Úclsr r r Ú unknown_decl s
|
z$BeautifulSoupHTMLParser.unknown_declc C s0 | j ¡ | j |¡ | |¡ | j t¡ dS )z\Handle a processing instruction.
|
|
:param data: The text of the instruction.
|
N)r- rJ r3 Z_document_might_be_xmlr r4 r r r Ú handle_pi s
|
|
z!BeautifulSoupHTMLParser.handle_piN)T)T)Ú__name__Ú
|
__module__Ú __qualname__Ú__doc__r+ r r r r& r! r" r3 rF rI rK rM rP rQ r r r r r $ s
|
7
|
(
|
r c sN e Zd ZdZdZdZeZeee gZ
|
dZ d fdd Z d ddZ d d
|
Z ZS ) r zpA Beautiful soup `TreeBuilder` that uses the `HTMLParser` parser,
|
found in the Python standard library.
|
FTNc sl t }dD ] }||kr
|
| |¡}|||< q
|
tt| jf | |pBg }|pJi }| |¡ d|d< ||f| _dS )a Constructor.
|
|
:param parser_args: Positional arguments to pass into
|
the BeautifulSoupHTMLParser constructor, once it's
|
invoked.
|
:param parser_kwargs: Keyword arguments to pass into
|
the BeautifulSoupHTMLParser constructor, once it's
|
invoked.
|
:param kwargs: Keyword arguments for the superclass constructor.
|
)r FÚconvert_charrefsN)Údictr Úsuperr r ÚupdateÚ parser_args)r rZ Z parser_kwargsr Zextra_parser_kwargsÚargr0 ©Ú __class__r r r * s
|
|
|
z HTMLParserTreeBuilder.__init__c c s\ t |tr |dddfV dS |g}|g}||g}t|||d|d}|j|j|j|jfV dS )aÜ Run any preliminary steps necessary to make incoming markup
|
acceptable to the parser.
|
|
:param markup: Some markup -- probably a bytestring.
|
:param user_specified_encoding: The user asked to try this encoding.
|
:param document_declared_encoding: The markup itself claims to be
|
in this encoding.
|
:param exclude_encodings: The user asked _not_ to try any of
|
these encodings.
|
|
:yield: A series of 4-tuples:
|
(markup, encoding, declared encoding,
|
has undergone character replacement)
|
|
Each 4-tuple represents a strategy for converting the
|
document to Unicode and parsing it. Each strategy will be tried
|
in turn.
|
NFT)Úknown_definite_encodingsÚuser_encodingsZis_htmlÚexclude_encodings)Ú
|
isinstanceÚstrr Úmarkupr= Zdeclared_html_encodingZ contains_replacement_characters) r rc Zuser_specified_encodingZdocument_declared_encodingr` r^ r_ Z try_encodingsZdammitr r r Úprepare_markupC s"
|
ûþz$HTMLParserTreeBuilder.prepare_markupc
|
C sh | j \}}t||}| j|_z| |¡ W n* tk
|
rT } z t|W 5 d}~X Y nX | ¡ g |_dS )z{Run some incoming markup through some parsing process,
|
populating the `BeautifulSoup` object in self.soup.
|
N)rZ r r- ÚfeedÚAssertionErrorr Úcloser )r rc r r ÚparserrE r r r re t s
|
|
zHTMLParserTreeBuilder.feed)NN)NNN)rR rS rT rU Zis_xmlZ picklableÚ
|
HTMLPARSERÚNAMEr r ÚfeaturesZTRACKS_LINE_NUMBERSr rd re Ú __classcell__r r r\ r r s
|
ÿ
|
1)rU Ú __license__Ú__all__Ú html.parserr ÚsysÚwarningsZ bs4.elementr r r r r Z
|
bs4.dammitr
|
r Z bs4.builderr r r r r ri r r r r r r Ú<module> s ÿ z
|
