U
|
H=®dð, ã @ sÚ d dl mZmZmZ d dlmZ d dlmZm Z d dl
|
m Z ddl m Z ddl mZ ddl mZmZ dd l mZmZmZ dd
|
l mZmZ dd l mZ dd lmZ dd lmZ eeZe dkrÂe Z ne Z G dd de Z dS )é )Úabsolute_importÚdivisionÚunicode_literals)Úunichr)ÚdequeÚ OrderedDict)Ú version_infoé )ÚspaceCharacters)Úentities)Ú asciiLettersÚasciiUpper2Lower)ÚdigitsÚ hexDigitsÚEOF)Ú
|
tokenTypesÚ tagTokenTypes)ÚreplacementCharacters)ÚHTMLInputStream)ÚTrie)é é c sd e Zd ZdZd fdd Zdd Zdd Zdd
|
d Zd d Zdd Z dd Z
|
dd Z dd Z dd Z dd Zdd Zd d Zd d Zd d! Zd"d# Zd$d% Zd&d' Zd(d) Zd*d+ Zd,d- Zd.d/ Zd0d1 Zd2d3 Zd4d5 Z d6d7 Z d8d9 Z d:d; Z d<d= Z d>d? Z!d@dA Z"dBdC Z#dDdE Z$dFdG Z%dHdI Z&dJdK Z'dLdM Z(dNdO Z)dPdQ Z*dRdS Z+dTdU Z,dVdW Z-dXdY Z.dZd[ Z/d\d] Z0d^d_ Z1d`da Z2dbdc Z3ddde Z4dfdg Z5dhdi Z6djdk Z7dldm Z8dndo Z9dpdq Z:drds Z;dtdu Z<dvdw Z=dxdy Z>dzd{ Z?d|d} Z@d~d ZAdd ZBdd ZCdd
ZDdd ZEdd ZFdd ZGdd ZHdd ZIdd ZJdd ZKdd ZL ZMS )Ú HTMLTokenizera This class takes care of tokenizing HTML.
|
|
* self.currentToken
|
Holds the token that is currently being processed.
|
|
* self.state
|
Holds a reference to the method to be invoked... XXX
|
|
* self.stream
|
Points to HTMLInputStream object.
|
Nc sF t |f|| _|| _d| _g | _| j| _d| _d | _t t
|
| ¡ d S )NF) r ÚstreamÚparserÚ
|
escapeFlagÚ lastFourCharsÚ dataStateÚstateÚescapeÚ currentTokenÚsuperr Ú__init__)Úselfr r Úkwargs©Ú __class__© úVD:\z\workplace\VsCode\pyvenv\venv\Lib\site-packages\pip/_vendor/html5lib/_tokenizer.pyr" ( s zHTMLTokenizer.__init__c c sP t g | _| ¡ rL| jjr6td | jj d¡dV q| jr
|
| j ¡ V q6q
|
dS )zä This is where the magic happens.
|
|
We do our usually processing through the states and when we have a token
|
to return we yield the token which pauses processing until the next token
|
is requested.
|
Ú
|
ParseErrorr ©ÚtypeÚdataN)r Ú
|
tokenQueuer r Úerrorsr ÚpopÚpopleft©r# r' r' r( Ú__iter__7 s
|
zHTMLTokenizer.__iter__c % C s t }d}|rt}d}g }| j ¡ }||krH|tk rH| |¡ | j ¡ }q"td |¡|}|tkrt| }| j t
|
d dd|id¡ nbd| krd ks¦n |d
|
krÊd }| j t
|
d dd|id¡ n d | kràd ksn d| krødksn d| krdksn d| kr,dksn |t ddddddddd d d d d d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d
|
g#kr| j t
|
d dd|id¡ z t |}W n> t k
|
rè |d6 }t d|d? B t d7|d8@ B }Y nX |d9kr| j t
|
d d:d;¡ | j |¡ |S )<zøThis function returns either U+FFFD or the character based on the
|
decimal or hexadecimal representation. It also discards ";" if present.
|
If not present self.tokenQueue.append({"type": tokenTypes["ParseError"]}) is invoked.
|
é
|
é Ú r) z$illegal-codepoint-for-numeric-entityÚ charAsInt©r+ r, Údatavarsi Ø iÿß iÿÿ õ �r é é é é é iÐý iïý é iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ
|
iÿÿ
|
iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ i i Ü iÿ ú;z numeric-entity-without-semicolonr* )r r r Úcharr ÚappendÚintÚjoinr r- r Ú frozensetÚchrÚ
|
ValueErrorÚunget) r# ÚisHexÚallowedÚradixÚ charStackÚcr6 rA Úvr' r' r( ÚconsumeNumberEntityG s´
|
|
þ ÿ þ ÿÿ
|
þ
|
þ
|
ý
|
ý
|
øü ý &
|
ÿ z!HTMLTokenizer.consumeNumberEntityFc C sÄ d}| j ¡ g}|d tksB|d tddfksB|d k rV||d krV| j |d ¡ n |d dkr d}| | j ¡ ¡ |d dkrd}| | j ¡ ¡ |r¨|d tks¸|sÔ|d tkrÔ| j |d ¡ | |¡}n4| j t
|
d d
|
d ¡ | j | ¡ ¡ dd |¡ }nh|d tk rDt d |¡¡s0qD| | j ¡ ¡ q z$t d |d d
¡¡}t|}W n tk
|
r d }Y nX |d k r@|d d kr²| j t
|
d dd ¡ |d d kr|r|| tksð|| tksð|| dkr| j | ¡ ¡ dd |¡ }n.t| }| j | ¡ ¡ |d ||d
¡7 }n4| j t
|
d dd ¡ | j | ¡ ¡ dd |¡ }|r| jd d d |7 < n*|tkr¦d}nd}| j t
|
| |d ¡ d S )Nú&r ú<ú#Féÿÿÿÿ)ÚxÚXTr) zexpected-numeric-entityr* r5 r@ z named-entity-without-semicolonú=zexpected-named-entityr, r ÚSpaceCharactersÚ
|
Characters)r rA r
|
r rH rB r r rO r- r r/ rD Ú entitiesTrieÚhas_keys_with_prefixÚlongest_prefixÚlenÚKeyErrorr r r ) r# Ú allowedCharÚ fromAttributeÚoutputrL ÚhexÚ
|
entityNameÚ entityLengthÚ tokenTyper' r' r( Ú consumeEntity s~ ÿ
|
ÿ ÿ
|
ÿ ÿ
|
|
ÿ
|
ÿ
|
þ
|
ý ÿ
|
zHTMLTokenizer.consumeEntityc C s | j |dd dS )zIThis method replaces the need for "entityInAttributeValueState".
|
T)r^ r_ N)re )r# r^ r' r' r( ÚprocessEntityInAttributeß s z&HTMLTokenizer.processEntityInAttributec C sÔ | j }|d tkr¼|d t¡|d< |d td krp|d }t|}t|t|krh| |ddd
¡ ||d< |d td kr¼|d r| j td d d
|
¡ |d r¼| j td d d
|
¡ | j |¡ | j
|
| _ dS ) z®This method is a generic handler for emitting the tags. It also sets
|
the state to "data" because that's what's needed after a token has been
|
emitted.
|
r+ ÚnameÚStartTagr, NrS ÚEndTagr) zattributes-in-end-tagr* Ú selfClosingz self-closing-flag-on-end-tag) r r Ú translater r Ú attributeMapr\ Úupdater- rB r r )r# ÚtokenÚrawr, r' r' r( ÚemitCurrentTokenä s( ÿ ÿ z HTMLTokenizer.emitCurrentTokenc C sÈ | j ¡ }|dkr | j| _n¨|dkr.| j| _n|dkrd| j td dd¡ | j td dd¡ n`|tkrpdS |t kr| j td || j
|
t d
|
¡ d¡ n&| j
|
d ¡}| j td || d¡ d
|
S ) NrP rQ ú r) úinvalid-codepointr* rX FrW T©rP rQ rq ) r rA ÚentityDataStater Ú tagOpenStater- rB r r r
|
Ú
|
charsUntil©r# r, Úcharsr' r' r( r s.
|
|
|
ÿ ÿ
|
ÿ
|
ÿzHTMLTokenizer.dataStatec C s | ¡ | j| _dS ©NT)re r r r1 r' r' r( rt s z HTMLTokenizer.entityDataStatec C sÈ | j ¡ }|dkr | j| _n¨|dkr.| j| _n|tkr:dS |dkrp| j td dd¡ | j td d d¡ nT|t kr| j td
|
|| j
|
t d ¡ d¡ n&| j
|
d ¡}| j td || d¡ d S ) NrP rQ Frq r) rr r* rX r9 rW Trs ) r rA ÚcharacterReferenceInRcdatar ÚrcdataLessThanSignStater r- rB r r
|
rv rw r' r' r( Ú rcdataState" s.
|
|
|
ÿ ÿ
|
ÿ
|
ÿzHTMLTokenizer.rcdataStatec C s | ¡ | j| _dS ry )re r| r r1 r' r' r( rz ? s z(HTMLTokenizer.characterReferenceInRcdatac C s | j ¡ }|dkr | j| _nh|dkrR| j td dd¡ | j td dd¡ n2|tkr^dS | j d ¡}| j td || d¡ d
|
S © NrQ rq r) rr r* rX r9 F)rQ rq T) r rA ÚrawtextLessThanSignStater r- rB r r rv rw r' r' r( Ú rawtextStateD s"
|
|
ÿ ÿ
|
ÿzHTMLTokenizer.rawtextStatec C s | j ¡ }|dkr | j| _nh|dkrR| j td dd¡ | j td dd¡ n2|tkr^dS | j d ¡}| j td || d¡ d
|
S r} ) r rA ÚscriptDataLessThanSignStater r- rB r r rv rw r' r' r( ÚscriptDataStateV s"
|
|
ÿ ÿ
|
ÿz HTMLTokenizer.scriptDataStatec C sr | j ¡ }|tkrdS |dkrL| j td dd¡ | j td dd¡ n"| j td || j d¡ d¡ dS ) NFrq r) rr r* rX r9 T)r rA r r- rB r rv ©r# r, r' r' r( ÚplaintextStateh s
|
ÿ ÿ
|
ÿz HTMLTokenizer.plaintextStatec C s | j ¡ }|dkr | j| _nì|dkr.| j| _nÚ|tkrVtd |g ddd| _| j| _n²|dkr| j
|
td dd ¡ | j
|
td
|
d d ¡ | j | _nt|d krÈ| j
|
td d d ¡ | j |¡ | j | _n@| j
|
td dd ¡ | j
|
td
|
dd ¡ | j |¡ | j | _dS )Nú!ú/rh F)r+ rg r, rj ÚselfClosingAcknowledgedú>r) z'expected-tag-name-but-got-right-bracketr* rX z<>ú?z'expected-tag-name-but-got-question-markzexpected-tag-namerQ T)r rA ÚmarkupDeclarationOpenStater ÚcloseTagOpenStater r r Ú tagNameStater- rB r rH ÚbogusCommentStater r' r' r( ru w s@
|
|
|
ý
|
ÿ
|
ÿ
|
ÿ zHTMLTokenizer.tagOpenStatec C sÊ | j ¡ }|tkr0td |g dd| _| j| _n|dkrX| j td dd¡ | j | _nn|t
|
kr| j td dd¡ | j td d
|
d¡ | j | _n0| j td d d |id ¡ | j |¡ | j | _dS )Nri F©r+ rg r, rj r r) z*expected-closing-tag-but-got-right-bracketr* z expected-closing-tag-but-got-eofrX ú</z!expected-closing-tag-but-got-charr, r7 T) r rA r r r r r r- rB r r rH r r r' r' r( r s2
|
ÿ
|
ÿ
|
ÿ
|
þ z HTMLTokenizer.closeTagOpenStatec C s° | j ¡ }|tkr | j| _n|dkr.| ¡ n~|tkrV| j t d dd¡ | j
|
| _nV|dkrh| j | _nD|dkr| j t d dd¡ | j d d 7 < n| j d |7 < d
|
S ) Nr r) zeof-in-tag-namer* r
rq rr rg r9 T) r rA r
|
ÚbeforeAttributeNameStater rp r r- rB r r ÚselfClosingStartTagStater r r' r' r( r ² s&
|
|
|
ÿ
|
|
ÿzHTMLTokenizer.tagNameStatec C sP | j ¡ }|dkr"d| _| j| _n*| j td dd¡ | j |¡ | j | _dS ©Nr
r5 rX rQ r* T)
|
r rA ÚtemporaryBufferÚrcdataEndTagOpenStater r- rB r rH r| r r' r' r( r{ È s
|
|
z%HTMLTokenizer.rcdataLessThanSignStatec C sX | j ¡ }|tkr*| j|7 _| j| _n*| j td dd¡ | j |¡ | j
|
| _dS ©NrX r r* T) r rA r r ÚrcdataEndTagNameStater r- rB r rH r| r r' r' r( r Ó s
|
|
z#HTMLTokenizer.rcdataEndTagOpenStatec C s | j o | j d ¡ | j ¡ k}| j ¡ }|tkrT|rTtd | jg dd| _ | j| _n¨|dkr|rtd | jg dd| _ | j | _n||dkr´|r´td | jg dd| _ |
|
¡ | j | _nH|t krÌ| j|7 _n0| j td d| j d ¡ | j |¡ | j| _d
|
S © Nrg ri Fr r
r rX r r* T)r Úlowerr r rA r
|
r r r r rp r r r- rB rH r| ©r# Ú appropriater, r' r' r( r Þ s@
|
þ
|
þ
|
þ
|
ÿ z#HTMLTokenizer.rcdataEndTagNameStatec C sP | j ¡ }|dkr"d| _| j| _n*| j td dd¡ | j |¡ | j | _dS r )
|
r rA r ÚrawtextEndTagOpenStater r- rB r rH r r r' r' r( r~ ú s
|
|
z&HTMLTokenizer.rawtextLessThanSignStatec C sX | j ¡ }|tkr*| j|7 _| j| _n*| j td dd¡ | j |¡ | j
|
| _dS r ) r rA r r ÚrawtextEndTagNameStater r- rB r rH r r r' r' r( r s
|
|
z$HTMLTokenizer.rawtextEndTagOpenStatec C s | j o | j d ¡ | j ¡ k}| j ¡ }|tkrT|rTtd | jg dd| _ | j| _n¨|dkr|rtd | jg dd| _ | j | _n||dkr´|r´td | jg dd| _ |
|
¡ | j | _nH|t krÌ| j|7 _n0| j td d| j d ¡ | j |¡ | j| _d
|
S r )r r r r rA r
|
r r r r rp r r r- rB rH r r r' r' r( r s@
|
þ
|
þ
|
þ
|
ÿ z$HTMLTokenizer.rawtextEndTagNameStatec C sx | j ¡ }|dkr"d| _| j| _nR|dkrJ| j td dd¡ | j| _n*| j td dd¡ | j |¡ | j
|
| _dS ) Nr
r5 r rX z<!r* rQ T) r rA r ÚscriptDataEndTagOpenStater r- rB r ÚscriptDataEscapeStartStaterH r r r' r' r( r , s
|
|
|
z)HTMLTokenizer.scriptDataLessThanSignStatec C sX | j ¡ }|tkr*| j|7 _| j| _n*| j td dd¡ | j |¡ | j
|
| _dS r ) r rA r r ÚscriptDataEndTagNameStater r- rB r rH r r r' r' r( r : s
|
|
z'HTMLTokenizer.scriptDataEndTagOpenStatec C s | j o | j d ¡ | j ¡ k}| j ¡ }|tkrT|rTtd | jg dd| _ | j| _n¨|dkr|rtd | jg dd| _ | j | _n||dkr´|r´td | jg dd| _ |
|
¡ | j | _nH|t krÌ| j|7 _n0| j td d| j d ¡ | j |¡ | j| _d
|
S r )r r r r rA r
|
r r r r rp r r r- rB rH r r r' r' r( r E s@
|
þ
|
þ
|
þ
|
ÿ z'HTMLTokenizer.scriptDataEndTagNameStatec C sJ | j ¡ }|dkr2| j td dd¡ | j| _n| j |¡ | j| _dS ©Nú-rX r* T) r rA r- rB r Ú scriptDataEscapeStartDashStater rH r r r' r' r( r a s
|
|
z(HTMLTokenizer.scriptDataEscapeStartStatec C sJ | j ¡ }|dkr2| j td dd¡ | j| _n| j |¡ | j| _dS r ) r rA r- rB r Ú scriptDataEscapedDashDashStater rH r r r' r' r( r¡ k s
|
|
z,HTMLTokenizer.scriptDataEscapeStartDashStatec C s¶ | j ¡ }|dkr2| j td dd¡ | j| _n|dkrD| j| _nn|dkrz| j td dd¡ | j td dd¡ n8|tkr| j | _n&| j
|
d ¡}| j td || d¡ d
|
S ) Nr rX r* rQ rq r) rr r9 )rQ r rq T) r rA r- rB r ÚscriptDataEscapedDashStater Ú"scriptDataEscapedLessThanSignStater r rv rw r' r' r( ÚscriptDataEscapedStateu s(
|
|
|
ÿ ÿ
|
|
ÿz$HTMLTokenizer.scriptDataEscapedStatec C s¶ | j ¡ }|dkr2| j td dd¡ | j| _n|dkrD| j| _nn|dkr| j td dd¡ | j td dd¡ | j| _n0|t kr| j
|
| _n | j td |d¡ | j| _d S )
|
Nr rX r* rQ rq r) rr r9 T) r rA r- rB r r¢ r r¤ r¥ r r r r' r' r( r£ s&
|
|
|
ÿ ÿ
|
|
z(HTMLTokenizer.scriptDataEscapedDashStatec C sÖ | j ¡ }|dkr*| j td dd¡ n¨|dkr<| j| _n|dkrd| j td dd¡ | j| _nn|dkr¢| j td dd¡ | j td d d¡ | j| _n0|t kr´| j
|
| _n | j td |d¡ | j| _d
|
S ) Nr rX r* rQ r rq r) rr r9 T) r rA r- rB r r¤ r r r¥ r r r r' r' r( r¢ s*
|
|
|
ÿ ÿ
|
|
z,HTMLTokenizer.scriptDataEscapedDashDashStatec C s | j ¡ }|dkr"d| _| j| _n\|tkrT| j td d| d¡ || _| j | _n*| j td dd¡ | j
|
|¡ | j | _dS r ) r rA r Ú scriptDataEscapedEndTagOpenStater r r- rB r Ú scriptDataDoubleEscapeStartStaterH r¥ r r' r' r( r¤ ³ s
|
|
|
z0HTMLTokenizer.scriptDataEscapedLessThanSignStatec C sP | j ¡ }|tkr"|| _| j| _n*| j td dd¡ | j |¡ | j
|
| _dS r ) r rA r r Ú scriptDataEscapedEndTagNameStater r- rB r rH r¥ r r' r' r( r¦ Â s
|
|
z.HTMLTokenizer.scriptDataEscapedEndTagOpenStatec C s | j o | j d ¡ | j ¡ k}| j ¡ }|tkrT|rTtd | jg dd| _ | j| _n¨|dkr|rtd | jg dd| _ | j | _n||dkr´|r´td | jg dd| _ |
|
¡ | j | _nH|t krÌ| j|7 _n0| j td d| j d ¡ | j |¡ | j| _d
|
S r )r r r r rA r
|
r r r r rp r r r- rB rH r¥ r r' r' r( r¨ Í s@
|
þ
|
þ
|
þ
|
ÿ z.HTMLTokenizer.scriptDataEscapedEndTagNameStatec C s | j ¡ }|ttdB krR| j td |d¡ | j ¡ dkrH| j | _
|
q| j | _
|
nB|t kr| j td |d¡ | j|7 _n| j |¡ | j | _
|
dS ©N)r
r rX r* ÚscriptT)r rA r
|
rE r- rB r r r Ú scriptDataDoubleEscapedStater r¥ r rH r r' r' r( r§ é s
|
|
|
z.HTMLTokenizer.scriptDataDoubleEscapeStartStatec C sÒ | j ¡ }|dkr2| j td dd¡ | j| _n|dkrZ| j td dd¡ | j| _nt|dkr| j td dd¡ | j td dd¡ n>|tkr¸| j td d d¡ | j | _n| j td |d¡ d
|
S © Nr rX r* rQ rq r) rr r9 úeof-in-script-in-scriptT)
|
r rA r- rB r Ú scriptDataDoubleEscapedDashStater Ú(scriptDataDoubleEscapedLessThanSignStater r r r' r' r( r« ù s*
|
|
|
ÿ ÿ
|
ÿ
|
z*HTMLTokenizer.scriptDataDoubleEscapedStatec C sâ | j ¡ }|dkr2| j td dd¡ | j| _n¬|dkrZ| j td dd¡ | j| _n|dkr| j td dd¡ | j td dd¡ | j| _nF|t krÀ| j td d d¡ | j
|
| _n | j td |d¡ | j| _d
|
S r¬ ) r rA r- rB r Ú$scriptDataDoubleEscapedDashDashStater r¯ r« r r r r' r' r( r® s.
|
|
|
ÿ ÿ
|
ÿ
|
z.HTMLTokenizer.scriptDataDoubleEscapedDashStatec C s | j ¡ }|dkr*| j td dd¡ nÔ|dkrR| j td dd¡ | j| _n¬|dkrz| j td dd¡ | j| _n|dkr¸| j td dd¡ | j td d d¡ | j| _nF|t krà| j td d
|
d¡ | j
|
| _n | j td |d¡ | j| _d S ) Nr rX r* rQ r rq r) rr r9 r T) r rA r- rB r r¯ r r r« r r r r' r' r( r° % s2
|
|
|
ÿ ÿ
|
ÿ
|
z2HTMLTokenizer.scriptDataDoubleEscapedDashDashStatec C sP | j ¡ }|dkr8| j td dd¡ d| _| j| _n| j |¡ | j | _dS )Nr
rX r* r5 T)
|
r rA r- rB r r Ú scriptDataDoubleEscapeEndStater rH r« r r' r' r( r¯ > s
|
|
z6HTMLTokenizer.scriptDataDoubleEscapedLessThanSignStatec C s | j ¡ }|ttdB krR| j td |d¡ | j ¡ dkrH| j | _
|
q| j | _
|
nB|t kr| j td |d¡ | j|7 _n| j |¡ | j | _
|
dS r© )r rA r
|
rE r- rB r r r r¥ r r« r rH r r' r' r( r± I s
|
|
|
z,HTMLTokenizer.scriptDataDoubleEscapeEndStatec C s0 | j ¡ }|tkr$| j td¡ n|tkrJ| jd |dg¡ | j| _nâ|dkr\| ¡ nÐ|dkrn| j
|
| _n¾|dkrª| j t d dd ¡ | jd |dg¡ | j| _n|d
|
kræ| j t d d d ¡ | jd d dg¡ | j| _nF|t kr| j t d d d ¡ | j| _n | jd |dg¡ | j| _dS )NTr, r5 r r
)ú'ú"rV rQ r) ú#invalid-character-in-attribute-namer* rq rr r9 z#expected-attribute-name-but-got-eof)r rA r
|
rv r r rB ÚattributeNameStater rp r r- r r r r r' r' r( r Y s<
|
|
|
|
ÿ
|
ÿ
|
|
ÿ
|
z&HTMLTokenizer.beforeAttributeNameStatec C sä | j ¡ }d}d}|dkr&| j| _n.|tkr\| jd d d || j td¡ 7 < d}nø|dkrjd}nê|tkr|| j| _nØ|dkr| j | _nÆ|d krÌ| j
|
t d
|
d d ¡ | jd d d d 7 < d}n|dkr | j
|
t d
|
dd ¡ | jd d d |7 < d}nH|t kr6| j
|
t d
|
dd ¡ | j| _n | jd d d |7 < d}|rà| jd d d t¡| jd d d< | jd d d
D ]>\}}| jd d d |kr| j
|
t d
|
dd ¡ qÒq|rà| ¡ dS )NTFrV r, rS r r r
rq r) rr r* r9 ©r² r³ rQ r´ zeof-in-attribute-namezduplicate-attribute)r rA ÚbeforeAttributeValueStater r r rv r
|
ÚafterAttributeNameStater r- rB r r r rk r rp )r# r, ÚleavingThisStateÚ emitTokenrg Ú_r' r' r( rµ w s^
|
ÿ
|
|
ÿ
|
þ
|
ÿ
|
ÿ ÿ
|
z HTMLTokenizer.attributeNameStatec C sD | j ¡ }|tkr$| j td¡ n |dkr8| j| _n|dkrJ| ¡ nö|tkrp| jd |dg¡ | j
|
| _nÐ|dkr| j | _n¾|dkr¾| j t d d d
|
¡ | jd d dg¡ | j
|
| _n|d krú| j t d d d
|
¡ | jd |dg¡ | j
|
| _nF|tkr$| j t d dd
|
¡ | j| _n | jd |dg¡ | j
|
| _dS )NTrV r r, r5 r
rq r) rr r* r9 r¶ z&invalid-character-after-attribute-namez expected-end-of-tag-but-got-eof)r rA r
|
rv r· r rp r r rB rµ r r- r r r r r' r' r( r¸ s@
|
|
|
|
ÿ
|
ÿ
|
|
ÿ
|
z%HTMLTokenizer.afterAttributeNameStatec C sh | j ¡ }|tkr$| j td¡ n@|dkr8| j| _n,|dkrX| j| _| j |¡ n |dkrj| j| _nú|dkr| j
|
t d dd¡ | ¡ nÒ|d krÔ| j
|
t d d
|
d¡ | j d d d d7 < | j| _n|dkr| j
|
t d dd¡ | j d d d |7 < | j| _nL|tkrB| j
|
t d dd¡ | j| _n"| j d d d |7 < | j| _dS )NTr³ rP r² r r) z.expected-attribute-value-but-got-right-bracketr* rq rr r, rS r r9 )rV rQ ú`z"equals-in-unquoted-attribute-valuez$expected-attribute-value-but-got-eof)r rA r
|
rv Ú attributeValueDoubleQuotedStater ÚattributeValueUnQuotedStaterH Ú attributeValueSingleQuotedStater- rB r rp r r r r r' r' r( r· Í sF
|
|
ÿ
|
ÿ
|
|
ÿ
|
|
ÿ
|
z'HTMLTokenizer.beforeAttributeValueStatec C s¼ | j ¡ }|dkr | j| _n|dkr0| d¡ n|dkrj| j td dd¡ | jd d d d
|
7 < nN|t kr| j td d d¡ | j
|
| _n&| jd d d || j d ¡ 7 < d S )Nr³ rP rq r) rr r* r, rS r r9 z#eof-in-attribute-value-double-quote)r³ rP rq T© r rA ÚafterAttributeValueStater rf r- rB r r r r rv r r' r' r( r½ ï s&
|
|
ÿ ÿ
|
|
ÿz-HTMLTokenizer.attributeValueDoubleQuotedStatec C s¼ | j ¡ }|dkr | j| _n|dkr0| d¡ n|dkrj| j td dd¡ | jd d d d
|
7 < nN|t kr| j td d d¡ | j
|
| _n&| jd d d || j d ¡ 7 < d S )Nr² rP rq r) rr r* r, rS r r9 z#eof-in-attribute-value-single-quote)r² rP rq TrÀ r r' r' r( r¿ s&
|
|
ÿ ÿ
|
|
ÿz-HTMLTokenizer.attributeValueSingleQuotedStatec C s | j ¡ }|tkr | j| _nð|dkr0| d¡ nÜ|dkrB| ¡ nÊ|dkr|| j t d dd¡ | j
|
d d d |7 < n|d
|
kr¶| j t d d d¡ | j
|
d d d d 7 < nV|t krÞ| j t d d d¡ | j | _n.| j
|
d d d || j tdtB ¡ 7 < dS )NrP r )r³ r² rV rQ r¼ r) z0unexpected-character-in-unquoted-attribute-valuer* r, rS r rq rr r9 z eof-in-attribute-value-no-quotes)rP r r³ r² rV rQ r¼ rq T)r rA r
|
r r rf rp r- rB r r r r rv rE r r' r' r( r¾ s4
|
|
|
ÿ ÿ ÿ
|
|
ÿ
|
z)HTMLTokenizer.attributeValueUnQuotedStatec C s¢ | j ¡ }|tkr | j| _n|dkr.| ¡ np|dkr@| j| _n^|tkrt| j t
|
d dd¡ | j |¡ | j | _n*| j t
|
d dd¡ | j |¡ | j| _dS )Nr r
r) z$unexpected-EOF-after-attribute-valuer* z*unexpected-character-after-attribute-valueT) r rA r
|
r r rp r r r- rB r rH r r r' r' r( rÁ . s&
|
|
|
|
ÿ
|
ÿ z&HTMLTokenizer.afterAttributeValueStatec C s | j ¡ }|dkr&d| jd< | ¡ n^|tkrZ| j td dd¡ | j |¡ | j | _
|
n*| j td dd¡ | j |¡ | j | _
|
dS )Nr Trj r) z#unexpected-EOF-after-solidus-in-tagr* z)unexpected-character-after-solidus-in-tag) r rA r rp r r- rB r rH r r r r r' r' r( r B s
|
|
|
þ
|
ÿ z&HTMLTokenizer.selfClosingStartTagStatec C sD | j d¡}| dd¡}| j td |d¡ | j ¡ | j| _dS )Nr rq r9 ÚCommentr* T) r rv Úreplacer- rB r rA r r r r' r' r( r T s ÿ
|
z HTMLTokenizer.bogusCommentStatec C s | j ¡ g}|d dkrR| | j ¡ ¡ |d dkrPtd dd| _| j| _dS nò|d dkr¸d}dD ](}| | j ¡ ¡ |d |krfd } qqf|r¶td
|
dd d dd | _| j| _dS n|d d krD| jd k rD| jj j
|
rD| jj j
|
d j | jj j krDd}d D ].}| | j ¡ ¡ |d |krd } q2q|rD| j | _dS | j td dd¡ |rt| j | ¡ ¡ qZ| j| _dS )NrS r r r5 r* T)ÚdÚD))ÚoÚO©rM ÚC©ÚtÚT©ÚyÚY©ÚpÚP©ÚeÚEFÚDoctype)r+ rg ÚpublicIdÚsystemIdÚcorrectú[)rÉ rÅ ÚArÌ rÛ rÚ r) zexpected-dashes-or-doctype)r rA rB r r ÚcommentStartStater Ú doctypeStater ÚtreeÚ openElementsÚ namespaceÚdefaultNamespaceÚcdataSectionStater- rH r/ r )r# rL ÚmatchedÚexpectedr' r' r( r c s\ ýÿþý
|
ÿz(HTMLTokenizer.markupDeclarationOpenStatec C sØ | j ¡ }|dkr | j| _n¸|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ | j | j¡ | j| _nP|t krº| j td d
|
d¡ | j | j¡ | j| _n| jd |7 < | j
|
| _d S ) Nr rq r) rr r* r, r9 r úincorrect-commentúeof-in-commentT) r rA ÚcommentStartDashStater r- rB r r r r Ú commentStater r' r' r( rÜ s.
|
|
ÿ ÿ
|
ÿ
|
z HTMLTokenizer.commentStartStatec C sÜ | j ¡ }|dkr | j| _n¼|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ | j | j¡ | j| _nT|t krº| j td d
|
d¡ | j | j¡ | j| _n | jd d| 7 < | j
|
| _d S ) Nr rq r) rr r* r, õ -�r rå ræ T) r rA ÚcommentEndStater r- rB r r r r rè r r' r' r( rç § s.
|
|
ÿ ÿ
|
ÿ
|
z#HTMLTokenizer.commentStartDashStatec C s¦ | j ¡ }|dkr | j| _n|dkrN| j td dd¡ | jd d7 < nT|tkr| j td dd¡ | j | j¡ | j | _n | jd || j
|
d ¡ 7 < d
|
S ) Nr rq r) rr r* r, r9 ræ )r rq T) r rA ÚcommentEndDashStater r- rB r r r r rv r r' r' r( rè ¾ s$
|
|
ÿ ÿ
|
|
ÿzHTMLTokenizer.commentStatec C s® | j ¡ }|dkr | j| _n|dkrV| j td dd¡ | jd d7 < | j| _nT|t kr| j td dd¡ | j | j¡ | j
|
| _n | jd d| 7 < | j| _d S )
|
Nr rq r) rr r* r, ré zeof-in-comment-end-dashT) r rA rê r r- rB r r rè r r r r' r' r( rë Ð s$
|
|
ÿ
|
ÿ
|
z!HTMLTokenizer.commentEndDashStatec C s, | j ¡ }|dkr*| j | j¡ | j| _nþ|dkrd| j td dd¡ | jd d7 < | j| _nÄ|dkr| j td d d¡ | j | _n|d
|
kr¾| j td d d¡ | jd |7 < nj|t
|
krô| j td d d¡ | j | j¡ | j| _n4| j td d d¡ | jd d| 7 < | j| _dS )Nr rq r) rr r* r, u --�r z,unexpected-bang-after-double-dash-in-commentr z,unexpected-dash-after-double-dash-in-commentzeof-in-comment-double-dashzunexpected-char-in-commentz--T) r rA r- rB r r r r rè ÚcommentEndBangStater r r' r' r( rê ã s@
|
|
ÿ
|
ÿ
|
ÿ ÿ
|
ÿz HTMLTokenizer.commentEndStatec C sà | j ¡ }|dkr*| j | j¡ | j| _n²|dkrN| jd d7 < | j| _n|dkr| j td dd¡ | jd d 7 < | j | _nT|t
|
kr¾| j td d
|
d¡ | j | j¡ | j| _n | jd d| 7 < | j | _d S ) Nr r r, z--!rq r) rr r* u --!�z eof-in-comment-end-bang-stateT) r rA r- rB r r r rë r rè r r r' r' r( rì s,
|
|
|
ÿ
|
ÿ
|
z!HTMLTokenizer.commentEndBangStatec C s | j ¡ }|tkr | j| _nj|tkr\| j td dd¡ d| j d< | j | j ¡ | j
|
| _n*| j td dd¡ | j |¡ | j| _dS )Nr) ú!expected-doctype-name-but-got-eofr* FrÙ zneed-space-after-doctypeT) r rA r
|
ÚbeforeDoctypeNameStater r r- rB r r r rH r r' r' r( rÝ s
|
|
ÿ
|
|
ÿ zHTMLTokenizer.doctypeStatec C sÜ | j ¡ }|tkrnÄ|dkrT| j td dd¡ d| jd< | j | j¡ | j| _n|dkr| j td dd¡ d | jd
|
< | j | _nR|t
|
krÆ| j td d d¡ d| jd< | j | j¡ | j| _n|| jd
|
< | j | _d S ) Nr r) z+expected-doctype-name-but-got-right-bracketr* FrÙ rq rr r9 rg rí T) r rA r
|
r- rB r r r r ÚdoctypeNameStater r r' r' r( rî * s4
|
ÿ
|
|
ÿ
|
|
ÿ
|
|
|
z$HTMLTokenizer.beforeDoctypeNameStatec C s | j ¡ }|tkr2| jd t¡| jd< | j| _nØ|dkrh| jd t¡| jd< | j | j¡ | j
|
| _n¢|dkr¢| j t d dd¡ | jd d7 < | j | _nh|t krø| j t d dd¡ d | jd
|
< | jd t¡| jd< | j | j¡ | j
|
| _n| jd |7 < d S ) Nrg r rq r) rr r* r9 zeof-in-doctype-nameFrÙ T)r rA r
|
r rk r ÚafterDoctypeNameStater r- rB r r rï r r r' r' r( rï D s0
|
|
|
ÿ
|
ÿ
|
|
z HTMLTokenizer.doctypeNameStatec C sH | j ¡ }|tkrn.|dkr8| j | j¡ | j| _n |tkrd| jd< | j |¡ | j t
|
d dd¡ | j | j¡ | j| _nÀ|dkrÆd}d D ] }| j ¡ }||krd} q´q|rÄ| j | _dS nD|d
|
kr
|
d}d D ] }| j ¡ }||krØd} qøqØ|r
|
| j | _dS | j |¡ | j t
|
d d d |id¡ d| jd< | j | _dS )Nr FrÙ r) úeof-in-doctyper* rÐ T))ÚuÚU)ÚbÚB)ÚlÚL)ÚiÚIrÈ ©ÚsÚS)rÍ rú rÊ rÓ )ÚmÚMz*expected-space-or-right-bracket-in-doctyper, r7 )r rA r
|
r- rB r r r r rH r Ú afterDoctypePublicKeywordStateÚ afterDoctypeSystemKeywordStateÚbogusDoctypeState)r# r, rã rä r' r' r( rð ] sT
|
|
ÿ
|
|
|
|
þ
|
z#HTMLTokenizer.afterDoctypeNameStatec C s¨ | j ¡ }|tkr | j| _n|dkrP| j td dd¡ | j |¡ | j| _nT|t kr| j td dd¡ d| j
|
d< | j | j
|
¡ | j | _n| j |¡ | j| _dS © N)r² r³ r) úunexpected-char-in-doctyper* rñ FrÙ T) r rA r
|
Ú"beforeDoctypePublicIdentifierStater r- rB r rH r r r r r' r' r( rÿ s&
|
|
ÿ
|
ÿ
|
|
z,HTMLTokenizer.afterDoctypePublicKeywordStatec C sø | j ¡ }|tkrnà|dkr0d| jd< | j| _nÄ|dkrLd| jd< | j| _n¨|dkr| j t d dd¡ d | jd
|
< | j | j¡ | j
|
| _nh|t krÌ| j t d d d¡ d | jd
|
< | j | j¡ | j
|
| _n(| j t d d d¡ d | jd
|
< | j | _d S )Nr³ r5 r× r² r r) úunexpected-end-of-doctyper* FrÙ rñ r T) r rA r
|
r Ú(doctypePublicIdentifierDoubleQuotedStater Ú(doctypePublicIdentifierSingleQuotedStater- rB r r r r r r' r' r( r ¤ s:
|
|
|
|
|
ÿ
|
|
ÿ
|
|
ÿ
|
z0HTMLTokenizer.beforeDoctypePublicIdentifierStatec C sä | j ¡ }|dkr | j| _nÄ|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ d
|
| jd < | j | j¡ | j| _nR|t krÎ| j td d d¡ d
|
| jd < | j | j¡ | j| _n| jd |7 < d S )Nr³ rq r) rr r* r× r9 r r FrÙ rñ T©
|
r rA Ú!afterDoctypePublicIdentifierStater r- rB r r r r r r' r' r( r Á s0
|
|
ÿ ÿ
|
|
ÿ
|
|
z6HTMLTokenizer.doctypePublicIdentifierDoubleQuotedStatec C sä | j ¡ }|dkr | j| _nÄ|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ d
|
| jd < | j | j¡ | j| _nR|t krÎ| j td d d¡ d
|
| jd < | j | j¡ | j| _n| jd |7 < d S )Nr² rq r) rr r* r× r9 r r FrÙ rñ Tr r r' r' r( r Ù s0
|
|
ÿ ÿ
|
|
ÿ
|
|
z6HTMLTokenizer.doctypePublicIdentifierSingleQuotedStatec C s | j ¡ }|tkr | j| _nì|dkr<| j | j¡ | j| _nÌ|dkrn| j t d dd¡ d| jd< | j
|
| _n|dkr | j t d dd¡ d| jd< | j | _nh|t krà| j t d d d¡ d
|
| jd < | j | j¡ | j| _n(| j t d dd¡ d
|
| jd < | j | _d S ) Nr r³ r) r r* r5 rØ r² rñ FrÙ T)r rA r
|
Ú-betweenDoctypePublicAndSystemIdentifiersStater r- rB r r r Ú(doctypeSystemIdentifierDoubleQuotedStateÚ(doctypeSystemIdentifierSingleQuotedStater r r r' r' r( r ñ s>
|
|
|
ÿ
|
|
ÿ
|
|
ÿ
|
|
ÿ
|
z/HTMLTokenizer.afterDoctypePublicIdentifierStatec C sØ | j ¡ }|tkrnÀ|dkr4| j | j¡ | j| _n |dkrPd| jd< | j| _n|dkrld| jd< | j | _nh|t
|
kr¬| j t d dd¡ d | jd
|
< | j | j¡ | j| _n(| j t d d d¡ d | jd
|
< | j | _d S ) Nr r³ r5 rØ r² r) rñ r* FrÙ r T) r rA r
|
r- rB r r r r r r r r r r' r' r( r
|
s2
|
|
|
|
|
|
ÿ
|
|
ÿ
|
z;HTMLTokenizer.betweenDoctypePublicAndSystemIdentifiersStatec C s¨ | j ¡ }|tkr | j| _n|dkrP| j td dd¡ | j |¡ | j| _nT|t kr| j td dd¡ d| j
|
d< | j | j
|
¡ | j | _n| j |¡ | j| _dS r ) r rA r
|
Ú"beforeDoctypeSystemIdentifierStater r- rB r rH r r r r r' r' r( r ) s&
|
|
ÿ
|
ÿ
|
|
z,HTMLTokenizer.afterDoctypeSystemKeywordStatec C sø | j ¡ }|tkrnà|dkr0d| jd< | j| _nÄ|dkrLd| jd< | j| _n¨|dkr| j t d dd¡ d | jd
|
< | j | j¡ | j
|
| _nh|t krÌ| j t d d d¡ d | jd
|
< | j | j¡ | j
|
| _n(| j t d dd¡ d | jd
|
< | j | _d S ) Nr³ r5 rØ r² r r) r r* FrÙ rñ T) r rA r
|
r r r r r- rB r r r r r r' r' r( r = s:
|
|
|
|
|
ÿ
|
|
ÿ
|
|
ÿ
|
z0HTMLTokenizer.beforeDoctypeSystemIdentifierStatec C sä | j ¡ }|dkr | j| _nÄ|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ d
|
| jd < | j | j¡ | j| _nR|t krÎ| j td d d¡ d
|
| jd < | j | j¡ | j| _n| jd |7 < d S )Nr³ rq r) rr r* rØ r9 r r FrÙ rñ T©
|
r rA Ú!afterDoctypeSystemIdentifierStater r- rB r r r r r r' r' r( r Z s0
|
|
ÿ ÿ
|
|
ÿ
|
|
z6HTMLTokenizer.doctypeSystemIdentifierDoubleQuotedStatec C sä | j ¡ }|dkr | j| _nÄ|dkrN| j td dd¡ | jd d7 < n|dkr| j td d d¡ d
|
| jd < | j | j¡ | j| _nR|t krÎ| j td d d¡ d
|
| jd < | j | j¡ | j| _n| jd |7 < d S )Nr² rq r) rr r* rØ r9 r r FrÙ rñ Tr r r' r' r( r r s0
|
|
ÿ ÿ
|
|
ÿ
|
|
z6HTMLTokenizer.doctypeSystemIdentifierSingleQuotedStatec C s | j ¡ }|tkrn~|dkr4| j | j¡ | j| _n^|tkrt| j t d dd¡ d| jd< | j | j¡ | j| _n | j t d dd¡ | j
|
| _dS ) Nr r) rñ r* FrÙ r T) r rA r
|
r- rB r r r r r r r r' r' r( r s$
|
|
ÿ
|
|
ÿz/HTMLTokenizer.afterDoctypeSystemIdentifierStatec C sZ | j ¡ }|dkr*| j | j¡ | j| _n,|tkrV| j |¡ | j | j¡ | j| _n dS )Nr T) r rA r- rB r r r r rH r r' r' r( r s
|
|
|
z HTMLTokenizer.bogusDoctypeStatec C sô g }| | j d¡¡ | | j d¡¡ | j ¡ }|tkr>qq|dksJt|d dd
dkrv|d d d
|d< qq| |¡ qd |¡}| d¡}|dkrÎt|D ]}| j t
|
d d
|
d ¡ q¦| dd ¡}|rè| j t
|
d |d ¡ | j | _ dS )Nú]r rS éþÿÿÿz]]r5 rq r r) rr r* r9 rX T)rB r rv rA r ÚAssertionErrorrD ÚcountÚranger- r rà r r )r# r, rA Ú nullCountr» r' r' r( râ « s2
|
|
|
ÿ
|
ÿz HTMLTokenizer.cdataSectionState)N)NF)NÚ__name__Ú
|
__module__Ú __qualname__Ú__doc__r" r2 rO re rf rp r rt r| rz r r r ru r r r{ r r r~ r r r r r r r¡ r¥ r£ r¢ r¤ r¦ r¨ r§ r« r® r° r¯ r± r rµ r¸ r· r½ r¿ r¾ rÁ r r r rÜ rç rè rë rê rì rÝ rî rï rð rÿ r r r r r
|
r r r r r r râ Ú __classcell__r' r' r% r( r s H
|
P #
|
|
6 "- 3 r N) Ú
|
__future__r r r Zpip._vendor.sixr rF Ú collectionsr r Úsysr Ú constantsr
|
r r r r r r r r r Ú _inputstreamr Ú_trier rY Údictrl Úobjectr r' r' r' r( Ú<module> s
|
