U
|
H=®da~ ã @ sì d dl mZmZmZ d dlmZ d dlmZmZ d dl Z d dl
|
Z
|
d dl m Z m Z d dlmZ ddlmZmZmZmZ dd lmZ dd
|
lmZ ed d eD Zed d eD Zedd eD ZeeddgB Z dZ ej r e d dkrøe d¡dksüt e
|
!e dd
e"d d ¡Z#n
|
e
|
!e ¡Z#dddddddd d d d d d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4h Z$e
|
!d5¡Z%i Z&G d6d7 d7e'Z(d8d9 Z)G d:d; d;e'Z*G d<d= d=e*Z+G d>d? d?e,Z-G d@dA dAe'Z.G dBdC dCe'Z/dDdE Z0dS )Fé )Úabsolute_importÚdivisionÚunicode_literals)Ú text_type)Ú http_clientÚurllibN)ÚBytesIOÚStringIO)Ú webencodingsé )ÚEOFÚspaceCharactersÚ asciiLettersÚasciiUppercase)Ú_ReparseException)Ú_utilsc C s g | ]}| d ¡qS ©Úascii©Úencode©Ú.0Úitem© r úXD:\z\workplace\VsCode\pyvenv\venv\Lib\site-packages\pip/_vendor/html5lib/_inputstream.pyÚ
|
<listcomp> s r c C s g | ]}| d ¡qS r r r r r r r s c C s g | ]}| d ¡qS r r r r r r r s ó >ó <u [- - -Âï·-﷯￾￿ð¿¾ð¿¿ð¯¿¾ð¯¿¿ð¿¿¾ð¿¿¿ñ¿¾ñ¿¿ñ¿¾ñ¿¿ñ¯¿¾ñ¯¿¿ñ¿¿¾ñ¿¿¿ò¿¾ò¿¿ò¿¾ò¿¿ò¯¿¾ò¯¿¿ò¿¿¾ò¿¿¿ó¿¾ó¿¿ó¿¾ó¿¿ó¯¿¾ó¯¿¿ó¿¿¾ó¿¿¿ô¿¾ô¿¿]éÿÿÿÿú]z"\uD800-\uDFFF"iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ
|
iÿÿ
|
iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ iþÿ iÿÿ z[ - -/:-@\[-`{-~]c @ sH e Zd ZdZdd Zdd Zdd Zdd Zd
|
d Zd d Z dd Z
|
dS )ÚBufferedStreamzÆBuffering for streams that do not have buffering of their own
|
|
The buffer is implemented as a list of chunks on the assumption that
|
joining many strings will be slow since it is O(n**2)
|
c C s || _ g | _ddg| _d S )Nr r )ÚstreamÚbufferÚposition)Úselfr! r r r Ú__init__: s zBufferedStream.__init__c C s< d}| j d | jd
D ]}|t|7 }q|| jd 7 }|S ©Nr r )r" r# Úlen)r$ ÚposÚchunkr r r Útell? s
|
zBufferedStream.tellc C sT || ¡ kst|}d}t| j| |k rF|t| j| 8 }|d7 }q||g| _d S r& )Ú_bufferedBytesÚAssertionErrorr' r" r# )r$ r( ÚoffsetÚir r r ÚseekF s
|
zBufferedStream.seekc C sT | j s| |¡S | jd t| j krF| jd t| j d krF| |¡S | |¡S d S )Nr r r )r" Ú _readStreamr# r' Ú_readFromBuffer©r$ Úbytesr r r ÚreadO s
|
ÿ
|
zBufferedStream.readc C s t dd | jD S )Nc S s g | ] }t |qS r )r' r r r r r Y s z1BufferedStream._bufferedBytes.<locals>.<listcomp>)Úsumr" ©r$ r r r r+ X s z BufferedStream._bufferedBytesc C s< | j |¡}| j |¡ | jd d7 < t|| jd< |S r& )r! r4 r" Úappendr# r' )r$ r3 Údatar r r r0 [ s
|
zBufferedStream._readStreamc C sÐ |}g }| j d }| j d }|t| jk r²|dkr²|dks>t| j| }|t|| krl|}||| g| _ n"t|| }|t|g| _ |d7 }| ||||
¡ ||8 }d}q |rÆ| | |¡¡ d |¡S )Nr r ó )r# r' r" r, r7 r0 Újoin)r$ r3 ÚremainingBytesÚrvÚ bufferIndexÚ bufferOffsetÚ bufferedDataÚ bytesToReadr r r r1 b s&
|
|
|
z BufferedStream._readFromBufferN) Ú__name__Ú
|
__module__Ú __qualname__Ú__doc__r% r* r/ r4 r+ r0 r1 r r r r r 3 s r c K s t | tjs(t | tjjr.t | jtjr.d}n&t| drJt | d¡t }n
|
t | t }|rdd |D }|rvt
|
d| t | f|S t | f|S d S )NFr4 r c S s g | ]}| d ¡r|qS )Ú _encoding)Úendswith)r Úxr r r r s
|
z#HTMLInputStream.<locals>.<listcomp>z3Cannot set an encoding with a unicode input, set %r) Ú
|
isinstancer Ú HTTPResponser ÚresponseÚaddbaseÚfpÚhasattrr4 r Ú TypeErrorÚHTMLUnicodeInputStreamÚHTMLBinaryInputStream)ÚsourceÚkwargsÚ isUnicodeÚ encodingsr r r ÚHTMLInputStream} s þ ý
|
|
rU c @ sp e Zd ZdZdZdd Zdd Zdd Zd d
|
Zd d Z d d Z
|
dddZ dd Z dd Z d ddZdd ZdS ) rO ú×Provides a unicode stream of characters to the HTMLTokenizer.
|
|
This class takes care of character encoding and removing or replacing
|
incorrect byte-sequences and also provides column and line tracking.
|
|
i ( c C sZ t jsd| _n tddkr$| j| _n| j| _dg| _tddf| _| |¡| _
|
| ¡ dS )á¿ Initialises the HTMLInputStream.
|
|
HTMLInputStream(source, [encoding]) -> Normalized stream from source
|
for use by html5lib.
|
|
source can be either a file-object, local filename or a string.
|
|
The optional encoding parameter must be a string that indicates
|
the encoding. If specified, that encoding will be used,
|
regardless of any BOM or later declaration (such as in a meta
|
element)
|
|
Nu ô¿¿r r úutf-8Úcertain) r Úsupports_lone_surrogatesÚreportCharacterErrorsr' ÚcharacterErrorsUCS4ÚcharacterErrorsUCS2ÚnewLinesÚlookupEncodingÚ charEncodingÚ
|
openStreamÚ
|
dataStreamÚreset)r$ rQ r r r r% s
|
z HTMLUnicodeInputStream.__init__c C s. d| _ d| _d| _g | _d| _d| _d | _d S )NÚ r )r) Ú chunkSizeÚ chunkOffsetÚerrorsÚ prevNumLinesÚ prevNumColsÚ_bufferedCharacterr6 r r r rc ¾ s z HTMLUnicodeInputStream.resetc C s t |dr|}nt|}|S ©zvProduces a file object from source.
|
|
source can be either a file object, local filename or a string.
|
|
r4 )rM r ©r$ rQ r! r r r ra Ì s
|
z!HTMLUnicodeInputStream.openStreamc C sT | j }| dd|¡}| j| }| dd|¡}|dkr@| j| }n ||d }||fS )NÚ
|
r r r )r) Úcountrh Úrfindri )r$ r- r) ÚnLinesÚ positionLineÚ lastLinePosÚpositionColumnr r r Ú _positionÚ s
|
z HTMLUnicodeInputStream._positionc C s | | j¡\}}|d |fS )z:Returns (line, col) of the current position in the stream.r )rt rf )r$ ÚlineÚcolr r r r# å s z HTMLUnicodeInputStream.positionc C s6 | j | jkr| ¡ stS | j }| j| }|d | _ |S )zo Read one character from the stream or queue if available. Return
|
EOF when EOF is reached.
|
r )rf re Ú readChunkr r) )r$ rf Úcharr r r rx ê s
|
|
zHTMLUnicodeInputStream.charNc C sê |d kr| j }| | j¡\| _| _d| _d| _d| _| j |¡}| j rX| j | }d | _ n|s`dS t
|
|dkr®t |d }|dksd| krdkr®n n|d | _ |d d
}| j r¾| |¡ | d d
|
¡}| d d
|
¡}|| _t
|
|| _d S ) Nrd r Fr r é é Ø iÿÛ z
|
rm ú T)Ú_defaultChunkSizert re rh ri r) rf rb r4 rj r' Úordr[ Úreplace)r$ re r8 Úlastvr r r rw ù s0
|
|
|
|
z HTMLUnicodeInputStream.readChunkc C s( t tt |¡D ]}| j d¡ qd S )Núinvalid-codepoint)Úranger' Úinvalid_unicode_reÚfindallrg r7 )r$ r8 Ú_r r r r\ s z*HTMLUnicodeInputStream.characterErrorsUCS4c C s¶ d}t |¡D ]¢}|rqt| ¡ }| ¡ }t |||d
¡rrt |||d
¡}|tkrl| j
|
d¡ d}q|dkr |dkr |t |d kr | j
|
d¡ qd}| j
|
d¡ qd S )NFé r Trz iÿß r ) r Úfinditerr} ÚgroupÚstartr ÚisSurrogatePairÚsurrogatePairToCodepointÚnon_bmp_invalid_codepointsrg r7 r' )r$ r8 ÚskipÚmatchÚ codepointr( Úchar_valr r r r] # s" ÿz*HTMLUnicodeInputStream.characterErrorsUCS2Fc
|
C s zt ||f }W nh tk
|
rx |D ]}t|dk s$tq$d dd |D ¡}|sZd| }t d| ¡ }t ||f< Y nX g }| | j| j ¡}|dkr¦| j | j
|
krÖqøn0| ¡ }|| j
|
krÖ| | j| j |
¡ || _ qø| | j| j d
¡ | ¡ s~qøq~d |¡} | S )zð Returns a string of characters from the stream up to but not
|
including any character in 'characters' or EOF. 'characters' must be
|
a container that supports the 'in' method and iteration over its
|
characters.
|
é rd c S s g | ]}d t | qS )z\x%02x)r} )r Úcr r r r H s z5HTMLUnicodeInputStream.charsUntil.<locals>.<listcomp>z^%sz[%s]+N)ÚcharsUntilRegExÚKeyErrorr} r, r: ÚreÚcompiler r) rf re Úendr7 rw )
|
r$ Ú
|
charactersÚoppositeÚcharsr Úregexr< Úmr Úrr r r Ú
|
charsUntil: s0
|
|
z!HTMLUnicodeInputStream.charsUntilc C sT |t k rP| jdkr.|| j | _| jd7 _n"| jd8 _| j| j |ksPtd S r& )r rf r) re r, )r$ rx r r r Úungeti s
|
z HTMLUnicodeInputStream.unget)N)F)rA rB rC rD r| r% rc ra rt r# rx rw r\ r] r r r r r r rO s
|
&
|
/rO c @ sL e Zd ZdZdddZdd Zd d
|
Zdd d Zd d Zdd Z dd Z
|
dS )rP rV Nú windows-1252Tc C sn | |¡| _t | | j¡ d| _d| _|| _|| _|| _|| _ || _
|
| |¡| _ | j d dk sbt | ¡ dS )rW i éd r N)ra Ú rawStreamrO r% Ú numBytesMetaÚnumBytesChardetÚoverride_encodingÚtransport_encodingÚsame_origin_parent_encodingÚlikely_encodingÚdefault_encodingÚdetermineEncodingr` r, rc )r$ rQ r¤ r¥ r¦ r§ r¨ Ú
|
useChardetr r r r% s z HTMLBinaryInputStream.__init__c C s& | j d j | jd¡| _t | ¡ d S )Nr r~ )r` Ú
|
codec_infoÚ streamreaderr¡ rb rO rc r6 r r r rc ¬ s zHTMLBinaryInputStream.resetc C sL t |dr|}nt|}z| | ¡ ¡ W n tk
|
rF t|}Y nX |S rk )rM r r/ r* Ú Exceptionr rl r r r ra ° s
|
z HTMLBinaryInputStream.openStreamc C s | ¡ df}|d d k r |S t| jdf}|d d k r:|S t| jdf}|d d k rX|S | ¡ df}|d d k rt|S t| jdf}|d d k r¢|d j d¡s¢|S t| jdf}|d d k rÀ|S |rpzddl m
|
} W n t k
|
rê Y nX g }| }|j s<| j | j¡}t|ts t|s&q<| |¡ | |¡ qö| ¡ t|jd }| j d¡ |d k rp|dfS t| jdf}|d d k r|S tddfS )NrY r Ú tentativezutf-16)ÚUniversalDetectorÚencodingr )Ú detectBOMr_ r¤ r¥ ÚdetectEncodingMetar¦ ÚnameÚ
|
startswithr§ Ú%pip._vendor.chardet.universaldetectorr¯ Ú ImportErrorÚdoner¡ r4 r£ rH r3 r, r7 ÚfeedÚcloseÚresultr/ r¨ )r$ Úchardetr` r¯ ÚbuffersÚdetectorr" r° r r r r© Ã sR
|
|
z'HTMLBinaryInputStream.determineEncodingc C s | j d dkstt|}|d kr&d S |jdkrFtd}|d k stnT|| j d krf| j d df| _ n4| j d¡ |df| _ | ¡ td| j d |f d S )Nr rY ©úutf-16beúutf-16lerX r z Encoding changed from %s to %s)r` r, r_ r³ r¡ r/ rc r )r$ Ú newEncodingr r r ÚchangeEncoding s
|
|
z$HTMLBinaryInputStream.changeEncodingc
|
C sª t jdt jdt jdt jdt jdi}| j d¡}t|t s<t
|
| |dd
¡}d}|s~| |¡}d}|s~| |dd
¡}d }|r| j |¡ t |S | j d
|
¡ dS dS ) z©Attempts to detect at BOM at the start of the stream. If
|
an encoding can be determined from the BOM return the name of the
|
encoding otherwise return NonerX rÀ r¿ zutf-32lezutf-32beé Né r
r )ÚcodecsÚBOM_UTF8Ú BOM_UTF16_LEÚ BOM_UTF16_BEÚ BOM_UTF32_LEÚ BOM_UTF32_BEr¡ r4 rH r3 r, Úgetr/ r_ )r$ ÚbomDictÚstringr° r/ r r r r± s4 ý
|
z HTMLBinaryInputStream.detectBOMc C sV | j | j¡}t|ts tt|}| j d¡ | ¡ }|dk rR|j dkrRt
|
d}|S )z9Report the encoding declared by the meta element
|
r Nr¾ rX ) r¡ r4 r¢ rH r3 r, ÚEncodingParserr/ Ú getEncodingr³ r_ )r$ r" Úparserr° r r r r² 3 s z(HTMLBinaryInputStream.detectEncodingMeta)NNNNr T)T) rA rB rC rD r% rc ra r© r r± r² r r r r rP z s þ
|
*
|
>"rP c @ s e Zd ZdZdd Zdd Zdd Zdd Zd
|
d Zd d Z dd Z
|
dd Z e e e
|
Z dd Ze eZefddZdd Zdd Zdd Zd S ) Ú EncodingBytesz£String-like object with an associated position and various extra methods
|
If the position is ever greater than the string length then an exception is
|
raisedc C s t |tstt | | ¡ ¡S ©N)rH r3 r, Ú__new__Úlower©r$ Úvaluer r r rÓ F s zEncodingBytes.__new__c C s
|
d| _ d S )Nr )rt rÕ r r r r% J s zEncodingBytes.__init__c C s | S rÒ r r6 r r r Ú__iter__N s zEncodingBytes.__iter__c C s> | j d }| _ |t| kr"tn |dk r.t| ||d
S )Nr r ©rt r' Ú StopIterationrN ©r$ Úpr r r Ú__next__Q s zEncodingBytes.__next__c C s | ¡ S rÒ )rÜ r6 r r r ÚnextY s zEncodingBytes.nextc C sB | j }|t| krtn |dk r$t|d | _ }| ||d
S r& rØ rÚ r r r Úprevious] s zEncodingBytes.previousc C s | j t| krt|| _ d S rÒ ©rt r' rÙ )r$ r# r r r Ú setPositionf s zEncodingBytes.setPositionc C s* | j t| krt| j dkr"| j S d S d S )Nr rß r6 r r r Ú getPositionk s
|
|
zEncodingBytes.getPositionc C s | | j | j d
S ©Nr )r# r6 r r r ÚgetCurrentByteu s z EncodingBytes.getCurrentBytec C sH | j }|t| k r>| ||d
}||kr4|| _|S |d7 }q|| _dS )z Skip past a list of charactersr N©r# r' rt ©r$ r rÛ r r r r r z s
|
zEncodingBytes.skipc C sH | j }|t| k r>| ||d
}||kr4|| _|S |d7 }q|| _d S râ rä rå r r r Ú skipUntil s
|
zEncodingBytes.skipUntilc C s( | || j¡}|r$| jt|7 _|S )zÓLook for a sequence of bytes at the start of a string. If the bytes
|
are found return True and advance the position to the byte after the
|
match. Otherwise return False and leave the position alone)r´ r# r' )r$ r3 r< r r r Ú
|
matchBytes s zEncodingBytes.matchBytesc C s> z | || j¡t| d | _W n tk
|
r8 tY nX dS )zLook for the next sequence of bytes matching a given sequence. If
|
a match is found advance the position to the last byte of the matchr T)Úindexr# r' rt Ú
|
ValueErrorrÙ r2 r r r ÚjumpTo s
|
|
zEncodingBytes.jumpToN)rA rB rC rD rÓ r% r× rÜ rÝ rÞ rà rá Úpropertyr# rã Ú currentByteÚspaceCharactersBytesr ræ rç rê r r r r rÑ B s
|
rÑ c @ sX e Zd ZdZdd Zdd Zdd Zdd Zd
|
d Zd d Z dd Z
|
dd Z dd Z dS )rÎ z?Mini parser for detecting character encoding from meta elementsc C s t || _d| _dS )z3string - the data to work on for encoding detectionN)rÑ r8 r° ©r$ r8 r r r r% § s
|
zEncodingParser.__init__c
|
C sÖ d| j krd S d| jfd| jfd| jfd| jfd| jfd| jff}| j D ]}d}z| j d¡ W n tk
|
rz Y qÐY nX |D ]D\}}| j |¡rz| }W qÆW q tk
|
r d}Y qÆY qX q|sH qÐqH| j S ) Ns <metas <!--s </s <!s <?r TF)
|
r8 Ú handleCommentÚ
|
handleMetaÚhandlePossibleEndTagÚ handleOtherÚhandlePossibleStartTagrê rÙ rç r° )r$ ÚmethodDispatchr Ú keepParsingÚkeyÚmethodr r r rÏ ¬ s4
|
ú
|
|
zEncodingParser.getEncodingc C s | j d¡S )zSkip over commentss -->©r8 rê r6 r r r rï Ê s z EncodingParser.handleCommentc C sà | j jtkrdS d}d }| ¡ }|d kr,dS |d dkr\|d dk}|rÚ|d k rÚ|| _dS q|d dkr|d }t|}|d k rÚ|| _dS q|d dkrtt|d }| ¡ }|d k rt|}|d k r|rÖ|| _dS |}qd S ) NTFr s
|
http-equivr s content-typeó charsets content) r8 rì rí Ú getAttributer° r_ ÚContentAttrParserrÑ Úparse)r$ Ú hasPragmaÚpendingEncodingÚattrÚtentativeEncodingÚcodecÚ contentParserr r r rð Î s8 zEncodingParser.handleMetac C s
|
| d¡S )NF)ÚhandlePossibleTagr6 r r r ró ò s z%EncodingParser.handlePossibleStartTagc C s t | j | d¡S )NT)rÝ r8 r r6 r r r rñ õ s
|
z#EncodingParser.handlePossibleEndTagc C sb | j }|jtkr(|r$| ¡ | ¡ dS | t¡}|dkrD| ¡ n| ¡ }|d k r^| ¡ }qLdS )NTr )r8 rì ÚasciiLettersBytesrÞ rò ræ ÚspacesAngleBracketsrú )r$ ÚendTagr8 r rÿ r r r r ù s
|
|
|
|
z EncodingParser.handlePossibleTagc C s | j d¡S )Nr rø r6 r r r rò s zEncodingParser.handleOtherc C sô | j }| ttdgB ¡}|dks2t|dks2t|dkr>dS g }g }|dkrV|rVq¸nX|tkrj| ¡ }q¸nD|dkrd |¡dfS |tkr| | ¡ ¡ n|dkr¤dS | |¡ t
|
|}qF|dkrÖ| ¡ d |¡dfS t
|
| | ¡ }|dkrJ|}t
|
|}||kr"t
|
| d |¡d |¡fS |tkr<| | ¡ ¡ qô| |¡ qônJ|d krbd |¡dfS |tkr|| | ¡ ¡ n|dkrdS | |¡ t
|
|}|t krºd |¡d |¡fS |tkrÔ| | ¡ ¡ n|dkrâdS | |¡ qdS )
|
z_Return a name,value pair for the next attribute in the stream,
|
if one is found, or Noneó /Nr )r Nó =)r r r9 )ó 'ó "r ) r8 r rí Ú frozensetr' r, r: ÚasciiUppercaseBytesr7 rÔ rÝ rÞ r )r$ r8 r ÚattrNameÚ attrValueÚ quoteCharr r r rú sb
|
|
|
|
|
|
|
|
|
|
|
|
zEncodingParser.getAttributeN) rA rB rC rD r% rÏ rï rð ró rñ r rò rú r r r r rÎ ¤ s $rÎ c @ s e Zd Zdd Zdd ZdS )rû c C s t |tst|| _d S rÒ )rH r3 r, r8 rî r r r r% a s zContentAttrParser.__init__c C s zü| j d¡ | j jd7 _| j ¡ | j jdks<W d S | j jd7 _| j ¡ | j jdkrª| j j}| j jd7 _| j j}| j |¡r¢| j || j j
W S W d S nR| j j}z"| j t¡ | j || j j
W W S tk
|
rú | j |d
Y W S X W n tk
|
r Y d S X d S )Nrù r r )r
|
r )r8 rê r# r rì ræ rí rÙ )r$ Ú quoteMarkÚ oldPositionr r r rü e s.
|
|
zContentAttrParser.parseN)rA rB rC r% rü r r r r rû ` s rû c C sf t | tr0z| d¡} W n tk
|
r. Y dS X | dk r^z t | ¡W S tk
|
rZ Y dS X ndS dS )z{Return the python codec name corresponding to an encoding or None if the
|
string doesn't correspond to a valid encoding.r N)rH r3 ÚdecodeÚUnicodeDecodeErrorr
|
ÚlookupÚAttributeError)r° r r r r_ s
|
|
r_ )1Ú
|
__future__r r r Zpip._vendor.sixr Úpip._vendor.six.movesr r rÅ r Úior r Ú pip._vendorr
|
Ú constantsr r r r r rd r r rí r r r Ú invalid_unicode_no_surrogaterZ rn r, r Úevalr r Úascii_punctuation_rer Úobjectr rU rO rP r3 rÑ rÎ rû r_ r r r r Ú<module> s ÿþ
|
|
ú
|
J g Ib ='
|
