U
|
ý°dû8 ã @ s d Z ddlmZ ddlZddlmZmZmZ ddlm Z ddl
|
Z
|
ddl m Z m Z ddlmZmZmZ ddlmZ dd Ze
|
j d
|
d d d g¡dd Zdd Ze
|
j dddiddig¡dd Zdd Ze
|
j dddg¡d d Z d d Z e
|
j dd
|
d id!d"ig¡d#d$ Z e
|
j d%d&d'g¡d(d) Z d*d+ Z d,d- Z!e
|
j d.d d/g¡d0d1 Z"e
|
j dd2d3d4d5d6d7d8d9g¡e
|
j dd:d;d<d=d>g¡d?d@ Z#e
|
j dAej$ej%g¡dBdC Z&dDdE Z'e
|
j dFdGdHg¡e
|
j d
|
ddIg¡dJdK Z(dLdM Z)dNdO Z*e
|
j dPdQdR dSdR g¡dTdU Z+dVdW Z,e
|
j dPdXdR dYdR g¡e
|
j ddZd[g¡d\d] Z-d^d_ Z.d`da Z/dbdc Z0ddde Z1dfdg Z2dhdi Z3dS )jzø
|
Tests that apply specifically to the Python parser. Unless specifically
|
stated as a Python-specific issue, the goal is to eventually move as many of
|
these tests out of this module as soon as the C parser can accept further
|
arguments when parsing.
|
é )Ú annotationsN)ÚBytesIOÚStringIOÚ TextIOWrapper)ÚIterator)Ú ParserErrorÚ ParserWarning)Ú DataFrameÚIndexÚ
|
MultiIndexc C s@ d}| }t ddgddgd}|jt|d d}t ||¡ d S )Nz aob
|
1o2
|
3o4é é é é ©ÚaÚb©Úsep©r Úread_csvr ÚtmÚassert_frame_equal©Úpython_parser_onlyÚdataÚparserÚexpectedÚresult© r úed:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/tests/io/parser/test_python_parser_only.pyÚtest_default_separator s
|
r! Ú
|
skipfooterÚfoog ø?Tc C s< d}| }d}t jt|d |jt||d W 5 Q R X d S )Núa
|
1
|
2z skipfooter must be an integer©Úmatch©r" ©ÚpytestÚraisesÚ
|
ValueErrorr r )r r" r r Úmsgr r r Ú test_invalid_skipfooter_non_int, s
|
r- c C s< d}| }d}t jt|d |jt|dd W 5 Q R X d S )Nr$ z skipfooter cannot be negativer% éÿÿÿÿr' r( )r r r r, r r r Ú test_invalid_skipfooter_negative7 s
|
r/ Úkwargsr Ú delimiterú|c C sl d}| }|j t|fddi|}tdddgddd gd
|
d d ggd ddgtdddgddd}t ||¡ d S )Nz*index|A|B|C
|
foo|1|2|3
|
bar|4|5|6
|
baz|7|8|9
|
Ú index_colr r r r r é é é é é ÚAÚBÚCr# ÚbarÚbazÚindex©Úname©Úcolumnsr> ©r r r r
|
r r ©r r0 r r r r r r r Útest_sniff_delimiterA s ýrE c C sh d}| }|j t|dd dd}tdddgdd d
|
gd d d ggdddgtdddgddd}t ||¡ d S )NzX# comment line
|
index|A|B|C
|
# comment line
|
foo|1|2|3 # ignore | this
|
bar|4|5|6
|
baz|7|8|9
|
r ú#)r3 r Úcommentr r r r r4 r5 r6 r7 r8 r9 r: r; r# r< r= r> r? rA rC )r r r r r r r r Ú test_sniff_delimiter_commentR s ýrH Úencodingzutf-8c C s | }d}|d k r0| |¡}t|}t||d}nt|}|j|dd d|d}tdddgdd d
|
gd d d ggdddgtdddgddd}t ||¡ d S )NzFignore this
|
ignore this too
|
index|A|B|C
|
foo|1|2|3
|
bar|4|5|6
|
baz|7|8|9
|
)rI r r )r3 r ZskiprowsrI r r r r4 r5 r6 r7 r8 r9 r: r; r# r< r= r> r? rA ) Úencoder r r r r r
|
r r )r rI r r r r r r r Ú test_sniff_delimiter_encodingd s
|
ýrK c C s@ | }|j tdddgd d d}tdgdgd}t ||¡ d S )Nz1,2r r )ÚnamesÚheaderr r r r ©r r r r r )r r r r r r r Útest_single_line s rO r Znrowsr c C sT d}| }|j t|f|}tdddgdddgdd d
|
ggd d d gd}t ||¡ d S )Nz>A,B,C
|
1,2,3
|
4,5,6
|
7,8,9
|
want to skip this
|
also also skip this
|
r r r r r4 r5 r6 r7 r8 r9 r: r; )rB rN rD r r r Útest_skipfooter s
|
*rP zcompression,klass)ÚgzipÚGzipFile)Úbz2ÚBZ2Filec
|
C s¢ | }t |d}| ¡ }W 5 Q R X | dd¡}| |¡}t |¡}t||}t ¡ D} || dd}
|
|
|
|¡ W 5 Q R X |j| d|d} t
|
| |¡ W 5 Q R X d S )NÚrbó ,s ::Úwb)Úmodeú::)r Ú compression) ÚopenÚreadÚreplacer r) Z importorskipÚgetattrr Z ensure_cleanÚwriter ) r Zcsv1rZ Úklassr Úfr r ÚmoduleÚpathÚtmpr r r r Ú test_decompression_regex_sep s
|
|
|
|
re c C s~ d}| }t dddddgddd d
|
d gd d dddggdddddgtjdddgdddd gd d }|jt|d d }t ||¡ d S )!Na A B C D E
|
one two three four
|
a b 10.0032 5 -0.5109 -2.3358 -0.4645 0.05076 0.3640
|
a q 20 4 0.4473 1.4152 0.2834 1.00661 0.1744
|
x q 30 3 -0.6662 -0.5243 -0.3580 0.89145 2.5838gEØðôJYà¿grùé·¯Àgî|?5^ºÝ¿gÌÑã÷6ý©?g²ï§ÆK×?g à- Ü?gÉTÁ¨¤ö?gúíëÀ9#Ò?g9]ð?g46<½RÆ?gá ©Qå¿g^)ËÇà¿g¶óýÔxéÖ¿gI&Âì?g=U«@r9 r: r; ÚDÚE)r r gı.n£$@r4 )r Úqé r )Úxrh é r ZoneZtwoZthreeZfour)rL rA ú\s+r )r r Ú from_tuplesr r r r r r r r Ú#test_read_csv_buglet_4x_multi_index± s ý
|
þù rn c C sF d}| }t jddgtdtdd}|jt|dd}t ||¡ d S ) Nz) A B C
|
a b c
|
1 3 7 0 3 6
|
3 1 4 1 5 9)r r r6 r r r5 )r r r r r4 r8 ZabcABCÚabcrA rl r )r Z from_recordsÚlistr r r r r r r r Ú$test_read_csv_buglet_4x_multi_index2Ê s ýrq Ú
|
add_footerFc C s` d}| }t dddgi}|r.ddi}|d7 }ni }|jt|fdgdd |}t ||¡ d S )
|
Nz1#2
|
3#4r g333333ó?g333333 @r" r z
|
FooterrF )rL Údecimalr )r rr r r r r0 r r r r Ú test_skipfooter_with_decimalØ s
|
rt rY z#####z!!!Z123z#1!c5z%!c!dz@@#4:2z_!pd#_zutf-16z utf-16-bez utf-16-lezutf-32Úcp037c C sV t dgdgd}| }d| d }| |¡}|jt||ddg|d}t ||¡ d S ) Nr r r Ú1Ú2r r )r rL rI )r rJ r r r r )r r rI r r r Z encoded_datar r r r Ú$test_encoding_non_utf8_multichar_sepë s
|
ÿrx Úquotingc C s ddi}| }d}|t jkrTd}tjt|d |jt|fd|i| W 5 Q R X n8d}tjt|d |jt|fd|i| W 5 Q R X d S )Nr z,,za,,b
|
1,,a
|
2,,"2,,b"z"Expected 2 fields in line 3, saw 3r% ry z+ignored when a multi-char delimiter is used)ÚcsvÚ
|
QUOTE_NONEr) r* r r r )r ry r0 r r r, r r r Útest_multi_char_sep_quotesÿ s
|
&r| c C s` | }d}t ddgddgddgd}|jt|dd d d
|
}t ||¡ | ¡ }d |jks\td S ) Nza,b,c
|
0,1,2
|
3,4,5,6
|
7,8,9r r6 r r7 r r8 )r r ÚcÚwarn)rM r Ú on_bad_lineszSkipping line 3)r r r r r Z
|
readouterrÚerrÚAssertionError)r Zcapsysr r r r Zcapturedr r r Útest_none_delimiter s r r za
|
1
|
"b"az a,b,c
|
cat,foo,bar
|
dog,foo,"bazr c C sn | }|r:d}t jt|d |jt||d W 5 Q R X n0d}t jt|d |jt||d W 5 Q R X d S )Nz)parsing errors in the skipped footer rowsr% r' z%unexpected end of data|expected after©r) r* r r r )r r r" r r, r r r Útest_skipfooter_bad_row! s r c C s@ | }d}d}t jt|d |jt|dddd W 5 Q R X d S )Nz4ignore
|
A,B,C
|
1,2,3 # comment
|
1,2,3,4,5
|
2,3,4
|
footer
|
z"Expected 3 fields in line 4, saw 5r% r rF )rM rG r" r )r r r r, r r r Útest_malformed_skipfooter0 s
|
r
c C s$ | }G dd d}| |d¡ d S )Nc @ s8 e Zd ZddddZddddZdd Zd
|
d Zd S ) z5test_python_engine_file_no_next.<locals>.NoNextBufferÚNone)Úreturnc S s
|
|| _ d S ©N©r )ÚselfZcsv_datar r r Ú__init__B s z>test_python_engine_file_no_next.<locals>.NoNextBuffer.__init__r c S s
|
| j ¡ S r )r Ú__iter__©r r r r r E s z>test_python_engine_file_no_next.<locals>.NoNextBuffer.__iter__c S s | j S r r r r r r r\ H s z:test_python_engine_file_no_next.<locals>.NoNextBuffer.readc S s | j S r r r r r r ÚreadlineK s z>test_python_engine_file_no_next.<locals>.NoNextBuffer.readlineN)Ú__name__Ú
|
__module__Ú __qualname__r r r\ r r r r r Ú NoNextBufferA s r za
|
1)r )r r r r r r Ú test_python_engine_file_no_next> s r Ú bad_line_funcc C s ddgS ©Nrw Ú3r ©rj r r r Ú<lambda>Q ó r c C s | d d
S ©Nr r r r r r r Q r c C sH | }d}t |}|j||d}tdddgdddgd}t ||¡ d S )Núa,b
|
1,2
|
2,3,4,5,6
|
3,4
|
©r r r r r r ©r r r r r )r r r r Úbad_sior r r r r Útest_on_bad_lines_callableQ s r c sx | }d}t |}g ddd fdd }|j||d}tddd gdd d
|
gd }t ||¡ d d dddggksttd S )Nr z list[str])Úbad_liner c s | ¡ ddgS r )Úappend©r ©Úlstr r r k s
|
zHtest_on_bad_lines_callable_write_to_external_list.<locals>.bad_line_funcr r r r r r rw r Ú4Ú5Ú6)r r r r r r )r r r r r r r r r£ r Ú1test_on_bad_lines_callable_write_to_external_list` s r¨ c C s ddgS )Nr# r< r r r r r r u r c C s | d d
S r r r r r r r u r ú,Z111c C s¢ | }d| d| d| d| d| d }t |}|j||dd|d }d
|
d d d dd ddd g}tt||D ].\}\} }
|
t|
|
t||d d}
|
t | |
|
¡ qnd S )Nz
|
0z1
|
hiz there
|
foor< zbaz
|
goodzbye
|
r T)r Ú chunksizeÚiteratorr ÚhiZthere)Ú0rv r# ZgoodZbye©r> )r r Ú enumerateÚzipr Úranger r ) r r r r r r Z result_iterÚ expectedsÚir r r r r Ú(test_on_bad_lines_callable_iterator_trueu s6 ÿÿÿÿÿ ÿýr´ c sL | }d}t |}d fdd}tjt d |j||d W 5 Q R X d S )Nr zThis function is buggy.c s t d S r )r+ r¢ ©r, r r r s zEtest_on_bad_lines_callable_dont_swallow_errors.<locals>.bad_line_funcr% r )r r) r* r+ r )r r r r r r rµ r Ú.test_on_bad_lines_callable_dont_swallow_errors s r¶ c C sP | }d}t |}|jtd|dd d}tdddgddd gd
|
}t ||¡ d S ) Nr zLength of header or namesc S s | S r r r r r r r ¬ r z@test_on_bad_lines_callable_not_expected_length.<locals>.<lambda>r r r r r r ©r Úread_csv_check_warningsr r r r ©r r r r r r r r r Ú.test_on_bad_lines_callable_not_expected_length¡ s ÿrº c C sH | }d}t |}|j|dd d}tddgddgd }t ||¡ d S )
|
Nr c S s d S r r r r r r r ¼ r z9test_on_bad_lines_callable_returns_none.<locals>.<lambda>r r r r r r r r¹ r r r Ú'test_on_bad_lines_callable_returns_none² s r» c C sP | }d}t |}|j|dd d}tddgddgd d
|
d gd }t ||¡ d S ) Nza,b
|
1,2,3
|
4,5,6
|
c S s ddgS )NZ99r r r r r r Ê r z6test_on_bad_lines_index_col_inferred.<locals>.<lambda>r r r4 r r5 r r r r® r r¹ r r r Ú$test_on_bad_lines_index_col_inferredÁ s r¼ c C sH | }d}|j tdt|dd dd}tddgdd gd
|
}t ||¡ d S ) Nz
|
0.5,0.03
|
0.1,0.2,0.3,2
|
úLength of headerr© F)r rM r3 g à?g¹?g¸
ëQ¸?gÉ?)r r )r¸ r r r r r ©r r r r r r r r Ú$test_index_col_false_and_header_noneÏ s úr¿ c C sB | }t d}|jtd|ddd}tddddgi}t ||¡ d S ) Nza
|
a,b
|
c,d,e
|
f,g,hr½ ÚpythonF)Zenginer3 r r} ra r· r¾ r r r Ú?test_header_int_do_not_infer_multiindex_names_on_different_lineâ s ÿrÁ )4Ú__doc__Ú
|
__future__r rz Úior r r Útypingr r) Z pandas.errorsr r Zpandasr r
|
r Zpandas._testingZ_testingr r! ÚmarkZ parametrizer- r/ rE rH rK rO rP re rn rq rt rx Ú QUOTE_MINIMALr{ r| r r r
r r r¨ r´ r¶ rº r» r¼ r¿ rÁ r r r r Ú<module> st
|
|
|
|
|
|
ÿ
|
|
ÿ ÿ
|
|
|
