U
|
¬ý°d ã @ sð d dl Zd dlZd dlZd dlmZ ejdddZ d dl
|
m Z dd ej D Z e dd ejD 7 Z e ejd d
|
d dgd d g7 Z eje d d e D ddd Zdd Zdd Zdd Zdd Zdd Zdd Zejd d Zd d Zd!d" ZdS )#é NZpyarrowz1.0.1)Z
|
minversion)Ú pyarrow_array_to_numpy_and_maskc C s" g | ]}t jd dddg|dqS )é é é N©Údtype©ÚpdÚarray©Ú.0r © r úcd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/tests/arrays/masked/test_arrow_compat.pyÚ
|
<listcomp> s r c C s" g | ]}t jd dddg|dqS )g¹?gÉ?g333333Ó?Nr r r r r r r s TFÚbooleanr c C s g | ] }|j jqS r )r Úname)r Úar r r r s )ÚparamsÚidsc C s | j S )zi
|
Fixture returning parametrized array from given dtype, including integer,
|
float and boolean
|
)Úparam)Úrequestr r r Údata s r c C s> t | ¡}t j| jtd dt | jj¡d}| |¡s:td S )N)Zna_value©Útype) Úpar
|
Zto_numpyÚobjectÚfrom_numpy_dtyper Ú numpy_dtypeÚequalsÚAssertionError)r ÚarrÚexpectedr r r Útest_arrow_array s
|
þr" c C s` t d| i¡}t |¡}| d¡jt| jjks4t |
|
¡ }|d j| jksPt t ||¡ d S )Nr ) r Ú DataFramer ÚtableÚfieldr Ústrr r r Ú to_pandasÚtmÚassert_frame_equal©r Údfr$ Úresultr r r Útest_arrow_roundtrip" s
|
r- c
|
C sÔ dd } t jdd dgt ¡ d}t jdd dgt ¡ d}t jdd d
|
gt ¡ d}t j |||gd d d g¡}|j| d}tj dd dgdd}tj dd dgdd}tj dd d
|
gdd}t
|
|||d¡} t || ¡ d S )Nc S s, t j | ¡rt ¡ S t j | ¡r(t ¡ S d S )N)r ÚtypesZ
|
is_booleanr Z BooleanDtypeÚ
|
is_integerZ
|
Int64Dtype)Z
|
arrow_typer r r Ú types_mapper, s z<test_dataframe_from_arrow_types_mapper.<locals>.types_mapperTFr r r éÿÿÿÿr é ÚboolsÚintsÚ
|
small_ints)r0 r r ZInt64)r3 r4 r5 ) r r
|
Úbool_Úint64Zint8Z RecordBatchZ from_arraysr' r ZSeriesr# r( r) )
|
r0 Z bools_arrayZ
|
ints_arrayZsmall_ints_arrayZ record_batchr, r3 r4 r5 r! r r r Ú&test_dataframe_from_arrow_types_mapper+ s ÿ r8 c C s t d| dd
i¡}t |¡}| d¡jt| jjks<t tjtj
|
g | d¡jdg|j d}| ¡ }|d j| jks|t t ||¡ d S )Nr r r )Úschema)r r# r r$ r% r r& r r r Z chunked_arrayr9 r' r( r) r* r r r Ú test_arrow_load_from_zero_chunks@ s
|
ÿr: c C sN t ¡ } | tjddddd gdd¡}t jddddd gdd}t ||¡ d S ) Nr r r é r7 r ZUInt32r )r Z UInt32DtypeÚ__from_arrow__r r
|
r( Z assert_extension_array_equal)r r, r! r r r Útest_arrow_from_arrow_uintN s r= c C s t d| i¡}t |¡}| dd ¡ ¡ }|jdd
jdd}t ||¡ |
|
| d ¡}t |¡}| dd ¡ ¡ }|jdd
jdd}t ||¡ d S )Nr r T)Zdropr ) r r# r r$ Úslicer' ZilocZ reset_indexr( r) Zfillna)r r+ r$ r, r! Zdf2r r r Útest_arrow_slicedY s
|
|
r? c C s\ t | ¡}t |¡}tjddddg|d}t jdddg|d}t ddddg¡}||||fS ) zt
|
Fixture returning actual and expected dtype, pandas and numpy arrays and
|
mask from a given numpy dtype
|
r r r Nr r TF)Únpr r r r
|
)Zany_real_numpy_dtypeÚnp_dtypeZpa_typeÚpa_arrayÚ np_expectedÚ mask_expectedr r r Únp_dtype_to_arraysj s
|
|
rE c C s¼ | \}}}}t ||\}}t |dd
|¡ t ||¡ | ¡ d }| ¡ d }| ¡ d ¡ } t | d ¡}
|
tjj|j t
|
|||
|
g|j d} | ¡ t | |\}}t |dd
|¡ t ||¡ d|j j d } t | | ¡} t d¡}tjj|j t
|
||| g|j d d}| ¡ t ||\}}t |dd
|¡ t ||¡ tjg |d }tjg tjd }tjj|j d||g|j d}| ¡ t ||\}}t |dd
|¡ t ||¡ dS )
|
a&
|
Test conversion from pyarrow array to numpy array.
|
|
Modifies the pyarrow buffer to contain padding and offset, which are
|
considered valid buffers by pyarrow.
|
|
Also tests empty pyarrow arrays with non empty buffers.
|
See https://github.com/pandas-dev/pandas/issues/40896
|
Nr r r ó )r ÚlengthÚbuffersÚoffseté ó r )r r( Zassert_numpy_array_equalrH Z
|
to_pybytesr Z py_bufferÚArrayZ from_buffersr ÚlenrI ÚvalidateZ bit_widthr@ r
|
r6 )rE rA rB rC rD r ÚmaskZ mask_bufferZ data_bufferZdata_buffer_bytesZdata_buffer_trailZpa_array_trailrI Zdata_buffer_offsetZmask_buffer_offsetZpa_array_offsetZnp_expected_emptyZmask_expected_emptyr r r Ú$test_pyarrow_array_to_numpy_and_mask| sV
|
ü
|
ü ürP c C s: t | ¡ d¡}tjtd d | j |¡ W 5 Q R X d S )NÚstring)Úmatch)r r
|
ÚcastÚpytestZraisesÚ TypeErrorr r< )r r r r r Útest_from_arrow_type_error» s rV )Únumpyr@ rT Zpandasr Zpandas._testingZ_testingr( Z importorskipr Z%pandas.core.arrays.arrow._arrow_utilsr ZALL_INT_EA_DTYPESZarraysZFLOAT_EA_DTYPESr
|
Zfixturer r" r- r8 r: r= r? rE rP rV r r r r Ú<module> s(
|
|
?
|
