U
|
¬ý°dlE ã @ s U d dl mZ d dlZd dlZd dlmZ d dlZd dlZ d dl
|
m Z m Z m Z mZmZ d dlmZmZ ejejejejejdejejejej ej dej ej ej dej!de"iiZ#d e$d
|
< d6d d dddZ%d7dd dddZ&dd dddZ'ddddd Z(dd dd d Z)dddd d!Z*d"d# Z+dddd$d%Z,d8d&d'd(d)d*d+d,d-Z-d9d*d(d(d*d.d/d0Z.d:d1dd2d d3d4d5Z/dS );é )Ú annotationsN)ÚAny)ÚBufferÚColumnÚColumnNullTypeÚ DataFrameÚ DtypeKind)Ú ArrowCTypesÚ
|
Endianness)é é é é@ )r r r z dict[DtypeKind, dict[int, Any]]Ú
|
_NP_DTYPESTÚboolz pd.DataFrame)Ú
|
allow_copyÚreturnc C s2 t | tjr| S t| ds"tdt| j|dS )a¯
|
Build a ``pd.DataFrame`` from any DataFrame supporting the interchange protocol.
|
|
Parameters
|
----------
|
df : DataFrameXchg
|
Object supporting the interchange protocol, i.e. `__dataframe__` method.
|
allow_copy : bool, default: True
|
Whether to allow copying the memory to perform the conversion
|
(if false then zero-copy approach is requested).
|
|
Returns
|
-------
|
pd.DataFrame
|
Ú __dataframe__z#`df` does not support __dataframe__)r )Ú
|
isinstanceÚpdr ÚhasattrÚ
|
ValueErrorÚ_from_dataframer ©Údfr © r ú]d:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/core/interchange/from_dataframe.pyÚfrom_dataframe s
|
|
r Ú DataFrameXchgr c C s g }| ¡ D ]}t|}| |¡ q |s<t|dkr<tdt|dkrR|d }ntj|dddd}| j dd¡}|dk r||_ |S ) a
|
Build a ``pd.DataFrame`` from the DataFrame interchange object.
|
|
Parameters
|
----------
|
df : DataFrameXchg
|
Object supporting the interchange protocol, i.e. `__dataframe__` method.
|
allow_copy : bool, default: True
|
Whether to allow copying the memory to perform the conversion
|
(if false then zero-copy approach is requested).
|
|
Returns
|
-------
|
pd.DataFrame
|
é zHTo join chunks a copy is required which is forbidden by allow_copy=Falser TF)ZaxisZ ignore_indexÚcopyz pandas.indexN)
|
Z
|
get_chunksÚprotocol_df_chunk_to_pandasÚappendÚlenÚ RuntimeErrorr ÚconcatÚmetadataÚgetÚindex)r r Z
|
pandas_dfsÚchunkÚ pandas_dfZ index_objr r r r 7 s ÿ
|
r )r r c C s i }g }| ¡ D ]â}t|ts.td| d||krFtd| d| |¡}|jd }|tjtjtj tj
|
fkrt |\||< }nd|tj kr t |\||< }nH|tjkr¼t|\||< }n,|tjkrØt|\||< }ntd| d| |¡ qt |¡}||jd< |S )z¡
|
Convert interchange protocol chunk to ``pd.DataFrame``.
|
|
Parameters
|
----------
|
df : DataFrameXchg
|
|
Returns
|
-------
|
pd.DataFrame
|
zColumn z is not a stringz is not uniquer z
|
Data type z not handled yetZ _INTERCHANGE_PROTOCOL_BUFFERS)Z column_namesr Ústrr Zget_column_by_nameÚdtyper ÚINTÚUINTÚFLOATÚBOOLÚprimitive_column_to_ndarrayZ CATEGORICALÚ categorical_column_to_seriesÚSTRINGÚstring_column_to_ndarrayZDATETIMEÚdatetime_column_to_ndarrayÚNotImplementedErrorr" r r Úattrs)r ÚcolumnsÚbuffersÚnameÚcolr, Úbufr* r r r r! \ s6
|
|
|
ü
|
|
|
|
|
r! r ztuple[np.ndarray, Any])r; r c C s@ | ¡ }|d \}}t||| j| ¡ }t|| |d }||fS )aJ
|
Convert a column holding one of the primitive dtypes to a NumPy array.
|
|
A primitive type is one of: int, uint, float, bool.
|
|
Parameters
|
----------
|
col : Column
|
|
Returns
|
-------
|
tuple
|
Tuple of np.ndarray holding the data and the memory owner object
|
that keeps the memory alive.
|
ÚdataÚvalidity)Ú get_buffersÚbuffer_to_ndarrayÚoffsetÚsizeÚ set_nulls)r; r9 Ú data_buffÚ
|
data_dtyper= r r r r1 s
|
r1 ztuple[pd.Series, Any]c C s¬ | j }|d std|d }t|dr6t |j¡}ntd| ¡ }|d \}}t||| j| ¡ }||t
|
| }t j |||d d} t | ¡}
|
t|
|
| |d }
|
|
|
|fS )
|
a
|
Convert a column holding categorical data to a pandas Series.
|
|
Parameters
|
----------
|
col : Column
|
|
Returns
|
-------
|
tuple
|
Tuple of pd.Series holding the data and the memory owner object
|
that keeps the memory alive.
|
Z is_dictionaryz-Non-dictionary categoricals not supported yetÚ
|
categoriesÚ_colz}Interchanging categorical columns isn't supported yet, and our fallback of using the `col._col` attribute (a ndarray) failed.r= Z
|
is_ordered)rF Zorderedr> )Zdescribe_categoricalr6 r ÚnpÚarrayrG r? r@ rA rB r# r Z CategoricalÚSeriesrC ) r; Z categoricalZ
|
cat_columnrF r9 Z
|
codes_buffZ codes_dtypeÚcodesÚvaluesÚcatr= r r r r2 £ s*
|
ÿ ÿ
|
r2 c C s¤ | j \}}|tjtjtjfkr,t| d| ¡ }|d sDtd|d \}}|d dks`t|d tj tj
|
fksxtt j dtj tjf}t||d| ¡ d }|d \}} t|| | j| ¡ d d
|
}
|
d } |tjtjfkr|d sêtd |d \} } t| | | j| ¡ } |dkr| } d g| ¡ }t| ¡ D ]\}| d k rZ| | rZtj||< q4||
|
| |
|
|d
}t|}|jdd}|||< q4tj|dd|fS )a
|
Convert a column holding string data to a NumPy array.
|
|
Parameters
|
----------
|
col : Column
|
|
Returns
|
-------
|
tuple
|
Tuple of np.ndarray holding the data and the memory owner object
|
that keeps the memory alive.
|
z3 null kind is not yet supported for string columns.Úoffsetsz#String buffers must contain offsetsr= r r é r )rA Úlength)rP Nr> z*Validity buffers cannot be empty for maskszutf-8)ÚencodingÚobject©r, )Ú describe_nullr Ú NON_NULLABLEÚ USE_BITMASKÚ USE_BYTEMASKr6 r? ÚAssertionErrorr r3 Z LARGE_STRINGr r. ZUINT8r
|
ÚNATIVEr@ rB rA ÚrangerH ÚnanÚbytesÚdecodeZasarray)r; Ú null_kindÚ sentinel_valr9 rD Zprotocol_data_dtyperE r= Z offset_buffZ offset_dtyperN Únull_posÚ
|
valid_buffÚ valid_dtypeZstr_listÚiZunitsZ str_bytesÚstringr r r r4 Ò s^
|
ýÿ þ
|
ü
|
ÿ
|
|
r4 c C sÎ t d| ¡}|r\| d¡| d¡ }}|dkr6td|dkrF|d7 }| d| d¡}|S t d | ¡}|r¼| d¡}|d
|
kr| tj¡d d ¡}n"|d krª| d¡}ntd| |S td| dS )z4Parse datetime `format_str` to interpret the `data`.zts([smun]):(.*)r rO Ú z Timezones are not supported yetÚsz datetime64[ú]ztd([Dm])ÚDiQ z datetime64[s]Úmzdatetime64[ms]z Date unit is not supported: z DateTime kind is not supported: N)ÚreÚmatchÚgroupr6 ÚastyperH Úuint64)Ú
|
format_strr= Ztimestamp_metaÚunitÚtzZ date_metar r r Úparse_datetime_format_str+ s&
|
rr c C sx | ¡ }| j\}}}}|d \}}t|tj|d ttd|d tjf| j |
|
¡ }t ||}t || |d }||fS )a
|
Convert a column holding DateTime data to a NumPy array.
|
|
Parameters
|
----------
|
col : Column
|
|
Returns
|
-------
|
tuple
|
Tuple of np.ndarray holding the data and the memory owner object
|
that keeps the memory alive.
|
r= r r. r> ) r? r, r@ r r. Úgetattrr r
|
rY rA rB rr rC )r; r9 Ú_ro Údbufr, r= r r r r5 L s ü÷
|
r5 r z tuple[DtypeKind, int, str, str]Úintz
|
int | Nonez
|
np.ndarray)Úbufferr, rA rP r c C s |\}}}}t |i ¡ |d¡}|dkr8td| dtj |¡}t | j|| d t |¡¡} |dkr¢|dk s|t
|
dtjj | | j fd}
|
t |
|
||d dS tjj | | j |d fdS dS ) aÚ
|
Build a NumPy array from the passed buffer.
|
|
Parameters
|
----------
|
buffer : Buffer
|
Buffer to build a NumPy array from.
|
dtype : tuple
|
Data type of the buffer conforming protocol dtypes format.
|
offset : int, default: 0
|
Number of elements to offset from the start of the buffer.
|
length : int, optional
|
If the buffer is a bit-mask, specifies a number of bits to read
|
from the buffer. Has no effect otherwise.
|
|
Returns
|
-------
|
np.ndarray
|
|
Notes
|
-----
|
The returned array doesn't own the memory. The caller of this function is
|
responsible for keeping the memory owner object alive as long as
|
the returned NumPy array is being used.
|
NzConversion for ú is not yet supported.r r z1`length` must be specified for a bit-mask buffer.)Úshape)Úfirst_byte_offset)r r' r6 rH Z ctypeslibZas_ctypes_typeÚctypesÚcastÚptrÚPOINTERrX Zas_arrayÚbufsizeÚbitmask_to_bool_ndarray) rw r, rA rP ÚkindZ bit_widthrt Z column_dtypeZ ctypes_typeZ data_pointerZarrr r r r@ p s" ÿ ÿr@ )ÚbitmaskÚ mask_lengthrz r c
|
C s
|
|d }| |d
} |d; }t j|td}| d }d}td| |}t|D ]$}|d|| > @ rhd||< |d7 }qLt|| d D ]:} | | d }tdD ] }|d|> @ r²d||< |d7 }qqt| dkr| d }tt|| D ] }|d|> @ rüd||< |d7 }qä|S )a
|
Convert bit-mask to a boolean NumPy array.
|
|
Parameters
|
----------
|
bitmask : np.ndarray[uint8]
|
NumPy array of uint8 dtype representing the bitmask.
|
mask_length : int
|
Number of elements in the mask to interpret.
|
first_byte_offset : int, default: 0
|
Number of elements to offset from the start of the first byte.
|
|
Returns
|
-------
|
np.ndarray[bool]
|
r NrS r r Téÿÿÿÿ)rH Úzerosr ÚminrZ r# )
|
r r rz Z bytes_to_skipZ bool_maskÚvalZmask_idxZbits_in_first_byteÚjrc r r r r § s0
|
|
r znp.ndarray | pd.Seriesz5tuple[Buffer, tuple[DtypeKind, int, str, str]] | None)r= r; r> Úallow_modify_inplacec C sè |j \}}d}|tjkr(t | ¡|k}nj|tjtjfkrp|sDtd|\}}t|||j |
|
¡ }|dkr| }n"|tj tj fkrnt d| d|dk rät |¡rä|s°| ¡ } z d| |< W n& tk
|
râ | t¡} d| |< Y nX | S )aÆ
|
Set null values for the data according to the column null kind.
|
|
Parameters
|
----------
|
data : np.ndarray or pd.Series
|
Data to set nulls in.
|
col : Column
|
Column object that describes the `data`.
|
validity : tuple(Buffer, dtype) or None
|
The return value of ``col.buffers()``. We do not access the ``col.buffers()``
|
here to not take the ownership of the memory of buffer objects.
|
allow_modify_inplace : bool, default: True
|
Whether to modify the `data` inplace when zero-copy is possible (True) or always
|
modify a copy of the `data` (False).
|
|
Returns
|
-------
|
np.ndarray or pd.Series
|
Data with the nulls being set.
|
Nz/Expected to have a validity buffer for the maskr z
|
Null kind rx )rT r Z USE_SENTINELr rJ rV rW rX r@ rA rB rU ZUSE_NANr6 rH Úanyr Ú TypeErrorrm Úfloat) r= r; r> r r^ r_ r` ra rb r r r rC Ý s,
|
|
|
rC )T)T)r N)r )T)0Ú
|
__future__r r{ rj Útypingr ÚnumpyrH Zpandasr Z*pandas.core.interchange.dataframe_protocolr r r r r r Z pandas.core.interchange.utilsr r
|
r- Zint8Úint16Úint32Úint64r. Zuint8Zuint16Zuint32rn r/ Úfloat32Úfloat64r0 r r Ú__annotations__r r r! r1 r2 r4 rr r5 r@ r rC r r r r Ú<module> s@ ü %./Y!' ü8 ÿ: ü
|
