U
|
¬ý°d¹) ã @ sV d Z ddlmZ ddlZddlmZmZmZmZm Z ddl
|
Z ddl m Z ddlmZ ddlmZmZ ddlmZmZ dd lmZmZmZmZmZ er°dd
|
l m Z m Z m Z m Z m!Z! d Z"d d ddddZ#dde"dfdddddddddZ$de"fd dddd d d Z%d dddd!d"d#Z&de"dfd$ddddd%d&d'Z'de"dfd(ddddd%d)d*Z(dS )+z"
|
data hash pandas / numpy objects
|
é )Ú annotationsN)Ú TYPE_CHECKINGÚHashableÚIterableÚIteratorÚcast)Úlib)Úhash_object_array)Ú ArrayLikeÚnpt)Úis_categorical_dtypeÚ is_list_like)Ú ABCDataFrameÚABCExtensionArrayÚABCIndexÚ ABCMultiIndexÚ ABCSeries)Ú CategoricalÚ DataFrameÚIndexÚ
|
MultiIndexÚSeriesZ0123456789123456zIterator[np.ndarray]Úintznpt.NDArray[np.uint64])ÚarraysÚ num_itemsÚreturnc C sÌ z t | }W n$ tk
|
r0 tjg tjd Y S X t |g| ¡} t d¡}t |¡t d¡ }d}t| D ]:\}}|| }||N }||9 }|t d| | ¡7 }|}qj|d |ksºt d|t d¡7 }|S ) z¼
|
Parameters
|
----------
|
arrays : Iterator[np.ndarray]
|
num_items : int
|
|
Returns
|
-------
|
np.ndarray[uint64]
|
|
Should be the same as CPython's tupleobject.c
|
©ÚdtypeiCB ixV4 r iXB é zFed in wrong num_itemsiû| )
|
ÚnextÚ StopIterationÚnpÚarrayÚuint64Ú itertoolsÚchainZ
|
zeros_likeÚ enumerateÚAssertionError) r r ÚfirstZmultÚoutZlast_iÚiÚaZ inverse_i© r, úOd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/core/util/hashing.pyÚcombine_hash_arrays2 s"
|
r. TÚutf8zIndex | DataFrame | SeriesÚboolÚstrz
|
str | Noner )ÚobjÚindexÚencodingÚhash_keyÚ
|
categorizer c s ddl m} dkrtttr8|tdddS ttrptj j ddd}||ddd}n tt
|
rÜtj j ddd}|rÈ fd d
|
d D }t |g|¡} t | d }||jddd}n ttrj fd d
|
¡ D }
|
tj} |rL fdd
|
d D } | d7 } t |
|
| ¡} dd
|
| D }
|
t |
|
| }||jddd}ntdt |S )a~
|
Return a data hash of the Index/Series/DataFrame.
|
|
Parameters
|
----------
|
obj : Index, Series, or DataFrame
|
index : bool, default True
|
Include the index in the hash (if Series/DataFrame).
|
encoding : str, default 'utf8'
|
Encoding for data & key when strings.
|
hash_key : str, default _default_hash_key
|
Hash_key for string key to encode.
|
categorize : bool, default True
|
Whether to first categorize object arrays before hashing. This is more
|
efficient when the array contains duplicate values.
|
|
Returns
|
-------
|
Series of uint64, same length as the object
|
r )r Nr# F)r Úcopy©r7 )r3 r r7 c 3 s$ | ] }t jd djV qdS ©F)r3 r4 r5 r6 N©Úhash_pandas_objectr3 Ú_values©Ú.0Ú_©r6 r4 r5 r2 r, r- Ú <genexpr> s ùûz%hash_pandas_object.<locals>.<genexpr>©Né c 3 s" | ]\}}t |j V qd S rB )Ú
|
hash_arrayr< )r> r? Zseries)r6 r4 r5 r, r- rA s ÿc 3 s$ | ] }t jd djV qdS r9 r: r= r@ r, r- rA s ùûr c s s | ]
|
}|V qd S rB r, )r> Úxr, r, r- rA ¨ s z Unexpected type for hashing )Úpandasr Ú_default_hash_keyÚ
|
isinstancer Ú hash_tuplesr rD r< Úastyper r$ r% r. r3 r ÚitemsÚlenÚcolumnsÚ TypeErrorÚtype)r2 r3 r4 r5 r6 r ÚhZserZ
|
index_iterr Úhashesr Zindex_hash_generatorÚ_hashesr, r@ r- r; V sN
|
|
ÿ
|
ÿø
|
|
þ
|
ø
|
|
r; z+MultiIndex | Iterable[tuple[Hashable, ...]])Úvalsr4 r5 r c sz t | stdddlm m} t| ts6| | ¡n| fddtj D }fdd|D }t
|
|t |}|S )a
|
Hash an MultiIndex / listlike-of-tuples efficiently.
|
|
Parameters
|
----------
|
vals : MultiIndex or listlike-of-tuples
|
encoding : str, default 'utf8'
|
hash_key : str, default _default_hash_key
|
|
Returns
|
-------
|
ndarray[np.uint64] of hashed values
|
z'must be convertible to a list-of-tuplesr )r r c s( g | ] } j | j| d ddqS )FT©ZorderedZfastpath)ÚcodesÚlevels)r> Úlevel)r Úmir, r- Ú
|
<listcomp>Ò s ÿz hash_tuples.<locals>.<listcomp>c 3 s | ]}t | d V qdS )©r4 r5 N)Ú_hash_categorical)r> ÚcatrZ r, r- rA Ø s z hash_tuples.<locals>.<genexpr>) r rN rF r r rH r Ú from_tuplesÚrangeZnlevelsr. rL )rS r4 r5 r Zcat_valsrQ rP r, )r r4 r5 rX r- rI ² s
|
þ ÿrI r )r\ r4 r5 r c C sd t | jj¡}t|||dd}| ¡ }t|r<| | j¡}nt j t|dd}|
|
¡ r`t j ||< |S )a
|
Hash a Categorical by hashing its categories, and then mapping the codes
|
to the hashes
|
|
Parameters
|
----------
|
cat : Categorical
|
encoding : str
|
hash_key : str
|
|
Returns
|
-------
|
ndarray[np.uint64] of hashed values, same size as len(c)
|
F)r6 r# r ) r! ZasarrayÚ
|
categoriesr< rD ZisnarL ZtakerU ÚzerosÚanyr Zu8max)r\ r4 r5 ÚvaluesÚhashedÚmaskÚresultr, r, r- r[ à s
|
r[ r
|
)rS r4 r5 r6 r c C s~ t | dstd| j}t|r6td| } t| ||S t| trN| ¡ \} }n"t| t j
|
sptdt | j dt | |||S )aK
|
Given a 1d array, return an array of deterministic integers.
|
|
Parameters
|
----------
|
vals : ndarray or ExtensionArray
|
encoding : str, default 'utf8'
|
Encoding for data & key when strings.
|
hash_key : str, default _default_hash_key
|
Hash_key for string key to encode.
|
categorize : bool, default True
|
Whether to first categorize object arrays before hashing. This is more
|
efficient when the array contains duplicate values.
|
|
Returns
|
-------
|
ndarray[np.uint64, ndim=1]
|
Hashed values, same length as the vals.
|
r zmust pass a ndarray-liker z6hash_array requires np.ndarray or ExtensionArray, not z!. Use hash_pandas_object instead.)ÚhasattrrN r r r r[ rH r Z_values_for_factorizer! ZndarrayrO Ú__name__Ú _hash_ndarray)rS r4 r5 r6 r r? r, r, r- rD s
|
|
|
ÿrD z
|
np.ndarrayc C sr | j }t |tj¡r4tt | ¡dtt | ¡ S |tkrH| d¡} næt |j
|
tj tj frr| d¡jddd} n¼t |j
|
tjr¦|jdkr¦| d| j j ¡ d¡} n|rîdd lm}m}m} || dd
|
\}} |||| dd d }
|
t|
|
||S zt| ||} W n. tk
|
r, t| t¡ t¡||} Y nX | | d ? N } | t d¡9 } | | d? N } | t d¡9 } | | d? N } | S )z!
|
See hash_array.__doc__.
|
é Úu8Úi8Fr8 é Úur )r r Ú factorize)ÚsortTrT é l ¹eÉ9´ Âz é l ëb&ì&& é )r r! Z
|
issubdtypeZ
|
complex128rD ÚrealÚimagr0 rJ Ú
|
issubclassrO Z
|
datetime64Z timedelta64ÚviewÚnumberÚitemsizerF r r rn r[ r rN r1 Úobjectr# ) rS r4 r5 r6 r r r rn rU r_ r\ r, r, r- rh 9 s8 ÿ
|
rh ))Ú__doc__Ú
|
__future__r r$ Útypingr r r r r Únumpyr! Z pandas._libsr Zpandas._libs.hashingr Zpandas._typingr
|
r Zpandas.core.dtypes.commonr r Zpandas.core.dtypes.genericr r r r r rF r r r r r rG r. r; rI r[ rD rh r, r, r, r- Ú<module> s<
|
&û^ý.*ü3ü
|
