U
|
¬ý°dV} ã @ s¼ d Z ddlmZ ddlZddlZddlmZ ddlZddl m
|
Z
|
m Z m Z m Z mZ ddlZddlZddlZddlmZmZmZmZ ddlmZ ddl m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z& dd l'm(Z( dd
|
l)m*Z*m+Z+m,Z, dd l-m.Z. dd l/m0Z0 dd l1m2Z2 ddl3m4Z4m5Z5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;m<Z<m=Z=m>Z>m?Z?m@Z@mAZAmBZBmCZCmDZD ddlEmFZF ddlGmHZHmIZI ddlJmKZKmLZL ddlMmNZNmOZOmPZPmQZQmRZR ddlSmT mUZV ddlWmXZXmYZYmZZZ ddl[m\Z\ ddl]mT m^Z_ ddl`maZambZb ddlcmdZd ddlemfZf ddlgmhZh e
|
r6ddlMmiZi ddljmkZk ddlWmlZl ejmejnejoejnejpejqejrejsejtejuejvejwejxejyejzej{ej|ej}ej~ej}ejejejejejejej
ejejejiZe+d e*eddd dd!d!d"d#d#d#d$d$d$d%d&d'd(d)d*d+d,Zd)d)d-d.d/Zdd)d)d#d#d#d(d%d1d)d2 d3d4Zd
d!d!d#d#d#d$d$d%d$d7d)d8 d9d:ZG d;d< d<Zdd$d=d>d?d@dAZdBdBdCdDdDdEdFdGdHZG dIdJ dJeZd7dKdLdMZe5e4e6dNZdOd7dPdQdRZG dSdT dTeZdBd$d>dUdVdWZddCd$d>dUdXdYZd>dZd[d\Zd]d$d>d^d_d`ZddCdCd$dadbdcddZddeded$d=dfdgdhdiZdededjdkdldmZdndDdDd>dodpdqZdrdrdsd$dtdudvdwZd$dZdxdyZd$dZdzd{Z d!d)d|d}d~Z¡dCdCd%ddddZ¢dS )z
|
SQL-style merge routines
|
é )Ú annotationsN)Úpartial)Ú TYPE_CHECKINGÚHashableÚLiteralÚSequenceÚcast)Ú TimedeltaÚ hashtableÚjoinÚlib)Úis_range_indexer)
|
Ú AnyArrayLikeÚ ArrayLikeÚAxisIntÚDtypeObjÚ
|
IndexLabelÚJoinHowÚMergeHowÚShapeÚSuffixesÚnpt)Ú
|
MergeError)ÚAppenderÚ SubstitutionÚcache_readonly)Úfind_stack_level)ÚExtensionDtype)Úfind_common_type)Úensure_float64Ú ensure_int64Ú ensure_objectÚ is_array_likeÚis_boolÚ is_bool_dtypeÚis_categorical_dtypeÚis_dtype_equalÚis_extension_array_dtypeÚis_float_dtypeÚ
|
is_integerÚis_integer_dtypeÚ is_list_likeÚ is_numberÚis_numeric_dtypeÚis_object_dtypeÚneeds_i8_conversion)ÚDatetimeTZDtype)Ú ABCDataFrameÚ ABCSeries)ÚisnaÚna_value_for_dtype)Ú
|
ArrowDtypeÚ CategoricalÚIndexÚ
|
MultiIndexÚSeries)ÚArrowExtensionArrayÚBaseMaskedArrayÚExtensionArray)ÚNDArrayBackedExtensionArray)Ú ensure_wrapped_if_datetimelikeÚ extract_array)Ú
|
_merge_doc)Ú default_index©Úis_int64_overflow_possible)Ú DataFrame)Úgroupby)Ú DatetimeArrayz!
|
left : DataFrame or named Series)ÚindentsÚinnerF©Ú_xZ_yúDataFrame | Seriesr úIndexLabel | NoneÚboolr ú bool | Noneú
|
str | boolú
|
str | NonerD )ÚleftÚrightÚhowÚonÚleft_onÚright_onÚ
|
left_indexÚ right_indexÚsortÚsuffixesÚcopyÚ indicatorÚvalidateÚreturnc C s, t | ||||||||| | | d } | j|
|
dS )N)
|
rS rT rU rV rW rX rY rZ r\ r] ©r[ )Ú_MergeOperationÚ
|
get_result)rQ rR rS rT rU rV rW rX rY rZ r[ r\ r] Úop© rc úPd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/core/reshape/merge.pyÚmerge s ôre ©rQ rR c s> g }t | ttfs| g} |j| dd}d}tfdd| D rNj| dd}|jj|j|jdD ]¦\}}|dkrx} ntz |j
|
| ¡} W n^ t k
|
rê |j ¡ fddj D }
|
|j|
|
d } tt| | _| | ¡ Y qbY nX ||| } || | < | | ¡ qbd
|
d lm} | |d d } | j|d
|
j dd} | |fS )zÒ
|
groupby & merge; we are always performing a left-by type operation
|
|
Parameters
|
----------
|
by: field to group
|
left: DataFrame
|
right: DataFrame
|
merge_pieces: function for merging
|
F©rY Nc 3 s | ]}| j kV qd S ©N©Úcolumns)Ú.0Úitem©rR rc rd Ú <genexpr>µ s z%_groupby_and_merge.<locals>.<genexpr>)Úaxisc s g | ]}|t kr|qS rc )Úset)rk Úr)Úlcolsrc rd Ú
|
<listcomp>Á s z&_groupby_and_merge.<locals>.<listcomp>ri r ©ÚconcatT)Z ignore_index)rj r[ )Ú
|
isinstanceÚlistÚtuplerE ÚallZgrouperZ get_iteratorZ _selected_objro ÚtakeÚindicesÚKeyErrorrj ÚtolistZreindexÚrangeÚlenÚindexÚappendZpandas.core.reshape.concatru )ÚbyrQ rR Z merge_piecesÚpiecesZlbyZrbyÚkeyÚlhsÚrhsÚcolsÚmergedru Úresultrc )rr rR rd Ú_groupby_and_merge¡ s4
|
|
|
|
r Úouterr ) rQ rR rT rU rV Ú fill_methodrZ rS r^ c
|
sþ ddfdd |dk r4|dk r4t d|dk rt|trL|g}t| | j¡}
|
t|
|
dkrvt|
|
dt|| | fd d
|
\} } nh|dk rðt|trª|g}t| |j¡}
|
t|
|
dkrÔt|
|
d t||| fd d
|
\} } n
|
| |} | S ) a1
|
Perform a merge for ordered data with optional filling/interpolation.
|
|
Designed for ordered data like time series data. Optionally
|
perform group-wise merge (see examples).
|
|
Parameters
|
----------
|
left : DataFrame or named Series
|
right : DataFrame or named Series
|
on : label or list
|
Field names to join on. Must be found in both DataFrames.
|
left_on : label or list, or array-like
|
Field names to join on in left DataFrame. Can be a vector or list of
|
vectors of the length of the DataFrame to use a particular vector as
|
the join key instead of columns.
|
right_on : label or list, or array-like
|
Field names to join on in right DataFrame or vector/list of vectors per
|
left_on docs.
|
left_by : column name or list of column names
|
Group left DataFrame by group columns and merge piece by piece with
|
right DataFrame. Must be None if either left or right are a Series.
|
right_by : column name or list of column names
|
Group right DataFrame by group columns and merge piece by piece with
|
left DataFrame. Must be None if either left or right are a Series.
|
fill_method : {'ffill', None}, default None
|
Interpolation method for data.
|
suffixes : list-like, default is ("_x", "_y")
|
A length-2 sequence where each element is optionally a string
|
indicating the suffix to add to overlapping column names in
|
`left` and `right` respectively. Pass a value of `None` instead
|
of a string to indicate that the column name from `left` or
|
`right` should be left as-is, with no suffix. At least one of the
|
values must not be None.
|
|
how : {'left', 'right', 'outer', 'inner'}, default 'outer'
|
* left: use only keys from left frame (SQL: left outer join)
|
* right: use only keys from right frame (SQL: right outer join)
|
* outer: use union of keys from both frames (SQL: full outer join)
|
* inner: use intersection of keys from both frames (SQL: inner join).
|
|
Returns
|
-------
|
DataFrame
|
The merged DataFrame output type will be the same as
|
'left', if it is a subclass of DataFrame.
|
|
See Also
|
--------
|
merge : Merge with a database-style join.
|
merge_asof : Merge on nearest keys.
|
|
Examples
|
--------
|
>>> from pandas import merge_ordered
|
>>> df1 = pd.DataFrame(
|
... {
|
... "key": ["a", "c", "e", "a", "c", "e"],
|
... "lvalue": [1, 2, 3, 1, 2, 3],
|
... "group": ["a", "a", "a", "b", "b", "b"]
|
... }
|
... )
|
>>> df1
|
key lvalue group
|
0 a 1 a
|
1 c 2 a
|
2 e 3 a
|
3 a 1 b
|
4 c 2 b
|
5 e 3 b
|
|
>>> df2 = pd.DataFrame({"key": ["b", "c", "d"], "rvalue": [1, 2, 3]})
|
>>> df2
|
key rvalue
|
0 b 1
|
1 c 2
|
2 d 3
|
|
>>> merge_ordered(df1, df2, fill_method="ffill", left_by="group")
|
key lvalue group rvalue
|
0 a 1 a NaN
|
1 b 1 a 1.0
|
2 c 2 a 2.0
|
3 d 2 a 3.0
|
4 e 3 a 3.0
|
5 a 1 b NaN
|
6 b 1 b 1.0
|
7 c 2 b 2.0
|
8 d 2 b 3.0
|
9 e 3 b 3.0
|
rD ©r^ c
|
s t | | d}| ¡ S )N)rT rU rV rZ r rS )Ú _OrderedMergera )ÚxÚyrb )r rS rU rT rV rZ rc rd Ú_merger@ s ø
|
z merge_ordered.<locals>._mergerNz*Can only group either left or right framesr z not found in left columnsc s
|
| |S rh rc ©r r ©r rc rd Ú<lambda>V ó z merge_ordered.<locals>.<lambda>z not found in right columnsc s
|
|| S rh rc r r rc rd r ^ r ) Ú
|
ValueErrorrv Ústrrp Ú
|
differencerj r r| r ) rQ rR rT rU rV Úleft_byÚright_byr rZ rS Úcheckr Ú_rc )r r rS rU rT rV rZ rd Ú merge_orderedØ s0 h
|
|
|
ÿ
|
|
r TÚbackwardr ) rQ rR rT rU rV rW rX rZ Úallow_exact_matchesÚ directionr^ c C s. t | ||||||||| |
|
d| | | d}| ¡ S )a'#
|
Perform a merge by key distance.
|
|
This is similar to a left-join except that we match on nearest
|
key rather than equal keys. Both DataFrames must be sorted by the key.
|
|
For each row in the left DataFrame:
|
|
- A "backward" search selects the last row in the right DataFrame whose
|
'on' key is less than or equal to the left's key.
|
|
- A "forward" search selects the first row in the right DataFrame whose
|
'on' key is greater than or equal to the left's key.
|
|
- A "nearest" search selects the row in the right DataFrame whose 'on'
|
key is closest in absolute distance to the left's key.
|
|
The default is "backward" and is compatible in versions below 0.20.0.
|
The direction parameter was added in version 0.20.0 and introduces
|
"forward" and "nearest".
|
|
Optionally match on equivalent keys with 'by' before searching with 'on'.
|
|
Parameters
|
----------
|
left : DataFrame or named Series
|
right : DataFrame or named Series
|
on : label
|
Field name to join on. Must be found in both DataFrames.
|
The data MUST be ordered. Furthermore this must be a numeric column,
|
such as datetimelike, integer, or float. On or left_on/right_on
|
must be given.
|
left_on : label
|
Field name to join on in left DataFrame.
|
right_on : label
|
Field name to join on in right DataFrame.
|
left_index : bool
|
Use the index of the left DataFrame as the join key.
|
right_index : bool
|
Use the index of the right DataFrame as the join key.
|
by : column name or list of column names
|
Match on these columns before performing merge operation.
|
left_by : column name
|
Field names to match on in the left DataFrame.
|
right_by : column name
|
Field names to match on in the right DataFrame.
|
suffixes : 2-length sequence (tuple, list, ...)
|
Suffix to apply to overlapping column names in the left and right
|
side, respectively.
|
tolerance : int or Timedelta, optional, default None
|
Select asof tolerance within this range; must be compatible
|
with the merge index.
|
allow_exact_matches : bool, default True
|
|
- If True, allow matching with the same 'on' value
|
(i.e. less-than-or-equal-to / greater-than-or-equal-to)
|
- If False, don't match the same 'on' value
|
(i.e., strictly less-than / strictly greater-than).
|
|
direction : 'backward' (default), 'forward', or 'nearest'
|
Whether to search for prior, subsequent, or closest matches.
|
|
Returns
|
-------
|
DataFrame
|
|
See Also
|
--------
|
merge : Merge with a database-style join.
|
merge_ordered : Merge with optional filling/interpolation.
|
|
Examples
|
--------
|
>>> left = pd.DataFrame({"a": [1, 5, 10], "left_val": ["a", "b", "c"]})
|
>>> left
|
a left_val
|
0 1 a
|
1 5 b
|
2 10 c
|
|
>>> right = pd.DataFrame({"a": [1, 2, 3, 6, 7], "right_val": [1, 2, 3, 6, 7]})
|
>>> right
|
a right_val
|
0 1 1
|
1 2 2
|
2 3 3
|
3 6 6
|
4 7 7
|
|
>>> pd.merge_asof(left, right, on="a")
|
a left_val right_val
|
0 1 a 1
|
1 5 b 3
|
2 10 c 7
|
|
>>> pd.merge_asof(left, right, on="a", allow_exact_matches=False)
|
a left_val right_val
|
0 1 a NaN
|
1 5 b 3.0
|
2 10 c 7.0
|
|
>>> pd.merge_asof(left, right, on="a", direction="forward")
|
a left_val right_val
|
0 1 a 1.0
|
1 5 b 6.0
|
2 10 c NaN
|
|
>>> pd.merge_asof(left, right, on="a", direction="nearest")
|
a left_val right_val
|
0 1 a 1
|
1 5 b 6
|
2 10 c 7
|
|
We can use indexed DataFrames as well.
|
|
>>> left = pd.DataFrame({"left_val": ["a", "b", "c"]}, index=[1, 5, 10])
|
>>> left
|
left_val
|
1 a
|
5 b
|
10 c
|
|
>>> right = pd.DataFrame({"right_val": [1, 2, 3, 6, 7]}, index=[1, 2, 3, 6, 7])
|
>>> right
|
right_val
|
1 1
|
2 2
|
3 3
|
6 6
|
7 7
|
|
>>> pd.merge_asof(left, right, left_index=True, right_index=True)
|
left_val right_val
|
1 a 1
|
5 b 3
|
10 c 7
|
|
Here is a real-world times-series example
|
|
>>> quotes = pd.DataFrame(
|
... {
|
... "time": [
|
... pd.Timestamp("2016-05-25 13:30:00.023"),
|
... pd.Timestamp("2016-05-25 13:30:00.023"),
|
... pd.Timestamp("2016-05-25 13:30:00.030"),
|
... pd.Timestamp("2016-05-25 13:30:00.041"),
|
... pd.Timestamp("2016-05-25 13:30:00.048"),
|
... pd.Timestamp("2016-05-25 13:30:00.049"),
|
... pd.Timestamp("2016-05-25 13:30:00.072"),
|
... pd.Timestamp("2016-05-25 13:30:00.075")
|
... ],
|
... "ticker": [
|
... "GOOG",
|
... "MSFT",
|
... "MSFT",
|
... "MSFT",
|
... "GOOG",
|
... "AAPL",
|
... "GOOG",
|
... "MSFT"
|
... ],
|
... "bid": [720.50, 51.95, 51.97, 51.99, 720.50, 97.99, 720.50, 52.01],
|
... "ask": [720.93, 51.96, 51.98, 52.00, 720.93, 98.01, 720.88, 52.03]
|
... }
|
... )
|
>>> quotes
|
time ticker bid ask
|
0 2016-05-25 13:30:00.023 GOOG 720.50 720.93
|
1 2016-05-25 13:30:00.023 MSFT 51.95 51.96
|
2 2016-05-25 13:30:00.030 MSFT 51.97 51.98
|
3 2016-05-25 13:30:00.041 MSFT 51.99 52.00
|
4 2016-05-25 13:30:00.048 GOOG 720.50 720.93
|
5 2016-05-25 13:30:00.049 AAPL 97.99 98.01
|
6 2016-05-25 13:30:00.072 GOOG 720.50 720.88
|
7 2016-05-25 13:30:00.075 MSFT 52.01 52.03
|
|
>>> trades = pd.DataFrame(
|
... {
|
... "time": [
|
... pd.Timestamp("2016-05-25 13:30:00.023"),
|
... pd.Timestamp("2016-05-25 13:30:00.038"),
|
... pd.Timestamp("2016-05-25 13:30:00.048"),
|
... pd.Timestamp("2016-05-25 13:30:00.048"),
|
... pd.Timestamp("2016-05-25 13:30:00.048")
|
... ],
|
... "ticker": ["MSFT", "MSFT", "GOOG", "GOOG", "AAPL"],
|
... "price": [51.95, 51.95, 720.77, 720.92, 98.0],
|
... "quantity": [75, 155, 100, 100, 100]
|
... }
|
... )
|
>>> trades
|
time ticker price quantity
|
0 2016-05-25 13:30:00.023 MSFT 51.95 75
|
1 2016-05-25 13:30:00.038 MSFT 51.95 155
|
2 2016-05-25 13:30:00.048 GOOG 720.77 100
|
3 2016-05-25 13:30:00.048 GOOG 720.92 100
|
4 2016-05-25 13:30:00.048 AAPL 98.00 100
|
|
By default we are taking the asof of the quotes
|
|
>>> pd.merge_asof(trades, quotes, on="time", by="ticker")
|
time ticker price quantity bid ask
|
0 2016-05-25 13:30:00.023 MSFT 51.95 75 51.95 51.96
|
1 2016-05-25 13:30:00.038 MSFT 51.95 155 51.97 51.98
|
2 2016-05-25 13:30:00.048 GOOG 720.77 100 720.50 720.93
|
3 2016-05-25 13:30:00.048 GOOG 720.92 100 720.50 720.93
|
4 2016-05-25 13:30:00.048 AAPL 98.00 100 NaN NaN
|
|
We only asof within 2ms between the quote time and the trade time
|
|
>>> pd.merge_asof(
|
... trades, quotes, on="time", by="ticker", tolerance=pd.Timedelta("2ms")
|
... )
|
time ticker price quantity bid ask
|
0 2016-05-25 13:30:00.023 MSFT 51.95 75 51.95 51.96
|
1 2016-05-25 13:30:00.038 MSFT 51.95 155 NaN NaN
|
2 2016-05-25 13:30:00.048 GOOG 720.77 100 720.50 720.93
|
3 2016-05-25 13:30:00.048 GOOG 720.92 100 720.50 720.93
|
4 2016-05-25 13:30:00.048 AAPL 98.00 100 NaN NaN
|
|
We only asof within 10ms between the quote time and the trade time
|
and we exclude exact matches on time. However *prior* data will
|
propagate forward
|
|
>>> pd.merge_asof(
|
... trades,
|
... quotes,
|
... on="time",
|
... by="ticker",
|
... tolerance=pd.Timedelta("10ms"),
|
... allow_exact_matches=False
|
... )
|
time ticker price quantity bid ask
|
0 2016-05-25 13:30:00.023 MSFT 51.95 75 NaN NaN
|
1 2016-05-25 13:30:00.038 MSFT 51.95 155 51.97 51.98
|
2 2016-05-25 13:30:00.048 GOOG 720.77 100 NaN NaN
|
3 2016-05-25 13:30:00.048 GOOG 720.92 100 NaN NaN
|
4 2016-05-25 13:30:00.048 AAPL 98.00 100 NaN NaN
|
Úasof) rT rU rV rW rX r r r rZ rS Ú tolerancer r )Ú
|
_AsOfMergera )rQ rR rT rU rV rW rX r r r rZ r¢ r r rb rc rc rd Ú
|
merge_asofe s( ñr¤ c @ sÌ e Zd ZU dZdZded< ded< ded< ded < d
|
ed < d
|
ed < d ed< d ed< d
|
ed< ded< d
|
ed< ded< ded< ded< ded< ded < d_d#d#ddddd d
|
d
|
d
|
dddd$d%d&d'Zd(d)d)d*d+d,d-d.Zd`d*d+d/d0d1Zd+dd$d2d3d4Z e
|
dd5d6d7Z d+d+d8d9d:d;Z d+d+d<d=d>Z d+d$d<d?d@Zd+dAdAd$dBdCdDZdEd5dFdGZdHd5dIdJZdad(d(dLdMd(dNdOdPZdQd5dRdSZd$d5dTdUZd+d+dVd9dWdXZdYdZ Zd[d$d\d]d^Zd S )br` z
|
Perform a database (SQL) merge operation between two DataFrame or Series
|
objects using either columns as keys or their row indexes
|
re úMergeHow | Literal['asof']rS rL rT z!Sequence[Hashable | AnyArrayLike]rU rV rM rW rX r ro Úbm_axisrY r rZ r[ rO r\ rP r] zlist[Hashable]Ú
|
join_nameszlist[AnyArrayLike]Úright_join_keysÚleft_join_keysrH Né FTrI rK ÚNone)rQ rR rS rT rU rV ro rW rX rY rZ r\ r] r^ c C sh t |}t |}| | _| _| | _| _|| _|| _| jjdkrHd| nd| _t
|
|¡| _ | | _ |
|
| _ || _| | _| | _t|stdt| t| s¬tdt| |jj|jjkrÞd|jj d|jj d}t|| ||¡\| _| _d }| jd kr.| | j| j¡\| _| _| _}|g | _| _|| _| ¡ \| _ | _ | _ | ¡ | d k rd| !| ¡ d S )
|
Né rª r z/left_index parameter must be of type bool, not z0right_index parameter must be of type bool, not z0Not allowed to merge between different levels. (z levels on the left, z on the right)Úcross)"Ú_validate_operandrQ Ú orig_leftrR Ú
|
orig_rightrS r¦ Úndimro ÚcomÚmaybe_make_listrT rZ rY rW rX r\ r# r Útyperj Únlevelsr Ú_validate_left_right_onrU rV Ú_create_cross_configurationÚ_crossÚ_get_merge_keysr© r¨ r§ Ú_maybe_coerce_merge_keysÚ _validate)ÚselfrQ rR rS rT rU rV ro rW rX rY rZ r\ r] Ú_leftZ_rightÚmsgÚ cross_colrc rc rd Ú__init__ sX ÿ ÿÿ ûü
|
z_MergeOperation.__init__r7 znpt.NDArray[np.intp] | NonerN rD )Ú
|
join_indexÚ left_indexerÚ right_indexerr[ r^ c C sä | j dd
}| jdd
}t| j j| jj| j\}}|dk rpt|t|sp|jj||dddddd} | | ¡}||_
|
|dk r°t|t|s°|jj||dddddd}
|
| |
|
¡}||_
|
ddl m } ||_ ||_ | ||gd|d} | S ) z?
|
reindex along index and concat along columns.
|
Nrª FT)ro r[ Z
|
only_sliceZ
|
allow_dupsZ use_na_proxyr rt )ro r[ )rQ rR Ú_items_overlap_with_suffixÚ
|
_info_axisrZ r r Z_mgrZreindex_indexerZ _constructorr Úpandasru rj ) r¼ rÁ r rà r[ rQ rR ÚllabelsÚrlabelsZlmgrZrmgrru r rc rc rd Ú_reindex_and_concaté sN ÿù
|
|
ÿù
|
z#_MergeOperation._reindex_and_concat©r[ r^ c C s | j r | | j| j¡\| _| _| ¡ \}}}| j||||d}|j| | jd}| j r^| |¡}| |||¡ |
|
|¡ | || j ¡ |j| ddS )Nr_ )Úmethodre ) r\ Ú_indicator_pre_mergerQ rR Ú_get_join_inforÉ Z __finalize__Ú _merge_typeÚ_indicator_post_mergeÚ_maybe_add_join_keysÚ_maybe_restore_index_levelsÚ_maybe_drop_cross_columnr¸ )r¼ r[ rÁ r rà r rc rc rd ra ! s ÿ
|
|
z_MergeOperation.get_result)r r¿ r^ c C s |d k r||= d S rh rc )r¼ r r¿ rc rc rd rÒ 7 s z(_MergeOperation._maybe_drop_cross_columnr c C s8 t | jtr| jS t | jtr,| jr(dS d S tdd S )NZ_mergez<indicator option can only accept boolean or string arguments)rv r\ r rM r ©r¼ rc rc rd Ú_indicator_name= s ÿz _MergeOperation._indicator_nameztuple[DataFrame, DataFrame])rQ rR r^ c C s |j |j ¡}dD ]}||krtd| q| j|kr@td| ¡ }| ¡ }d|d< |d d¡|d< d|d< |d d¡|d< ||fS ) N)Ú_left_indicatorÚ_right_indicatorzECannot use `indicator=True` option when data contains a column named z:Cannot use name of an existing column for indicator columnrª rÕ Úint8r¬ rÖ )rj Úunionr rÔ r[ Úastype)r¼ rQ rR rj Úirc rc rd rÌ H s" ÿ
|
ÿz$_MergeOperation._indicator_pre_merge)r r^ c C s| |d d¡|d< |d d¡|d< t|d |d dddgd|| j< || j j dd d
|
g¡|| j< |jddgdd }|S ) NrÕ r rÖ rª r¬ é )Ú
|
categoriesZ left_onlyZ
|
right_onlyZboth)Úlabelsro )Zfillnar6 rÔ ÚcatZrename_categoriesZdrop)r¼ r rc rc rd rÏ c s þ ÿþ
|
z%_MergeOperation._indicator_post_mergec C sn g }t | j| j| jD ]@\}}}| j |¡r| j |¡r||kr||jjkr| |¡ q|rj|j
|
|dd dS )a
|
Restore index levels specified as `on` parameters
|
|
Here we check for cases where `self.left_on` and `self.right_on` pairs
|
each reference an index level in their respective DataFrames. The
|
joined columns corresponding to these pairs are then restored to the
|
index of `result`.
|
|
**Note:** This method has side effects. It modifies `result` in-place
|
|
Parameters
|
----------
|
result: DataFrame
|
merge result
|
|
Returns
|
-------
|
None
|
T©ZinplaceN) Úzipr§ rU rV r¯ Ú_is_level_referencer° r Únamesr Ú set_index)r¼ r Znames_to_restoreÚnameÚleft_keyÚ right_keyrc rc rd rÑ r s& ÿ
|
üÿø õ
|
ô z+_MergeOperation._maybe_restore_index_levelsznp.ndarray | None)r r rà r^ c sÖ d }d }t dd | jD s tt| j| j| j}t|D ]\}\}} t|| sVq:d\}
|
} kr&|d k rD|d k rD| j krÎ|d kr|dk
|
¡ }|rÌ| j | } t j | j j sÌ| j j}
|
nV| jkrD|d krî|dk
|
¡ }|rD| j| }
|
t j | j j sD| j j} n |d k rD| j| }
|
| j | } |
|
d k sV| d k r:|
|
d krl j} n&t|
|
dd}
|
t|
|
j } tj|
|
|| d} | d kr¨ j}n&t| dd}t|j }tj|||d}|dk}| ¡ rðt| |j }nJ|d k r|dk ¡ rt| | j }n"t| | |¡ t| j |j g} ¡r\t |jd< q: ¡r¶tjt r¦ _ fd d
|
jj D }j |dd nt d _q: |pÊd | ¡ q:d S )Nc s s | ]}t |V qd S rh )r" ©rk r rc rc rd rn ¦ s z7_MergeOperation._maybe_add_join_keys.<locals>.<genexpr>)NNéÿÿÿÿT©Ú extract_numpy©Ú
|
fill_value)Údtyper c s$ g | ] }|kr j |¡n qS rc )r Úget_level_values)rk Z
|
level_name©Zkey_colrä r rc rd rs ó s þÿz8_MergeOperation._maybe_add_join_keys.<locals>.<listcomp>rß )rä Úkey_)!ry r© ÚAssertionErrorrà r§ rU rV Ú enumerateÚ _should_fillrQ Úanyr¨ r& rí Ú_valuesrR r? r4 ÚalgosÚtake_ndr7 Úwherer Z_is_label_referencer9 r rá rv r8 rä râ rã Úinsert)r¼ r r rà Zleft_has_missingZright_has_missingÚkeysrÚ ÚlnameÚrnameZ take_leftZ
|
take_rightZlvalsZlfillZrvalsZtakerZrfillZ mask_leftZ result_dtypeZidx_listrc rï rd rÐ s
|
|
|
|
|
ÿ
|
|
ÿ
|
|
|
|
|
|
|
|
ÿ üz$_MergeOperation._maybe_add_join_keysú1tuple[npt.NDArray[np.intp], npt.NDArray[np.intp]]c C s t | j| j| j| jdS )úreturn the join indexers©rY rS )Úget_join_indexersr© r¨ rY rS rÓ rc rc rd Ú_get_join_indexers s ÿz"_MergeOperation._get_join_indexerszFtuple[Index, npt.NDArray[np.intp] | None, npt.NDArray[np.intp] | None]c C s² | j dkst| jj| j }| jj| j }| jr`| jr`| j dkr`|j|| j d| j d\}}}n | jr| j dkrt
|
||| j | j d\}}}nò| jr¸| j dkr¸t
|
||| j | j d\}}}nÆ| ¡ \}}| jrt| jdkrö| j| jj| jj|dd }n| jj |¡}nx| jrr| j dkr6| j| jj| jj|dd }n:t| jdkrb| j| jj| jj|dd }n| jj |¡}n tt|}t|dkr¨t|ts¨td |j¡}|||fS )
|
Nr r¡ T)rS Zreturn_indexersrY rQ rg rR r ©rS )rS rñ rQ Zaxesro rR rW rX r rY Ú_left_join_on_indexr© r¨ r r Ú_create_join_indexr rz rA rv r8 Z set_namesrä )r¼ Úleft_axÚright_axrÁ r rà rc rc rd rÍ sl ÿ ÿ ÿ ü üü z _MergeOperation._get_join_inforQ únpt.NDArray[np.intp]r )r Ú other_indexÚindexerrS r^ c C sR | j |dfkrHt|tsH|dk}t |¡rHt|jdd}| t|g¡}| |¡S )a«
|
Create a join index by rearranging one index to match another
|
|
Parameters
|
----------
|
index : Index being rearranged
|
other_index : Index used to supply values not found in index
|
indexer : np.ndarray[np.intp] how to rearrange index
|
how : str
|
Replacement is only necessary if indexer based on other_index.
|
|
Returns
|
-------
|
Index
|
r rè F)Úcompat)
|
rS rv r8 Únprô r4 rí r r7 rz )r¼ r r r rS Úmaskrì rc rc rd r C s
|
z"_MergeOperation._create_join_indexú=tuple[list[AnyArrayLike], list[AnyArrayLike], list[Hashable]]c s\ g }g }g }g }g }| j | j fdd}fdd}t| jrÜt| jrÜt| j| jD ]t\}} ||røtt|}| |¡ || r®tt| } | | ¡ | d¡ nHtt | } | dk rÜ|
|
| ¡¡ | | ¡ n| j ¡ | j j ¡ q`|| sxtt | } | dk r(|
|
| ¡¡ n | j ¡ |dk r|| krtt |}t dkrl| | ¡ n
|
| |¡ ntt| } | | ¡ |dk r¼tt |}|
|
|¡¡ | |¡ q`| j ¡ | j j ¡ q`nNt| jr| jD ]T}
|
||
|
r tt|
|
}
|
| |
|
¡ | d¡ n$tt |
|
}
|
|
|
|
|
¡¡ | |
|
¡ qît| jj trvdd t| jj j| jj jD }n | jj jg}n¦t| jr*| jD ]T}
|
||
|
rÄtt|
|
}
|
| |
|
¡ | d¡ n$tt |
|
}
|
|
|
|
|
¡¡ | |
|
¡ qt| j j tr dd t| j j j| j j jD }n | j j jg}|r>| j |¡| _ |rR| j |¡| _|||fS ) zÚ
|
Note: has side effects (copy/delete key columns)
|
|
Parameters
|
----------
|
left
|
right
|
on
|
|
Returns
|
-------
|
left_keys, right_keys, join_names
|
c s t | ot| t kS rh ©r" r ©r )rQ rc rd r r z1_MergeOperation._get_merge_keys.<locals>.<lambda>c s t | ot| t kS rh r r rm rc rd r r Nr c S s g | ]\}}|j |¡qS rc ©rõ rz ©rk ÚlevZ lev_codesrc rc rd rs Ì s ÿz3_MergeOperation._get_merge_keys.<locals>.<listcomp>c S s g | ]\}}|j |¡qS rc r r rc rc rd rs á s ÿ)rQ rR Ú_anyrU rV rà r r r r Ú_get_label_or_level_valuesr rä r rv r8 ÚlevelsÚcodesrõ Z_drop_labels_or_levels) r¼ Ú left_keysÚ
|
right_keysr§ Z
|
right_dropZ left_dropZis_lkeyZis_rkeyÚlkÚrkÚkrc rf rd r¹ e s
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
ÿþ
|
|
|
|
|
ÿþ z _MergeOperation._get_merge_keysc
|
C s
|
t | j| j| jD ]ð\}}}t|r.t|rt|s@t|r@qt|dd}t|dd}t|j}t|j}t|j}t|j}|rª|rªt t
|
|}t t
|
|}| |¡rÄqn|sÄ|r´nt |j|jrÄqd|j d|j d}t |jrt |jr|jj|jjkrqt|jrxt|jrxtjddF | |j¡} t |¡ }
|
|| k} | |
|
¡ sltjdtt d W 5 Q R X qt|jrêt|jrêtjddF | |j¡} t |¡ }
|
|| k} | |
|
¡ sÞtjdtt d W 5 Q R X qtj|d
|
d tj|d
|
d krvqnh|r t |jsvt |jr6|r6n@|rHt |jsZt |jrØ|rØtj|d
|
d } tj|d
|
d } d d ddg}dddddg}| |kr¦| |kr¦n0| |krº| |ksÎ| |krv| |krvt |nt |jrút |jsút |n|t |js t |jr t |nZt |jt rBt |jt sBt |n4t |jt sht |jt rht |n|rv|rvq|| j!j"kr¾|rt t
|
|j#jnt$}| j! %¡ | _!| j!| |¡| j!|< || j&j"kr|rÞt t
|
|j#jnt$}| j& %¡ | _&| j&| |¡| j&|< qd S )NTré zYou are trying to merge on ú and z9 columns. If you wish to proceed you should use pd.concatÚignore)ÚinvalidzjYou are merging on int and float columns where the float values are not equal to their int representation.)Ú
|
stacklevelF)ZskipnaÚintegerz mixed-integerÚbooleanÚemptyÚstringÚunicodeÚmixedÚbytes)'rà r© r¨ r§ r r? r% rí r. r r6 Z#_categories_match_up_to_permutationr& r- Úkindr* r( r ÚerrstaterÙ Úisnanry ÚwarningsÚwarnÚ UserWarningr r Z infer_dtyper$ r r/ rv r0 rQ rj rÜ Úobjectr[ rR )r¼ r r rä Z lk_is_catZ rk_is_catZ lk_is_objectZ rk_is_objectr¾ Zcastedr ÚmatchZ inferred_leftZinferred_rightZ
|
bool_typesÚ string_typesÚtyprc rc rd rº ò sÔ ÿ
|
|
|
|
|
|
|
ÿ û û ÿ
|
ÿÿÿÿ ÿÿýý
|
|
|
ÿ
|
ÿ
|
z(_MergeOperation._maybe_coerce_merge_keysz)tuple[DataFrame, DataFrame, JoinHow, str]c C s6 dt ¡ }d}|jf |di|jf |di||fS )a
|
Creates the configuration to dispatch the cross operation to inner join,
|
e.g. adding a join column and resetting parameters. Join column is added
|
to a new object, no inplace modification
|
|
Parameters
|
----------
|
left : DataFrame
|
right : DataFrame
|
|
Returns
|
-------
|
a tuple (left, right, how, cross_col) representing the adjusted
|
DataFrames with cross_col, the merge operation set to inner and the column
|
to join over.
|
Z_cross_rH rª )ÚuuidÚuuid4Zassign)r¼ rQ rR r¿ rS rc rc rd r· s üz+_MergeOperation._create_cross_configurationc C s t |¡}t |¡}| jdkrP| jsD| jsD|d k sD|d k sD| jd k rLtdn| jd kr2|d kr2|d kr2| jr| jrd\}}n¨| jrtdn| jr¦tdn| jj}| j j}|
|
|¡}t |dkrðtd| d| d | j d
|
| j |j |d d j r|j |d d j s&td t| | }}n"| jd k r~|d k sR|d k rZtd| jsj| jrrtd| j }}nÖ|d k rê| jrtd| js²|d kr²tdt |}| jrTt || j jjkrÞtdd g| }nj|d k rT| jrtd| js |d kr tdt |}| jrTt || jjjkrJtdd g| }| jdkrzt |t |krztd||fS )Nr zMCan not pass on, right_on, left_on or set right_index=True or left_index=True)rc rc z&Must pass right_on or right_index=Truez$Must pass left_on or left_index=Truer z>No common columns to perform merge on. Merge options: left_on=z , right_on=z , left_index=z, right_index=rH r zData columns not unique: zSCan only pass argument "on" OR "left_on" and "right_on", not a combination of both.zYCan only pass argument "on" OR "left_index" and "right_index", not a combination of both.z:Can only pass argument "left_on" OR "left_index" not both.z&Must pass "right_on" OR "right_index".zDlen(left_on) must equal the number of levels in the index of "right"z<Can only pass argument "right_on" OR "right_index" not both.z$Must pass "left_on" OR "left_index".zDlen(right_on) must equal the number of levels in the index of "left"z%len(right_on) must equal len(left_on))r² r³ rS rW rX rT r rQ rj rR Ú intersectionr r Ú is_uniqueÚreprr rµ r )r¼ rU rV Z left_colsZ
|
right_colsZ common_colsÚnrc rc rd r¶ ¬ s
|
|
|
ÿþýüûÿ
|
|
|
|
ÿÿþ ÿÿ
|
ÿÿ
|
ÿÿ
|
z'_MergeOperation._validate_left_right_onr )r] r^ c C s¼ | j r| jjj}nt | j¡j}| jr2| jjj}nt | j ¡j}|dkrr|sX|sXt
|
d|sdt
|
d|s¸t
|
dnF|dkr|s¸t
|
dn0|dkr|s¸t
|
dn|d kr¨nt d
|
| d d S ) N)Z
|
one_to_onez1:1zQMerge keys are not unique in either left or right dataset; not a one-to-one mergezAMerge keys are not unique in left dataset; not a one-to-one mergezBMerge keys are not unique in right dataset; not a one-to-one merge)Z one_to_manyz1:mzBMerge keys are not unique in left dataset; not a one-to-many merge)Z many_to_onezm:1zCMerge keys are not unique in right dataset; not a many-to-one merge)Z many_to_manyzm:mú"z" is not a valid argument. Valid arguments are:
|
- "1:1"
|
- "1:m"
|
- "m:1"
|
- "m:m"
|
- "one_to_one"
|
- "one_to_many"
|
- "many_to_one"
|
- "many_to_many") rW r¯ r r4 r8 Ú from_arraysr© rX r° r¨ r r )r¼ r] Z left_uniqueZ right_uniquerc rc rd r» sD ÿÿÿÿÿ
|
ÿz_MergeOperation._validate) rH NNNrª FFTrI FN)T)rQ )Ú__name__Ú
|
__module__Ú __qualname__Ú__doc__rÎ Ú__annotations__rÀ rÉ ra rÒ r rÔ rÌ rÏ rÑ rÐ r rÍ r r¹ rº r· r¶ r» rc rc rc rd r` z sd
|
ò*U8
|
+cB û" Zr` r¥ rý )rY rS r^ c s: t t kstdt d }t d }|dkr^ dkrFt S s dkrt|dS n,|dkr dkrtt S s dkrt|dS fd d
|
tt D }t| }d d
|
|D \} }
|
} t| |
|
t| \} } t| | d \} } }t
|
|¡} d kr|d< t j t j dd t jd }|| | |f|S )ac
|
|
Parameters
|
----------
|
left_keys : ndarray, Index, Series
|
right_keys : ndarray, Index, Series
|
sort : bool, default False
|
how : {'inner', 'outer', 'left', 'right'}, default 'inner'
|
|
Returns
|
-------
|
np.ndarray[np.intp]
|
Indexer into the left_keys.
|
np.ndarray[np.intp]
|
Indexer into the right_keys.
|
z/left_key and right_keys must be the same lengthr )rQ rH r )rR r T)rR rH r )rQ r Fc 3 s& | ] }t | | d V qdS )rÿ N)Ú_factorize_keys©rk r6 ©rS r r rY rc rd rn k s ÿz$get_join_indexers.<locals>.<genexpr>c s s | ]}t |V qd S rh ©rw rç rc rc rd rn p s rÿ rf rY c [ s t j|| |f|d d d
S )Nrè )ÚlibjoinÚleft_outer_join)r r ÚcountÚkwargsrc rc rd r s ÿÿþz#get_join_indexers.<locals>.<lambda>)rH rQ rR r )r rñ Ú_get_empty_indexerÚ _get_no_sort_one_missing_indexerr~ rà Ú_get_join_keysrx r> Úcpr[ rB Z
|
inner_joinrC Zfull_outer_join)r r rY rS rE Zleft_nZright_nÚmappedÚzippedÚllabÚrlabÚshapeÚlkeyÚrkeyrD Z join_funcrc r@ rd r A sF ÿþ
|
|
þ
|
|
úù
|
r r8 r7 r z8tuple[list[Index], npt.NDArray[np.intp], list[Hashable]])rQ rR rÁ ÚlindexerÚrindexerr^ c C s¶ ddddd}||}|j }|j}|j} |D ]|}
|
|
|
| jkrF| } |} n|} |} | j |
|
¡} | j | }| j| }| dkr||}ntj|| dd}||g }||g }| |
|
g } q.||| fS ) aø
|
*this is an internal non-public method*
|
|
Returns the levels, labels and names of a multi-index to multi-index join.
|
Depending on the type of join, this method restores the appropriate
|
dropped levels of the joined multi-index.
|
The method relies on lindexer, rindexer which hold the index positions of
|
left and right, where a join was feasible
|
|
Parameters
|
----------
|
left : MultiIndex
|
left index
|
right : MultiIndex
|
right index
|
dropped_level_names : str array
|
list of non-common level names
|
join_index : Index
|
the index of the join between the
|
common levels of left and right
|
lindexer : np.ndarray[np.intp]
|
left indexer
|
rindexer : np.ndarray[np.intp]
|
right indexer
|
|
Returns
|
-------
|
levels : list of Index
|
levels of combined multiindexes
|
labels : np.ndarray[np.intp]
|
labels of combined multiindexes
|
names : List[Hashable]
|
names of combined multiindex levels
|
|
r7 r8 )r r^ c S s( t | tr| S tj| jg| jgdS d S )N)râ )rv r8 r8 rõ rä )r rc rc rd Ú_convert_to_multiindex· s
|
z@restore_dropped_levels_multijoin.<locals>._convert_to_multiindexNrè rë )r r râ r rö r÷ )rQ rR Zdropped_level_namesrÁ rQ rR rS Z join_levelsZ
|
join_codesr§ Zdropped_level_nameÚidxr Zname_idxZrestore_levelsr Z restore_codesrc rc rd Ú restore_dropped_levels_multijoin s* ,
|
|
|
|
|
|
rU c @ sH e Zd ZdZddddddd d d
|
d d d dd ddZddddddZdS )r Z ordered_mergeNFrª rI r rK rL rM r r rP zJoinHow | Literal['asof']r« ) rQ rR rT rU rV rW rX ro rZ r rS r^ c C s, |
|
| _ tj| ||||||||| | dd d S )NT) rT rU rW rX rV ro rS rZ rY )r r` rÀ ) r¼ rQ rR rT rU rV rW rX ro rZ r rS rc rc rd rÀ ç s ôz_OrderedMerge.__init__TrN rD rÊ c
|
C s | ¡ \}}}t| jj| jj| j\}}| jdkrr|d krBtdtt j
|
|tt j
|
| }}t |¡}t |¡}n|}|}| j ||||d} | | ||¡ | S )NZffillzleft_indexer cannot be Noner_ )rÍ rÄ rQ rÅ rR rZ r Ú TypeErrorr r ZndarrayrB Z ffill_indexerrÉ rÐ )
|
r¼ r[ rÁ r rà rÇ rÈ Zleft_join_indexerZright_join_indexerr rc rc rd ra s2 ÿ
|
ÿ
|
ÿz_OrderedMerge.get_result) NNNFFrª rI Nr )T)r9 r: r; rÎ rÀ ra rc rc rc rd r ä s ô& r )r c C s d| d}t t|d S )NZ
|
asof_join_Z
|
_on_X_by_Y)ÚgetattrrB )r rä rc rc rd Ú_asof_by_function# s rX )Úint64_tÚdoubler- r )rí r^ c C s t | r dS t| rdS dS dS )z2Upcast a dtype to 'int64_t', 'double', or 'object'rY rZ r- N)r* r( ©rí rc rc rd Ú_get_cython_type_upcast/ s
|
r\ c sl e Zd ZdZd d d d
|
d
|
d
|
d d d d d ddd dddddZ fddZdd fdd Zddd d Z ZS ) r£ Z
|
asof_mergeNFrª rI Tr¡ r rK rL rM r r rP zLiteral['asof']r r« )rQ rR rT rU rV rW rX ro rZ r[ r rS r r r^ c C sJ || _ | | _|
|
| _|| _|| _|| _tj| |||||||| || |d d S )N) rT rU rV rW rX ro rS rZ r )r r r r¢ r r r rÀ )r¼ rQ rR rT rU rV rW rX r r r ro rZ r[ r rS r¢ r r rc rc rd rÀ < s( ôz_AsOfMerge.__init__c sb t ||¡\}}t|dkr,| js,tdt|dkrF| jsFtd| jrbt| jjt rbtd| jr~t| j
|
jt r~td| j d k r²| j d k s| j d k r¤td| j | _ | _ | j d krÎ| j d k rÎtd| j d k rê| j d krêtd| js8|d }t|r |j}n*|| jjkr(| j |¡jn | jj |¡}n
|
| jjj}| js|d }t|rd|j}n*|| j
|
jkr| j
|
|¡jn | j
|
j |¡}n
|
| j
|
jj}t|s®t|rÌtd
|
t| d t| d | j d k r>t| j sî| j g| _ t| j s| j g| _ t| j t| j kr"td | j t| }| j t| }| jdkrZtd| j ||fS )Nrª z can only asof on a key for leftz can only asof on a key for rightz left can only have one indexz right can only have one indexz(Can only pass by OR left_by and right_byzmissing left_byzmissing right_byr zIncompatible merge dtype, r z$, both sides must have numeric dtypez(left_by and right_by must be same length)r ZforwardZnearestzdirection invalid: )Úsuperr¶ r rW r rX rv rQ r r8 rR r r r r" rí rj r rî r. r5 r+ rw r )r¼ rU rV Z left_on_0Zlo_dtypeZ
|
right_on_0Zro_dtype©Ú __class__rc rd r¶ g sf
|
|
|
ÿ ý
|
|
|
ÿ ý
|
ÿ
|
|
z"_AsOfMerge._validate_left_right_onr r c s¸ t ¡ \}}}tt||D ]\}\}}t|j|js t|jrrt|jrrd| dt|j dt|j d}n$d| dt|j dt|j d}t|q | j d k r| j
|
rÂt t | j j}n|d }d| j dt|j }t|r t| j tjst|| j td k rtd
|
npt|rRt| j s<t|| j d k rtd
|
n<t|rt| j spt|| j d k rtd
|
ntd t| js®d | j }t||||fS ) Nzincompatible merge keys [z] r z), both sides category, but not equal onesz, must be the same typerè zincompatible tolerance z, must be compat with type r ztolerance must be positivez'key must be integer, timestamp or floatz,allow_exact_matches must be boolean, passed )r] r¹ rò rà r& rí r% r5 r r¢ rW r r rQ r r/ rv ÚdatetimeÚ timedeltar r* r) r( r, r# r ) r¼ r© r¨ r§ rÚ r r r¾ Últr^ rc rd r¹ ¶ sH
|
"ÿ"ÿ
|
ÿ
|
|
|
|
|
|
|
ÿz_AsOfMerge._get_merge_keysrý c C sö dddd}| j r | jjjn| jd }| jr8| jjjn| jd }| j}t |j
|
sd}t | ¡ rrt d| dt | d t |j
|
s¸d
|
}t | ¡ rªt d| dt | d t|r |d k røt|}|jjd kròt|}| |j¡}|j}| d ¡}| d ¡}| jd k rÂ| j r6| jr6| j}| j}n | jdd
}| jdd
}t|dkrr|d } |d }
|
n||} ||}
|
t| j} t| } | | }| |
|
}t| j } | ||||| j |S t| j } | |j dt f ||d d | j |dS d S )rþ ú
|
np.ndarrayr c s` dd fdd| D } t tjdt|
}dd | D }t t||}t t t| |¡S )z4unlike np.transpose, this returns an array of tuplesc S s. t | s | S t| } t| tr$| jS t | ¡S rh )r' r? rv r= Ú_ndarrayr Zasarray©Úobjrc rc rd Ú injection s
|
|
z>_AsOfMerge._get_join_indexers.<locals>.flip.<locals>.injectionc s g | ] } |qS rc rc rç ©rg rc rd rs s z?_AsOfMerge._get_join_indexers.<locals>.flip.<locals>.<listcomp>Nc S s g | ]
|
}|j qS rc r[ rç rc rc rd rs s )rw r# Úascii_lowercaser rà r Úarray)ÚxsrÝ ZdtypesZlabeled_dtypesrc rh rd Úflip s z+_AsOfMerge._get_join_indexers.<locals>.fliprè rQ z"Merge keys contain null values on z sidez keys must be sortedrR N)ÚmÚMÚi8r rª Z_tF) rW rQ r rõ r© rX rR r¨ r¢ r7 Zis_monotonic_increasingr3 rô r r/ r rí r' r> Zas_unitÚunitÚ_valueÚviewr r r\ Ú _type_castersrX r r r- )r¼ rl Z left_valuesZ right_valuesr¢ ZsideZlvsZleft_by_valuesZright_by_valuesZlbvZrbvZby_typeZby_type_casterÚfuncrc rc rd r ÿ sv ÿÿ
|
|
|
|
|
|
|
|
ú
|
|
ùz _AsOfMerge._get_join_indexers)NNNFFNNNrª rI TNr¡ NTr ) r9 r: r; rÎ rÀ r¶ r¹ r Ú __classcell__rc rc r^ rd r£ 9 s* í,+ OIr£ )r rY r^ c s$ fddt jD }t| }dd |D \}}}rRtttj| j}ndd }tt| j}tD ]t\} }
|
j| dk} | ¡ rr|
|
|| || d k } | j
|
dksÂ| d | d ksÒ|| d7 < || d || | < qrt ||t |\} }t | |d \} }}tj| ||d S )
|
Nc 3 s& | ] }t j| | d V qdS )rg N)r> r r? ©r Ú join_keysrY rc rd rn s ÿz*_get_multiindex_indexer.<locals>.<genexpr>c s s | ]}t |V qd S rh rA rç rc rc rd rn s c S s | j ddddS )Nro FT©Zsubokr[ )rÙ )Úarc rc rd r r z)_get_multiindex_indexer.<locals>.<lambda>rè rª r rg )r~ rµ rà rw Úmapr rz r rò rô ÚsizerH rx r> rB rC )rw r rY rJ rK ZrcodesZlcodesrN Zi8copyrÚ Újoin_keyr ry rO rP rD rc rv rd Ú_get_multiindex_indexer s& þr} c C s( t | |j|d\}}}tj||||dS )Nrg )r> rõ rB rC )r| r rY rå ræ rD rc rc rd Ú_get_single_indexer« s r~ r c C s t jg t jdt jg t jdfS )zReturn empty join indexers.r[ )r rj Úintprc rc rc rd rF ³ s þrF Úint)r6 Ú left_missingr^ c C s6 t j| t jd}t j| dt jd}|r.||fS ||fS )aÏ
|
Return join indexers where all of one side is selected without sorting
|
and none of the other side is selected.
|
|
Parameters
|
----------
|
n : int
|
Length of indexers to create.
|
left_missing : bool
|
If True, the left indexer will contain only -1's.
|
If False, the right indexer will contain only -1's.
|
|
Returns
|
-------
|
np.ndarray[np.intp]
|
Left indexer
|
np.ndarray[np.intp]
|
Right indexer
|
r[ rè )rN rì rí )r Zaranger Úfull)r6 r rT Z idx_missingrc rc rd rG » s
|
rG z?tuple[Index, npt.NDArray[np.intp] | None, npt.NDArray[np.intp]])r r rY r^ c C s t |dkr@t|tr$t ||jks,tdt|||d\}}n|d }t|||d\}}|snt | t |kr| |¡}|||fS | d |fS )Nrª zIf more than one join key is given then 'right_ax' must be a MultiIndex and the number of join keys must be the number of levels in right_axrg r )r rv r8 rµ rñ r} r~ rz )r r rw rY r rà ZjkeyrÁ rc rc rd r Ø s& ÿ ÿÿ ÿ
|
|
r r z6tuple[npt.NDArray[np.intp], npt.NDArray[np.intp], int])r r rY rS r^ c C s¶ t | ddd} t |ddd}t| jtrNt|jtrNtd| j} td|j}n°t| jr¬t|jr¬t| j|jr¬t| ts~t t|tst |
|
|¡}t | j } t |j }nRt| t rþt| j|jrþt| tsþt| jtræt| jjsþ| ¡ \} }| ¡ \}}t| |\}} }|tt| t|}t| trft|ts@t |j| j| jd}|j|j|jd}nrt| trÄt|tst |j| jd| jjd| ¡ d}|j|jd| jjd| ¡ d}n| | ¡}| |¡}|jt t j ¡ksöt |j|jt t j ¡kst |j| ¡ } |r<|j ¡ }
|
t!|
|
||\}}|dk} | "¡ } |dk} | "¡ }| sh|r| r|t #|| | ¡ |rt #|| | ¡ | d7 } |dkr¬||| fS ||| fS ) u?
|
Encode left and right keys as enumerated types.
|
|
This is used to get the join indexers to be used when merging DataFrames.
|
|
Parameters
|
----------
|
lk : array-like
|
Left key.
|
rk : array-like
|
Right key.
|
sort : bool, defaults to True
|
If True, the encoding is done such that the unique elements in the
|
keys are sorted.
|
how : {âleftâ, ârightâ, âouterâ, âinnerâ}, default âinnerâ
|
Type of merge.
|
|
Returns
|
-------
|
np.ndarray[np.intp]
|
Left (resp. right if called with `key='right'`) labels, as enumerated type.
|
np.ndarray[np.intp]
|
Right (resp. left if called with `key='right'`) labels, as enumerated type.
|
int
|
Number of unique elements in union of left and right labels.
|
|
See Also
|
--------
|
merge : Merge DataFrame or named Series objects
|
with a database-style join.
|
algorithms.factorize : Encode the object as an enumerated type
|
or categorical variable.
|
|
Examples
|
--------
|
>>> lk = np.array(["a", "c", "b"])
|
>>> rk = np.array(["a", "c"])
|
|
Here, the unique values are `'a', 'b', 'c'`. With the default
|
`sort=True`, the encoding will be `{0: 'a', 1: 'b', 2: 'c'}`:
|
|
>>> pd.core.reshape.merge._factorize_keys(lk, rk)
|
(array([0, 2, 1]), array([0, 2]), 3)
|
|
With the `sort=False`, the encoding will correspond to the order
|
in which the unique elements first appear: `{0: 'a', 1: 'c', 2: 'b'}`:
|
|
>>> pd.core.reshape.merge._factorize_keys(lk, rk, sort=False)
|
(array([0, 1, 2]), array([0, 1]), 3)
|
T)rê Z extract_rangerF )r rª )Zna_valuerí rè rR )$r? rv rí r0 r rd r% r& r6 rñ Z_encode_with_my_categoriesr r r< r; r5 r- Ú numpy_dtypeZ_values_for_factorizeÚ#_convert_arrays_and_get_rizer_klassÚmaxr Z factorizeÚ_dataZ_maskr: Zto_numpyr3 r r Z get_countÚuniquesZto_arrayÚ _sort_labelsrô Zputmask)r r rY rS r ÚklassZrizerrL rM rD r ZlmaskZlanyZrmaskZranyrc rc rd r> ö sz 9 ÿþ ý
|
|
|
|
þ
|
ý ÿ ÿ
|
|
|
|
|
r> z:tuple[type[libhashtable.Factorizer], ArrayLike, ArrayLike])r r r^ c C sú t | jrÚt| |st| j|jg}t|tr| ¡ }t| tsR|j| |dd} n
|
| |¡} t|tsx|j||dd}q| |¡}n| |¡} | |¡}t| t
|
r°t | jj }qðt| jt rÌt | jjj }qðt | jj }ntj}t| } t|}|| |fS )NF)rí r[ )r- rí r& r rv r Zconstruct_array_typer< Z_from_sequencerÙ r; Ú _factorizersr´ r5 r Ú libhashtableÚObjectFactorizerr! )r r rí Úclsr rc rc rd r s,
|
|
|
|
|
|
|
|
|
r rc )r rQ rR r^ c C sL t |}t ||g¡}tj| |dd\}}|d |
||d
}}||fS )NT)Zuse_na_sentinel)r r Z concatenaterö Z safe_sort) r rQ rR ZllengthrÝ r Z
|
new_labelsZnew_leftZ new_rightrc rc rd r ¬ s
|
r z%list[npt.NDArray[np.int64 | np.intp]]r z3tuple[npt.NDArray[np.int64], npt.NDArray[np.int64]])rL rM rN rY r^ c
|
|
s( t fddtt ddD }tj d|
dd}|| d jdddd }||d jdddd }td|D ]H}tjd
|
d | | }W 5 Q R X || | | 7 }||| | 7 }qr|t krÐ||fS t|||d \}}} |g| |d
} |g||d
}| f |d
t| | |S ) Nc 3 s" | ]}t d |
s|V qd S rh rB )rk r ©rN rc rd rn ¿ s þz!_get_join_keys.<locals>.<genexpr>r rè rª ro r[ Frx r )Údividerg ) Únextr~ r r ÚprodrÙ r( r> rH )
|
rL rM rN rY ZnlevZstriderO rP rÚ rD rc r rd rH ¸ s$ þ rH c C s t | trt |tsdS | |kS )NT)rv r )rû rü rc rc rd ró Ý s ró c C s | d k ot j| S rh )r² Z any_not_noner rc rc rd r ã s r )rf r^ c C sJ t | tr| S t | tr2| jd kr*td| ¡ S tdt| dd S )Nz$Cannot merge a Series without a namez.Can only merge Series or DataFrame objects, a z was passed)rv r1 r2 rä r Zto_framerV r´ re rc rc rd r® ç s
|
|
|
ÿr® ztuple[Index, Index])rQ rR rZ r^ c s
|
t |ddrt|tr*tdt| d| |¡ t dkrH| |fS |\}}|sf|sftd fdd}t||d }t||d }| |¡}| |¡} g }
|
|j
|
sÂ|| ¡ | ¡ @ ¡ }
|
| j
|
sè|
|
| | ¡ | ¡ @ ¡ ¡ |
|
rtd
|
t|
|
d || fS ) z¶
|
Suffixes type validation.
|
|
If two indices overlap, add suffixes to overlapping entries.
|
|
If corresponding suffix is empty, the entry is simply converted to string.
|
|
F)Z
|
allow_setszPassing 'suffixes' as a z:, is not supported. Provide 'suffixes' as a tuple instead.r z)columns overlap but no suffix specified: c s | kr |dk r | | S | S )a?
|
Rename the left and right indices.
|
|
If there is overlap, and suffix is not None, add
|
suffix, otherwise, leave it as-is.
|
|
Parameters
|
----------
|
x : original column name
|
suffix : str or None
|
|
Returns
|
-------
|
x : renamed column name
|
Nrc )r Úsuffix©Z to_renamerc rd Úrenamer
|
s z+_items_overlap_with_suffix.<locals>.renamer)r z1Passing 'suffixes' which cause duplicate columns z is not allowed.)r+ rv ÚdictrV r´ r3 r r r Z_transform_indexr4 Z
|
duplicatedr} Úextendr rp ) rQ rR rZ ZlsuffixZrsuffixr ZlrenamerZrrenamerrÇ rÈ Zdupsrc r rd rÄ ô s2 ÿ
|
|
|
ÿrÄ ) rH NNNFFFrI NFN)NNNNNNrI r ) NNNFFNNNrI NTr )FrH )F)F)TrH )£r< Ú
|
__future__r r[ rI r` Ú functoolsr r# Útypingr r r r r r1 r* Únumpyr Z pandas._libsr r
|
r r rB r Zpandas._libs.libr Zpandas._typingr r r r r r r r r r Z pandas.errorsr Zpandas.util._decoratorsr r r Zpandas.util._exceptionsr Zpandas.core.dtypes.baser Zpandas.core.dtypes.castr Zpandas.core.dtypes.commonr r r! r" r# r$ r% r& r' r( r) r* r+ r, r- r. r/ Zpandas.core.dtypes.dtypesr0 Zpandas.core.dtypes.genericr1 r2 Zpandas.core.dtypes.missingr3 r4 rÆ r5 r6 r7 r8 r9 Zpandas.core.algorithmsÚcoreZ
|
algorithmsrö Zpandas.core.arraysr: r; r< Zpandas.core.arrays._mixinsr= Zpandas.core.commonÚcommonr² Zpandas.core.constructionr> r? Zpandas.core.framer@ Zpandas.core.indexes.apirA Zpandas.core.sortingrC rD Z pandas.corerE rF Úint64ZInt64FactorizerZlonglongÚint32ZInt32FactorizerÚint16ZInt16Factorizerr× ZInt8FactorizerZuint64ZUInt64FactorizerZuint32ZUInt32FactorizerZuint16ZUInt16FactorizerZuint8ZUInt8FactorizerZbool_Úfloat64ZFloat64FactorizerÚfloat32ZFloat32FactorizerZ complex64ZComplex64FactorizerZ
|
complex128ZComplex128FactorizerZobject_r r re r r r¤ r` r rU r rX rs r\ r£ r} r~ rF rG r r> r r rH ró r r® rÄ rc rc rc rd Ú<module> s" 0 L ñ
|
ó. : ö ò$ Q üJY?ý
|
O& ÿ ÿ! ü % %
|
