U
|
¬ý°dgI ã @ sF d dl mZ d dlZd dlmZmZ d dlZd dlm Z d dl
|
m Z m Z d dl mZ d dlmZ d dlm mZ d dlmZ d dlm mZ d d lmZm Z d d
|
l m Z d d l m Z d d l!m"Z" d d l#m$Z$ erìd dl%m&Z& d dl'm(Z( e e"d ddd d$dddddddZ)d%ddddd d Z*d&dd d dd!d"d#Z+dS )'é )Ú annotationsN)Ú TYPE_CHECKINGÚHashable)ÚAppender)Úis_extension_array_dtypeÚ is_list_like)Ú concat_compat)Únotna)Ú Categorical)ÚIndexÚ
|
MultiIndex)Úconcat)Ú tile_compat)Ú _shared_docs)Ú
|
to_numeric)Ú AnyArrayLike)Ú DataFrameÚmeltz pd.melt(df, zDataFrame.melt)ZcallerÚotherÚvalueTr r Úbool)ÚframeÚ
|
value_nameÚ ignore_indexÚreturnc sL t jtr dd jD }n
|
t j}| jkrBtd| d|d k r°t|sZ|g}q´t jtrzt |tsztdq´t|}tt |¡ |¡}|j
|
s´t dt| ng }|d k rpt|sÎ|g}nVt jtrît |tsîtdn6t|}tt |¡ |¡}|j
|
s$t dt| |d k rF j |¡ || ¡} nt j || ¡¡} jd d
| f n ¡ |d k r j |¡ _|d krt jtrìt jjtt jjkrÐ jj}nd d tt jjD }n jjd k r jjnd
|
g}t |tr|g} j\}
|
} | t|8 } i } |D ]f} | ¡}t|r| d krnt |g| d d }nt |g |j|j d}nt |j!| ¡}|| | < q8|| |g } jd d kræt fddt jd D j"| |< n j! #d¡| |< t$|D ]&\}} t % j &|¡¡ '|
|
¡| | < qþ j(| |d}|sHt) j*| |_*|S )Nc S s g | ]}|D ]}|q qS © r )Ú.0ÚcÚxr r úOd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/core/reshape/melt.pyÚ
|
<listcomp>3 s zmelt.<locals>.<listcomp>z value_name (z3) cannot match an element in the DataFrame columns.z>id_vars must be a list of tuples when columns are a MultiIndexz:The following 'id_vars' are not present in the DataFrame: zAvalue_vars must be a list of tuples when columns are a MultiIndexz=The following 'value_vars' are not present in the DataFrame: c S s g | ]}d | qS )Z variable_r ©r Úir r r r s s Úvariabler T)r )ÚnameÚdtypeé c s g | ]} j d d
|f qS ©N)Úilocr! ©r r r r s ÚF©Úcolumns)+Ú
|
isinstancer, r ÚlistÚ
|
ValueErrorr r ÚcomÚflattenÚ
|
differenceÚemptyÚKeyErrorZget_level_valuesZ get_indexerÚalgosÚuniqueZget_indexer_forr( ÚcopyÚlenÚnamesÚsetÚranger$ ÚstrÚshapeÚpopr r Útyper% ÚnpÚtileÚ_valuesÚvaluesZravelÚ enumerateZ
|
asanyarrayZ_get_level_valuesÚrepeatÚ _constructorr Úindex)r Úid_varsÚ
|
value_varsÚvar_namer Z col_levelr ÚcolsÚmissingÚidxÚNÚKÚmdataÚcolZid_dataZmcolumnsr" Úresultr r) r r & s
|
|
|
ÿÿ ÿ
|
ÿ ÿ
|
ÿ
|
|
|
ÿ
|
|
|
|
ÿ )ÚdataÚdropnar c sT t |tr$t| ¡ }t| ¡ }n t| \}}ttjdd |D }t j |¡}t
|
|d }|D ]}t
|
||krht dqhi } g }
|
t||D ]0\} } fdd| D } t | | | < |
|
| ¡ q|D ]}t | j|¡| |< qÊ|rBtjt
|
| |
|
d td|
|
D ]}t| | M q
|
¡ sBfdd | ¡ D } j| ||
|
d
|
S ) aÔ
|
Reshape wide-format data to long. Generalized inverse of DataFrame.pivot.
|
|
Accepts a dictionary, ``groups``, in which each key is a new column name
|
and each value is a list of old column names that will be "melted" under
|
the new column name as part of the reshape.
|
|
Parameters
|
----------
|
data : DataFrame
|
The wide-format DataFrame.
|
groups : dict
|
{new_name : list_of_columns}.
|
dropna : bool, default True
|
Do not include columns whose entries are all NaN.
|
|
Returns
|
-------
|
DataFrame
|
Reshaped DataFrame.
|
|
See Also
|
--------
|
melt : Unpivot a DataFrame from wide to long format, optionally leaving
|
identifiers set.
|
pivot : Create a spreadsheet-style pivot table as a DataFrame.
|
DataFrame.pivot : Pivot without aggregation that can handle
|
non-numeric data.
|
DataFrame.pivot_table : Generalization of pivot that can handle
|
duplicate values for one index/column pair.
|
DataFrame.unstack : Pivot based on the index values instead of a
|
column.
|
wide_to_long : Wide panel to long format. Less flexible but more
|
user-friendly than melt.
|
|
Examples
|
--------
|
>>> data = pd.DataFrame({'hr1': [514, 573], 'hr2': [545, 526],
|
... 'team': ['Red Sox', 'Yankees'],
|
... 'year1': [2007, 2007], 'year2': [2008, 2008]})
|
>>> data
|
hr1 hr2 team year1 year2
|
0 514 545 Red Sox 2007 2008
|
1 573 526 Yankees 2007 2008
|
|
>>> pd.lreshape(data, {'year': ['year1', 'year2'], 'hr': ['hr1', 'hr2']})
|
team year hr
|
0 Red Sox 2007 514
|
1 Yankees 2007 573
|
2 Red Sox 2008 545
|
3 Yankees 2008 526
|
c s s | ]}t |V qd S r' )r: )r r r r r Ú <genexpr>Ü s zlreshape.<locals>.<genexpr>r z$All column lists must be same lengthc s g | ]} | j qS r )rB ©r rQ )rS r r r é s z lreshape.<locals>.<listcomp>)r% c s i | ]\}}|| qS r r )r ÚkÚv)Úmaskr r Ú
|
<dictcomp>ö s z lreshape.<locals>.<dictcomp>r+ )r- Údictr. ÚkeysrC Úzipr: Úunionr, r2 r8 r/ r Úappendr@ rA rB Zonesr r ÚallÚitemsrF )rS ÚgroupsrT r\ rC Zall_colsZid_colsrO ÚseqrP Z
|
pivot_colsÚtargetr9 Z to_concatrQ r r )rS rY r Úlreshape¡ s2 5
|
|
|
re Ú ú\d+r< )ÚdfÚsepÚsuffixr c sP ddddddddddddt s4gnttfd d
|
jD r\td t slgnt ¡ ¡ rtd fd dD }dd |D }tt j ¡ |¡} fddt |D } | d j
|
| dd
dd}
|
t dkr* | ¡
|
|
|
¡} | S | j |
|
¡ d g ¡} | S )ax
|
Unpivot a DataFrame from wide to long format.
|
|
Less flexible but more user-friendly than melt.
|
|
With stubnames ['A', 'B'], this function expects to find one or more
|
group of columns with format
|
A-suffix1, A-suffix2,..., B-suffix1, B-suffix2,...
|
You specify what you want to call this suffix in the resulting long format
|
with `j` (for example `j='year'`)
|
|
Each row of these wide variables are assumed to be uniquely identified by
|
`i` (can be a single column name or a list of column names)
|
|
All remaining variables in the data frame are left intact.
|
|
Parameters
|
----------
|
df : DataFrame
|
The wide-format DataFrame.
|
stubnames : str or list-like
|
The stub name(s). The wide format variables are assumed to
|
start with the stub names.
|
i : str or list-like
|
Column(s) to use as id variable(s).
|
j : str
|
The name of the sub-observation variable. What you wish to name your
|
suffix in the long format.
|
sep : str, default ""
|
A character indicating the separation of the variable names
|
in the wide format, to be stripped from the names in the long format.
|
For example, if your column names are A-suffix1, A-suffix2, you
|
can strip the hyphen by specifying `sep='-'`.
|
suffix : str, default '\\d+'
|
A regular expression capturing the wanted suffixes. '\\d+' captures
|
numeric suffixes. Suffixes with no numbers could be specified with the
|
negated character class '\\D+'. You can also further disambiguate
|
suffixes, for example, if your wide variables are of the form A-one,
|
B-two,.., and you have an unrelated column A-rating, you can ignore the
|
last one by specifying `suffix='(!?one|two)'`. When all suffixes are
|
numeric, they are cast to int64/float64.
|
|
Returns
|
-------
|
DataFrame
|
A DataFrame that contains each stub name as a variable, with new index
|
(i, j).
|
|
See Also
|
--------
|
melt : Unpivot a DataFrame from wide to long format, optionally leaving
|
identifiers set.
|
pivot : Create a spreadsheet-style pivot table as a DataFrame.
|
DataFrame.pivot : Pivot without aggregation that can handle
|
non-numeric data.
|
DataFrame.pivot_table : Generalization of pivot that can handle
|
duplicate values for one index/column pair.
|
DataFrame.unstack : Pivot based on the index values instead of a
|
column.
|
|
Notes
|
-----
|
All extra variables are left untouched. This simply uses
|
`pandas.melt` under the hood, but is hard-coded to "do the right thing"
|
in a typical case.
|
|
Examples
|
--------
|
>>> np.random.seed(123)
|
>>> df = pd.DataFrame({"A1970" : {0 : "a", 1 : "b", 2 : "c"},
|
... "A1980" : {0 : "d", 1 : "e", 2 : "f"},
|
... "B1970" : {0 : 2.5, 1 : 1.2, 2 : .7},
|
... "B1980" : {0 : 3.2, 1 : 1.3, 2 : .1},
|
... "X" : dict(zip(range(3), np.random.randn(3)))
|
... })
|
>>> df["id"] = df.index
|
>>> df
|
A1970 A1980 B1970 B1980 X id
|
0 a d 2.5 3.2 -1.085631 0
|
1 b e 1.2 1.3 0.997345 1
|
2 c f 0.7 0.1 0.282978 2
|
>>> pd.wide_to_long(df, ["A", "B"], i="id", j="year")
|
... # doctest: +NORMALIZE_WHITESPACE
|
X A B
|
id year
|
0 1970 -1.085631 a 2.5
|
1 1970 0.997345 b 1.2
|
2 1970 0.282978 c 0.7
|
0 1980 -1.085631 d 3.2
|
1 1980 0.997345 e 1.3
|
2 1980 0.282978 f 0.1
|
|
With multiple id columns
|
|
>>> df = pd.DataFrame({
|
... 'famid': [1, 1, 1, 2, 2, 2, 3, 3, 3],
|
... 'birth': [1, 2, 3, 1, 2, 3, 1, 2, 3],
|
... 'ht1': [2.8, 2.9, 2.2, 2, 1.8, 1.9, 2.2, 2.3, 2.1],
|
... 'ht2': [3.4, 3.8, 2.9, 3.2, 2.8, 2.4, 3.3, 3.4, 2.9]
|
... })
|
>>> df
|
famid birth ht1 ht2
|
0 1 1 2.8 3.4
|
1 1 2 2.9 3.8
|
2 1 3 2.2 2.9
|
3 2 1 2.0 3.2
|
4 2 2 1.8 2.8
|
5 2 3 1.9 2.4
|
6 3 1 2.2 3.3
|
7 3 2 2.3 3.4
|
8 3 3 2.1 2.9
|
>>> l = pd.wide_to_long(df, stubnames='ht', i=['famid', 'birth'], j='age')
|
>>> l
|
... # doctest: +NORMALIZE_WHITESPACE
|
ht
|
famid birth age
|
1 1 1 2.8
|
2 3.4
|
2 1 2.9
|
2 3.8
|
3 1 2.2
|
2 2.9
|
2 1 1 2.0
|
2 3.2
|
2 1 1.8
|
2 2.8
|
3 1 1.9
|
2 2.4
|
3 1 1 2.2
|
2 3.3
|
2 1 2.3
|
2 3.4
|
3 1 2.1
|
2 2.9
|
|
Going from long back to wide just takes some creative use of `unstack`
|
|
>>> w = l.unstack()
|
>>> w.columns = w.columns.map('{0[0]}{0[1]}'.format)
|
>>> w.reset_index()
|
famid birth ht1 ht2
|
0 1 1 2.8 3.4
|
1 1 2 2.9 3.8
|
2 1 3 2.2 2.9
|
3 2 1 2.0 3.2
|
4 2 2 1.8 2.8
|
5 2 3 1.9 2.4
|
6 3 1 2.2 3.3
|
7 3 2 2.3 3.4
|
8 3 3 2.1 2.9
|
|
Less wieldy column names are also handled
|
|
>>> np.random.seed(0)
|
>>> df = pd.DataFrame({'A(weekly)-2010': np.random.rand(3),
|
... 'A(weekly)-2011': np.random.rand(3),
|
... 'B(weekly)-2010': np.random.rand(3),
|
... 'B(weekly)-2011': np.random.rand(3),
|
... 'X' : np.random.randint(3, size=3)})
|
>>> df['id'] = df.index
|
>>> df # doctest: +NORMALIZE_WHITESPACE, +ELLIPSIS
|
A(weekly)-2010 A(weekly)-2011 B(weekly)-2010 B(weekly)-2011 X id
|
0 0.548814 0.544883 0.437587 0.383442 0 0
|
1 0.715189 0.423655 0.891773 0.791725 1 1
|
2 0.602763 0.645894 0.963663 0.528895 1 2
|
|
>>> pd.wide_to_long(df, ['A(weekly)', 'B(weekly)'], i='id',
|
... j='year', sep='-')
|
... # doctest: +NORMALIZE_WHITESPACE
|
X A(weekly) B(weekly)
|
id year
|
0 2010 0 0.548814 0.437587
|
1 2010 1 0.715189 0.891773
|
2 2010 1 0.602763 0.963663
|
0 2011 0 0.544883 0.383442
|
1 2011 1 0.423655 0.791725
|
2 2011 1 0.645894 0.528895
|
|
If we have many columns, we could also use a regex to find our
|
stubnames and pass that list on to wide_to_long
|
|
>>> stubnames = sorted(
|
... set([match[0] for match in df.columns.str.findall(
|
... r'[A-B]\(.*\)').values if match != []])
|
... )
|
>>> list(stubnames)
|
['A(weekly)', 'B(weekly)']
|
|
All of the above examples have integers as suffixes. It is possible to
|
have non-integers as suffixes.
|
|
>>> df = pd.DataFrame({
|
... 'famid': [1, 1, 1, 2, 2, 2, 3, 3, 3],
|
... 'birth': [1, 2, 3, 1, 2, 3, 1, 2, 3],
|
... 'ht_one': [2.8, 2.9, 2.2, 2, 1.8, 1.9, 2.2, 2.3, 2.1],
|
... 'ht_two': [3.4, 3.8, 2.9, 3.2, 2.8, 2.4, 3.3, 3.4, 2.9]
|
... })
|
>>> df
|
famid birth ht_one ht_two
|
0 1 1 2.8 3.4
|
1 1 2 2.9 3.8
|
2 1 3 2.2 2.9
|
3 2 1 2.0 3.2
|
4 2 2 1.8 2.8
|
5 2 3 1.9 2.4
|
6 3 1 2.2 3.3
|
7 3 2 2.3 3.4
|
8 3 3 2.1 2.9
|
|
>>> l = pd.wide_to_long(df, stubnames='ht', i=['famid', 'birth'], j='age',
|
... sep='_', suffix=r'\w+')
|
>>> l
|
... # doctest: +NORMALIZE_WHITESPACE
|
ht
|
famid birth age
|
1 1 one 2.8
|
two 3.4
|
2 one 2.9
|
two 3.8
|
3 one 2.2
|
two 2.9
|
2 1 one 2.0
|
two 3.2
|
2 one 1.8
|
two 2.8
|
3 one 1.9
|
two 2.4
|
3 1 one 2.2
|
two 3.3
|
2 one 2.3
|
two 3.4
|
3 one 2.1
|
two 2.9
|
r< z list[str])Ústubri rj r c s> dt |¡ t |¡ | d}t |¡ fdd| jD S )Nú^ú$c s g | ]} |¡r|qS r )ÚmatchrV ©Úpatternr r r ì s
|
z7wide_to_long.<locals>.get_var_names.<locals>.<listcomp>)ÚreÚescapeÚcompiler, )rh rk ri rj Úregexr ro r Ú get_var_namesé s
|
z#wide_to_long.<locals>.get_var_names)rk ri c S sp t | ||| |¡|d}t|| ||< || jjt || ¡ddd||< t|| dd||< | ||g ¡S )N)rH rI r rJ rf T)rt Úignore)Úerrors) r Úrstripr
|
r< Úreplacerq rr r Ú set_index)rh rk r" ÚjrI ri Znewdfr r r Ú melt_stubî s û$z wide_to_long.<locals>.melt_stubc 3 s | ]}| kV qd S r' r rV )Ú stubnamesr r rU s z wide_to_long.<locals>.<genexpr>z,stubname can't be identical to a column namez3the id variables need to uniquely identify each rowc s g | ]} |qS r r )r rk )rh ru ri rj r r r s z wide_to_long.<locals>.<listcomp>c S s g | ]}|D ]}|q qS r r )r ZsublistÚer r r r s c s" g | ]\}} ||qS r r )r ÚsrX )rh r" r{ r| ri r r r s r r& NÚouter)Úhow)Úon)r r. Úanyr, r/ Z
|
duplicatedr: Útolistr2 r] Újoinr8 rz ÚmergeZ reset_index) rh r} r" r{ ri rj rI Zvalue_vars_flattenedrH Z_meltedZmeltedÚnewr )rh ru r" r{ r| ri r} rj r Ú wide_to_longû s. o "r )NNNr NT)T)rf rg ),Ú
|
__future__r rq Útypingr r Únumpyr@ Zpandas.util._decoratorsr Zpandas.core.dtypes.commonr r Zpandas.core.dtypes.concatr Zpandas.core.dtypes.missingr Zpandas.core.algorithmsÚcoreZ
|
algorithmsr5 Zpandas.core.arraysr
|
Zpandas.core.commonÚcommonr0 Zpandas.core.indexes.apir r Zpandas.core.reshape.concatr Zpandas.core.reshape.utilr Zpandas.core.shared_docsr Zpandas.core.tools.numericr Zpandas._typingr Zpandasr r re r r r r r Ú<module> s< ùz[ ÿ
|
