U
|
¬ý°dÿD ã @ sþ d dl mZ d dlmZmZ d dlZd dlZd dlmZm Z m
|
Z
|
d dl Z d dl mZ d dlmZmZ d dlZd dlmZ ddd d
|
d Zd)dddddddZddddddddZddddddZd*ddd d d d Zd+dd!d"d#d#d$ddd%d& d'd(ZdS ),é )Ú annotations)ÚabcÚ defaultdictN)ÚAnyÚ DefaultDictÚIterable©Úconvert_json_to_lines)Ú IgnoreRaiseÚScalar)Ú DataFrameÚstr)ÚsÚreturnc C s0 | d dks | d dkr | S | dd
} t | S )zJ
|
Helper function that converts JSON lists to line delimited JSON.
|
r ú[éÿÿÿÿú]é r )r © r úPd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/io/json/_normalize.pyÚconvert_to_line_delimits s r Ú Ú.Úintz
|
int | None)ÚprefixÚsepÚlevelÚ max_levelc
|
C sâ d}t | tr| g} d}g }| D ]°}t |¡}| ¡ D ]\} }
|
t | tsPt| } |dkr^| } n || | } t |
|
tr|dk r ||kr |dkr6| | ¡}
|
|
|
|| < q6| | ¡}
|
| t|
|
| ||d |¡ q6| |¡ q |rÞ|d S |S )a
|
A simplified json_normalize
|
|
Converts a nested dict into a flat dict ("record"), unlike json_normalize,
|
it does not attempt to extract a subset of the data.
|
|
Parameters
|
----------
|
ds : dict or list of dicts
|
prefix: the prefix, optional, default: ""
|
sep : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
level: int, optional, default: 0
|
The number of levels in the json string.
|
|
max_level: int, optional, default: None
|
The max depth to normalize.
|
|
Returns
|
-------
|
d - dict or list of dicts, matching `ds`
|
|
Examples
|
--------
|
>>> nested_to_record(
|
... dict(flat1=1, dict1=dict(c=1, d=2), nested=dict(e=dict(c=1, d=2), d=2))
|
... )
|
{'flat1': 1, 'dict1.c': 1, 'dict1.d': 2, 'nested.e.c': 1, 'nested.e.d': 2, 'nested.d': 2}
|
FTr Nr )
|
Ú
|
isinstanceÚdictÚcopyÚdeepcopyÚitemsr ÚpopÚupdateÚnested_to_recordÚappend) Údsr r r r Z singletonZnew_dsÚdZnew_dÚkÚvZnewkeyr r r r% * s8 ,
|
|
|
|
ÿÿ
|
|
r% r údict[str, Any])ÚdataÚ
|
key_stringÚnormalized_dictÚ separatorr c C sz t | trn| ¡ D ]X\}}| | | }|sZtjdk rPddlm} |||}n
|
| |¡}t||||d qn| ||< |S )a3
|
Main recursive function
|
Designed for the most basic use case of pd.json_normalize(data)
|
intended as a performance improvement, see #15621
|
|
Parameters
|
----------
|
data : Any
|
Type dependent on types contained within nested Json
|
key_string : str
|
New key (with separator(s) in) for data
|
normalized_dict : dict
|
The new normalized/flattened Json dict
|
separator : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
)é é r )Ú removeprefix©r, r- r. r/ )r r r" ÚsysÚ version_infoZpandas.util._str_methodsr2 Ú_normalise_json)r, r- r. r/ ÚkeyÚvalueZnew_keyr2 r r r r6 { s
|
|
|
ü
|
r6 )r, r/ r c C s8 dd | ¡ D }tdd | ¡ D di |d}||S )aw
|
Order the top level keys and then recursively go to depth
|
|
Parameters
|
----------
|
data : dict or list of dicts
|
separator : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
|
Returns
|
-------
|
dict or list of dicts, matching `normalised_json_object`
|
c S s i | ]\}}t |ts||qS r ©r r ©Ú.0r) r* r r r Ú
|
<dictcomp>¸ s
|
z+_normalise_json_ordered.<locals>.<dictcomp>c S s i | ]\}}t |tr||qS r r9 r: r r r r< º s
|
r r3 )r" r6 )r, r/ Z top_dict_Z nested_dict_r r r Ú_normalise_json_ordered© s ür= zdict | list[dict]zdict | list[dict] | Any)r' r r c s@ i }t | tr t| d}n t | tr< fdd| D }|S |S )a
|
A optimized basic json_normalize
|
|
Converts a nested dict into a flat dict ("record"), unlike
|
json_normalize and nested_to_record it doesn't do anything clever.
|
But for the most basic use cases it enhances performance.
|
E.g. pd.json_normalize(data)
|
|
Parameters
|
----------
|
ds : dict or list of dicts
|
sep : str, default '.'
|
Nested records will generate names separated by sep,
|
e.g., for sep='.', { 'foo' : { 'bar' : 0 } } -> foo.bar
|
|
Returns
|
-------
|
frame : DataFrame
|
d - dict or list of dicts, matching `normalised_json_object`
|
|
Examples
|
--------
|
>>> _simple_json_normalize(
|
... {
|
... "flat1": 1,
|
... "dict1": {"c": 1, "d": 2},
|
... "nested": {"e": {"c": 1, "d": 2}, "d": 2},
|
... }
|
... )
|
{'flat1': 1, 'dict1.c': 1, 'dict1.d': 2, 'nested.e.c': 1, 'nested.e.d': 2, 'nested.d': 2}
|
|
)r, r/ c s g | ]}t | d qS )©r )Ú_simple_json_normalize)r; Úrowr> r r Ú
|
<listcomp>ò s z*_simple_json_normalize.<locals>.<listcomp>)r r r= Úlist)r' r Znormalised_json_objectZnormalised_json_listr r> r r? Â s +
|
|
r? Úraisezstr | list | Nonez"str | list[str | list[str]] | Nonez
|
str | Noner
|
r ) r, Ú record_pathÚmetaÚ meta_prefixÚ record_prefixÚerrorsr r r c sþ d"dddddfdd ddd d
|
fd d t | trD| sDt S t | trV| g} n$t | tjrvt | tsvt| } nt|d kr²|d kr²|d kr² d kr²d kr²tt| dS |d krât dd | D rÚt
|
| d} t| S t |tsò|g}|d krg }nt |ts|g}dd |D g
|
g t t fdd D d#ddd
|
f
|
dd | |i dd t
|
} d k r¢|j fd d d } ¡ D ]N\} }
|
|d k rÄ|| } | |krÞtd | d tj|
|
td! ¡|| < qª|S )$a´
|
Normalize semi-structured JSON data into a flat table.
|
|
Parameters
|
----------
|
data : dict or list of dicts
|
Unserialized JSON objects.
|
record_path : str or list of str, default None
|
Path in each object to list of records. If not passed, data will be
|
assumed to be an array of records.
|
meta : list of paths (str or list of str), default None
|
Fields to use as metadata for each record in resulting table.
|
meta_prefix : str, default None
|
If True, prefix records with dotted (?) path, e.g. foo.bar.field if
|
meta is ['foo', 'bar'].
|
record_prefix : str, default None
|
If True, prefix records with dotted (?) path, e.g. foo.bar.field if
|
path to records is ['foo', 'bar'].
|
errors : {'raise', 'ignore'}, default 'raise'
|
Configures error handling.
|
|
* 'ignore' : will ignore KeyError if keys listed in meta are not
|
always present.
|
* 'raise' : will raise KeyError if keys listed in meta are not
|
always present.
|
sep : str, default '.'
|
Nested records will generate names separated by sep.
|
e.g., for sep='.', {'foo': {'bar': 0}} -> foo.bar.
|
max_level : int, default None
|
Max number of levels(depth of dict) to normalize.
|
if None, normalizes all levels.
|
|
Returns
|
-------
|
frame : DataFrame
|
Normalize semi-structured JSON data into a flat table.
|
|
Examples
|
--------
|
>>> data = [
|
... {"id": 1, "name": {"first": "Coleen", "last": "Volk"}},
|
... {"name": {"given": "Mark", "family": "Regner"}},
|
... {"id": 2, "name": "Faye Raker"},
|
... ]
|
>>> pd.json_normalize(data)
|
id name.first name.last name.given name.family name
|
0 1.0 Coleen Volk NaN NaN NaN
|
1 NaN NaN NaN Mark Regner NaN
|
2 2.0 NaN NaN NaN NaN Faye Raker
|
|
>>> data = [
|
... {
|
... "id": 1,
|
... "name": "Cole Volk",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... {"name": "Mark Reg", "fitness": {"height": 130, "weight": 60}},
|
... {
|
... "id": 2,
|
... "name": "Faye Raker",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... ]
|
>>> pd.json_normalize(data, max_level=0)
|
id name fitness
|
0 1.0 Cole Volk {'height': 130, 'weight': 60}
|
1 NaN Mark Reg {'height': 130, 'weight': 60}
|
2 2.0 Faye Raker {'height': 130, 'weight': 60}
|
|
Normalizes nested data up to level 1.
|
|
>>> data = [
|
... {
|
... "id": 1,
|
... "name": "Cole Volk",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... {"name": "Mark Reg", "fitness": {"height": 130, "weight": 60}},
|
... {
|
... "id": 2,
|
... "name": "Faye Raker",
|
... "fitness": {"height": 130, "weight": 60},
|
... },
|
... ]
|
>>> pd.json_normalize(data, max_level=1)
|
id name fitness.height fitness.weight
|
0 1.0 Cole Volk 130 60
|
1 NaN Mark Reg 130 60
|
2 2.0 Faye Raker 130 60
|
|
>>> data = [
|
... {
|
... "state": "Florida",
|
... "shortname": "FL",
|
... "info": {"governor": "Rick Scott"},
|
... "counties": [
|
... {"name": "Dade", "population": 12345},
|
... {"name": "Broward", "population": 40000},
|
... {"name": "Palm Beach", "population": 60000},
|
... ],
|
... },
|
... {
|
... "state": "Ohio",
|
... "shortname": "OH",
|
... "info": {"governor": "John Kasich"},
|
... "counties": [
|
... {"name": "Summit", "population": 1234},
|
... {"name": "Cuyahoga", "population": 1337},
|
... ],
|
... },
|
... ]
|
>>> result = pd.json_normalize(
|
... data, "counties", ["state", "shortname", ["info", "governor"]]
|
... )
|
>>> result
|
name population state shortname info.governor
|
0 Dade 12345 Florida FL Rick Scott
|
1 Broward 40000 Florida FL Rick Scott
|
2 Palm Beach 60000 Florida FL Rick Scott
|
3 Summit 1234 Ohio OH John Kasich
|
4 Cuyahoga 1337 Ohio OH John Kasich
|
|
>>> data = {"A": [1, 2]}
|
>>> pd.json_normalize(data, "A", record_prefix="Prefix.")
|
Prefix.0
|
0 1
|
1 2
|
|
Returns normalized data with columns prefixed with the given string.
|
Fr+ z
|
list | strÚboolzScalar | Iterable)ÚjsÚspecÚextract_recordr c
|
sª | }z:t |tr4|D ] }|dkr(t||| }qn|| }W nf tk
|
r¤ } zH|rftd| d| dkr|tj W Y ¢ S td| d| d|W 5 d}~X Y nX |S )z Internal function to pull fieldNzKey zS not found. If specifying a record_path, all elements of data should have the path.Úignorez) not found. To replace missing values of z% with np.nan, pass in errors='ignore')r rB ÚKeyErrorÚnpÚnan)rJ rK rL ÚresultÚfieldÚe)rH r r Ú _pull_field s.
|
|
ÿýÿýz#json_normalize.<locals>._pull_fieldrB )rJ rK r c sF | |dd}t |tsBt |¡r(g }nt| d| d| d|S )z¶
|
Internal function to pull field for records, and similar to
|
_pull_field, but require to return list. And will raise error
|
if has non iterable value.
|
T)rL z has non list value z
|
for path z. Must be list or null.)r rB ÚpdZisnullÚ TypeError)rJ rK rQ )rT r r Ú _pull_records¡ s
|
|
ÿz%json_normalize.<locals>._pull_recordsNr> c s s | ]}d d | ¡ D V qdS )c S s g | ]}t |tqS r r9 )r; Úxr r r rA Î s z,json_normalize.<locals>.<genexpr>.<listcomp>N)Úvalues)r; Úyr r r Ú <genexpr>Î s z!json_normalize.<locals>.<genexpr>©r r c S s g | ]}t |tr|n|gqS r )r rB )r; Úmr r r rA à s z"json_normalize.<locals>.<listcomp>c s g | ]} |¡qS r )Újoin)r; Úvalr> r r rA ç s r r ÚNone)r r c s t | tr| g} t|dkr| D ]^}t D ]*\}}|d t|kr.||d ||< q.||d |dd
||d d q n| D ]}||d } fdd|D } t|¡ t D ]B\}}|d t|kræ|| }n|||d
}| |¡ qÄ |¡ qd S )Nr r r ©r c s( g | ] }t |tr t| d n|qS )r\ )r r r% )r; Úr)r r r r rA ö s þÿz>json_normalize.<locals>._recursive_extract.<locals>.<listcomp>)r r ÚlenÚzipr& Úextend) r, ÚpathZ seen_metar Úobjr_ r7 ZrecsZmeta_val)
|
Ú_metarT rW Ú_recursive_extractÚlengthsr Ú meta_keysÚ meta_valsÚrecordsr r r ri é s(
|
( ü
|
z*json_normalize.<locals>._recursive_extractra c s | S )Nr )rX )rG r r Ú<lambda> ó z json_normalize.<locals>.<lambda>)ÚcolumnszConflicting metadata name z , need distinguishing prefix )Zdtype)F)r )r rB r r r r r ÚNotImplementedErrorr? Úanyr% r Úrenamer" Ú
|
ValueErrorrO ÚarrayÚobjectÚrepeat) r, rD rE rF rG rH r r rQ r) r* r ) rh rT rW ri rH rj r rk rl rG rm r r Újson_normalize÷ sf ÿ
|
|
ÿþýüû
|
|
(
|
|
|
|
ÿ rx )r r r N)r )NNNNrC r N)Ú
|
__future__r Ú collectionsr r r r4 Útypingr r r ÚnumpyrO Zpandas._libs.writersr Zpandas._typingr
|
r ZpandasrU r r r% r6 r= r? rx r r r r Ú<module> s6 ûQ. þ7 ø
|
