U
|
ý°dÿy ã @ s® d dl Z d dlZd dlZd dlmZmZmZmZ d dl m
|
Z d dl m Z ejdd Zejdd Zejdd Zejd
|
d Zejd d ZG dd dZG dd dZdS )é N)Ú DataFrameÚIndexÚSeriesÚjson_normalize)Únested_to_recordc C sl ddddddddgdd d
|
d dd d dgdgddddddgddddddddgdgdgS )NÚUSAÚ
|
Californiaú San Franciscoé90 )ÚnameÚpopú Los Angelesé:0 )r ÚcitiesÚOhioÚColumbuséÒ Ú ClevelandéÔ )ÚcountryÚstatesÚGermanyÚBayernÚMuniché;0 úNordrhein-WestfalenÚ DuesseldorféÖ ÚKoelné× © r r r úZd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/tests/io/json/test_normalize.pyÚ deep_nested s. þþþþøþþþþþìr" c C sL dddddddddgdd id
|
d d d dddddgddiddd gS )NÚDader
|
©r Ú
|
populationÚBrowardé@ ú
|
Palm Beaché`ê Úgovernorú
|
Rick ScottÚFLÚFlorida)ÚcountiesÚinfoÚ shortnameÚstateÚSummitr ÚCuyahogaé9 ú John KasichÚOHr r r r r r! Ú
|
state_data8 s ýø þùõr7 c C s dd idddddddgS ) Nr/ ú
|
11/08/1993ú
|
26/05/2012©Z
|
created_atZ last_updatedÚJaneÚDoe©ÚfirstÚ last_name©r/ Z author_namer r r r r! Úauthor_missing_dataQ s
|
þþrA c C sF dddddddgdd d
|
igid d d ddddgdd digidgS )NÚAliceéZ% ú
|
Morris St.Ú Massillonr6 éf® )ÚnumberÚstreetÚcityr1 Úzipr Ú city_nameú Foo York City)r Ú addressesÚprevious_residencesé! ú
|
Spring St.Ú ElizabethtonÚTNé Ú
|
Barmingham)rM rN r r r r r! Úmissing_metadata\ s( ûÿ õûÿ öòrU c C s" ddiddddddd id
|
gS ) z@
|
input data to test json_normalize with max_level param
|
ÚNameÚUser001ú Some textÚID001ÚName001©ZIdrV ©Ú TextFieldÚ UserFieldÚaÚb©Ú CreatedByÚLookupÚImager r r r r! Úmax_level_test_input_data{ s þúÿre c @ s e Zd Zdd Zdd Zdd Zej ddd idd
|
igd d fddd idd
|
igidd fd d e fd d e fg¡d d Z
|
dd Z dd Z dd Z dd Zdd Zdd Zdd Zd d Zd d Zd!d" Zd#d$ Zd%d& Zej d'd d(d)d*d+d,d-idd.id/d(d)d*d+d,d-idd.id/gfd
|
d(d)d*d,d-idd.id0d(d)d*d,d-idd.id0gfg¡d1d2 Zd3d4 Zd5d6 Zej d7d8d9d:d;d<g¡d=d> Zd?d@ ZdAdB Z dCdD Z d S )EÚTestJSONNormalizec C sL dddddddddd d
|
dd d d dg}t |}t|}t ||¡ d S )Né é é )r_ r` Úcé é é é é é é
|
é é ©r r ÚtmÚassert_frame_equal©ÚselfZrecsÚresultÚexpectedr r r! Útest_simple_records s
|
|
|
|
üz%TestJSONNormalize.test_simple_recordsc C s t |d d}t|d d }t ||¡ t |d}g }|D ]}| |d ¡ q<t|}t ||¡ t |ddd}t ddg¡ ddg¡|d< t ||¡ d S ) Nr r. r1 ©Úmetar- r ri rh )r r ru rv ÚextendÚnpÚarrayÚrepeat©rx r7 ry rz Úrecr r r! Útest_simple_normalize s
|
z'TestJSONNormalize.test_simple_normalizec C s t g }t }t ||¡ d S )Nrt ©rx ry rz r r r! Útest_empty_array s z"TestJSONNormalize.test_empty_arrayz!data, record_path, exception_typer_ r rg Nz{"a": [{"a": 0}, {"a": 1}]}c C s` |d k r2t j|tjd t||d W 5 Q R X n*t||d}tddgdgd}t ||¡ d S )N©Úmatch©Ú record_pathr rg r_ ©Úcolumns)ÚpytestÚraisesru ZEMPTY_STRING_PATTERNr r rv )rx Údatar Zexception_typery rz r r r! Útest_accepted_input² s
|
z%TestJSONNormalize.test_accepted_inputc C sþ t ddddi}tddggddgd}t | |¡|¡ t ddddidd }tddggd
|
d gd}t | |¡|¡ t ddddid d }tddggd dgd}t | |¡|¡ t |ddgdddggdd}tddddg ¡ }|j ¡ |¡sút d S )NÚArg rh )r ÚBzA.AzA.Br Ú_©ÚsepZA_AZA_Bu Ïu AÏAu AÏBr r r r )r} r r Z states_name)
|
r r ru rv Z reindex_liker Z sort_valuesr ÚequalsÚAssertionError)rx r" ry rz r r r! Ú$test_simple_normalize_with_separatorÄ s"
|
üz6TestJSONNormalize.test_simple_normalize_with_separatorc C sN ddgdddd}t |dd}tddgddggdd d
|
gd }t ||¡ d S ) Nrg rh )ri rk )Zb_1Zb_2)r_ r` Ú__r r_ Zb__b_1Zb__b_2r rt ©rx r ry rz r r r! Ú'test_normalize_with_multichar_separatorÛ s z9TestJSONNormalize.test_normalize_with_multichar_separatorc C s< t dddgiddd}tdgdggdgd}t ||¡ d S )Nr rg rh zPrefix.)Ú record_prefixzPrefix.0r rt r
r r r! Ú test_value_array_record_prefixâ s z0TestJSONNormalize.test_value_array_record_prefixc C sj dddddddddd dgd
|
d }t |d d gd}tddgddgdd ggddgd}t ||¡ d S )Nr- r+ r# r
|
r$ r& r' r( r) )r* r. ©r1 r/ r/ r. r r r% r rt r r r r! Ú test_nested_object_record_pathè s ýþþ þz0TestJSONNormalize.test_nested_object_record_pathc
|
C s t |ddgdddggd}dgd dgd d
|
d
|
d d d d d gdddddddgdddddddgd }t||jd }t ||¡ d S ) Nr r r r r| r rk r ri r r r r r r r r r r r r
|
r r r r r r )r z states.namer r r ©r r r ru rv )rx r" ry Úex_datarz r r r! Útest_more_deeply_nestedü s4
|
ÿù
|
ù ìz)TestJSONNormalize.test_more_deeply_nestedc C sÎ ddddiddddd dd
|
d dgd d dddiddddddgd g}t |dddddgg}ddd
|
ddgdgd d gd dddddgdgd dgd dd d ddgd}t||jd}t ||¡ d S ) Nr- r, r* r+ r# r
|
r$ r& r' r( r) )r1 r0 r/ r. r r6 r5 r2 r r3 r4 r. r1 r0 r/ ri rh )r r1 r0 ú info.governorr% r r )rx r ry r¡ rz r r r! Útest_shallow_nested s: ýü þüõ ÿ ûz%TestJSONNormalize.test_shallow_nestedc C s t |dgddddggdd}dd d
|
d d gd ddddgdgd dgd dgd dgd dgd dgd d}t|}t ||¡ d S )Nr. r1 r0 r/ r* Úignore©r r r} Úerrorsr# r& r( r2 r3 r
|
r' r) r r4 r- ri r rh r, r6 r+ r5 )r r% r1 r0 r£ rt )rx r7 ry r¡ rz r r r! Ú-test_nested_meta_path_with_nested_record_path> s ü ûz?TestJSONNormalize.test_nested_meta_path_with_nested_record_pathc C s| ddddddddgdg}d }t jt|d
|
t|d d d gd W 5 Q R X t|d d d gdd}dD ]}||ksftqfd S )NÚhelloÚthereÚ somethingÚelse©ÚfooÚbarÚ
|
something2Úelse2©r® r¯ r z?Conflicting metadata name (foo|bar), need distinguishing prefixr r r® r¯ r| r} ©r} Z meta_prefix©ZmetafooZmetabarr® r¯ )r r Ú
|
ValueErrorr r )rx r Úmsgry Úvalr r r! Útest_meta_name_conflictR s þýÿ z)TestJSONNormalize.test_meta_name_conflictc C s` ddddddddgdg}d d
|
g}t |d |d d }|d d
|
gksFtdD ]}||ksJtqJd S )Nr© rª r« r¬ r r° r± r² r® r¯ r r} r³ r´ )r r )rx r ÚCOLUMNSry r· r r r! Ú test_meta_parameter_not_modifiedg s þýÿ z2TestJSONNormalize.test_meta_parameter_not_modifiedc C s t |d d}t|d d }t ||¡ t |dddd}g }|D ]}| |d ¡ qBt|}|jdd d}t d d
|
g¡ d d g¡|d< t ||¡ d S ) Nr r. r1 Úcounty_)r} r c S s d| S )Nr» r )Úxr r r! Ú<lambda> ó z6TestJSONNormalize.test_record_prefix.<locals>.<lambda>r r- r ri rh ) r r ru rv r~ Úrenamer r r r r r r! Útest_record_prefix{ s ÿ z$TestJSONNormalize.test_record_prefixc C sR d d¡}d d¡ddgdddgdd d
|
gi}t|}tt |¡}t ||¡ d S ) NsK [{"Ãnicøde":0,"sub":{"A":1, "B":2}},{"Ãnicøde":1,"sub":{"A":3, "B":4}}]Úutf8s Ãnicøder rg zsub.Ari zsub.Brh rk )Údecoder r ÚjsonÚloadsru rv )rx ZtestjsonZtestdatarz ry r r r! Útest_non_ascii_key s ÿý ýz$TestJSONNormalize.test_non_ascii_keyc C sJ t |}tjtjtjtjtjdd dddddg}t|}t ||¡ d S )N)r/ úinfo.created_atúinfo.last_updatedúauthor_name.firstúauthor_name.last_namer8 r9 r; r< ©r r Únanr ru rv )rx rA ry r¡ rz r r r! Útest_missing_field s ûûøz$TestJSONNormalize.test_missing_fieldzmax_level,expectedrX rY rZ r[ rV rW r` )r] r^ rb rd )r] z UserField.IdzUserField.Namerb rd c C s~ ddiddddddddddgdd id
|
d d d dd gdg}t |dgdgdgg|d}t||jjd}t ||¡ d S )NrV rW rX rY rZ r[ r\ r_ r` r« r¬ r r° r± )rb rc rd Útagsrc rb rd )r r} Ú max_level)r r )r r r Úvaluesru Ú assert_equal)rx rÎ rz Ú
|
test_inputry Z expected_dfr r r! Ú test_max_level_with_records_path³ s, -þþû
|
þóÿ
|
üz2TestJSONNormalize.test_max_level_with_records_pathc C s: t dddiig}t ddddiigid}t ||¡ d S )Nr r rg Údummy)r ru rv )rx Zdf1Zdf2r r r! Ú!test_nested_flattening_consistentü s z3TestJSONNormalize.test_nested_flattening_consistentc C sF t d|ddddigdgdgd}tddidgd }t ||¡ d S )
|
NÚTexasr r- Úirh r/ r r )Úindex)r r ru rÐ )rx Z nulls_fixturery rz r r r! Útest_nonetype_record_path s þûz+TestJSONNormalize.test_nonetype_record_pathÚvalueÚfalseÚtruez{}Ú1z"text"c C s\ t |¡}d|d}d}| d| d| d}tjt|d t|g|gd W 5 Q R X d S ) NrÕ r r/ z has non list value z
|
for path z. Must be list or null.r r )rà rÄ r r Ú TypeErrorr )rx rÙ Z parsed_valuerÑ Z test_pathr¶ r r r! Ú test_non_list_record_path_errors s
|
|
ÿz2TestJSONNormalize.test_non_list_record_path_errorsc C sL d}t t |¡dgdgd}tdgdgtjdgtdd }t ||¡ d S )
|
Nz,[{"id": 99, "data": [{"one": 1, "two": 2}]}]r Úid)r r} rg rh éc )Zdtype)ZoneZtworß ) r rà rÄ r r r Úobjectru rv r r r r! Útest_meta_non_iterable s ÿz(TestJSONNormalize.test_meta_non_iterablec s6 fdd}t | }t d d }t ||¡ d S )Nc 3 s d d E d H d S )Nr r. r r ©r7 r r! Úgenerator_data+ s z8TestJSONNormalize.test_generator.<locals>.generator_datar r. rt )rx r7 rä ry rz r rã r! Útest_generator( s
|
z TestJSONNormalize.test_generatorc C sJ dddiddd}t |dd}tdddggd d
|
d gd }t ||¡ d S ) Nrq Úl3r )Za1Úl2rk )Z_idÚggr r rè Z_id_a1Z _id_l2_l3r rt r r r r! Ú'test_top_column_with_leading_underscore3 s z9TestJSONNormalize.test_top_column_with_leading_underscore) Ú__name__Ú
|
__module__Ú __qualname__r{ r r r ÚmarkÚ parametrizeÚNotImplementedErrorr r r r r r¢ r¤ r¨ r¸ rº rÀ rÅ rÌ rÒ rÔ rØ rÞ râ rå ré r r r r! rf s üþ
|
$üüùþûûøþîþ)
|
|
|
rf c @ sÚ e Zd Zdd Zdd Zdd Zdd Zd d
|
Zd d Zd d Z dd Z
|
dd Z dd Z dd Z ej dddddd d d gfd d diddd d!d"d#d id$gfd%dddd d!d d&gfg¡d'd( Zd)d* Zd+d, ZdS )-ÚTestNestedToRecordc C s0 ddddddg}t |}|}||ks,td S )Nrg rh )Úflat1Úflat2ri rk )Zflat3rò ©r r rw r r r! Útest_flat_stays_flat= s z'TestNestedToRecord.test_flat_stays_flatc C s4 ddddd}t |}dddd}||ks0td S )Nrg rh ©rj Úd)rñ Údict1)údict1.cúdict1.drñ ró r r r r! Ú test_one_level_deep_flattensC s z/TestNestedToRecord.test_one_level_deep_flattensc C sH dddddddddd}t |}ddddddd}||ksDtd S )Nrg rh rõ )Úerö )rñ r÷ Únested)rø rù rñ znested.dz
|
nested.e.cz
|
nested.e.dró r r r r! Útest_nested_flattensK s ýú z'TestNestedToRecord.test_nested_flattensc C s2 d}t jt|d t|dddd W 5 Q R X d S )Nú^Key 'name' not found. To replace missing values of 'name' with np.nan, pass in errors='ignore'r rM r Úraiser¦ ©r r ÚKeyErrorr ©rx rU r¶ r r r! Útest_json_normalize_errors^ s ÿüz-TestNestedToRecord.test_json_normalize_errorsc C s^ t |dddd}ddddd d
|
gd d d ddtjgg}ddddddg}t||d}t ||¡ d S )NrM r r¥ r¦ rC rD rE r6 rF rB rO rP rQ rR rS rG rH rI r1 rJ r rÊ ©rx rU ry r¡ r rz r r r! Útest_missing_metao s ÿþ z$TestNestedToRecord.test_missing_metac C sª dd ddiddigd}t |dddd ggd
|
d }ddtjgddtjgg}ddd g}t||d d ti¡}t ||¡ tj t
|
dd t |dddd ggdd W 5 Q R X d S )Nr® r rg rh )r} Ú nested_metarÙ rÙ r} r Zleafr¥ )r r} r§ znested_meta.leafr z'leaf' not foundr rÿ ) r r rË r Zastyperá ru rv r r r )rx r ry r¡ r rz r r r! Útest_missing_nested_meta~ s(
|
ü
|
ÿ
|
üz+TestNestedToRecord.test_missing_nested_metac C s6 d}t jt|d t|ddgddd W 5 Q R X d S )Nrþ r rN r r rÿ r¦ r r r r r! Ú5test_missing_meta_multilevel_record_path_errors_raise s ÿüzHTestNestedToRecord.test_missing_meta_multilevel_record_path_errors_raisec C sJ t |ddgddd}ddgdtjgg}d dg}t||d
|
}t ||¡ d S ) NrN r r r¥ r¦ rL rB rT rK r rÊ r r r r! Ú6test_missing_meta_multilevel_record_path_errors_ignore© s üþ zITestNestedToRecord.test_missing_meta_multilevel_record_path_errors_ignorec C sX d ddddddddd ddg}t |}d ddd
|
dd ddd g}||ksTtd S ) NZSmithZ Appleseedr= r@ r8 r9 r: r; r< )r/ rÈ rÉ )rÈ rÉ rÆ rÇ ró r r r r! Útest_donot_drop_nonevalues» s" þþýüúz-TestNestedToRecord.test_donot_drop_nonevaluesc
|
C sP d ddd d d dddddiid}t |}d d d d dddd }||ksLtd S )
|
Nr r1 ç gH@ç À@Àç ;@)rß Úregionr¼ ÚyÚz©rß z town.info©rß Úlocation)rß úlocation.country.state.idz#location.country.state.town.info.idú'location.country.state.town.info.regionú"location.country.state.town.info.xú"location.country.state.town.info.yú"location.country.state.town.info.zró r r r r! Ú$test_nonetype_top_level_bottom_levelÕ s0 ûþÿÿþù z7TestNestedToRecord.test_nonetype_top_level_bottom_levelc C sT d d d d d dddddddd}t |}d d d d d dddd }||ksPtd S )
|
Nr r r )r r¼ r r r )rß r1 )rß r r )rß z location.idzlocation.country.idr r r r r ró r r r r! Ú test_nonetype_multiple_levelsõ s0 üþþþþø
|
z0TestNestedToRecord.test_nonetype_multiple_levelszmax_level, expectedNrW rX rY rZ r` )úCreatedBy.NameúLookup.TextFieldzLookup.UserField.IdzLookup.UserField.NameúImage.ar rV r[ r\ r_ ra rg )r r zLookup.UserFieldr c C s t ||d}||kstd S )N©rÎ ró )rx rÎ rz re Úoutputr r r! Útest_with_max_level s + z&TestNestedToRecord.test_with_max_levelc C sh d}dddddddd d
|
d d id dd idiig}dddd d
|
d dg}t ||d}||ksdtd S )Néd rb ÚuserZLeoZThomson)Z firstnameZLastNameÚfatherZ Father001Z Father002Z Father003rV Z Father004)r r# )rV r# )r Z family_tree)z CreatedBy.user.name.firstnamez CreatedBy.user.name.LastNamez&CreatedBy.user.family_tree.father.namez-CreatedBy.user.family_tree.father.father.Namez4CreatedBy.user.family_tree.father.father.father.namez;CreatedBy.user.family_tree.father.father.father.father.Namer ró )rx rÎ Z
|
input_datarz r r r r! Útest_with_large_max_levelE s6 þþþÿþÿÿÿúÿ
|
z,TestNestedToRecord.test_with_large_max_levelc C s ddddidddddiddd d
|
didd }t |}dddg|_t|}tdddgddd gd tjtjgtjd tjgtjtjdgd}t ||¡ d S )Nrg ZFoor_ )rß r Úelementsrh ÚBarr` ri ZBazrj )r rg rh g ð?g @g @)rß r z
|
elements.az
|
elements.bz
|
elements.c)r r× r r r rË ru rv )rx r Úsry rz r r r! Útest_series_non_zero_indexj s ý ûÿ z-TestNestedToRecord.test_series_non_zero_index)rê rë rì rô rú rý r r r r r r
|
r r r rí rî r r$ r( r r r r! rð < sX "ûÿþ þúÿþüÿþæþ)
|
%rð )rà Únumpyr r Zpandasr r r r Zpandas._testingZ_testingru Zpandas.io.json._normalizer Zfixturer" r7 rA rU re rf rð r r r r! Ú<module> s(
|
&
|
|
|
|
|
3
|
