U
|
ý°dD' ã @ s8 d Z ddlmZ ddlZddlZddlZddlZddlZ ddl
|
Z
|
ddl
|
m Z m Z m Z mZmZmZmZmZmZmZmZmZmZmZ ddlmZ ddlm Z m Z m Z m Z m Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/ dd Z0d d
|
Z1d d Z2d d Z3dd Z4dd Z5dd Z6dd Z7e8dkr4e7 dS )a
|
self-contained to write legacy storage pickle files
|
|
To use this script. Create an environment where you want
|
generate pickles, say its for 0.20.3, with your pandas clone
|
in ~/pandas
|
|
. activate pandas_0.20.3
|
cd ~/pandas/pandas
|
|
$ python -m tests.io.generate_legacy_storage_files tests/io/data/legacy_pickle/0.20.3/ pickle
|
|
This script generates a storage file for the current arch, system,
|
and python version
|
pandas version: 0.20.3
|
output dir : pandas/pandas/tests/io/data/legacy_pickle/0.20.3/
|
storage format: pickle
|
created pickle file: 0.20.3_x86_64_darwin_3.5.2.pickle
|
|
The idea here is you are using the *current* version of the
|
generate_legacy_storage_files with an *older* version of pandas to
|
generate a pickle file. We will then check this file into a current
|
branch, and test using test_pickle.py. This will load the *older*
|
pickles and test versus the current data that is generated
|
(with main). These are then compared.
|
|
If we have cases where we changed the signature (e.g. we renamed
|
offset -> freq in Timestamp). Then we have to conditionally execute
|
in the generate_legacy_storage_files.py to make it
|
run under the older AND the newer version.
|
|
é )Ú timedeltaN)Ú CategoricalÚ DataFrameÚIndexÚ
|
MultiIndexÚNaTÚPeriodÚ
|
RangeIndexÚSeriesÚ TimestampÚ bdate_rangeÚ
|
date_rangeÚinterval_rangeÚ period_rangeÚtimedelta_range)Ú SparseArray)ÚFY5253Ú BusinessDayÚ BusinessHourÚCustomBusinessDayÚ
|
DateOffsetÚDayÚEasterÚHourÚLastWeekOfMonthÚMinuteÚ
|
MonthBeginÚMonthEndÚ QuarterBeginÚ
|
QuarterEndÚSemiMonthBeginÚ SemiMonthEndÚWeekÚ WeekOfMonthÚ YearBeginÚYearEndc C sH t j} t jdt jd}| |dd
< | |dd
< tt|dd}d|_|S ) Né ©Zdtypeé é éÿÿÿÿÚblock©ÚkindÚbseries)ÚnpÚnanÚarangeÚfloat64r
|
r Úname)r0 Úarrr. © r5 údd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/tests/io/generate_legacy_storage_files.pyÚ_create_sp_seriesV s r7 c C s\ t j} t jdt jd}| |dd
< | |dd
< tdt|d}tt|dd |d
|
}d |_|S ) Nr& r' r( r) r* ú1/1/2011©Úperiodsr+ r, ©ÚindexZ btsseries) r/ r0 r1 r2 r Úlenr
|
r r3 )r0 r4 Z
|
date_indexr. r5 r5 r6 Ú_create_sp_tsseriesc s r> c C s| t j} | | | dddddddg
|
ddd| | | ddddg
|
t d¡ t j¡dddddd| | | | g
|
d }td
|
dd }t||d t¡S ) Nr é é é é é é é
|
©ÚAÚBÚCÚDr8 r9 r; ) r/ r0 r1 ÚastypeÚint64r r Úapplyr )r0 ÚdataÚdatesr5 r5 r6 Ú_create_sp_frameq s ü rP c s ddddt jgdddddgddd d
|
d gtd d dddtdddgd tdtddd} tt d¡tdddtddddtt jdt jdtt jdt j dt
|
ddddd }t d|d < t ddd|d < d t jttd d d!d!ddd"d"gd#d$d#d$d#d$d#d$gg d%d&gd'i}t d( t d) t d* tt d¡ t j¡tdddd+tt d ¡ t j¡t jttddd,d,d,gd-d.d-d.d gg d#d$gd'd+tt d ¡ t j¡d(d)d/d0d(gd+ttdd d!gttdd dttdd d1d2ttd3gd d4
|
}t }td5|_t|d6 |d6 d d7t|d8 |d8 d d7t fd9d:d;D tt d ¡ t j¡t d ¡ t j¡d7t jttd d d!d!d!gd#d$d#d$d<gg d%d&gd'd+tt d=¡ d d-¡ t j¡d(d)d(gd>td(tdd gittdd d!gt d-¡ t j¡d7|ttd?d1d@tdAdBd@d7td d+ttd?d1d@tdAdBd@tdAdCd@dDtd d+dE
|
}ttdFtt dG¡tt dH¡dI}tdJttdJd1d@dK}tddLtdMdNdOt t dPdQdRt dSdMdTdUt dVdWt dPdXt!dYdXt"dt#dt$dt%dt&dt'dt(dt)dt)d,dZdd[t*d-d.d\t+dd-d]t,d,dMd^d_d`t- t.dt/dda}|||| |t0 t1 dbd6t2 i|||dc
|
S )dzcreate the pickle datag g ð?g @g @r r? Zfoo1Zfoo2Zfoo3Zfoo4Zfoo5z1/1/2009rC r9 Z20100101Zfoo)rG rH rI rJ ÚEZ20130101Z2012ÚM)Ú timestampÚperiodrE z
|
2013-01-01)Úfreqr: r' z00:00:00Z30T)ÚintÚdaterT ÚfloatZuintr ÚrangeÚintervalZreg2ÚbarZbazZquxZoneZtwoÚfirstÚsecond)ÚnamesrG rH rQ r; r@ rA rB rI rJ z
|
US/Eastern)r: ÚtzZ2000Q1)
|
rX rV ÚmixedÚtsÚmiÚdupÚcatÚdtZdt_tzrT ZABCDArX )rG rH rV c s i | ]}| | qS r5 r5 )Ú.0Úk©rN r5 r6 Ú
|
<dictcomp>¿ s z create_data.<locals>.<dictcomp>rF Zthreer& )ÚcolumnsZ20130102)r_ Z20130603ZCETÚUTC)rG rH rI )
|
rX rV r` rb rc Z
|
cat_onecolZ cat_and_floatZ mixed_dupZ dt_mixed_tzsZ dt_mixed2_tzsZabcdefgiè i' )Zint8Úint16Úint32z
|
2011-01-01)ÚnormalZnatr_ )ZyearsrD iÀ )ÚhourÚ nanosecondsé )Úseconds)ÚoffsetTz15:14)Ú normalizeÚnÚendzMon Fri)Zweekmask)Z day_of_monthé F)rt Úweekday)Úweekrx )ru rx r( Úlast)ru rx Z startingMonthZ variation)r ZDateOffset_h_nsr r r r r! r r r r r r$ r% r" Z Week_Tuesr# r r r r r )rX ra )
|
ÚseriesÚframer< Úscalarsrb Z sp_seriesZsp_framerd rS Úoffsets)3r/ r0 r r r r r1 r r2 Zuint64r r r r Ú from_tuplesÚtupleÚzipr
|
rK rL r r Úlistrj ZreshaperY r r r r r r r r! r r r r r r$ r% r" r# r r r r r r7 r> rP ) r} r< rb r{ Z mixed_dup_dfr| rd rS Úoffr5 rh r6 Ú create_data s
|
|
û
|
ú þÿÿ÷ÿ
|
|
|
|
ÿ" ÿþ"ï
|
$ þÿÿ÷þ ÿ þÿ
|
|
þû
|
|
|
ýúÜ/
|
ý
|
ý
|
|
|
ê ör c C s
|
t } | S )N)r rh r5 r5 r6 Úcreate_pickle_data s r
c C s4 d ttjtt ¡ tt ¡ ¡ tt ¡ g¡S )NÚ_) ÚjoinÚstrÚpandasÚ __version__ÚplÚmachineÚsystemÚlowerÚpython_versionr5 r5 r5 r6 Ú platform_name# s
|
|
üÿr c C s t j}td td| td| td t d}ttj | |¡d}t t
|
|tj ¡ W 5 Q R X td| d S )NzUThis script generates a storage file for the current arch, system, and python versionz pandas version: z output dir : z storage format: picklez.pickleÚwbzcreated pickle file: ) r r Úprintr ÚopenÚosÚpathr ÚpickleÚdumpr
ÚDEFAULT_PROTOCOL)Ú
|
output_dirÚversionÚpthÚfhr5 r5 r6 Úwrite_legacy_pickles. s ÿ r c C sr t j dd¡ dtt j kr(dks4n t d¡ tt jd } tt jd }|dkrdt| d n
|
t d
|
¡ d S ) Nr Ú.rA rB zhSpecify output directory and storage type: generate_legacy_storage_files.py <output_dir> <storage_type> r? r@ r )r z&storage_type must be one of {'pickle'})Úsysr Úinsertr= ÚargvÚexitr r )r Z storage_typer5 r5 r6 Úwrite_legacy_fileA s ÿ r£ Ú__main__)9Ú__doc__Údatetimer r r Úplatformr r Únumpyr/ r r r r r r r r r
|
r r r r r r Z pandas.arraysr Zpandas.tseries.offsetsr r r r r r r r r r r r r r r r! r" r# r$ r% r7 r> rP r r
r r r£ Ú__name__r5 r5 r5 r6 Ú<module> s* " @ X
|
|
