U
|
¬ý°dú ã @ sZ U d Z ddlmZ ddlmZ ddlZddlmZ ddlZddl Z ddl
|
Z
|
ddl m Z ddl mZmZmZmZmZmZmZmZmZ ddlZddlmZ ddlZdd l m Z dd
|
l m Z dd l m!Z!m"Z"m#Z#m$Z$m%Z% dd l&m'Z'm(Z(m)Z)m*Z* dd l+m,Z,m-Z- ddl.m/Z/ ddl0m1Z1m2Z2m3Z3m4Z4 ddl5m6Z6m7Z7m8Z8m9Z9m:Z:m;Z;m<Z< ddl=m>Z> ddl?m@Z@ ddlAmBZB ddlCmDZD ddlEmFZF ddlGmHZH ddlImJZJ er¨ddl mKZK dZLdZMdZNd ZOd ZPd ZQd eM d eN d eO d eP d eHd! d" d eHd# d$eQ d%ZRd&eM d eN d'ZSd(eM d eN d eO d eHd! d eHd# d)eQ d ZTd*d+d,d-d.d/d0d1d2g ZUe d3d4d4¡ZVd5eWd6< d7d8d9d:ZXd7d;d7d<d=d>ZYd?ZZd5eWd@< dAZ[d5eWdB< dCZ\d5eWdD< dEZ]d5eWdF< dGZ^d5eWdH< dIdIdJdKdLZ_G dMdN dNZ`G dOdP dPe`ZaG dQdR dRZbG dSdT dTZcG dUdV dVecejdZee,eRdWdWddXdWddWddXdYddZ d[d\d\d]d\d\d^d\d_d\d`dadbdc dddeZfd;d;dfdgdhZgdidjdidkdldmZhd;dndodpdqZidrdsdrdtdudvZjdnd7djdwdxdyZkdd7djd\d;d{d|d}Zle-eHd# eHd~ d dG dd decZmdnd7d\djddd
ZnddjddkddZoG dd dZpG dd demZqG dd deqZrdS )a°
|
Module contains tools for processing Stata files into DataFrames
|
|
The StataReader below was originally written by Joe Presbrey as part of PyDTA.
|
It has been extended and improved by Skipper Seabold from the Statsmodels
|
project who also developed the StataWriter and was finally added to pandas in
|
a once again improved version.
|
|
You can find more information on http://presbrey.mit.edu/PyDTA and
|
https://www.statsmodels.org/devel/
|
é )Ú annotations)ÚabcN)ÚBytesIO)Ú TracebackType) ÚIOÚ TYPE_CHECKINGÚAnyÚAnyStrÚCallableÚFinalÚHashableÚSequenceÚcast)Ú relativedelta)Ú infer_dtype)Úmax_len_string_array)ÚCompressionOptionsÚFilePathÚ
|
ReadBufferÚStorageOptionsÚ WriteBuffer)Ú CategoricalConversionWarningÚInvalidColumnNameÚPossiblePrecisionLossÚValueLabelTypeMismatch)ÚAppenderÚdoc)Úfind_stack_level)Ú ensure_objectÚis_categorical_dtypeÚis_datetime64_dtypeÚis_numeric_dtype)Ú CategoricalÚ DatetimeIndexÚNaTÚ TimestampÚisnaÚ to_datetimeÚ to_timedelta)Ú BooleanDtype)Ú IntegerDtype)Ú DataFrame)ÚIndex)ÚSeries)Ú _shared_docs)Ú
|
get_handle)ÚLiteralzîVersion of given Stata file is {version}. pandas supports importing versions 105, 108, 111 (Stata 7SE), 113 (Stata 8/9), 114 (Stata 10/11), 115 (Stata 12), 117 (Stata 13), 118 (Stata 14/15/16),and 119 (Stata 15/16, over 32,767 variables).zÌconvert_dates : bool, default True
|
Convert date variables to DataFrame time values.
|
convert_categoricals : bool, default True
|
Read value labels and convert columns to Categorical/Factor variables.a index_col : str, optional
|
Column to set as index.
|
convert_missing : bool, default False
|
Flag indicating whether to convert missing values to their Stata
|
representations. If False, missing values are replaced with nan.
|
If True, columns containing missing values are returned with
|
object data types and missing values are represented by
|
StataMissingValue objects.
|
preserve_dtypes : bool, default True
|
Preserve Stata datatypes. If False, numeric data are upcast to pandas
|
default types for foreign data (float64 or int64).
|
columns : list or None
|
Columns to retain. Columns will be returned in the given order. None
|
returns all columns.
|
order_categoricals : bool, default True
|
Flag indicating whether converted categorical data are ordered.zzchunksize : int, default None
|
Return StataReader object for iterations, returns chunks with
|
given number of lines.z=iterator : bool, default False
|
Return StataReader object.zúNotes
|
-----
|
Categorical variables read through an iterator may not have the same
|
categories and dtype. This occurs when a variable stored in a DTA
|
file is associated to an incomplete set of value labels that only
|
label a strict subset of the values.a>
|
Read Stata file into DataFrame.
|
|
Parameters
|
----------
|
filepath_or_buffer : str, path object or file-like object
|
Any valid string path is acceptable. The string could be a URL. Valid
|
URL schemes include http, ftp, s3, and file. For file URLs, a host is
|
expected. A local file could be: ``file://localhost/path/to/table.dta``.
|
|
If you want to pass in a path object, pandas accepts any ``os.PathLike``.
|
|
By file-like object, we refer to objects with a ``read()`` method,
|
such as a file handle (e.g. via builtin ``open`` function)
|
or ``StringIO``.
|
Ú
|
Zdecompression_optionsÚfilepath_or_bufferÚstorage_optionszª
|
|
Returns
|
-------
|
DataFrame or StataReader
|
|
See Also
|
--------
|
io.stata.StataReader : Low-level reader for Stata data files.
|
DataFrame.to_stata: Export Stata data files.
|
|
a
|
|
Examples
|
--------
|
|
Creating a dummy stata for this example
|
|
>>> df = pd.DataFrame({'animal': ['falcon', 'parrot', 'falcon', 'parrot'],
|
... 'speed': [350, 18, 361, 15]}) # doctest: +SKIP
|
>>> df.to_stata('animals.dta') # doctest: +SKIP
|
|
Read a Stata dta file:
|
|
>>> df = pd.read_stata('animals.dta') # doctest: +SKIP
|
|
Read a Stata dta file in 10,000 line chunks:
|
|
>>> values = np.random.randint(0, 10, size=(20_000, 1), dtype="uint8") # doctest: +SKIP
|
>>> df = pd.DataFrame(values, columns=["i"]) # doctest: +SKIP
|
>>> df.to_stata('filename.dta') # doctest: +SKIP
|
|
>>> with pd.read_stata('filename.dta', chunksize=10000) as itr: # doctest: +SKIP
|
>>> for chunk in itr:
|
... # Operate on a single chunk, e.g., chunk.mean()
|
... pass # doctest: +SKIP
|
zReads observations from Stata file, converting them into a dataframe
|
|
Parameters
|
----------
|
nrows : int
|
Number of lines to read from data file, if None read whole file.
|
z
|
|
Returns
|
-------
|
DataFrame
|
zæClass for reading Stata dta files.
|
|
Parameters
|
----------
|
path_or_buf : path (string), buffer or path object
|
string, path object (pathlib.Path or py._path.local.LocalPath) or object
|
implementing a binary read() functions.
|
z
|
|
ú%tcú%tCú%tdú%dú%twú%tmú%tqú%thú%tyé¨ é r Ú stata_epochr- ©Úreturnc sl t jjt jj t jt ddd¡ j t jt ddd¡ jd d d d d d ddfdd }ddfd
|
d }dd fd d }t | ¡}d}| ¡ rÈd}| ¡ d| |< |
|
tj ¡} | d¡röt }| }|||d} nd| d¡r0tjdt d t| td} |r,t| |< | S | d¡rTt }| }
|
|||
|
d} n| d¡rt j| d } | d d }
|
|| |
|
} nÔ| d ¡r¸t j| d } | d d } || | } n¢| d ¡rît j| d } | d d d } || | } nl| d!¡r$t j| d" } | d" d# d } || | } n6| d$¡rJ| } t | ¡}|| |} ntd%| d&|rht| |< | S )'a
|
Convert from SIF to datetime. https://www.stata.com/help.cgi?datetime
|
|
Parameters
|
----------
|
dates : Series
|
The Stata Internal Format date to convert to datetime according to fmt
|
fmt : str
|
The format to convert to. Can be, tc, td, tw, tm, tq, th, ty
|
Returns
|
|
Returns
|
-------
|
converted : Series
|
The converted dates
|
|
Examples
|
--------
|
>>> dates = pd.Series([52])
|
>>> _stata_elapsed_date_to_datetime_vec(dates , "%tw")
|
0 1961-01-01
|
dtype: datetime64[ns]
|
|
Notes
|
-----
|
datetime/c - tc
|
milliseconds since 01jan1960 00:00:00.000, assuming 86,400 s/day
|
datetime/C - tC - NOT IMPLEMENTED
|
milliseconds since 01jan1960 00:00:00.000, adjusted for leap seconds
|
date - td
|
days since 01jan1960 (01jan1960 = 0)
|
weekly date - tw
|
weeks since 1960w1
|
This assumes 52 weeks in a year, then adds 7 * remainder of the weeks.
|
The datetime value is the start of the week in terms of days in the
|
year, not ISO calendar weeks.
|
monthly date - tm
|
months since 1960m1
|
quarterly date - tq
|
quarters since 1960q1
|
half-yearly date - th
|
half-years since 1960h1 yearly
|
date - ty
|
years since 0000
|
r= r> é i éè r- r@ c sX | ¡ k r,| ¡ kr,td| | ddS t| dd}tdd t| |D |dS dS ) zú
|
Convert year and month to datetimes, using pandas vectorized versions
|
when the date range falls within the range supported by pandas.
|
Otherwise it falls back to a slower but more robust method
|
using datetime.
|
éd z%Y%m©ÚformatÚindexNc S s g | ]\}}t ||d ¡qS ©r> )Údatetime)Ú.0ÚyÚm© rM úFd:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/io/stata.pyÚ
|
<listcomp>* s zX_stata_elapsed_date_to_datetime_vec.<locals>.convert_year_month_safe.<locals>.<listcomp>©rG )ÚmaxÚminr' Úgetattrr- Úzip)ÚyearÚmonthrG ©ÚMAX_YEARÚMIN_YEARrM rN Úconvert_year_month_safe s ÿzD_stata_elapsed_date_to_datetime_vec.<locals>.convert_year_month_safec sd | ¡ d k r4| ¡ kr4t| ddt|dd S t| dd}dd t| |D }t||d
|
S dS ) z{
|
Converts year (e.g. 1999) and days since the start of the year to a
|
datetime or datetime64 Series
|
r> ú%YrE Úd©ÚunitrG Nc S s, g | ]$\}}t |d d ¡tt|d qS )r> ©Údays)rI r Úint)rJ rK r\ rM rM rN rO 6 s ÿzW_stata_elapsed_date_to_datetime_vec.<locals>.convert_year_days_safe.<locals>.<listcomp>rP )rQ rR r' r( rS rT r- )rU r` rG ÚvaluerW rM rN Úconvert_year_days_safe- s þzC_stata_elapsed_date_to_datetime_vec.<locals>.convert_year_days_safec s° t |dd}|dkrL| ¡ ks,| ¡ k r fdd|D }t||dS nH|dkr| ¡ ksl| ¡ k r fdd|D }t||dS ntd t t||d
|
} | S ) z¾
|
Convert base dates and deltas to datetimes, using pandas vectorized
|
versions if the deltas satisfy restrictions required to be expressed
|
as dates in pandas.
|
rG Nr\ c s g | ]} t t|d qS )r_ ©r ra ©rJ r\ ©ÚbaserM rN rO E s zS_stata_elapsed_date_to_datetime_vec.<locals>.convert_delta_safe.<locals>.<listcomp>rP Úmsc s" g | ]} t t|d d qS )rC )Ú microsecondsrd re rf rM rN rO I s zformat not understoodr] )rS rQ rR r- Ú
|
ValueErrorr' r( )rg Zdeltasr^ rG Úvalues)Ú MAX_DAY_DELTAÚ MAX_MS_DELTAÚ MIN_DAY_DELTAÚ MIN_MS_DELTArf rN Úconvert_delta_safe< s
|
ÿ z?_stata_elapsed_date_to_datetime_vec.<locals>.convert_delta_safeFTg ð?©r4 Útcrh ©r5 ZtCz9Encountered %tC format. Leaving in Stata Internal Format.©Ú
|
stacklevel©Údtype)r6 Útdr7 r\ r\ ©r8 Útwé4 é ©r9 Útmé ©r: Útqé é ©r; Úthé é ©r< Útyz Date fmt ú not understood)r% rR rU rQ rI r` ÚnpÚisnanÚanyZ _reset_cacherÚastypeÚint64Ú
|
startswithr? ÚwarningsÚwarnr r- Úobjectr$ Z ones_likerj )ÚdatesÚfmtrZ rc rp Zbad_locsZhas_bad_valuesrg rh Ú
|
conv_datesr` rU rV Z quarter_monthZ first_monthrM )rl rm rX rn ro rY rN Ú#_stata_elapsed_date_to_datetime_vecê sp .
|
|
þ
|
r Ústr)r r rA c s | j d d d%dddd fdd }t| }| j | ¡ rjt| } t| rbtt| |< nt| |< |dkr|| d d
|
}|jd }nD|d kr¬tj d t
|
d | }n$|dkrÎ|| d d
|
}|j }n|dkr|| d d d}d|j tj |j d }nÎ|dkr4|| d d}d|j tj |j d }n|dkrj|| d d}d|j tj |j d d }nf|dkr¢|| d d}d|j tj |j d k t¡ }n.|d krÀ|| d d}|j }ntd | d t|tjd }t d!d"¡d# }|||< t|d$S )&aX
|
Convert from datetime to SIF. https://www.stata.com/help.cgi?datetime
|
|
Parameters
|
----------
|
dates : Series
|
Series or array containing datetime.datetime or datetime64[ns] to
|
convert to the Stata Internal Format given by fmt
|
fmt : str
|
The format to convert to. Can be, tc, td, tw, tm, tq, th, ty
|
l "R: rC FÚbool)ÚdeltarU r` c s\ i }t | jr|r:| tt d¡ }|j tj¡d |d< |sB|rbt | }|j
|
j |d< |j
|
j |d< |r| tj¡t |d dd tj¡ }| |d< n¸t| d d
|
d krH|râ| jt }d d dfdd }t |¡} | ||d< |r| dd ¡}
|
|
|
jd |d< |
|
j|d d |d< |rPddddd} t | ¡} | | |d< ntdt|dS )NÚnsrC r rU rV r[ rE r` F©ZskipnarI zdatetime.timedeltaÚfloat©ÚxrA c s | j d| j | j S )Ni@B )r` Úsecondsri ©r )Ú
|
US_PER_DAYrM rN Úf¶ s zC_datetime_to_stata_elapsed_vec.<locals>.parse_dates_safe.<locals>.fc S s d| j | j S )NrD )rU rV r¡ rM rM rN Ú<lambda>¼ ó zJ_datetime_to_stata_elapsed_vec.<locals>.parse_dates_safe.<locals>.<lambda>rD zdatetime.datetimera c S s | t | jdd¡ jS )Nr> )rI rU r` r¡ rM rM rN ÚgÁ s zC_datetime_to_stata_elapsed_vec.<locals>.parse_dates_safe.<locals>.gzZColumns containing dates must contain either datetime64, datetime.datetime or null values.rP )r rw r% r? Zas_unitÚ_valuesÚviewr r r# Ú_datarU rV r' r Z vectorizeÚapplyrj r+ ) r r rU r` r\ Z
|
time_deltaZ
|
date_indexZ
|
days_in_nsr£ ÚvZ
|
year_monthr¦ ©Z
|
NS_PER_DAYr¢ rG rM rN Úparse_dates_safe sF
|
ÿþ
|
|
|
ÿz8_datetime_to_stata_elapsed_vec.<locals>.parse_dates_saferq T)r rs z'Stata Internal Format tC not supported.rt )r6 rx ry )rU r` r{ r| r} )rU r r> r r r r r r r úFormat z! is not a known Stata date formatrv ú<dó àr rP )FFF)rG r& r r- r r' r? r r r r rU r` rV r ra rj r Úfloat64ÚstructÚunpack)r r r Zbad_locr\ r Ú missing_valuerM r¬ rN Ú _datetime_to_stata_elapsed_vec s\ ÿ . þ
|
|
|
|
"
|
rµ zÔ
|
Fixed width strings in Stata .dta files are limited to 244 (or fewer)
|
characters. Column '{0}' does not satisfy this restriction. Use the
|
'version=117' parameter to write the newer (Stata 13 and later) format.
|
Ú excessive_string_length_errorz
|
Column converted from {0} to {1}, and some data are outside of the lossless
|
conversion range. This may result in a loss of precision in the saved data.
|
Úprecision_loss_doczä
|
Stata value labels (pandas categories) must be strings. Column {0} contains
|
non-string labels which will be converted to strings. Please check that the
|
Stata data file created has not lost information due to duplicate labels.
|
Úvalue_label_mismatch_doca;
|
Not all pandas column names were valid Stata variable names.
|
The following replacements have been made:
|
|
{0}
|
|
If this is not what you expect, please make sure you have Stata-compliant
|
column names in your DataFrame (strings only, max 32 characters, only
|
alphanumerics and underscores, no Stata reserved words)
|
Úinvalid_name_docaï
|
One or more series with value labels are not fully labeled. Reading this
|
dataset with an iterator results in categorical variable with different
|
categories. This occurs since it is not possible to know all possible values
|
until the entire dataset has been read. To avoid this warning, you can either
|
read dataset without an iterator, or manually convert categorical data by
|
``convert_categoricals`` to False and then accessing the variable labels
|
through the value_labels method of the reader.
|
Ú categorical_conversion_warningr+ ©ÚdatarA c C s d}t jt jt jft jt jt jft jt jt jft jt jt jft j t jt j
|
ff}t dd¡d }t dd¡d }| D ]ü}t | | jttf}| | }| ¡ }|rô| | ¡ } | ¡ rÚt | | jtrÈdnd}
|
|
|
| j| |f< | | | | jj¡| |< | | j} |D ]} | | d kr| | ¡ t | d ¡jkr<| d } n| d } | d t jkrr| | ¡ d
|
krrt d d ¡}| | | ¡| |< q| t jkrÐ| | ¡ d ks¸| | ¡ dk rD| | t j¡| |< nt| t jkr| | ¡ dks | | ¡ dk rD| | t j¡| |< n,| t jkr¤| | ¡ dkr^| | ¡ dkr^| | t j¡| |< nD| | t j
|
¡| |< | | ¡ d
|
ks| | ¡ dkrDt dd ¡}n | t jt j
|
fkrDt | | ¡ ¡ rÚt d| d| | ¡ } | t jkr| |kr| | t j
|
¡| |< n2| t j
|
krD| |krDt d| d| d| d|rr| ¡ rrt j | | jj }|| j||f< qr|rt!j"|t#t$ d | S )a-
|
Checks the dtypes of the columns of a pandas DataFrame for
|
compatibility with the data types and ranges supported by Stata, and
|
converts if necessary.
|
|
Parameters
|
----------
|
data : DataFrame
|
The DataFrame to check and convert
|
|
Notes
|
-----
|
Numeric columns in Stata must be one of int8, int16, int32, float32 or
|
float64, with some additional value restrictions. int8 and int16 columns
|
are checked for violations of the value restrictions and upcast if needed.
|
int64 data is not usable in Stata, and so it is downcast to int32 whenever
|
the value are in the int32 range, and sidecast to float64 when larger than
|
this range. If the int64 values are outside of the range of those
|
perfectly representable as float64 values, a warning is raised.
|
|
bool columns are cast to int8. uint columns are converted to int of the
|
same size if there is no loss in precision, otherwise are upcast to a
|
larger type. uint64 is currently not supported since it is concerted to
|
object in a DataFrame.
|
Ú ú<fó ÿÿÿ~r r¯ ó ÿÿÿÿÿÿßFr> r l Úuint64r± rD éÿÿÿéä éÿÿéäÿÿé lüÿÿÿ r zColumn zM contains infinity or -infinitywhich is outside the range supported by Stata.z has a maximum value (z() outside the range supported by Stata (ú)rt )%r Zbool_Úint8Úuint8Úint16Zuint16Úint32Zuint32r rÁ r± r² r³ Ú
|
isinstancerw r* r) r& r Úlocr Z numpy_dtyperQ Ziinfor· rF rR Úfloat32Úisinfrj ÚStataMissingValueÚBASE_MISSING_VALUESÚnamer r r r )r¼ ÚwsZconversion_dataÚ float32_maxÚ float64_maxÚcolZis_nullable_intÚorigZ orig_missingÚ missing_locZfvrw Zc_datarb ÚsentinelrM rM rN Ú_cast_to_stata_types' s ûú
|
|
|
$ $ $$
|
ÿ
|
ÿýrÚ c @ s< e Zd ZdZdddddddZd d
|
Zd d d ddZdS )ÚStataValueLabelzé
|
Parse a categorical column and prepare formatted output
|
|
Parameters
|
----------
|
catarray : Series
|
Categorical Series to encode
|
encoding : {"latin-1", "utf-8"}
|
Encoding to use for value labels.
|
úlatin-1r- ú Literal[('latin-1', 'utf-8')]ÚNone)ÚcatarrayÚencodingrA c C s^ |dkrt d|j| _|| _|jj}ttt t
|
|¡|| _ | j j dd d | ¡ d S )N©rÜ úutf-8ú%Only latin-1 and utf-8 are supported.c S s | d S ©Nr rM r¡ rM rM rN r¤ ² r¥ z*StataValueLabel.__init__.<locals>.<lambda>©Úkey)rj rÒ ÚlabnameÚ _encodingÚcatÚ
|
categoriesÚlistrT r ZarangeÚlenÚ value_labelsÚsortÚ_prepare_value_labels)Úselfrß rà rê rM rM rN Ú__init__§ s ÿzStataValueLabel.__init__c C s4 d| _ g | _d| _tjg tjd| _tjg tjd| _d| _g }g }| j D ]}|d }t
|
|t st |}t j t | j¡tt d | | j¡}| | j ¡ | j t|d 7 _ | |d ¡ | j |¡ | jd7 _qJ| j dkrìtdtj|tjd| _tj|tjd| _dd| j d| j | j | _d S )
|
zEncode value labels.r rv r> rt i } zaStata value labels for a single variable must have a combined length less than 32,000 characters.é r N)Útext_lenÚtxtÚnr ÚarrayrË ÚoffÚvalrì rí rÌ r r r r¸ rF rç r r Úencoderè Úappendrj )rð Úoffsetsrk ÚvlÚcategoryrM rM rN rï ¶ s<
|
|
|
ý
|
ÿz%StataValueLabel._prepare_value_labelsr Úbytes)Ú byteorderrA c C s | j }t }d}| t |d | j¡¡ t| jdd
|¡}|dkrLdnd}t ||d }| |¡ t
|
dD ]}| t d |¡¡ qp| t |d | j ¡¡ | t |d | j ¡¡ | j D ]}| t |d |¡¡ q¾| jD ]} | t |d | ¡¡ qà| jD ]}
|
| |
|
| ¡ q| ¡ S )
|
a!
|
Generate the binary representation of the value labels.
|
|
Parameters
|
----------
|
byteorder : str
|
Byte order of the output
|
|
Returns
|
-------
|
value_label : bytes
|
Bytes containing the formatted value label
|
ó ÚiNé )râ Úutf8é r> r Úc)rè r Úwriter² Úpackrì r rç rù Ú
|
_pad_bytesÚrangerõ ró r÷ rø rô Úgetvalue) rð rÿ rà ÚbioÚ null_byterç Zlab_lenr Úoffsetrb ÚtextrM rM rN Úgenerate_value_labelâ s&
|
|
|
|
z$StataValueLabel.generate_value_labelN)rÜ )Ú__name__Ú
|
__module__Ú __qualname__Ú__doc__rñ rï r rM rM rM rN rÛ s
|
ÿ,rÛ c @ s& e Zd ZdZd ddddddd Zd
|
S ) ÚStataNonCatValueLabela
|
Prepare formatted version of value labels
|
|
Parameters
|
----------
|
labname : str
|
Value label name
|
value_labels: Dictionary
|
Mapping of values to labels
|
encoding : {"latin-1", "utf-8"}
|
Encoding to use for value labels.
|
rÜ r údict[float, str]rÝ rÞ )rç rí rà rA c C s> |dkrt d|| _|| _t| ¡ dd d| _| ¡ d S )Nrá rã c S s | d S rä rM r¡ rM rM rN r¤ 1 r¥ z0StataNonCatValueLabel.__init__.<locals>.<lambda>rå )rj rç rè ÚsortedÚitemsrí rï )rð rç rí rà rM rM rN rñ % s ÿz StataNonCatValueLabel.__init__N)rÜ )r r r r rñ rM rM rM rN r s ür c @ s( e Zd ZU dZi Zded< dZded< eD ]4Zdee< edd D ] Z de
|
d
|
e ee e < q@q*d Z d ed < e dd¡d Zded< ed D ]dZ e de ¡d Zdee< e dkrÈee e
|
d
|
e 7 < e de de¡¡d e Ze de¡Z qdZd ed< e dd¡d Zed D ]hZ e de¡d Zdee< e dkrTee e
|
d
|
e 7 < e de de¡¡d e Ze de¡Zqddde de ¡d e de¡d d Zded < d d d d!d"Zed#d$d%d&Zed d$d'd(Zd#d$d)d*Zd#d$d+d,Zd-d.d/d0d1Zed2d d3d4d5Z d6S )7rÐ a
|
An observation's missing value.
|
|
Parameters
|
----------
|
value : {int, float}
|
The Stata missing value code
|
|
Notes
|
-----
|
More information: <https://www.stata.com/help.cgi?missing>
|
|
Integer missing values make the code '.', '.a', ..., '.z' to the ranges
|
101 ... 127 (for int8), 32741 ... 32767 (for int16) and 2147483621 ...
|
2147483647 (for int32). Missing values for floating point data types are
|
more complex but the pattern is simple to discern from the following table.
|
|
np.float32 missing values (float in Stata)
|
0000007f .
|
0008007f .a
|
0010007f .b
|
...
|
00c0007f .x
|
00c8007f .y
|
00d0007f .z
|
|
np.float64 missing values (double in Stata)
|
000000000000e07f .
|
000000000001e07f .a
|
000000000002e07f .b
|
...
|
000000000018e07f .x
|
000000000019e07f .y
|
00000000001ae07f .z
|
r ÚMISSING_VALUES)ée éå éåÿÿr ÚbasesÚ.r> é é` ó rþ Ú float32_basez<is r ra Ú incrementr¾ r° Ú float64_baseÚqs r¯ r r r )rÈ rÊ rË rÎ r± rÑ r rÞ ©rb rA c C s. || _ |dk rt|nt|}| j| | _d S )Nl )Ú_valuera r r Ú_str©rð rb rM rM rN rñ s zStataMissingValue.__init__r r@ c C s | j S )z²
|
The Stata representation of the missing value: '.', '.a'..'.z'
|
|
Returns
|
-------
|
str
|
The representation of the missing value.
|
)r' ©rð rM rM rN Ústring s
|
zStataMissingValue.stringc C s | j S )z³
|
The binary representation of the missing value.
|
|
Returns
|
-------
|
{int, float}
|
The binary representation of the missing value.
|
)r& r) rM rM rN rb s
|
zStataMissingValue.valuec C s | j S ©N)r* r) rM rM rN Ú__str__ s zStataMissingValue.__str__c C s t | d| dS )Nú(rÇ )Útyper) rM rM rN Ú__repr__¡ s zStataMissingValue.__repr__r r )ÚotherrA c C s& t |t| o$| j|jko$| j|jkS r+ )rÌ r. r* rb )rð r0 rM rM rN Ú__eq__¤ s
|
|
ÿ
|
ýzStataMissingValue.__eq__únp.dtype)rw rA c C s |j tjkr| jd }nh|j tjkr0| jd }nP|j tjkrH| jd }n8|j tjkr`| jd }n |j tjkrx| jd }ntd|S )NrÈ rÊ rË rÎ r± zUnsupported dtype) r. r rÈ rÑ rÊ rË rÎ r± rj )Úclsrw rb rM rM rN Úget_base_missing_value« s z(StataMissingValue.get_base_missing_valueN) r r r r r Ú__annotations__r Úbr r Úchrr! r² r³ r" ræ r Z int_valuer# rÑ rñ Úpropertyr* rb r, r/ r1 Ú classmethodr4 rM rM rM rN rÐ 6 sR
|
%
|
û rÐ c @ s e Zd ZddddZdS )Ú StataParserrÞ r@ c C s¾ t tttdddd tddD dt tj¡fdt tj¡fdt tj¡fdt tj ¡fd t tj
|
¡fg | _ t tj ¡t tj
|
¡t tj ¡t tj¡t tj¡t tj¡d
|
| _ tttdtd | _d d ddddd
|
| _d}d}d}d}dddt t d|¡d ¡t t d|¡d ¡ft
|
t d|¡d ¡t
|
t d|¡d ¡fd | _ddddd d | _d d d t t dd!¡d ¡t
|
t dd"¡d ¡d | _d#d$d%d&d'd(d)| _d*| _d S )+Nr> éõ c S s g | ]}t d t| ¡qS )Úa)r rw r )rJ r rM rM rN rO Î s z(StataParser.__init__.<locals>.<listcomp>éû éü éý éþ éÿ )é éöÿ é÷ÿ éøÿ éùÿ éúÿ ZbhlfdÚQr\ r£ ÚlÚhr6 s ÿÿÿþr¿ s ÿÿÿÿÿÿïÿrÀ )r rD )rÄ rà )rÆ rÅ r¾ r r¯ )r6 rJ rI r£ r\ )éb éi él éf rD r r r r r° Úi1Úi2Úi4Zf4Zf8Úu8)r6 rJ rI r£ r\ rH )<Z aggregaterö ÚbooleanÚbreakÚbyteZcaseZcatchÚclassZ colvectorÚcomplexÚconstÚcontinueÚdefaultZdelegateÚdeleteZdoÚdoubleÚelseZ eltypedefÚendÚenumÚexplicitÚexportZexternalr ÚforZfriendÚfunctionÚglobalÚgotoÚifÚinlinera ÚlocalÚlongÚNULLZpragmaZ protectedZquadZ rowvectorÚshortZtypedefÚtypenameZvirtualZ_allZ_NZ_skipÚ_bÚ_pizstr#ÚinZ_predZstrLZ_coefZ_rcZusingZ_consZ_seÚwithZ_n)Údictrë rT r r rw rÈ rÊ rË rÎ r± Ú DTYPE_MAPrÉ Ú DTYPE_MAP_XMLÚtupleÚTYPE_MAPÚ TYPE_MAP_XMLr² r³ Ú VALID_RANGEÚOLD_TYPE_MAPPINGr ÚNUMPY_TYPE_MAPÚRESERVED_WORDS)rð Z float32_minrÔ Z float64_minrÕ rM rM rN rñ ½ sz "ûÿÿ
|
|
|
|
|
|
úù þþøû ÿû
|
ú
|
zStataParser.__init__N)r r r rñ rM rM rM rN r: ¼ s r: c sø e Zd ZU eZded< ddddd ddd
|
dd d d dd fdd ZddddZddddZd dddZ ddddd d d Z
|
ddd d Z ddd!d"Z d#dd$d%Z d#dd&d'Zd#dd(d)Zd#dd*d+Zd#dd,d-Zd#dd.d/Zd#dd0d1Zd#dd2d3Zd4dd5d6Zd#d7d8d9d:Zddd;d<Zddd=d>Zd#d?d@dAdBZdCddDdEZdCddFdGZdCddHdIZ dCddJdKZ d#ddLdMZ dNddOdPZ dNddQdRZ d#ddSdTZ!d4ddUdVdWZ"dXddYdZZ#d[d#d\d]d^Z$d4dNd_d`daZ%dddbdcZ&dddddeZ'dfddgdhZ(dd dfdidjdkZ)e*e+dd dldld dldld
|
dldfdm dndoZ,dfddfdpdqdrZ-dfdfdsdtduZ.dfdvdfdwdxdyZ/dfdzdvddfd{d|d}Z0e1dNdd~dZ2e1dNdddZ3ddddZ4dzdd
dZ5 Z6S )Ú StataReaderz IO[bytes]Ú _path_or_bufTNFÚinferú FilePath | ReadBuffer[bytes]r ú
|
str | NoneúSequence[str] | Noneú
|
int | Noner r rÞ ) Ú path_or_bufÚ convert_datesÚconvert_categoricalsÚ index_colÚconvert_missingÚpreserve_dtypesÚcolumnsÚorder_categoricalsÚ chunksizeÚ compressionr3 rA c sÖ t ¡ g | _|| _|| _|| _|| _|| _|| _|| _ || _
|
|
|
| _ | | _ d| _ | | _d| _d| _| jd krvd| _nt| tr| dkrtdd | _d| _d| _d| _d| _d| _d| _d | _d| _ t t j | _ d S )Nr½ Fr> r z.chunksize must be a positive integer when set.)!Úsuperrñ Ú
|
_col_sizesÚ_convert_datesÚ_convert_categoricalsÚ
|
_index_colÚ_convert_missingÚ_preserve_dtypesÚ_columnsÚ_order_categoricalsÚ_original_path_or_bufÚ _compressionÚ_storage_optionsrè Ú
|
_chunksizeÚ_using_iteratorÚ_enteredrÌ ra rj Ú _close_fileÚ_has_string_dataZ_missing_valuesÚ_can_read_value_labelsÚ_column_selector_setÚ_value_labels_readÚ
|
_data_readÚ_dtypeÚ _lines_readÚ_set_endiannessÚsysrÿ Ú_native_byteorder) rð r r r r
r r r r r r r3 ©Ú __class__rM rN rñ ` s<
|
|
zStataReader.__init__r@ c C s t | ds| ¡ dS )zK
|
Ensure the file has been opened and its header data read.
|
r| N)ÚhasattrÚ
|
_open_filer) rM rM rN Ú _ensure_open s
|
zStataReader._ensure_openc C s | j stjdtt d t| jd| jd| jd}t |j
|
drX|j
|
¡ rX|j
|
| _ |j | _n*| t|j
|
¡ | _ W 5 Q R X | j j | _| ¡ | ¡ dS )z^
|
Open the file (with compression options, etc.), and read header information.
|
zStataReader is being used without using a context manager. Using StataReader as a context manager is the only supported method.rt ÚrbF)r3 Úis_textr ÚseekableN)r r r ÚResourceWarningr r/ r r r r¨ Úhandler r| Úcloser r ÚreadÚ _read_headerÚ _setup_dtype)rð ÚhandlesrM rM rN r© s* üû
|
|
zStataReader._open_filec C s
|
d| _ | S )zenter context managerT)r r) rM rM rN Ú __enter__¹ s zStataReader.__enter__ztype[BaseException] | NonezBaseException | NonezTracebackType | None)Úexc_typeÚ exc_valueÚ tracebackrA c C s | j r| ¡ d S r+ )r )rð r¶ r· r¸ rM rM rN Ú__exit__¾ s zStataReader.__exit__c C s$ t jdtt d | jr | ¡ dS )zØClose the handle if its open.
|
|
.. deprecated: 2.0.0
|
|
The close method is not part of the public API.
|
The only supported way to use StataReader is to use it as a context manager.
|
z¹The StataReader.close() method is not part of the public API and will be removed in a future version without notice. Using StataReader as a context manager is the only supported method.rt N)r r Ú FutureWarningr r r) rM rM rN r° Ç s ûzStataReader.closec C s | j dk rd| _nd| _dS )zC
|
Set string encoding which depends on file version
|
év rÜ râ N)Ú_format_versionrè r) rM rM rN Ú _set_encodingÙ s
|
zStataReader._set_encodingra c C s t d| j d¡¡d S )Nr6 r> r ©r² r³ r| r± r) rM rM rN Ú
|
_read_int8â s zStataReader._read_int8c C s t d| j d¡¡d S )NÚBr> r r¾ r) rM rM rN Ú _read_uint8å s zStataReader._read_uint8c C s t | j d| j d¡¡d S )NÚHr r ©r² r³ Ú
|
_byteorderr| r± r) rM rM rN Ú _read_uint16è s zStataReader._read_uint16c C s t | j d| j d¡¡d S )NÚIr r rà r) rM rM rN Ú _read_uint32ë s zStataReader._read_uint32c C s t | j d| j d¡¡d S )NrH rò r rà r) rM rM rN Ú _read_uint64î s zStataReader._read_uint64c C s t | j d| j d¡¡d S )NrJ r r rà r) rM rM rN Ú _read_int16ñ s zStataReader._read_int16c C s t | j d| j d¡¡d S )Nr r r rà r) rM rM rN Ú _read_int32ô s zStataReader._read_int32c C s t | j d| j d¡¡d S )Nr$ rò r rà r) rM rM rN Ú _read_int64÷ s zStataReader._read_int64rþ c C s t d| j d¡¡d S )Nr r> r r¾ r) rM rM rN Ú _read_char8ú s zStataReader._read_char8ztuple[int, ...])ÚcountrA c C s& t | j d| | j d| ¡¡S )NrJ r rà )rð rÍ rM rM rN Ú_read_int16_countý s þz StataReader._read_int16_countc sX ¡ }|dkr ¡ n
|
|¡ tdd jD dk _ fdd jD _d S )Nó <c S s g | ]}t |tkr|qS rM )r. ra ©rJ r rM rM rN rO
|
s z,StataReader._read_header.<locals>.<listcomp>r c s g | ]} |¡qS rM )Ú _calcsize©rJ Útypr) rM rN rO s )rÌ Ú_read_new_headerÚ_read_old_headerrì Ú_typlistr r )rð Ú
|
first_charrM r) rN r² s
|
|
zStataReader._read_headerc C s | j d¡ t| j d¡| _| jdkr:ttj| jd| ¡ | j d¡ | j d¡dkrbdnd| _| j d ¡ | jd
|
kr| ¡ n|
|
¡ | _ | j d ¡ | ¡ | _ | j d ¡ | ¡ | _| j d ¡ | ¡ | _| j d¡ | j d¡ | j d¡ | ¡ d | _| ¡ d | _| ¡ d | _| ¡ d | _| ¡ d | _| ¡ | _| j d¡ | ¡ d | _| ¡ d | _| ¡ d | _ | | j¡\| _ | _ | j | j¡ | !¡ | _"| j | j¡ | #| j d ¡d d
| _$| j | j¡ | %¡ | _&| j | j¡ | '¡ | _(| j | j¡ | )¡ | _*d S )Nr r ©éu r» éw ©Úversioné s MSFú>ú<é r» r| é é é rò é é
|
é r é r> éÿÿÿÿ)+r| r± ra r¼ rj Ú_version_errorrF r½ rÄ rÅ rÇ Ú_nvarÚ _get_nobsÚ_nobsÚ_get_data_labelÚ _data_labelÚ_get_time_stampÚ _time_stamprË Z_seek_vartypesZ_seek_varnamesZ_seek_sortlistZ _seek_formatsÚ_seek_value_label_namesÚ_get_seek_variable_labelsZ_seek_variable_labelsÚ_data_locationÚ _seek_strlsÚ_seek_value_labelsÚ _get_dtypesrÖ Ú _dtyplistÚseekÚ _get_varlistÚ_varlistrÎ Ú_srtlistÚ _get_fmtlistÚ_fmtlistÚ _get_lbllistÚ_lbllistÚ_get_variable_labelsÚ_variable_labelsr) rM rM rN rÔ sP
|
ÿ
|
|
|
|
|
|
|
z StataReader._read_new_headerz,tuple[list[int | str], list[str | np.dtype]])Ú seek_vartypesrA c sx j |¡ fddtjD }dddfdd fdd|D }dd dfd
|
d fd d|D }||fS ) Nc s g | ] } ¡ qS rM )rÅ ©rJ Ú_r) rM rN rO K s z+StataReader._get_dtypes.<locals>.<listcomp>ra ú int | str)rÓ rA c
|
sR | dkr | S z j | W S tk
|
rL } ztd| d|W 5 d }~X Y nX d S )Néý ú cannot convert stata types [ú])rv ÚKeyErrorrj ©rÓ Úerrr) rM rN r£ M s z"StataReader._get_dtypes.<locals>.fc s g | ] } |qS rM rM rÐ )r£ rM rN rO U s zstr | np.dtypec
|
sV | dkrt | S z j| W S tk
|
rP } ztd| d|W 5 d }~X Y nX d S )Nr z cannot convert stata dtype [r )r rs r rj r
|
r) rM rN r¦ W s z"StataReader._get_dtypes.<locals>.gc s g | ] } |qS rM rM rÐ )r¦ rM rN rO _ s )r| rø r rê )rð r Z raw_typlistÚtyplistÚdtyplistrM )r£ r¦ rð rN rö G s zStataReader._get_dtypesz list[str]c s, j dk rdnd fddtjD S )Nr» é! é c s g | ]} j ¡¡qS rM ©Ú_decoder| r± r ©r6 rð rM rN rO f s z,StataReader._get_varlist.<locals>.<listcomp>©r¼ r rê r) rM r rN rù c s zStataReader._get_varlistc sN j dkrd n$j dkr d nj dkr0d nd fdd tjD S )
|
Nr» é9 éq é1 éh r r| c s g | ]} j ¡¡qS rM r r r rM rN rO s s z,StataReader._get_fmtlist.<locals>.<listcomp>r r) rM r rN rü i s
|
|
|
zStataReader._get_fmtlistc s> j dkrd nj dkr d nd fddtjD S )Nr» r rM r ræ c s g | ]} j ¡¡qS rM r r r rM rN rO } s z,StataReader._get_lbllist.<locals>.<listcomp>r r) rM r rN rþ v s
|
|
zStataReader._get_lbllistc sd j dkr$ fddt jD }n< j dkrH fddt jD }n fddt jD }|S )Nr» c s g | ]} j d ¡¡qS )iA r r r) rM rN rO s z4StataReader._get_variable_labels.<locals>.<listcomp>rL c s g | ]} j d ¡¡qS )éQ r r r) rM rN rO
s c s g | ]} j d ¡¡qS )r r r r) rM rN rO s r )rð ZvlblistrM r) rN r s
|
|
ÿ
|
|
ÿ
|
ÿz StataReader._get_variable_labelsc C s | j dkr| ¡ S | ¡ S d S )Nr» )r¼ rÈ rÇ r) rM rM rN rë s
|
zStataReader._get_nobsr c C sz | j dkr$| ¡ }| | j |¡¡S | j dkrH| ¡ }| | j |¡¡S | j dkrd| | j d¡¡S | | j d¡¡S d S )Nr» rÙ rL r r )r¼ rÅ r r| r± r¿ ©rð ZstrlenrM rM rN rí s
|
|
|
zStataReader._get_data_labelc C sn | j dkr$| ¡ }| j |¡ d¡S | j dkrH| ¡ }| | j |¡¡S | j dkrd| | j d¡¡S t d S )Nr» râ rÙ r é )r¼ r¿ r| r± Údecoder rj r rM rM rN rï s
|
|
|
zStataReader._get_time_stampc C sN | j dkr.| j d¡ | jd| j d d S | j dkrD| ¡ d S t d S )NrÙ rò r é é r» )r¼ r| r± rñ rê rË rj r) rM rM rN rò ¬ s
|
|
z%StataReader._get_seek_variable_labels)r× rA c
|
s¢ t |d _ jdkr*ttj jd ¡ ¡ dkrBdnd _ ¡ _ j
|
d¡ ¡ _ ¡ _ ¡ _ ¡ _ jdkrªdd j
|
j ¡D }nV j
|
j ¡}tj|tjd
|
}g }|D ].}| jkrð| j| ¡ qÐ| |d ¡ qÐz fd d |D _W nJ tk
|
rb } z*d dd |D ¡}td| d|W 5 d }~X Y nX z fdd |D _W nJ tk
|
rÆ } z*d dd |D ¡}td| d|W 5 d }~X Y nX jdkrð fdd t j D _ n fdd t j D _ j d ¡d d
_ ¡ _ !¡ _" #¡ _$ jdkr ¡ } jdkrl %¡ }
|
n &¡ }
|
| dkrq j
|
|
|
¡ qN j '¡ _(d S )Nr )r rL rM éo r ér és rÛ r> rÞ rß rM c S s g | ] }t |qS rM )ra ©rJ r rM rM rN rO Ê s z0StataReader._read_old_header.<locals>.<listcomp>rv é c s g | ]} j | qS rM )ru rÒ r) rM rN rO Ö s ú,c S s g | ] }t |qS rM ©r rÐ rM rM rN rO Ø s r r c s g | ]} j | qS rM )rr rÒ r) rM rN rO Û s c S s g | ] }t |qS rM r$ rÐ rM rM rN rO Ý s z cannot convert stata dtypes [c s g | ]} j d ¡¡qS )r r r r) rM rN rO á s c s g | ]} j d ¡¡qS )ræ r r r) rM rN rO å s rè r ))ra r¼ rj ré rF r½ r¿ rÄ Z _filetyper| r± rÅ rê rë rì rí rî rï rð r Ú
|
frombufferrÉ rx rú rÖ Újoinr÷ r rú rÎ rû rü rý rþ rÿ r r rÊ rÉ Útellró ) rð r× r ÚbufZtyplistbÚtpr Z invalid_typesZinvalid_dtypesZ data_typeZdata_lenrM r) rN rÕ ¸ sd
|
|
|
|
|
|
|
|
$$
|
ÿ
|
|
ÿ
|
|
|
|
|
z StataReader._read_old_headerr2 c C s | j dk r| j S g }t| jD ]\\}}|| jkr`tt|}| d| | j | j| f¡ q | d| d| f¡ q t |¡| _ | j S )z"Map between numpy and state dtypesNÚsÚS)
|
r¡ Ú enumeraterÖ ry r r rú rÄ r rw )rð Údtypesr rÓ rM rM rN r³ s
|
|
|
& zStataReader._setup_dtyper ©r rA c C s t |tr|S t | j| ¡S r+ )rÌ ra r² ÚcalcsizerÄ ©rð r rM rM rN rÑ s
|
zStataReader._calcsize©r* rA c C sd | d¡d }z| | j¡W S tk
|
r^ | j}d| d}tj|tt d | d¡ Y S X d S )Nr r z@
|
One or more strings in the dta file could not be decoded using zÍ, and
|
so the fallback encoding of latin-1 is being used. This can happen when a file
|
has been incorrectly encoded by Stata or some other software. You should verify
|
the string values returned are correct.rt rÜ )Ú partitionr rè ÚUnicodeDecodeErrorr r ÚUnicodeWarningr )rð r* rà ÚmsgrM rM rN r s ÿýzStataReader._decodec C sê | ¡ | jrd S | jdkr,d| _i | _d S | jdkrF| j | j¡ n.| jd k sTt| j | jj
|
}| j | j | ¡ d| _i | _| jdkr| j d¡dkrqà| j d¡}|s²qà| jdkrÐ| | j d¡¡}n| | j d¡¡}| j d ¡ | ¡ }| ¡ }tj| j d| ¡| j d
|
|d }tj| j d| ¡| j d
|
|d }t |¡}|| }|| }| j |¡} i | j|< t|D ]H}
|
|
|
|d k r||
|
d n|} | | ||
|
|
¡| j| ||
|
< q~| jdkr| j d ¡ qd| _d S )NrM TrÙ é s </valr r r r rQ ©rw rÍ r> r )rª r r¼ Ú_value_label_dictr| rø rõ r¡ ÚAssertionErrorrì Úitemsizeró r± r rÇ r r% rÄ Zargsortr ) rð r Zslengthrç rõ Ztxtlenr÷ rø Úiirô r r^ rM rM rN Ú_read_value_labels- sb
|
|
|
|
|
ÿ
|
ÿ
|
|
ÿ
|
z StataReader._read_value_labelsc C s | j | j¡ ddi| _| j d¡dkr,q| jdkr@| ¡ }nr| j d¡}| jdkrZdnd}| jd kr|d
|
|
|d d|
}n |d
|
|
|d | d
}t d |¡d
|
}|
|
¡ }| ¡ }| j |¡}|d krì|d
|
d
| j ¡}nt|}|| jt|< qd S )NÚ0r½ r s GSOrÙ r r» r rß r r rH é rè )r| rø rô ÚGSOr± r¼ rÈ rÄ r² r³ rÁ rÇ r rè r )rð Zv_or( Zv_sizerÓ ÚlengthÚvaZ
|
decoded_varM rM rN Ú _read_strlsf s&
|
|
|
|
zStataReader._read_strlsr+ c C s d| _ | j| jdS )NT©Únrows)r r± r r) rM rM rN Ú__next__
s zStataReader.__next__)ÚsizerA c C s |dkr| j }| j|dS )a
|
Reads lines from Stata file and returns as dataframe
|
|
Parameters
|
----------
|
size : int, defaults to None
|
Number of lines to read. If None, reads whole file.
|
|
Returns
|
-------
|
DataFrame
|
NrC )r r± )rð rF rM rM rN Ú get_chunk s zStataReader.get_chunkz bool | None) rD r r r
r r r r rA c s | ¡ | jdkr2|d kr2d| _d| _t| jdS |d kr@| j}|d krN| j}|d kr\| j}|d krj| j }|d krx| j
|
}|d kr| j }|d kr| j }|d kr¢| j}| j dkrÀ| jsÀd| _| ¡ | jd k sÎt| j} | j| j | j }
|
|| j } t| |
|
} | dkr|r| ¡ t| j| j } | j | j| ¡ t|| j| j } tj| j | ¡| | d}| j| 7 _| j| jkrd| _d| _| j | j kr | ¡ ¡ }|r®| ¡ t!|dkrÊt| jd}nt "|¡}t#| j|_$|d krt%| j| | j}t#||_&|d k r | '||¡}t(|| j)D ]0\}}t*|t+kr(|| j,| j-dd||< q(| .|¡}t /dd | j0D ¡d }|j&}d }g }|D ]}| j0| d k r|j$| }|| j1} | t 1t2¡krþ| | j0| krþd}| 3|t4|| || j0| f¡ n| 3||| f¡ q|r(t 5t6|¡}~| 7||¡}|rd
|
d d d d t / fdd| j8D ¡d }|D ](}|j$| }t9|| | j8| ||< qn|r¾| j dkr¾| :|| j;| j<|¡}|svg }d }|D ]}|| j1} | t 1tj=¡t 1tj>¡fkrt 1tj?¡} d}n8| t 1tj@¡t 1tjA¡t 1tjB¡fkrFt 1tjC¡} d}| 3||| D| ¡f¡ qÐ|rvt 5t6|¡}|d k r| E| F|¡¡}|S )Nr T©r rÙ r7 )Z convert_dtypec S s g | ] }|d k qS r+ rM )rJ ZdtyprM rM rN rO ú s z$StataReader.read.<locals>.<listcomp>Fr r r c s t fddtD S )Nc 3 s | ]} |¡V qd S r+ )r )rJ r r¡ rM rN Ú <genexpr> s z;StataReader.read.<locals>.any_startswith.<locals>.<genexpr>)r Ú _date_formatsr¡ rM r¡ rN Úany_startswith s z(StataReader.read.<locals>.any_startswithc s g | ] } |qS rM rM rÐ ©rK rM rN rO s rM )Grª rì r r r+ rú r r r r r r r r¼ r rB r¡ r9 r¢ r: rR r< Ú StopIterationr| rø ró r r% r± rÄ r¥ ZbyteswapÚ newbyteorderrì Z from_recordsr, r r rG Ú_do_select_columnsrT rÖ r. ra rª r Ú _insert_strlsÚwherer÷ rw r rú r- Ú from_dictrq Ú_do_convert_missingrý r Ú_do_convert_categoricalsr8 rÿ Úfloat16rÎ r± rÈ rÊ rË r r Z set_indexÚpop)rð rD r r r
r r r r rw Z max_read_lenÚread_lenr Ú
|
read_linesÚraw_datar¼ ÚrngrÖ rÓ Zcols_ÚixZrequires_type_conversionÚdata_formattedr ÚcolsZ retyped_dataÚconvertrM rL rN r± sê
|
|
|
|
ÿ
|
|
|
|
|
|
|
ÿ
|
ÿ ÿ
|
|
|
|
ý
|
zStataReader.read)r¼ r rA c C sR i }t |D ] \}}| j| }|| jkr,q tt|}| j| \}}|| } | j}
|
|
|
|k |
|
|kB } | ¡ slq |rØt t | ¡¡d } tj
|
| | dd\} }t | t d}t | D ]&\}}t |}| ||k }||j|< q®nJ| j}|tjtjfkrôtj}t | |d}|jjd s| ¡ }tj|j| < |||< q |rN| ¡ D ]\}}|||< q:|S )Nr T)Zreturn_inverserv Z WRITEABLE)r, rÖ rw r r r§ r r ZnonzeroZasarrayÚuniquer- r rÐ Úilocrw rÎ r± ÚflagsÚcopyÚnanr )rð r¼ r Ú replacementsr Úcolnamer ZnminZnmaxZseriesZsvalsÚmissingrØ ZumissingZ umissing_locÚ replacementÚjZumr´ rÍ rw rÖ rb rM rM rN rS 9 s@
|
|
|
|
z StataReader._do_convert_missingr» c sl t drt jdkr |S t jD ]@\}}|dkr8q& fdd|jd d
|f D |jd d
|f< q&|S )Nr? r rH c s g | ]} j t| qS rM )r? r )rJ Úkr) rM rN rO o s z-StataReader._insert_strls.<locals>.<listcomp>)r¨ rì r? r, rÖ r` )rð r¼ r rÓ rM r) rN rP h s 0zStataReader._insert_strlsú Sequence[str])r¼ r rA c C sÞ | j sÖt|}t|t|kr&td| |j¡}|rRd t|¡}td| g }g }g }g } |D ]P}
|
|j |
|
¡} | | j
|
| ¡ | | j | ¡ | | j | ¡ | | j | ¡ qf|| _
|
|| _ || _ | | _ d| _ || S )Nz"columns contains duplicate entriesz, z<The following columns were not found in the Stata data set: T)r Úsetrì rj Ú
|
differencer r& rë Zget_locrú r÷ rÖ rý rÿ ) rð r¼ r Z
|
column_setÚ unmatchedZjoinedr r ÚfmtlistÚlbllistrÖ r rM rM rN rO r s4 ÿ z StataReader._do_select_columnszdict[str, dict[float, str]])r¼ Úvalue_label_dictro r rA c C s¨ t | ¡ }g }t||D ]v\}}||kr|| } t t | ¡ ¡}
|
|| } | |
|
¡} | jrn| ¡ rn|
|
} n | jrtj t
|
t t d d} t | | |d}| dkrÔg }|jD ]&}|| krÆ| | | ¡ qª| |¡ qªn t | ¡ }z| |¡}W np tk
|
r^ } zPt|dd ¡ }t |j|dk }dd |¡ }d | d
|
| d}t||W 5 d}~X Y nX t||jdd }| ||f¡ q| ||| f¡ qtt|dd}|S ) zC
|
Converts categorical columns to Categorical type.
|
rt N)rê ZorderedF)rb r> zQ--------------------------------------------------------------------------------
|
r1 z
|
Value labels for column a are not unique. These cannot be converted to
|
pandas categoricals.
|
|
Either read the file with `convert_categoricals` set to False or use the
|
low level interface in `StataReader` to separately read the values and the
|
value_labels.
|
|
The repeated labels are:
|
)rG rb )rë ÚkeysrT r rö Úisinr Úallr r rº r r r" rê rú rk Zrename_categoriesrj r- Z value_countsrG r& r+ rq )rð r¼ rp ro r rí Zcat_converted_datarÖ Úlabelrü rq ÚcolumnZ key_matchesZinitial_categoriesZcat_datarê rý r ZvcZ repeated_catsZrepeatsr5 Z
|
cat_seriesrM rM rN rT s\
|
|
|
ý ÿ
|
ÿ ÷ z$StataReader._do_convert_categoricalsc C s | ¡ | jS )z2
|
Return data label of Stata file.
|
)rª rî r) rM rM rN Ú
|
data_labelß s zStataReader.data_labelc C s | ¡ | jS )z2
|
Return time stamp of Stata file.
|
)rª rð r) rM rM rN Ú
|
time_stampç s zStataReader.time_stampzdict[str, str]c C s | ¡ tt| j| jS )z
|
Return a dict associating each variable name with corresponding label.
|
|
Returns
|
-------
|
dict
|
)rª rq rT rú r r) rM rM rN Úvariable_labelsï s zStataReader.variable_labelsc C s | j s| ¡ | jS )z
|
Return a nested dict associating each variable name to its value and label.
|
|
Returns
|
-------
|
dict
|
)r r< r8 r) rM rM rN rí ú s zStataReader.value_labels)
|
TTNFTNTNr} N)N)NNNNNNNN)7r r r Ú_stata_reader_docr r5 rñ rª r© rµ r¹ r° r½ r¿ rÁ rÅ rÇ rÈ rÉ rÊ rË rÌ rÎ r² rÔ rö rù rü rþ r rë rí rï rò rÕ r³ rÑ r r< rB rE rG r Ú_read_method_docr± rS rP rO rT r8 rv rw rx rí Ú __classcell__rM rM r¦ rN r{ [ s
|
ô*3 8 K9 ÷" /
|
M r{ TFr} ) r r r
r r r r r Úiteratorr r3 r~ r r r r r r zDataFrame | StataReader) r2 r r r
r r r r r r| r r3 rA c C sN t | ||||||||| |
|
d } | s&|r*| S | | ¡ W 5 Q R £ S Q R X d S )N)
|
r r r
r r r r r r3 r )r{ r± ) r2 r r r
r r r r r r| r r3 ÚreaderrM rM rN Ú
|
read_stata s" õr~ )Ú
|
endiannessrA c C s4 | ¡ dkrdS | ¡ dkr dS td| dd S )N)rß Úlittlerß )rÞ ÚbigrÞ z Endianness r )Úlowerrj )r rM rM rN r£ - s
|
r£ r ra )rÒ r@ rA c C s2 t | tr | d|t| S | d|t| S )zQ
|
Take a char string and pads it with null bytes until it's length chars.
|
r ú )rÌ rþ rì ©rÒ r@ rM rM rN r 6 s
|
r r2 r. c C s( | dkrt t j¡S td| ddS )zK
|
Convert from one of the stata date formats to a type in TYPE_MAP.
|
)rr r4 rx r6 rz r8 r~ r9 r r: r
r; r r< r® z not implementedN)r rw r± ÚNotImplementedError)r rM rM rN Ú _convert_datetime_to_stata_type? s r rq zlist[Hashable])r ÚvarlistrA c C sv i }| D ]h}| | d¡s*d| | | |< ||krL| | |¡| | i¡ qt|ts^td| || | i¡ q|S )Nú%z0convert_dates key must be a column or an integer)r ÚupdaterG rÌ ra rj )r r Znew_dictræ rM rM rN Ú_maybe_convert_to_int_keysX s
|
r )rw ru rA c C s | j tjkr$tt|j}t|dS | j tjkr4dS | j tjkrDdS | j tj krTdS | j tj
|
krddS | j tj krtdS t d| dd S )
|
aõ
|
Convert dtype types to stata types. Returns the byte of the given ordinal.
|
See TYPE_MAP and comments for an explanation. This is also explained in
|
the dta spec.
|
1 - 244 are strings of this length
|
Pandas Stata
|
251 - for int8 byte
|
252 - for int16 int
|
253 - for int32 long
|
254 - for float32 float
|
255 - for double double
|
|
If there are dates to convert, then dtype will already have the correct
|
type inserted.
|
r> rA r@ r? r> r= ú
|
Data type ú not supported.N© r. r Úobject_r r r§ rQ r± rÎ rË rÊ rÈ r
)rw ru r: rM rM rN Ú_dtype_to_stata_typef s
|
r r )ru Ú dta_versionÚ
|
force_strlrA c C sÀ |dk rd}n d}|rdS | j tjkrntt|j}||krX|dkrHdS tt |j ¡dt
|
t |d d S | tj kr|dS | tj krd S | tjkrd
|
S | tjtjfkr¬d S td | d dS )a¢
|
Map numpy dtype to stata's default format for this type. Not terribly
|
important since users can change this in Stata. Semantics are
|
|
object -> "%DDs" where DD is the length of the string. If not a string,
|
raise ValueError
|
float64 -> "%10.0g"
|
float32 -> "%9.0g"
|
int64 -> "%9.0g"
|
int32 -> "%12.0g"
|
int16 -> "%8.0g"
|
int8 -> "%8.0g"
|
strl -> "%9s"
|
rÙ éô r z%9sr r> r* z%10.0gz%9.0gz%12.0gz%8.0gr r N)r. r r r r r§ rj r¶ rF rÒ r rQ r± rÎ rË rÈ rÊ r
)rw ru r r Ú max_str_lenr: rM rM rN Ú_dtype_to_default_stata_fmt s*
|
|
|
r Úcompression_optionsÚfname)r3 r c s, e Zd ZU dZdZdZded< dcdd d
|
d d d dddd ddddd fddZdddddZddd d d Z d d d d!d"Z
|
d d d d#d$Z d d d d%d&Z dd'd(d)Z ddd*d+d,Zd d d d-d.Zd/dd0d1d2Zd dd d3d4Zdd'd5d6Zdd'd7d8Zdd'd9d:Zdd'd;d<Zdd'd=d>Zdd'd?d@Zdd'dAdBZdd'dCdDZdd'dEdFZddddddGdHdIZdd'dJdKZ dd'dLdMZ dd'dNdOZ dd'dPdQZ dd'dRdSZ dd'dTdUZ!d d d dVdWZ"dXd'dYdZZ#dXdd[d\d]Z$e%ddd^d_d`Z&ddd^dadbZ' Z(S )eÚ StataWritera½
|
A class for writing Stata binary dta files
|
|
Parameters
|
----------
|
fname : path (string), buffer or path object
|
string, path object (pathlib.Path or py._path.local.LocalPath) or
|
object implementing a binary write() functions. If using a buffer
|
then the buffer will not be automatically closed after the file
|
is written.
|
data : DataFrame
|
Input to save
|
convert_dates : dict
|
Dictionary mapping columns containing datetime types to stata internal
|
format to use when writing the dates. Options are 'tc', 'td', 'tm',
|
'tw', 'th', 'tq', 'ty'. Column can be either an integer or a name.
|
Datetime columns that do not have a conversion type specified will be
|
converted to 'tc'. Raises NotImplementedError if a datetime column has
|
timezone information
|
write_index : bool
|
Write the index to Stata dataset.
|
byteorder : str
|
Can be ">", "<", "little", or "big". default is `sys.byteorder`
|
time_stamp : datetime
|
A datetime to use as file creation date. Default is the current time
|
data_label : str
|
A label for the data set. Must be 80 characters or smaller.
|
variable_labels : dict
|
Dictionary containing columns as keys and variable labels as values.
|
Each label must be 80 characters or smaller.
|
{compression_options}
|
|
.. versionadded:: 1.1.0
|
|
.. versionchanged:: 1.4.0 Zstandard support.
|
|
{storage_options}
|
|
.. versionadded:: 1.2.0
|
|
value_labels : dict of dicts
|
Dictionary containing columns as keys and dictionaries of column value
|
to labels as values. The combined length of all labels for a single
|
variable must be 32,000 characters or smaller.
|
|
.. versionadded:: 1.4.0
|
|
Returns
|
-------
|
writer : StataWriter instance
|
The StataWriter instance has a write_file method, which will
|
write the file to the given `fname`.
|
|
Raises
|
------
|
NotImplementedError
|
* If datetimes contain timezone information
|
ValueError
|
* Columns listed in convert_dates are neither datetime64[ns]
|
or datetime.datetime
|
* Column dtype is not representable in Stata
|
* Column listed in convert_dates is not in DataFrame
|
* Categorical label contains more than 32,000 characters
|
|
Examples
|
--------
|
>>> data = pd.DataFrame([[1.0, 1]], columns=['a', 'b'])
|
>>> writer = StataWriter('./data_file.dta', data)
|
>>> writer.write_file()
|
|
Directly write a zip file
|
>>> compression = {{"method": "zip", "archive_name": "data_file.dta"}}
|
>>> writer = StataWriter('./data_file.zip', data, compression=compression)
|
>>> writer.write_file()
|
|
Save a DataFrame with dates
|
>>> from datetime import datetime
|
>>> data = pd.DataFrame([[datetime(2000,1,1)]], columns=['date'])
|
>>> writer = StataWriter('./date_data_file.dta', data, {{'date' : 'tw'}})
|
>>> writer.write_file()
|
r rÜ rÝ rè NTr} ©rí ú FilePath | WriteBuffer[bytes]r+ údict[Hashable, str] | Noner r údatetime.datetime | Noner r ú'dict[Hashable, dict[float, str]] | NonerÞ ) r r¼ r Ú write_indexrÿ rw rv rx r r3 rí rA c s® t ¡ || _|d kr i n|| _|| _|| _|| _|| _| | _g | _ t
|
j g t d| _ | | _d | _i | _| |¡ |
|
| _|d krtj}t|| _|| _t
|
jt
|
jt
|
jd| _d S )Nrv )r? r> r= ) r rñ r¼ r Ú _write_indexrð rî r Ú_non_cat_value_labelsÚ _value_labelsr rö r Ú_has_value_labelsr Ú _output_fileÚ_converted_namesÚ_prepare_pandasr3 r¤ rÿ r£ rÄ Ú_fnamerË rÊ rÈ Ztype_converters) rð r r¼ r r rÿ rw rv rx r r3 rí r¦ rM rN rñ s(
|
|
|
zStataWriter.__init__r )Úto_writerA c C s | j j | | j¡¡ dS )zS
|
Helper to call encode before writing to file for Python 3 compat.
|
N)r´ r¯ r rù rè )rð r¦ rM rM rN Ú_write7 s zStataWriter._writerþ r% c C s | j j |¡ dS )z?
|
Helper to assert file is open before writing.
|
N)r´ r¯ r r( rM rM rN Ú _write_bytes= s zStataWriter._write_byteszlist[StataNonCatValueLabel]r» c C s g }| j dkr|S | j ¡ D ]x\}}|| jkr:| j| }n$||jkrNt|}ntd| dt|| js|td| dt ||| j
|
}| |¡ q |S )zc
|
Check for value labels provided for non-categorical columns. Value
|
labels
|
Nz Can't create value labels for z!, it wasn't found in the dataset.z6, value labels can only be applied to numeric columns.) r r r£ r r r r! rw rj r rè rú )rð r¼ Únon_cat_value_labelsrç Úlabelsre ÚsvlrM rM rN Ú _prepare_non_cat_value_labelsC s$
|
|
|
|
|
ÿ
|
ÿ z)StataWriter._prepare_non_cat_value_labelsc
|
s@ fdd D }t |s S | jt |¡O _tj}g }t |D ]ê\}}|r t | | jd}| j
|
|¡ | j j j }|tjkrtd | j j j ¡ } | ¡ ||krþ|tjkrÌt tj¡}n$|tjkrät tj¡}n t tj¡}tj| |d} ||| | dk< |
|
|| f¡ qF|
|
| | f¡ qFt t|¡S )z
|
Check for categorical columns, retain categorical information for
|
Stata file and convert categorical data to int
|
c s g | ]}t | jqS rM )r rw ©rJ rÖ ©r¼ rM rN rO i s z5StataWriter._prepare_categoricals.<locals>.<listcomp>)rà zCIt is not possible to export int64-based categorical data to Stata.rv rè )r r¡ r rö rÐ r4 rT rÛ rè r rú ré Úcodesrw r rj r§ rb rQ rÈ rÊ rË r± r+ rR rq )
|
rð r¼ Zis_catr4 r\ rÖ Z
|
col_is_catr« rw rk rM r® rN Ú_prepare_categoricalsd s6
|
ÿ
|
|
z!StataWriter._prepare_categoricalsc C sZ |D ]P}|| j }|tjtjfkr|tjkr8| jd }n
|
| jd }|| |¡||< q|S )z
|
Checks floating point data columns for nans, and replaces these with
|
the generic Stata for missing value (.)
|
r£ r\ )rw r rÎ r± r Úfillna)rð r¼ r rw rg rM rM rN Ú _replace_nans s
|
|
|
zStataWriter._replace_nansr@ c C s dS )z No-op, forward compatibilityNrM r) rM rM rN Ú_update_strl_names s z StataWriter._update_strl_names©rÒ rA c C sR |D ]H}|dk s|dkr|dk s(|dkr|dk s8|dkr|dkr| |d¡}q|S )a
|
Validate variable names for Stata export.
|
|
Parameters
|
----------
|
name : str
|
Variable name
|
|
Returns
|
-------
|
str
|
The validated name with invalid characters replaced with
|
underscores.
|
|
Notes
|
-----
|
Stata 114 and 117 support ascii characters in a-z, A-Z, 0-9
|
and _.
|
ÚAÚZr< Úzr= Ú9r )Úreplace©rð rÒ r rM rM rN Ú_validate_variable_name¢ s" ÿÿþþýýüz#StataWriter._validate_variable_namec C s¦ i }t |j}|dd
}d}t|D ]Ê\}}|}t|tsDt|}| |¡}|| jkr`d| }d|d krxdkrn nd| }|dtt|d
}||ksè| |¡dkràdt| | }|dtt|d
}|d7 }q¢|||< |||< q&t
|
||_| j r<t ||D ],\} }
|
| |
|
kr| j |
|
| j | < | j |
|
= q|rg } | ¡ D ]"\}}| d| } | | ¡ qNt d | ¡¡} tj| tt d
|
|| _| ¡ |S ) aÌ
|
Checks column names to ensure that they are valid Stata column names.
|
This includes checks for:
|
* Non-string names
|
* Stata keywords
|
* Variables that start with numbers
|
* Variables with names that are too long
|
|
When an illegal variable name is detected, it is converted, and if
|
dates are exported, the variable name is propagated to the date
|
conversion dictionary
|
Nr r r= r¸ r r> z -> z
|
rt )rë r r, rÌ r r» rz rR rì rÍ r, r rT r rú r¹ rF r& r r r r r£ r³ )rð r¼ Zconverted_namesr Zoriginal_columnsZduplicate_var_idrh rÒ Ú orig_namer ÚoZconversion_warningr5 rÓ rM rM rN Ú_check_column_namesÀ sR
|
|
|
|
|
|
|
|
ýz StataWriter._check_column_namesr- ©r- rA c C sR g | _ g | _| ¡ D ]8\}}| j t|| j| ¡ | j t|| j| ¡ qd S r+ )rn r r rú r r¼ r )rð r- rÖ rw rM rM rN Ú_set_formats_and_types
|
s
|
z"StataWriter._set_formats_and_typesc
|
C st | ¡ }| jr$| ¡ }t|tr$|}| |¡}t|}| |¡}t d|j
|
d ¡| _ | |¡}dd |D }|j |¡}| j |O _ | j |¡ | |¡}|j
|
\| _| _|| _|j ¡ | _|j}|D ]&}|| jkrÖqÆt|| rÆd| j|< qÆt| j| j| _| jD ]"}t| j| } t | ¡||< q| ¡ | |¡ | jd k rp| jD ]"}t|t rL| j| | j |< qLd S )NFr> c S s g | ]
|
}|j qS rM )rç )rJ r« rM rM rN rO +
|
s z/StataWriter._prepare_pandas.<locals>.<listcomp>rr )!rb r Z reset_indexrÌ r+ r¾ rÚ r² r ÚrepeatÚshaper¡ r¬ r rr r Úextendr° ÚnobsÚnvarr¼ Útolistr r- r r r r rw Ú_encode_stringsrÀ ra rn )
|
rð r¼ Útempr© Znon_cat_columnsZhas_non_cat_val_labelsr- rÖ ræ Únew_typerM rM rN r¤
|
sJ
|
|
|
|
|
|
ÿ
|
|
|
zStataWriter._prepare_pandasc C sÀ | j }t| dg }t| jD ]\}}||ks ||kr6q | j| }|j}|jtjkr t|dd}|dkst |dks|j
|
}t d| d| j| j | j¡}tt|j| jkr || j|< q dS ) zõ
|
Encode strings in dta-specific encoding
|
|
Do not encode columns marked for date conversion or for strL
|
conversion. The strL converter independently handles conversion and
|
also accepts empty string arrays.
|
Ú _convert_strlTr r* r zColumn `a ` cannot be exported.
|
|
Only string-like object arrays
|
containing all strings or a mix of strings and None can be exported.
|
Object arrays containing only null values are prohibited. Other object
|
types cannot be exported and must first be converted to one of the
|
supported types.N)r rS r, r¼ rw r. r r r rì rÒ rj r rù rè r r r§ Ú_max_string_length) rð r Ú convert_strlr rÖ ru rw Zinferred_dtypeÚencodedrM rM rN rÇ R
|
s,
|
ÿÿ ÿÿzStataWriter._encode_stringsc C s t | jd| jd| jdt| _| jjd dk rV| jjt | _| j_| jj
|
| jj¡ z| j | j | j d | ¡ | ¡ | ¡ | ¡ | ¡ | ¡ | ¡ | ¡ | ¡ | ¡ }| |¡ | ¡ | ¡ | ¡ | ¡ | ¡ W n t k
|
r } zz| j ¡ t | jt t!j"frtt!j# $| j¡rtzt! %| j¡ W n2 t&k
|
rr t'j(d| j dt)t* d Y nX |W 5 d}~X Y nX W 5 Q R X dS )
|
z>
|
Export DataFrame object to Stata dta format.
|
ÚwbF)r r¬ r3 ÚmethodN)rv rw z!This save was not successful but z. could not be deleted. This file is not valid.rt )+r/ r¥ r r3 r´ r r¯ r r¢ Zcreated_handlesrú Ú _write_headerrî rð Ú
|
_write_mapÚ_write_variable_typesÚ_write_varnamesÚ_write_sortlistÚ_write_formatsÚ_write_value_label_namesÚ_write_variable_labelsÚ_write_expansion_fieldsÚ_write_characteristicsÚ _prepare_dataÚ _write_dataÚ _write_strlsÚ_write_value_labelsÚ_write_file_close_tagÚ_closeÚ Exceptionr° rÌ r ÚosÚPathLikeÚpathÚisfileÚunlinkÚOSErrorr r r® r )rð ÚrecordsÚexcrM rM rN Ú
|
write_filew
|
s\ û ÿ
|
|
ÿ ü zStataWriter.write_filec C sF | j dk rBt| jjts t| jj| j }| j_| jj | ¡ ¡ dS )zÌ
|
Close the file if it was created by the writer.
|
|
If a buffer or file-like object was passed in, for example a GzipFile,
|
then leave this file open for the caller to close.
|
N)r¢ rÌ r´ r¯ r r9 r r
|
)rð r rM rM rN rß ¬
|
s
|
zStataWriter._closec C s dS ©úNo-op, future compatibilityNrM r) rM rM rN rÑ ¹
|
s zStataWriter._write_mapc C s dS rê rM r) rM rM rN rÞ ¼
|
s z!StataWriter._write_file_close_tagc C s dS rê rM r) rM rM rN rÙ ¿
|
s z"StataWriter._write_characteristicsc C s dS rê rM r) rM rM rN rÜ Â
|
s zStataWriter._write_strlsc C s | tdd¡ dS )z"Write 5 zeros for expansion fieldsr½ r6 N)r§ r r) rM rM rN rØ Å
|
s z#StataWriter._write_expansion_fieldsc C s" | j D ]}| | | j¡¡ qd S r+ )r r¨ r rÄ )rð rü rM rM rN rÝ É
|
s
|
z StataWriter._write_value_labels©rv rw rA c C sH | j }| t dd¡¡ | |dkr(dp*d¡ | d¡ | d¡ | t |d | j¡d d
¡ | t |d | j¡d d
|
¡ |d kr¢| | td d ¡¡ n | | t|d d
d ¡¡ |d krÔt j
|
¡ }nt |t j sèt d ddddddddddddg }dd t |D }| d ¡||j | d ¡ }| | |¡¡ d S ) Nr6 r rÞ úúr rJ r r r r½ éP ú"time_stamp should be datetime typeÚJanÚFebÚMarÚAprÚMayÚJunÚJulÚAugÚSepÚOctÚNovÚDecc S s i | ]\}}|d |qS rH rM ©rJ r rV rM rM rN Ú
|
<dictcomp>ü
|
s z-StataWriter._write_header.<locals>.<dictcomp>ú%d ú %Y %H:%M)rÄ r¨ r² r r§ rÅ rÄ Ú_null_terminate_bytesr rI ÚnowrÌ rj r, ÚstrftimerV )rð rv rw rÿ ÚmonthsÚ month_lookupÚtsrM rM rN rÐ Í
|
sJ
|
|
ÿ ôÿþÿzStataWriter._write_headerc C s" | j D ]}| t d|¡¡ qd S )NrÀ )r r¨ r² r )rð rÓ rM rM rN rÒ s
|
z!StataWriter._write_variable_typesc C s6 | j D ]*}| |¡}t|d d
d}| |¡ qd S )Nr r )r Ú_null_terminate_strr r§ )rð rÒ rM rM rN rÓ s
|
|
zStataWriter._write_varnamesc C s" t dd| jd }| |¡ d S )Nr½ r r> )r rÅ r§ )rð ZsrtlistrM rM rN rÔ s zStataWriter._write_sortlistc C s | j D ]}| t|d¡ qd S )Nr )rn r§ r r0 rM rM rN rÕ s
|
zStataWriter._write_formatsc C s` t | jD ]P}| j| rJ| j| }| |¡}t|d d
d}| |¡ q
|
| tdd¡ q
|
d S )Nr r r½ )r rÅ r¡ r r r r§ )rð r rÒ rM rM rN rÖ s
|
|
|
z$StataWriter._write_value_label_namesc C s¤ t dd}| jd kr2t| jD ]}| |¡ q d S | jD ]f}|| jkr| j| }t|dkrdtdtdd |D }|std| t |d¡ q8| |¡ q8d S )Nr½ r rï ú.Variable labels must be 80 characters or fewerc s s | ]}t |d k V qdS )é N)Úordr! rM rM rN rI 4 s z5StataWriter._write_variable_labels.<locals>.<genexpr>zKVariable labels must contain only characters that can be encoded in Latin-1) r r r rÅ r§ r¼ rì rj rs )rð Úblankr rÖ rt Z is_latin1rM rM rN r× & s"
|
|
|
|
|
ÿz"StataWriter._write_variable_labelsc C s |S )rë rM )rð r¼ rM rM rN Ú_convert_strls> s zStataWriter._convert_strlsz np.recarrayc C s | j }| j}| j}| jd k rNt|D ](\}}||kr$t|| | j| ||< q$| |¡}i }| jtt j
|
k}t|D ]\}}|| }|| j krÒ|| d¡j t|fd||< d| } | ||< || | ¡||< qt|| j}
|
|sì|
|
| j¡}
|
|
|
||< qt|jd|dS )Nr½ )Úargsr+ F)rG Z column_dtypes)r¼ r r r, rµ rn r rÄ r£ r¤ rÿ rË r± rª r r rw rN Z
|
to_records) rð r¼ r r r rÖ r- Znative_byteorderrÓ Ústyperw rM rM rN rÚ B s2
|
ÿ
|
|
|
|
|
|
zStataWriter._prepare_data)rç rA c C s | | ¡ ¡ d S r+ )r¨ Útobytes©rð rç rM rM rN rÛ b s zStataWriter._write_datar1 c C s | d7 } | S )Nr rM )r* rM rM rN r e s z StataWriter._null_terminate_strc C s | |¡ | j¡S r+ )r rù rè )rð r* rM rM rN r j s z!StataWriter._null_terminate_bytes)NTNNNNr} N)NN))r r r r rË rè r5 rñ r§ r¨ r¬ r° r² r³ r» r¾ rÀ r¤ rÇ ré rß rÑ rÞ rÙ rÜ rØ rÝ rÐ rÒ rÓ rÔ rÕ rÖ r× r rÚ rÛ Ú staticmethodr r r{ rM rM r¦ rN r · s`
|
R õ ó,&!* IB%5 ý7 r )rw ru r rA c C s |rdS | j tjkr<tt|j}t|d}|dkr8|S dS | j tjkrLdS | j tjkr\dS | j tj krldS | j tj
|
kr|dS | j tj krdS t d | d
|
d S ) a
|
Converts dtype types to stata types. Returns the byte of the given ordinal.
|
See TYPE_MAP and comments for an explanation. This is also explained in
|
the dta spec.
|
1 - 2045 are strings of this length
|
Pandas Stata
|
32768 - for object strL
|
65526 - for int8 byte
|
65527 - for int16 int
|
65528 - for int32 long
|
65529 - for float32 float
|
65530 - for double double
|
|
If there are dates to convert, then dtype will already have the correct
|
type inserted.
|
rB r> r rC rD rE rF rG r r Nr )rw ru r r: rM rM rN Ú_dtype_to_stata_type_117n s&
|
r ú str | bytesrþ c C s( t | trt| d} | d|t| S )zU
|
Takes a bytes instance and pads it with null bytes until it's length chars.
|
râ r )rÌ r rþ rì r rM rM rN Ú_pad_bytes_new s
|
|
r c @ sV e Zd ZdZddddddd d
|
d Zd dd ddZddddZdddddZdS )ÚStataStrLWriteraÑ
|
Converter for Stata StrLs
|
|
Stata StrLs map 8 byte values to strings which are stored using a
|
dictionary-like format where strings are keyed to two values.
|
|
Parameters
|
----------
|
df : DataFrame
|
DataFrame to convert
|
columns : Sequence[str]
|
List of columns names to convert to StrL
|
version : int, optional
|
dta version. Currently supports 117, 118 and 119
|
byteorder : str, optional
|
Can be ">", "<", "little", or "big". default is `sys.byteorder`
|
|
Notes
|
-----
|
Supports creation of the StrL block of a dta file for dta versions
|
117, 118 and 119. These differ in how the GSO is stored. 118 and
|
119 store the GSO lookup value as a uint32 and a uint64, while 117
|
uses two uint32s. 118 and 119 also encode all strings as unicode
|
which is required by the format. 117 uses 'latin-1' a fixed width
|
encoding that extends the 7-bit ascii table with an additional 128
|
characters.
|
rÙ Nr+ rj ra r rÞ )Údfr rÜ rÿ rA c C s |dkrt d|| _|| _|| _ddi| _|d kr:tj}t|| _d}d}d| _ |dkrjd }d}d
|
| _ n|d krxd }nd }ddd| | _
|
|| _ || _ d S )NrØ z,Only dta versions 117, 118 and 119 supportedr½ ©r r rÆ rH râ rÙ r rÜ r» r r6 r rò ) rj Z_dta_verr r Ú
|
_gso_tabler¤ rÿ r£ rÄ rè Ú_o_offetÚ _gso_o_typeÚ _gso_v_type)rð r r rÜ rÿ Z
|
gso_v_typeZ
|
gso_o_typeZo_sizerM rM rN rñ ¾ s,
|
|
zStataStrLWriter.__init__ztuple[int, int])ræ rA c C s |\}}|| j | S r+ )r )rð ræ r« r½ rM rM rN Ú _convert_keyß s z StataStrLWriter._convert_keyz,tuple[dict[str, tuple[int, int]], DataFrame]r@ c sü | j }| j}t|j || j } fdd| jD }tj|jtjd}t| ¡ D ]x\}\}}t|D ]b\} \}
|
} ||
|
} | dkrdn| } |
|
| d¡} | dkr¶| d |d f} | || < | | ¡||| f< qfqRt| jD ] \}}
|
|dd
|f ||
|
< qÖ||fS )aø
|
Generates the GSO lookup table for the DataFrame
|
|
Returns
|
-------
|
gso_table : dict
|
Ordered dictionary using the string found as keys
|
and their lookup position (v,o) as values
|
gso_df : DataFrame
|
DataFrame where strl columns have been converted to
|
(v,o) values
|
|
Notes
|
-----
|
Modifies the DataFrame in-place.
|
|
The DataFrame returned encodes the (v,o) values as uint64s. The
|
encoding depends on the dta version, and can be expressed as
|
|
enc = v + o * 2 ** (o_size * 8)
|
|
so that v is stored in the lower bits and o is in the upper
|
bits. o_size is
|
|
* 117: 4
|
* 118: 6
|
* 119: 5
|
c s g | ]}| |¡fqS rM rP r rH rM rN rO s z2StataStrLWriter.generate_table.<locals>.<listcomp>rv Nr½ r> ) r r rë r r Úemptyr rÁ r, ZiterrowsÚgetr )rð Ú gso_tableZgso_dfÚselectedZ col_indexrq r½ ÚidxÚrowrh rÖ r« rø ræ r rM rH rN Úgenerate_tableã s$
|
|
z StataStrLWriter.generate_tablezdict[str, tuple[int, int]]rþ )r rA c C sò t }tdd}t | jd d¡}t | jd d¡}| j| j }| j| j }| jd }| ¡ D ]\} }
|
|
|
dkrpq^|
|
\} } | |¡ | t || ¡¡ | t || ¡¡ | |¡ t| d} | t |t | d ¡¡ | | ¡ | |¡ q^|
|
¡ S )
|
aç
|
Generates the binary blob of GSOs that is written to the dta file.
|
|
Parameters
|
----------
|
gso_table : dict
|
Ordered dictionary (str, vo)
|
|
Returns
|
-------
|
gso : bytes
|
Binary content of dta file to be placed between strl tags
|
|
Notes
|
-----
|
Output format depends on dta version. 117 uses two uint32s to
|
express v and o while 118+ uses a uint32 for v and a uint64 for o.
|
r? ÚasciirÀ r> r rÆ r râ r> ) r rþ r² r rÄ r r r r rì r
|
)rð r r ZgsoZgso_typeÚnullZv_typeZo_typeZlen_typeZstrlZvor« r½ Z utf8_stringrM rM rN Ú generate_blob s(
|
|
|
|
|
|
z StataStrLWriter.generate_blob)rÙ N)r r r r rñ r r# r& rM rM rM rN r ¡ s û!3r c sl e Zd ZdZdZdZdFdddd d
|
d d d d d
|
dddddd fddZeddddddZddd d d Z dGd d dd d d!Z
|
dd"d#d$Z dd"d%d&Z dd"d'd(Z dd"d)d*Zdd"d+d,Zdd"d-d.Zdd"d/d0Zdd"d1d2Zdd"d3d4Zdd"d5d6Zdd"d7d8Zdd"d9d:Zdd"d;d<Zdd"d=d>Zd d d?d@dAZdBddCdDdEZ ZS )HÚStataWriter117a«
|
A class for writing Stata binary dta files in Stata 13 format (117)
|
|
Parameters
|
----------
|
fname : path (string), buffer or path object
|
string, path object (pathlib.Path or py._path.local.LocalPath) or
|
object implementing a binary write() functions. If using a buffer
|
then the buffer will not be automatically closed after the file
|
is written.
|
data : DataFrame
|
Input to save
|
convert_dates : dict
|
Dictionary mapping columns containing datetime types to stata internal
|
format to use when writing the dates. Options are 'tc', 'td', 'tm',
|
'tw', 'th', 'tq', 'ty'. Column can be either an integer or a name.
|
Datetime columns that do not have a conversion type specified will be
|
converted to 'tc'. Raises NotImplementedError if a datetime column has
|
timezone information
|
write_index : bool
|
Write the index to Stata dataset.
|
byteorder : str
|
Can be ">", "<", "little", or "big". default is `sys.byteorder`
|
time_stamp : datetime
|
A datetime to use as file creation date. Default is the current time
|
data_label : str
|
A label for the data set. Must be 80 characters or smaller.
|
variable_labels : dict
|
Dictionary containing columns as keys and variable labels as values.
|
Each label must be 80 characters or smaller.
|
convert_strl : list
|
List of columns names to convert to Stata StrL format. Columns with
|
more than 2045 characters are automatically written as StrL.
|
Smaller columns can be converted by including the column name. Using
|
StrLs can reduce output file size when strings are longer than 8
|
characters, and either frequently repeated or sparse.
|
{compression_options}
|
|
.. versionadded:: 1.1.0
|
|
.. versionchanged:: 1.4.0 Zstandard support.
|
|
value_labels : dict of dicts
|
Dictionary containing columns as keys and dictionaries of column value
|
to labels as values. The combined length of all labels for a single
|
variable must be 32,000 characters or smaller.
|
|
.. versionadded:: 1.4.0
|
|
Returns
|
-------
|
writer : StataWriter117 instance
|
The StataWriter117 instance has a write_file method, which will
|
write the file to the given `fname`.
|
|
Raises
|
------
|
NotImplementedError
|
* If datetimes contain timezone information
|
ValueError
|
* Columns listed in convert_dates are neither datetime64[ns]
|
or datetime.datetime
|
* Column dtype is not representable in Stata
|
* Column listed in convert_dates is not in DataFrame
|
* Categorical label contains more than 32,000 characters
|
|
Examples
|
--------
|
>>> data = pd.DataFrame([[1.0, 1, 'a']], columns=['a', 'b', 'c'])
|
>>> writer = pd.io.stata.StataWriter117('./data_file.dta', data)
|
>>> writer.write_file()
|
|
Directly write a zip file
|
>>> compression = {"method": "zip", "archive_name": "data_file.dta"}
|
>>> writer = pd.io.stata.StataWriter117(
|
... './data_file.zip', data, compression=compression
|
... )
|
>>> writer.write_file()
|
|
Or with long strings stored in strl format
|
>>> data = pd.DataFrame([['A relatively long string'], [''], ['']],
|
... columns=['strls'])
|
>>> writer = pd.io.stata.StataWriter117(
|
... './data_file_with_long_strings.dta', data, convert_strl=['strls'])
|
>>> writer.write_file()
|
r rÙ NTr} r r r+ r r r r úSequence[Hashable] | Noner r r rÞ ) r r¼ r r rÿ rw rv rx rÌ r r3 rí rA c sL g | _ | d k r| j | ¡ t j||||||||| |
|
| d i | _d| _d S )N)rÿ rw rv rx rí r r3 r¥ )rÊ rà r rñ Ú_mapÚ
|
_strl_blob) rð r r¼ r r rÿ rw rv rx rÌ r r3 rí r¦ rM rN rñ ± s$ õ zStataWriter117.__init__r r rþ )rø ÚtagrA c C s< t | trt| d} td| d d| td| d d S )z Surround val with <tag></tag>râ rß rÞ z</)rÌ r rþ )rø r+ rM rM rN Ú_tagÖ s
|
|
zStataWriter117._tag)r+ rA c C s& | j jdk st| j j ¡ | j|< dS )z.Update map location for tag with file positionN)r´ r¯ r9 r' r) )rð r+ rM rM rN Ú _update_mapÝ s zStataWriter117._update_maprì c C sÒ | j }| tdd¡ t }| | tt| jdd¡¡ | | |dkrNdpPdd¡¡ | jdkrfd nd
|
}| | t || | j
|
¡d ¡¡ | jd krd
|
nd }| | t || | j ¡d¡¡ |dk rÐ|dd
nd}| | j ¡}| jd krîdnd } t || t|¡}
|
|
|
| }| | |d¡¡ |dkr6tj ¡ }nt|tjsLtddddddddd d d d d g } d!d" t| D } | d#¡| |j | d$¡ } d%t| d }| | |d&¡¡ | | | ¡ d'¡¡ dS )(zWrite the file headerz <stata_dta>râ ÚreleaserÞ ZMSFZLSFrÿ r» r rÆ ÚKrÙ rH ÚNNrï r½ rÀ rt rð rñ rò ró rô rõ rö r÷ rø rù rú rû rü c S s i | ]\}}|d |qS rH rM rý rM rM rN rþ s z0StataWriter117._write_header.<locals>.<dictcomp>rÿ r ó Ú timestampÚheader)rÄ r¨ rþ r r r, r Ú _dta_versionr² r rÅ rÄ rù rè rì rI r rÌ rj r, r rV r
|
)rð rv rw rÿ r Z nvar_typeZ nobs_sizert Z encoded_labelZ
|
label_sizeZ label_lenr r r Zstata_tsrM rM rN rÐ â sT
|
ôÿþÿz StataWriter117._write_headerr@ c C s | j s2d| jj ¡ ddddddddddddd| _ | jj | j d ¡ t }| j ¡ D ] }| t | j
|
d |¡¡ qV| | | ¡ d¡¡ dS )zÇ
|
Called twice during file write. The first populates the values in
|
the map with 0s. The second call writes the final map locations when
|
all blocks have been written.
|
r )Z
|
stata_dataÚmapÚvariable_typesÚvarnamesÚsortlistÚformatsÚvalue_label_namesrx Úcharacteristicsr¼ Ústrlsrí Ústata_data_closeú end-of-filer5 rH N)r) r´ r¯ r' rø r rk r r² r rÄ r¨ r, r
|
)rð r rø rM rM rN rÑ s*
|
òzStataWriter117._write_mapc C sN | d¡ t }| jD ] }| t | jd |¡¡ q| | | ¡ d¡¡ d S )Nr6 r )
|
r- r r r r² r rÄ r¨ r, r
|
)rð r rÓ rM rM rN rÒ : s
|
|
|
z$StataWriter117._write_variable_typesc C sz | d¡ t }| jdkr dnd}| jD ]6}| |¡}t|d d
| j¡|d }| |¡ q(| |
|
| ¡ d¡¡ d S )Nr7 rÙ r r r> ) r- r r4 r r r rù rè r r¨ r, r
|
)rð r Zvn_lenrÒ rM rM rN rÓ A s
|
|
|
z StataWriter117._write_varnamesc C s@ | d¡ | jdk rdnd}| | d| | jd d¡¡ d S )Nr8 rÚ r r r r> )r- r4 r¨ r, rÅ )rð Z sort_sizerM rM rN rÔ L s
|
z StataWriter117._write_sortlistc C s` | d¡ t }| jdkr dnd}| jD ] }| t| | j¡|¡ q(| | |
|
¡ d¡¡ d S )Nr9 rÙ r r ) r- r r4 rn r r rù rè r¨ r, r
|
)rð r Zfmt_lenr rM rM rN rÕ Q s
|
|
z StataWriter117._write_formatsc C s | d¡ t }| jdkr dnd}t| jD ]N}d}| j| rH| j| }| |¡}t|d d
| j
|
¡|d }| |¡ q,| | | ¡ d¡¡ d S )Nr: rÙ r r r½ r> )r- r r4 r rÅ r¡ r r r rù rè r r¨ r, r
|
)rð r Úvl_lenr rÒ Ú encoded_namerM rM rN rÖ Y s
|
|
|
|
z'StataWriter117._write_value_label_namesc C s" | d¡ t }| jdkr dnd}td|d }| jd krnt| jD ]}| |¡ qD| | |
|
¡ d¡¡ d S | j D ]}|| jkrü| j| }t |dkr t dz| | j¡}W n4 tk
|
rä } zt d| j |W 5 d }~X Y nX | t||d ¡ qt| |¡ qt| | |
|
¡ d¡¡ d S ) Nrx rÙ rï i@ r½ r> r zDVariable labels must contain only characters that can be encoded in )r- r r4 r r r rÅ r r¨ r, r
|
r¼ rì rj rù rè ÚUnicodeEncodeError) rð r r? r r rÖ rt rÍ r rM rM rN r× h s2
|
|
|
|
|
|
ÿý z%StataWriter117._write_variable_labelsc C s | d¡ | | dd¡¡ d S )Nr; r¥ )r- r¨ r, r) rM rM rN rÙ s
|
z%StataWriter117._write_characteristicsc C s0 | d¡ | d¡ | | ¡ ¡ | d¡ d S )Nr¼ s <data>s </data>)r- r¨ r r rM rM rN rÛ s
|
|
zStataWriter117._write_datac C s" | d¡ | | | jd¡¡ d S )Nr< )r- r¨ r, r* r) rM rM rN rÜ s
|
zStataWriter117._write_strlsc C s dS )zNo-op in dta 117+NrM r) rM rM rN rØ s z&StataWriter117._write_expansion_fieldsc C sX | d¡ t }| jD ]&}| | j¡}| |d¡}| |¡ q| | | ¡ d¡¡ d S )Nrí Zlbl) r- r r r rÄ r, r r¨ r
|
)rð r rü ZlabrM rM rN rÝ s
|
|
z"StataWriter117._write_value_labelsc C s( | d¡ | tdd¡ | d¡ d S )Nr= z </stata_dta>râ r> )r- r¨ rþ r) rM rM rN rÞ ¢ s
|
z$StataWriter117._write_file_close_tagc C s8 | j ¡ D ](\}}|| jkr
|
| j |¡}|| j|< q
|
dS )z
|
Update column names for conversion to strl if they might have been
|
changed to comply with Stata naming rules
|
N)r£ r rÊ rG )rð r× Únewr! rM rM rN r³ § s
|
z!StataWriter117._update_strl_namesr» c sJ fddt |D }|rFt|| jd}| ¡ \}}|}| |¡ _|S )zg
|
Convert columns to StrLs if either very large or in the
|
convert_strl variable
|
c s, g | ]$\}} j | d ks$| jkr|qS )rB )r rÊ )rJ r rÖ r) rM rN rO · s
|
þz1StataWriter117._convert_strls.<locals>.<listcomp>rÛ )r, r r4 r# r& r* )rð r¼ Z convert_colsZsswÚtabZnew_datarM r) rN r ² s
|
þ z StataWriter117._convert_strlsr- r¿ c C sj g | _ g | _| ¡ D ]P\}}|| jk}t|| j| | j|d}| j |¡ | j t|| j| |¡ qd S )N)r r ) r rn r rÊ r r¼ r4 rú r )rð r- rÖ rw r r rM rM rN rÀ Ä s
|
ü ÿz%StataWriter117._set_formats_and_types) NTNNNNNr} N)NN) r r r r rË r4 rñ r r, r- rÐ rÑ rÒ rÓ rÔ rÕ rÖ r× rÙ rÛ rÜ rØ rÝ rÞ r³ r rÀ r{ rM rM r¦ rN r' V sJ W ôò.% ý: r' c sf e Zd ZU dZdZded< d ddd d
|
d d d dd d ddddddd fddZdddddZ ZS ) ÚStataWriterUTF8u
|
Stata binary dta file writing in Stata 15 (118) and 16 (119) formats
|
|
DTA 118 and 119 format files support unicode string data (both fixed
|
and strL) format. Unicode is also supported in value labels, variable
|
labels and the dataset label. Format 119 is automatically used if the
|
file contains more than 32,767 variables.
|
|
Parameters
|
----------
|
fname : path (string), buffer or path object
|
string, path object (pathlib.Path or py._path.local.LocalPath) or
|
object implementing a binary write() functions. If using a buffer
|
then the buffer will not be automatically closed after the file
|
is written.
|
data : DataFrame
|
Input to save
|
convert_dates : dict, default None
|
Dictionary mapping columns containing datetime types to stata internal
|
format to use when writing the dates. Options are 'tc', 'td', 'tm',
|
'tw', 'th', 'tq', 'ty'. Column can be either an integer or a name.
|
Datetime columns that do not have a conversion type specified will be
|
converted to 'tc'. Raises NotImplementedError if a datetime column has
|
timezone information
|
write_index : bool, default True
|
Write the index to Stata dataset.
|
byteorder : str, default None
|
Can be ">", "<", "little", or "big". default is `sys.byteorder`
|
time_stamp : datetime, default None
|
A datetime to use as file creation date. Default is the current time
|
data_label : str, default None
|
A label for the data set. Must be 80 characters or smaller.
|
variable_labels : dict, default None
|
Dictionary containing columns as keys and variable labels as values.
|
Each label must be 80 characters or smaller.
|
convert_strl : list, default None
|
List of columns names to convert to Stata StrL format. Columns with
|
more than 2045 characters are automatically written as StrL.
|
Smaller columns can be converted by including the column name. Using
|
StrLs can reduce output file size when strings are longer than 8
|
characters, and either frequently repeated or sparse.
|
version : int, default None
|
The dta version to use. By default, uses the size of data to determine
|
the version. 118 is used if data.shape[1] <= 32767, and 119 is used
|
for storing larger DataFrames.
|
{compression_options}
|
|
.. versionadded:: 1.1.0
|
|
.. versionchanged:: 1.4.0 Zstandard support.
|
|
value_labels : dict of dicts
|
Dictionary containing columns as keys and dictionaries of column value
|
to labels as values. The combined length of all labels for a single
|
variable must be 32,000 characters or smaller.
|
|
.. versionadded:: 1.4.0
|
|
Returns
|
-------
|
StataWriterUTF8
|
The instance has a write_file method, which will write the file to the
|
given `fname`.
|
|
Raises
|
------
|
NotImplementedError
|
* If datetimes contain timezone information
|
ValueError
|
* Columns listed in convert_dates are neither datetime64[ns]
|
or datetime.datetime
|
* Column dtype is not representable in Stata
|
* Column listed in convert_dates is not in DataFrame
|
* Categorical label contains more than 32,000 characters
|
|
Examples
|
--------
|
Using Unicode data and column names
|
|
>>> from pandas.io.stata import StataWriterUTF8
|
>>> data = pd.DataFrame([[1.0, 1, 'á´¬']], columns=['a', 'β', 'Ä'])
|
>>> writer = StataWriterUTF8('./data_file.dta', data)
|
>>> writer.write_file()
|
|
Directly write a zip file
|
>>> compression = {"method": "zip", "archive_name": "data_file.dta"}
|
>>> writer = StataWriterUTF8('./data_file.zip', data, compression=compression)
|
>>> writer.write_file()
|
|
Or with long strings stored in strl format
|
|
>>> data = pd.DataFrame([['á´ relatively long Åtring'], [''], ['']],
|
... columns=['strls'])
|
>>> writer = StataWriterUTF8('./data_file_with_long_strings.dta', data,
|
... convert_strl=['strls'])
|
>>> writer.write_file()
|
râ zLiteral['utf-8']rè NTr} r r r+ r r r r r( r r r r rÞ )r r¼ r r rÿ rw rv rx rÌ rÜ r r3 rí rA c s~ |
|
d kr |j d dkrdnd}
|
n0|
|
dkr2tdn |
|
dkrP|j d dkrPtdt j||||||||| | | | d |
|
| _d S ) Nr> iÿ r» rÚ )r» rÚ z"version must be either 118 or 119.zKYou must use version 119 for data sets containing more than32,767 variables)
|
r r rÿ rw rv rx rí rÌ r r3 )r rj r rñ r4 )rð r r¼ r r rÿ rw rv rx rÌ rÜ r r3 rí r¦ rM rN rñ : s.
|
ÿôzStataWriterUTF8.__init__r r´ c C s |D ]v}t |dk rL|dk s$|dkrL|dk s4|dkrL|dk sD|dkrL|dksndt | krdd k snn |d
|
kr| |d¡}q|S ) aÕ
|
Validate variable names for Stata export.
|
|
Parameters
|
----------
|
name : str
|
Variable name
|
|
Returns
|
-------
|
str
|
The validated name with invalid characters replaced with
|
underscores.
|
|
Notes
|
-----
|
Stata 118+ support most unicode characters. The only limitation is in
|
the ascii range where the characters supported are a-z, A-Z, 0-9 and _.
|
r rµ r¶ r< r· r= r¸ r éÀ > õ Ãõ ÷)r
|
r¹ rº rM rM rN r» f s4
|
þýýüüûûú øø ÷ z'StataWriterUTF8._validate_variable_name)
|
NTNNNNNNr} N) r r r r rè r5 rñ r» r{ rM rM r¦ rN rD Õ s
|
b óñ0,rD )r F)sr Ú
|
__future__r Ú collectionsr rI Úior rá r² r¤ Útypesr Útypingr r r r r
|
r r r r r Zdateutil.relativedeltar Únumpyr Zpandas._libs.libr Zpandas._libs.writersr Zpandas._typingr r r r r Z pandas.errorsr r r r Zpandas.util._decoratorsr r Zpandas.util._exceptionsr Zpandas.core.dtypes.commonr r r r! Zpandasr" r# r$ r% r& r' r( Zpandas.core.arrays.booleanr) Zpandas.core.arrays.integerr* Zpandas.core.framer+ Zpandas.core.indexes.baser, Zpandas.core.seriesr- Zpandas.core.shared_docsr. Zpandas.io.commonr/ r0 ré Z _statafile_processing_params1Z _statafile_processing_params2Z_chunksize_paramsZ_iterator_paramsZ _reader_notesZ_read_stata_docrz ry rJ r? r5 r rµ r¶ r· r¸ r¹ rº rÚ rÛ r rÐ r: ÚIteratorr{ r~ r£ r r r r r r r r r r' rD rM rM rM rN Ú<module> s , $ ÿñðïî
|
íì á:ùøø ÷
|
ö õ ôò 'l t| 4ó,$ % ÿ-
|
þ 8* 6
|
