U
|
¬ý°di³ ã @ s" U d Z ddlmZ ddlmZ ddlZddlmZmZ ddl Z ddl
|
Z
|
ddl Z ddl m Z ddlmZ ddlmZmZmZmZmZmZmZmZmZmZ ddlZddl Z dd l m Z m Z dd
|
l!m"Z"m#Z$ dd l%m&Z& dd l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-m.Z. dd l/m0Z0 ddl1m2Z2 ddl3m4Z4m5Z5m6Z6m7Z7m8Z8 ddl9m:Z: ddl;m<Z< ddl=m>Z>m?Z?m@Z@mAZAmBZBmCZCmDZDmEZEmFZFmGZGmHZHmIZImJZJ ddlKmLZL ddlMmNZNmOZOmPZPmQZQmRZRmSZSmTZTmUZUmVZVmWZW ddlXmYZYmZZZm[Z[ ddl\m] m^Z_ ddl`maZambZb ddlcmdZd ddlemfZf ddlgmhZhmiZi ddljmkZk ddllmmZmmnZn erZdd lompZpmqZqmrZr dd lgmsZs d Ztd Zud d! Zvd"d#d$d%d&Zwd'd( ZxeaZyd)d*d+d,Zzd-Z{d.e|d/< d0Z}d.e|d1< d2Z~d.e|d3< d4d4d5d5d6ZeNdgiZd7Zd.e|d8< d9Zd.e|d:< e d;¡8 e jd<d=ee j
d> e jd?dee d4d5dg¡d> W 5 Q R X dad=ad@dA Zd¼dEd#dFd#dGd"dHd"dHdIdJdKd#d#dLdMdNdOZd½dEd#d#dQdGdGdRdHdGdS dTdUZdVdVdHdWdXdYZG dZd[ d[ZG d\d] d]ZG d^d_ d_ZG d`da daeZG dbdc dceZG ddde deeZG dfdg dgeZG dhdi diZG djdk dkeZG dldm dmeZG dndo doeZG dpdq dqeZG drds dseZG dtdu dueZG dvdw dweZG dxdy dyeZG dzd{ d{eZG d|d} d}eZG d~d deZG dd deZ d¾ddddd
ddZ¡dddddZ¢ed¿dddHddddZ£edÀddLdHddddZ£dÁdddHddddZ£d#dd#d#d_dddZ¤d#d#d#ddddZ¥d#dddd d¡Z¦dd#d#dd¢d£d¤Z§dd#d#dd¢d¥d¦Z¨dd#d#d#d§d¨d©Z©d#d#d#dªd«d¬Zªd#dHdd®d¯Z«d#d°d#d±d²d³Z¬d#d#d´dµd¶Zdd·d¸d¹Z®G dºd» d»Z¯dS )ÂzY
|
High level interface to PyTables for reading and writing pandas data structures
|
to disk
|
é )Ú annotations)ÚsuppressN)ÚdateÚtzinfo)Údedent)Ú TracebackType)
|
Ú TYPE_CHECKINGÚAnyÚCallableÚFinalÚHashableÚIteratorÚLiteralÚSequenceÚcastÚoverload)ÚconfigÚ
|
get_option)ÚlibÚwriters)Ú timezones)Ú AnyArrayLikeÚ ArrayLikeÚAxisIntÚDtypeArgÚFilePathÚShapeÚnpt)Úimport_optional_dependency)Ú patch_pickle)ÚAttributeConflictWarningÚClosedFileErrorÚIncompatibilityWarningÚPerformanceWarningÚPossibleDataLossError)Úcache_readonly)Úfind_stack_level) Ú ensure_objectÚ is_bool_dtypeÚis_categorical_dtypeÚis_complex_dtypeÚis_datetime64_dtypeÚis_datetime64tz_dtypeÚis_extension_array_dtypeÚis_integer_dtypeÚ is_list_likeÚis_object_dtypeÚis_string_dtypeÚis_timedelta64_dtypeÚneeds_i8_conversion)Úarray_equivalent)
|
Ú DataFrameÚ DatetimeIndexÚIndexÚ
|
MultiIndexÚ PeriodIndexÚ
|
RangeIndexÚSeriesÚTimedeltaIndexÚconcatÚisna)Ú CategoricalÚ DatetimeArrayÚ PeriodArray)Ú PyTablesExprÚmaybe_expression)Ú extract_array)Ú ensure_index)Ú ArrayManagerÚ BlockManager)Ústringify_path)ÚadjoinÚ pprint_thing)ÚColÚFileÚNode)ÚBlockz0.15.2úUTF-8c C s t | tjr| d¡} | S )z(if we have bytes, decode them to unicoderO )Ú
|
isinstanceÚnpÚbytes_Údecode)Ús© rU úId:\z\workplace\vscode\pyvenv\venv\Lib\site-packages\pandas/io/pytables.pyÚ_ensure_decoded s
|
rW ú
|
str | NoneÚstr)ÚencodingÚreturnc C s | d kr t } | S ©N)Ú_default_encoding©rZ rU rU rV Ú_ensure_encoding s r_ c C s t | trt| } | S )zÓ
|
Ensure that an index / column name is a str (python 3); otherwise they
|
may be np.string dtype. Non-string dtypes are passed through unchanged.
|
|
https://github.com/pandas-dev/pandas/issues/13492
|
)rP rY ©ÚnamerU rU rV Ú _ensure_str s
|
rb Úint©Ú scope_levelc sV |d t | ttfr* fdd| D } nt| r>t| d} | dksNt| rR| S dS )zÔ
|
Ensure that the where is a Term or a list of Term.
|
|
This makes sure that we are capturing the scope of variables that are
|
passed create the terms here with a frame_level=2 (we are 2 levels down)
|
é c s0 g | ](}|d k rt |r(t| d dn|qS )Nrf rd )rC ÚTerm)Ú.0Zterm©ÚlevelrU rV Ú
|
<listcomp>¯ s þz _ensure_term.<locals>.<listcomp>rd N)rP ÚlistÚtuplerC rg Úlen)Úwherere rU ri rV Ú _ensure_term¤ s
|
þ rp z¨
|
where criteria is being ignored as this version [%s] is too old (or
|
not-defined), read the file in and write it out to a new file to upgrade (with
|
the copy_to method)
|
r Úincompatibility_doczu
|
the [%s] attribute of the existing index is [%s] which conflicts with the new
|
[%s], resetting the attribute to None
|
Úattribute_conflict_docz
|
your performance may suffer as PyTables will pickle object types that it cannot
|
map directly to c-types [inferred_type->%s,key->%s] [items->%s]
|
Úperformance_docÚfixedÚtable)Úfrt Útru z;
|
: boolean
|
drop ALL nan rows when appending to a table
|
Ú
|
dropna_docz~
|
: format
|
default format writing format, if None, then
|
put will default to 'fixed' and append will default to 'table'
|
Ú
|
format_doczio.hdfZ dropna_tableF)Z validatorÚdefault_formatc C s8 t d kr4dd l} | a tt | jjdkaW 5 Q R X t S )Nr Ústrict)Ú
|
_table_modÚtablesr ÚAttributeErrorÚfileZ_FILE_OPEN_POLICYÚ!_table_file_open_policy_is_strict)r} rU rU rV Ú_tablesè s
|
|
ÿ r ÚaTr{ zFilePath | HDFStoreúDataFrame | Seriesú
|
int | NoneÚboolúint | dict[str, int] | Noneú bool | Noneú Literal[True] | list[str] | NoneÚNone)Ú path_or_bufÚkeyÚvalueÚmodeÚ complevelÚcomplibÚappendÚformatÚindexÚ min_itemsizeÚdropnaÚ data_columnsÚerrorsrZ r[ c
|
s |r$ f
|
dd}n f
|
dd}t | } t| trzt| |||d}|| W 5 Q R X n|| dS )z+store this object, close it if we opened itc s | j d
|
S )N)r r r Únan_repr r r rZ )r ©Ústore©
|
r r rZ r r r r r r r rU rV Ú<lambda> s özto_hdf.<locals>.<lambda>c s | j d
|
S )N)r r r r r r rZ r ©Úputr r rU rV r s ö)r r r N)rH rP rY ÚHDFStore)r r r r r r r r r r r r r r rZ rv r rU r rV Úto_hdfþ s
|
ÿr Úrzstr | list | Nonezlist[str] | None) r r r ro ÚstartÚstopÚcolumnsÚiteratorÚ chunksizec
|
K s |dkrt d| d|dk r,t|dd}t| trN| jsDtd| } d} nvt| } t| tshtd zt j
|
| ¡} W n t t fk
|
r d} Y nX | sªt d
|
| d t| f||d |
|
} d } zx|dkr"| ¡ }t|dkrìt d|d }|dd
D ]}t||s t dq |j}| j||||||| | dW S t t tfk
|
r t| ts|tt | ¡ W 5 Q R X Y nX dS )a"
|
Read from the store, close it if we opened it.
|
|
Retrieve pandas object stored in file, optionally based on where
|
criteria.
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
Parameters
|
----------
|
path_or_buf : str, path object, pandas.HDFStore
|
Any valid string path is acceptable. Only supports the local file system,
|
remote URLs and file-like objects are not supported.
|
|
If you want to pass in a path object, pandas accepts any
|
``os.PathLike``.
|
|
Alternatively, pandas accepts an open :class:`pandas.HDFStore` object.
|
|
key : object, optional
|
The group identifier in the store. Can be omitted if the HDF file
|
contains a single pandas object.
|
mode : {'r', 'r+', 'a'}, default 'r'
|
Mode to use when opening the file. Ignored if path_or_buf is a
|
:class:`pandas.HDFStore`. Default is 'r'.
|
errors : str, default 'strict'
|
Specifies how encoding and decoding errors are to be handled.
|
See the errors argument for :func:`open` for a full list
|
of options.
|
where : list, optional
|
A list of Term (or convertible) objects.
|
start : int, optional
|
Row number to start selection.
|
stop : int, optional
|
Row number to stop selection.
|
columns : list, optional
|
A list of columns names to return.
|
iterator : bool, optional
|
Return an iterator object.
|
chunksize : int, optional
|
Number of rows to include in an iteration when using an iterator.
|
**kwargs
|
Additional keyword arguments passed to HDFStore.
|
|
Returns
|
-------
|
object
|
The selected object. Return type depends on the object stored.
|
|
See Also
|
--------
|
DataFrame.to_hdf : Write a HDF file from a DataFrame.
|
HDFStore : Low-level access to HDF files.
|
|
Examples
|
--------
|
>>> df = pd.DataFrame([[1, 1.0, 'a']], columns=['x', 'y', 'z']) # doctest: +SKIP
|
>>> df.to_hdf('./store.h5', 'data') # doctest: +SKIP
|
>>> reread = pd.read_hdf('./store.h5') # doctest: +SKIP
|
)r úr+r zmode zG is not allowed while performing a read. Allowed modes are r, r+ and a.Nrf rd z&The HDFStore must be open for reading.Fz5Support for generic buffers has not been implemented.zFile z does not exist)r r Tr z]Dataset(s) incompatible with Pandas data types, not table, or no datasets found in HDF5 file.z?key must be provided when HDF5 file contains multiple datasets.)ro r¡ r¢ r£ r¤ r¥ Ú
|
auto_close)Ú
|
ValueErrorrp rP r Úis_openÚOSErrorrH rY ÚNotImplementedErrorÚosÚpathÚexistsÚ TypeErrorÚFileNotFoundErrorÚgroupsrn Ú_is_metadata_ofÚ _v_pathnameÚselectÚKeyErrorr r~ Úclose)r r r r ro r¡ r¢ r£ r¤ r¥ Úkwargsr r§ r® r± Zcandidate_only_groupZgroup_to_checkrU rU rV Úread_hdf6 sj O
|
ÿ
|
|
ÿ
|
|
ÿ ÿø
|
|
r¸ rM )ÚgroupÚ parent_groupr[ c C sF | j |j krdS | }|j dkrB|j}||kr:|jdkr:dS |j}qdS )zDCheck if a given group is a metadata group for a given parent_group.Frf ÚmetaT)Z_v_depthZ _v_parentÚ_v_name)r¹ rº ÚcurrentÚparentrU rU rV r² Ñ s
|
r² c @ s( e Zd ZU dZded< ded< ddd d
|
d d d dZddddZedd ZeddddZ ddddZ
|
dd dddZ dd dd d Z dd d d Z dd
|
dd!d"Zd#dd$d%Zddd&d'Zd dd(d)Zd*d+d,d d-d.d/Zddd1d2d3d4Zd5dd6d7Zd8dd9d:Zddd d;d<d=Zd dd>d?Zed
|
dd@dAZdd
|
d dBdCdDZdddEdFZddd
|
d
|
dGdHdIZddd d dJdKdLZ dddd d dMdNdOZ dd
|
d
|
dPdQdRZ dddUd
|
d
|
d dVdWdd
|
d
|
d dX dYdZZ ddd dd[d\Z dddUd]d
|
d dVd^dWdd d_
|
d`daZ!ddbd
|
d dcdddeZ"d dd dfd dgdhdiZ#djddkdlZ$d¡ddndodpdqZ%ddrddsdtZ&dduddvdwZ'd¢dd
|
d d
|
d
|
d dydzd{Z(ddd|d}Z)d~d Z*dddddZ+d£ddddud
ddZ,d¤ddUd]d
|
d dVd
|
dd
|
d d
|
ddZ-ddddZ.dd
|
ddddZ/dddddZ0dS )¥r aa
|
Dict-like IO interface for storing pandas objects in PyTables.
|
|
Either Fixed or Table format.
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
Parameters
|
----------
|
path : str
|
File path to HDF5 file.
|
mode : {'a', 'w', 'r', 'r+'}, default 'a'
|
|
``'r'``
|
Read-only; no data can be modified.
|
``'w'``
|
Write; a new file is created (an existing file with the same
|
name would be deleted).
|
``'a'``
|
Append; an existing file is opened for reading and writing,
|
and if the file does not exist it is created.
|
``'r+'``
|
It is similar to ``'a'``, but the file must already exist.
|
complevel : int, 0-9, default None
|
Specifies a compression level for data.
|
A value of 0 or None disables compression.
|
complib : {'zlib', 'lzo', 'bzip2', 'blosc'}, default 'zlib'
|
Specifies the compression library to be used.
|
As of v0.20.2 these additional compressors for Blosc are supported
|
(default if no compressor specified: 'blosc:blosclz'):
|
{'blosc:blosclz', 'blosc:lz4', 'blosc:lz4hc', 'blosc:snappy',
|
'blosc:zlib', 'blosc:zstd'}.
|
Specifying a compression library which is not available issues
|
a ValueError.
|
fletcher32 : bool, default False
|
If applying compression use the fletcher32 checksum.
|
**kwargs
|
These parameters will be passed to the PyTables open_file method.
|
|
Examples
|
--------
|
>>> bar = pd.DataFrame(np.random.randn(10, 4))
|
>>> store = pd.HDFStore('test.h5')
|
>>> store['foo'] = bar # write to HDF5
|
>>> bar = store['foo'] # retrieve
|
>>> store.close()
|
|
**Create or load HDF5 file in-memory**
|
|
When passing the `driver` option to the PyTables open_file method through
|
**kwargs, the HDF5 file is loaded or created in-memory and will only be
|
written when closed:
|
|
>>> bar = pd.DataFrame(np.random.randn(10, 4))
|
>>> store = pd.HDFStore('test.h5', driver='H5FD_CORE')
|
>>> store['foo'] = bar
|
>>> store.close() # only now, data is written to disk
|
z File | NoneÚ_handlerY Ú_moder NFr r
r )r r Ú
|
fletcher32r[ c K s² d|krt dtd}|d k r@||jjkr@t d|jj d|d krX|d k rX|jj}t|| _|d krnd}|| _d | _|r|nd| _ || _
|
|| _ d | _ | j f d|i| d S ) Nr z-format is not a defined argument for HDFStorer} zcomplib only supports z compression.r r r )r¨ r ÚfiltersZ all_complibsZdefault_complibrH Ú_pathrÀ r¿ Ú
|
_complevelÚ_complibÚ _fletcher32Ú_filtersÚopen)Úselfr r r r rÁ r· r} rU rU rV Ú__init__$ s& ÿ
|
zHDFStore.__init__©r[ c C s | j S r\ ©rà ©rÉ rU rU rV Ú
|
__fspath__E s zHDFStore.__fspath__c C s | ¡ | jdk st| jjS )zreturn the root nodeN)Ú_check_if_openr¿ ÚAssertionErrorÚrootrÍ rU rU rV rÑ H s z HDFStore.rootc C s | j S r\ rÌ rÍ rU rU rV ÚfilenameO s zHDFStore.filename©r c C s
|
| |¡S r\ )Úget©rÉ r rU rU rV Ú __getitem__S s zHDFStore.__getitem__©r r[ c C s | ||¡ d S r\ r )rÉ r r rU rU rV Ú __setitem__V s zHDFStore.__setitem__c C s
|
| |¡S r\ )ÚremoverÕ rU rU rV Ú __delitem__Y s zHDFStore.__delitem__r` c C sF z | |¡W S ttfk
|
r$ Y nX tdt| j d| ddS )z$allow attribute access to get storesú'z' object has no attribute 'N)rÔ rµ r! r~ ÚtypeÚ__name__)rÉ ra rU rU rV Ú __getattr__\ s ÿzHDFStore.__getattr__c C s4 | |¡}|dk r0|j}|||dd
fkr0dS dS )zx
|
check for existence of this key
|
can match the exact pathname or the pathnm w/o the leading '/'
|
Nrf TF)Úget_noder³ )rÉ r Únodera rU rU rV Ú __contains__f s
|
zHDFStore.__contains__rc c C s t | ¡ S r\ )rn r± rÍ rU rU rV Ú__len__r s zHDFStore.__len__c C s t | j}t| d| dS )Nú
|
File path: Ú
|
)rJ rà rÜ )rÉ ÚpstrrU rU rV Ú__repr__u s
|
zHDFStore.__repr__c C s | S r\ rU rÍ rU rU rV Ú __enter__y s zHDFStore.__enter__ztype[BaseException] | NonezBaseException | NonezTracebackType | None)Úexc_typeÚ exc_valueÚ tracebackr[ c C s | ¡ d S r\ )r¶ )rÉ rè ré rê rU rU rV Ú__exit__| s zHDFStore.__exit__Úpandasú list[str])Úincluder[ c C s^ |dkrdd | ¡ D S |dkrJ| jdk s0tdd | jjddd D S td
|
| d dS ) a#
|
Return a list of keys corresponding to objects stored in HDFStore.
|
|
Parameters
|
----------
|
|
include : str, default 'pandas'
|
When kind equals 'pandas' return pandas objects.
|
When kind equals 'native' return native HDF5 Table objects.
|
|
.. versionadded:: 1.1.0
|
|
Returns
|
-------
|
list
|
List of ABSOLUTE path-names (e.g. have the leading '/').
|
|
Raises
|
------
|
raises ValueError if kind has an illegal value
|
rì c S s g | ]
|
}|j qS rU ©r³ ©rh ÚnrU rU rV rk s z!HDFStore.keys.<locals>.<listcomp>ZnativeNc S s g | ]
|
}|j qS rU rï rð rU rU rV rk s ú/ÚTable)Ú classnamez8`include` should be either 'pandas' or 'native' but is 'rÛ )r± r¿ rÐ Z
|
walk_nodesr¨ )rÉ rî rU rU rV Úkeys s ÿ
|
ÿz HDFStore.keysz Iterator[str]c C s t | ¡ S r\ )Úiterrõ rÍ rU rU rV Ú__iter__¦ s zHDFStore.__iter__zIterator[tuple[str, list]]c c s | ¡ D ]}|j|fV qdS )z'
|
iterate on key->group
|
N)r± r³ )rÉ ÚgrU rU rV Úitems© s zHDFStore.items)r r[ c K sº t }| j|krR| jdkr$|dkr$n(|dkrL| jrLtd| j d| j d|| _| jr`| ¡ | jr| jdkrt j| j| j| j d| _
|
t r | jr d }t ||j | j| jf|| _d
|
S ) a9
|
Open the file in the specified mode
|
|
Parameters
|
----------
|
mode : {'a', 'w', 'r', 'r+'}, default 'a'
|
See HDFStore docstring or tables.open_file for info about modes
|
**kwargs
|
These parameters will be passed to the PyTables open_file method.
|
)r Úw)r r¦ )rú zRe-opening the file [z ] with mode [z ] will delete the current file!r )rÁ zGCannot open HDF5 file, which is already opened, even in read-only mode.N)r rÀ r© r$ rà r¶ rÄ ÚFiltersrÅ rÆ rÇ r r¨ Ú open_filer¿ )rÉ r r· r} ÚmsgrU rU rV rÈ ° s.
|
ÿ ÿ
|
ÿz HDFStore.openc C s | j dk r| j ¡ d| _ dS )z0
|
Close the PyTables file handle
|
N)r¿ r¶ rÍ rU rU rV r¶ Ý s
|
|
zHDFStore.closec C s | j dkrdS t| j jS )zF
|
return a boolean indicating whether the file is open
|
NF)r¿ r
ZisopenrÍ rU rU rV r© å s
|
zHDFStore.is_open)Úfsyncr[ c C s@ | j dk r<| j ¡ |r<tt t | j ¡ ¡ W 5 Q R X dS )aó
|
Force all buffered modifications to be written to disk.
|
|
Parameters
|
----------
|
fsync : bool (default False)
|
call ``os.fsync()`` on the file handle to force writing to disk.
|
|
Notes
|
-----
|
Without ``fsync=True``, flushing may not guarantee that the OS writes
|
to disk. With fsync, the operation will block until the OS claims the
|
file has been written; however, other caching layers may still
|
interfere.
|
N)r¿ Úflushr rª r¬ rþ Úfileno)rÉ rþ rU rU rV rÿ î s
|
|
|
|
zHDFStore.flushc
|
C sJ t : | |¡}|dkr*td| d| |¡W 5 Q R £ S Q R X dS )zÑ
|
Retrieve pandas object stored in file.
|
|
Parameters
|
----------
|
key : str
|
|
Returns
|
-------
|
object
|
Same type as object stored in file.
|
NúNo object named ú in the file)r rß rµ Ú _read_group©rÉ r r¹ rU rU rV rÔ s
|
|
z HDFStore.get)r r¤ r§ c st | |¡} | dkr"td| dt|dd}| | ¡ ¡ fdd}
|
t| |
|
|j|||||d
|
} | ¡ S ) aÖ
|
Retrieve pandas object stored in file, optionally based on where criteria.
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
Parameters
|
----------
|
key : str
|
Object being retrieved from file.
|
where : list or None
|
List of Term (or convertible) objects, optional.
|
start : int or None
|
Row number to start selection.
|
stop : int, default None
|
Row number to stop selection.
|
columns : list or None
|
A list of columns that if not None, will limit the return columns.
|
iterator : bool or False
|
Returns an iterator.
|
chunksize : int or None
|
Number or rows to include in iteration, return an iterator.
|
auto_close : bool or False
|
Should automatically close the store when finished.
|
|
Returns
|
-------
|
object
|
Retrieved object from file.
|
Nr r rf rd c s j | || dS )N)r¡ r¢ ro r£ ©Úread©Ú_startÚ_stopÚ_where©r£ rT rU rV ÚfuncQ s z HDFStore.select.<locals>.func©ro Únrowsr¡ r¢ r¤ r¥ r§ )rß rµ rp Ú_create_storerÚ
|
infer_axesÚ TableIteratorr Ú
|
get_result) rÉ r ro r¡ r¢ r£ r¤ r¥ r§ r¹ r ÚitrU r rV r´ s( .
|
|
ö zHDFStore.select©r r¡ r¢ c C s8 t |dd}| |¡}t|ts(td|j|||dS )a
|
return the selection as an Index
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
|
Parameters
|
----------
|
key : str
|
where : list of Term (or convertible) objects, optional
|
start : integer (defaults to None), row number to start selection
|
stop : integer (defaults to None), row number to stop selection
|
rf rd z&can only read_coordinates with a table©ro r¡ r¢ )rp Ú
|
get_storerrP ró r¯ Úread_coordinates)rÉ r ro r¡ r¢ ÚtblrU rU rV Úselect_as_coordinatesd s
|
|
|
z HDFStore.select_as_coordinates)r Úcolumnr¡ r¢ c C s, | |¡}t|ts td|j|||dS )a~
|
return a single column from the table. This is generally only useful to
|
select an indexable
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
Parameters
|
----------
|
key : str
|
column : str
|
The column of interest.
|
start : int or None, default None
|
stop : int or None, default None
|
|
Raises
|
------
|
raises KeyError if the column is not found (or key is not a valid
|
store)
|
raises ValueError if the column can not be extracted individually (it
|
is part of a data block)
|
|
z!can only read_column with a table©r r¡ r¢ )r rP ró r¯ Ú read_column)rÉ r r r¡ r¢ r rU rU rV Ú select_column s #
|
|
zHDFStore.select_column)r¤ r§ c
|
sv t |dd}t|ttfr.t|dkr.|d }t|trRj||||||| dS t|ttfshtdt|sxtd|dkr|d }fdd |D |¡}
|
d} t
|
|
|
|fgt |¡D ]\\} } | dkràt d
|
| d | jsøtd | j d | dkr
|
| j} qÀ| j| krÀtdqÀdd D }dd |D ¡ fdd}t|
|
||| ||||| d
|
}|jddS )aÙ
|
Retrieve pandas objects from multiple tables.
|
|
.. warning::
|
|
Pandas uses PyTables for reading and writing HDF5 files, which allows
|
serializing object-dtype data with pickle when using the "fixed" format.
|
Loading pickled data received from untrusted sources can be unsafe.
|
|
See: https://docs.python.org/3/library/pickle.html for more.
|
|
Parameters
|
----------
|
keys : a list of the tables
|
selector : the table to apply the where criteria (defaults to keys[0]
|
if not supplied)
|
columns : the columns I want back
|
start : integer (defaults to None), row number to start selection
|
stop : integer (defaults to None), row number to stop selection
|
iterator : bool, return an iterator, default False
|
chunksize : nrows to include in iteration, return an iterator
|
auto_close : bool, default False
|
Should automatically close the store when finished.
|
|
Raises
|
------
|
raises KeyError if keys or selector is not found or keys is empty
|
raises TypeError if keys is not a list or tuple
|
raises ValueError if the tables are not ALL THE SAME DIMENSIONS
|
rf rd r )r ro r£ r¡ r¢ r¤ r¥ r§ zkeys must be a list/tuplez keys must have a non-zero lengthNc s g | ]} |¡qS rU )r ©rh ÚkrÍ rU rV rk ð s z/HDFStore.select_as_multiple.<locals>.<listcomp>zInvalid table [ú]zobject [z>] is not a table, and cannot be used in all select as multiplez,all tables must have exactly the same nrows!c S s g | ]}t |tr|qS rU )rP ró ©rh ÚxrU rU rV rk s
|
c S s h | ]}|j d d qS ©r )Únon_index_axes©rh rw rU rU rV Ú <setcomp> s z.HDFStore.select_as_multiple.<locals>.<setcomp>c s* fddD }t |dd ¡ S )Nc s g | ]}|j d qS )©ro r£ r¡ r¢ r r% )r r r
|
r£ rU rV rk s ÿz=HDFStore.select_as_multiple.<locals>.func.<locals>.<listcomp>F)ÚaxisÚverify_integrity)r= Ú _consolidate)r r r
|
Úobjs)r( r£ Útblsr rV r
|
s þz)HDFStore.select_as_multiple.<locals>.funcr T©Ú coordinates)rp rP rl rm rn rY r´ r¯ r¨ r Ú itertoolsÚchainÚziprµ Úis_tableÚpathnamer Úpopr r )rÉ rõ ro Úselectorr£ r¡ r¢ r¤ r¥ r§ rT r rw r Z_tblsr r rU )r( r£ rÉ r, rV Úselect_as_multiple¬ sd +
|
ø
|
ÿ
|
|
|
ö zHDFStore.select_as_multipleTr{ r r r ) r r r r r r r r Ú track_timesr r[ c C sH |dkrt dpd}| |¡}| j||||||||| |
|
| | | |d dS )aó
|
Store object in HDFStore.
|
|
Parameters
|
----------
|
key : str
|
value : {Series, DataFrame}
|
format : 'fixed(f)|table(t)', default is 'fixed'
|
Format to use when storing object in HDFStore. Value can be one of:
|
|
``'fixed'``
|
Fixed format. Fast writing/reading. Not-appendable, nor searchable.
|
``'table'``
|
Table format. Write as a PyTables Table structure which may perform
|
worse but allow more flexible operations like searching / selecting
|
subsets of the data.
|
index : bool, default True
|
Write DataFrame index as a column.
|
append : bool, default False
|
This will force Table format, append the input data to the existing.
|
data_columns : list of columns or True, default None
|
List of columns to create as data columns, or True to use all columns.
|
See `here
|
<https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#query-via-data-columns>`__.
|
encoding : str, default None
|
Provide an encoding for strings.
|
track_times : bool, default True
|
Parameter is propagated to 'create_table' method of 'PyTables'.
|
If set to False it enables to have the same h5 files (same hashes)
|
independent on creation time.
|
dropna : bool, default False, optional
|
Remove missing values.
|
|
.. versionadded:: 1.1.0
|
Núio.hdf.default_formatrt ) r r r r r r r r rZ r r7 r )r Ú_validate_formatÚ_write_to_group)rÉ r r r r r r r r r r rZ r r7 r rU rU rV r % s& 4
|
òz HDFStore.putc
|
C sà t |dd}z| |¡}W n tk
|
r0 Y np tk
|
rD Y n\ tk
|
r } z>|dk rftd|| |¡}|dk r|jdd W Y ¢dS W 5 d}~X Y nX t |||¡r¾|j
|
jdd n |j sÌtd|j |||dS dS ) a:
|
Remove pandas object partially by specifying the where condition
|
|
Parameters
|
----------
|
key : str
|
Node to remove or delete rows from
|
where : list of Term (or convertible) objects, optional
|
start : integer (defaults to None), row number to start selection
|
stop : integer (defaults to None), row number to stop selection
|
|
Returns
|
-------
|
number of rows removed (or None if not a Table)
|
|
Raises
|
------
|
raises KeyError if key is not a valid store
|
|
rf rd Nz5trying to remove a node with a non-None where clause!T©Ú recursivez7can only remove with where on objects written as tablesr ) rp r rµ rÐ Ú Exceptionr¨ rß Z _f_removeÚcomÚall_noner¹ r2 Údelete)rÉ r ro r¡ r¢ rT Úerrrà rU rU rV rÙ m s2 ÿþ
|
ÿzHDFStore.removezbool | list[str]r )
|
r r r r r r r r r r[ c C sl | dk rt d|dkr td}|dkr4tdp2d}| |¡}| j|||||||||
|
| | | ||||d dS )a
|
Append to Table in file.
|
|
Node must already exist and be Table format.
|
|
Parameters
|
----------
|
key : str
|
value : {Series, DataFrame}
|
format : 'table' is the default
|
Format to use when storing object in HDFStore. Value can be one of:
|
|
``'table'``
|
Table format. Write as a PyTables Table structure which may perform
|
worse but allow more flexible operations like searching / selecting
|
subsets of the data.
|
index : bool, default True
|
Write DataFrame index as a column.
|
append : bool, default True
|
Append the input data to the existing.
|
data_columns : list of columns, or True, default None
|
List of columns to create as indexed data columns for on-disk
|
queries, or True to use all columns. By default only the axes
|
of the object are indexed. See `here
|
<https://pandas.pydata.org/pandas-docs/stable/user_guide/io.html#query-via-data-columns>`__.
|
min_itemsize : dict of columns that specify minimum str sizes
|
nan_rep : str to use as str nan representation
|
chunksize : size to chunk the writing
|
expectedrows : expected TOTAL row size of this table
|
encoding : default None, provide an encoding for str
|
dropna : bool, default False, optional
|
Do not write an ALL nan row to the store settable
|
by the option 'io.hdf.dropna_table'.
|
|
Notes
|
-----
|
Does *not* check if data being appended overlaps with existing
|
data in the table, so be careful
|
Nz>columns is not a supported keyword in append, try data_columnszio.hdf.dropna_tabler8 ru )r Úaxesr r r r r r r¥ Ú expectedrowsr r rZ r )r¯ r r9 r: )rÉ r r r rB r r r r r£ r r r¥ rC r r rZ r rU rU rV r ¦ s6 ;ÿ
|
ðzHDFStore.appendÚdict)Údr r[ c s¬ |dk rt dt|ts"td||kr2tdtttjttt d }d} g }
|
|
|
¡ D ]0\} dkr| dk rtd| } qh|
|
¡ qh| dk rÖj | } | t|
|
¡} t| | ¡} | | ¡|| < |dkræ|| }|r*fdd| ¡ D }t|}|D ]}| |¡}qj| | d d¡}|
|
¡ D ]h\} | |krT|nd}j |d
|
}|dk r fd d |
|
¡ D nd}| j| |f||d | q>dS )a
|
Append to multiple tables
|
|
Parameters
|
----------
|
d : a dict of table_name to table_columns, None is acceptable as the
|
values of one node (this will get all the remaining columns)
|
value : a pandas object
|
selector : a string that designates the indexable table; all of its
|
columns will be designed as data_columns, unless data_columns is
|
passed, in which case these are used
|
data_columns : list of columns to create as data columns, or True to
|
use all columns
|
dropna : if evaluates to True, drop rows from all tables if any single
|
row in each table has all NaN. Default False.
|
|
Notes
|
-----
|
axes parameter is currently not accepted
|
|
Nztaxes is currently not accepted as a parameter to append_to_multiple; you can create the tables independently insteadzQappend_to_multiple must have a dictionary specified as the way to split the valuez=append_to_multiple requires a selector that is in passed dictr z<append_to_multiple can only have one value in d that is Nonec 3 s | ]} | j d djV qdS )Úall)ÚhowN)r r )rh Úcols)r rU rV Ú <genexpr>I s z.HDFStore.append_to_multiple.<locals>.<genexpr>r ©r( c s i | ]\}}| kr||qS rU rU ©rh r r )ÚvrU rV Ú
|
<dictcomp>Y s z/HDFStore.append_to_multiple.<locals>.<dictcomp>)r r )r¯ rP rD r¨ rl ÚsetÚrangeÚndimÚ _AXES_MAPrÜ rù ÚextendrB Ú
|
differencer7 ÚsortedÚ get_indexerÚtakeÚvaluesÚnextÚ intersectionÚlocr4 Úreindexr )rÉ rE r r5 r rB r r· r( Z
|
remain_keyZ remain_valuesr ÚorderedZorddZidxsZ valid_indexr r ÚdcÚvalÚfilteredrU )rL r rV Úappend_to_multipleþ sZ ÿ
|
ÿÿ&ÿ
|
|
ÿýzHDFStore.append_to_multiplerX )r ÚoptlevelÚkindr[ c C sB t | |¡}|dkr dS t|ts.td|j|||d dS )aà
|
Create a pytables index on the table.
|
|
Parameters
|
----------
|
key : str
|
columns : None, bool, or listlike[str]
|
Indicate which columns to create an index on.
|
|
* False : Do not create any indexes.
|
* True : Create indexes on all columns.
|
* None : Create indexes on all columns.
|
* listlike : Create indexes on the given columns.
|
|
optlevel : int or None, default None
|
Optimization level, if None, pytables defaults to 6.
|
kind : str or None, default None
|
Kind of index, if None, pytables defaults to "medium".
|
|
Raises
|
------
|
TypeError: raises if the node is not a table
|
Nz1cannot create table index on a Fixed format store)r£ ra rb )r r rP ró r¯ Ú create_index)rÉ r r£ ra rb rT rU rU rV Úcreate_table_index_ s
|
|
zHDFStore.create_table_indexrl c C s< t | ¡ | jdk s ttdk s(tdd | j ¡ D S )zÂ
|
Return a list of all the top-level nodes.
|
|
Each node returned is not a pandas storage object.
|
|
Returns
|
-------
|
list
|
List of objects.
|
Nc S sP g | ]H}t |tjjst|jd dsHt|ddsHt |tjjr|jdkr|qS )Ú pandas_typeNru ) rP r| ÚlinkÚLinkÚgetattrÚ_v_attrsru ró r¼ )rh rø rU rU rV rk s
|
ùz#HDFStore.groups.<locals>.<listcomp>)r rÏ r¿ rÐ r| Ú walk_groupsrÍ rU rU rV r± s þzHDFStore.groupsrò z*Iterator[tuple[str, list[str], list[str]]])ro r[ c c s¼ t | ¡ | jdk s ttdk s(t| j |¡D ]}t|jdddk rLq4g }g }|j ¡ D ]B}t|jdd}|dkrt
|
|tj j r | |j¡ q^| |j¡ q^|j d¡||fV q4dS )aS
|
Walk the pytables group hierarchy for pandas objects.
|
|
This generator will yield the group path, subgroups and pandas object
|
names for each group.
|
|
Any non-pandas PyTables objects that are not a group will be ignored.
|
|
The `where` group itself is listed first (preorder), then each of its
|
child groups (following an alphanumerical order) is also traversed,
|
following the same procedure.
|
|
Parameters
|
----------
|
where : str, default "/"
|
Group where to start walking.
|
|
Yields
|
------
|
path : str
|
Full path to a group (without trailing '/').
|
groups : list
|
Names (strings) of the groups contained in `path`.
|
leaves : list
|
Names (strings) of the pandas objects contained in `path`.
|
Nre rò )r rÏ r¿ rÐ r| rj rh ri Z _v_childrenrW rP r¹ ÚGroupr r¼ r³ Úrstrip)rÉ ro rø r± ZleavesÚchildre rU rU rV Úwalk£ s z HDFStore.walkz Node | Nonec C s | ¡ | d¡sd| }| jdk s(ttdk s4tz| j | j|¡}W n tjjk
|
rb Y dS X t |tj
|
s|tt ||S )z9return the node with the key or None if it does not existrò N) rÏ Ú
|
startswithr¿ rÐ r| rß rÑ Ú
|
exceptionsZNoSuchNodeErrorrP rM rÜ )rÉ r rà rU rU rV rß Ó s
|
zHDFStore.get_nodeúGenericFixed | Tablec C s8 | |¡}|dkr"td| d| |¡}| ¡ |S )z<return the storer object for a key, raise if not in the fileNr r )rß rµ r r )rÉ r r¹ rT rU rU rV r ã s
|
|
zHDFStore.get_storerrú )r Ú propindexesr rÁ Ú overwriter[ c C sÎ t |||||d} |dkr&t| ¡ }t|ttfs:|g}|D ]}
|
| |
|
¡} | dk r>|
|
| krj|rj| |
|
¡ | |
|
¡} t| tr¶d} |rdd | j D } | j
|
|
|
| | t | dd| j d q>| j |
|
| | j d q>| S ) a;
|
Copy the existing store to a new file, updating in place.
|
|
Parameters
|
----------
|
propindexes : bool, default True
|
Restore indexes in copied file.
|
keys : list, optional
|
List of keys to include in the copy (defaults to all).
|
overwrite : bool, default True
|
Whether to overwrite (remove and replace) existing nodes in the new store.
|
mode, complib, complevel, fletcher32 same as in HDFStore.__init__
|
|
Returns
|
-------
|
open file handle of the new store
|
)r r r rÁ NFc S s g | ]}|j r|jqS rU )Ú
|
is_indexedra ©rh r rU rU rV rk s z!HDFStore.copy.<locals>.<listcomp>r )r r rZ r^ )r rl rõ rP rm r rÙ r´ ró rB r rh rZ r )rÉ r r rr rõ r r rÁ rs Z new_storer rT Údatar rU rU rV Úcopyí s> ÿ
|
|
|
|
|
ûz HDFStore.copyc
|
C s
|
t | j}t| d| d}| jrþt| ¡ }t|rôg }g }|D ]}z<| |¡}|dk r| t |j pj|¡ | t |p|d¡ W qD t
|
k
|
r Y qD t k
|
rÞ } z(| |¡ t |} | d| d¡ W 5 d}~X Y qDX qD|t d||7 }n|d7 }n|d 7 }|S )
|
zg
|
Print detailed information on the store.
|
|
Returns
|
-------
|
str
|
rã rä Nzinvalid_HDFStore nodez[invalid_HDFStore node: r é ÚEmptyzFile is CLOSED) rJ rà rÜ r© rT rõ rn r r r3 rÐ r= rI )
|
rÉ r ÚoutputZlkeysrõ rW r rT ZdetailZdstrrU rU rV Úinfo( s.
|
|
|
&
|
z HDFStore.infoc C s | j st| j dd S )Nz file is not open!)r© r! rà rÍ rU rU rV rÏ R s zHDFStore._check_if_open)r r[ c
|
C sJ zt | ¡ }W n4 tk
|
rD } ztd| d|W 5 d}~X Y nX |S )z validate / deprecate formatsz#invalid HDFStore format specified [r N)Ú _FORMAT_MAPÚlowerrµ r¯ )rÉ r rA rU rU rV r9 V s
|
$zHDFStore._validate_formatrO zDataFrame | Series | None)r rZ r r[ c
|
C s: |dk r t |ttfs tdtt|jdd}tt|jdd}|dkr¶|dkrt tdk sdt t|dds~t |tj
|
j rd}d}q¶tdn$t |tr¢d }nd
|
}|dkr¶|d 7 }d|kr,t t d }z || } W nD tk
|
r }
|
z$td | dt| d| |
|
W 5 d}
|
~
|
X Y nX | | |||dS |dkrÆ|dk rÆ|dkrt|dd} | dk rÆ| jdkrrd}n| jdkrÆd}nB|dkrÆt|dd} | dk rÆ| jdkr¶d}n| jdkrÆd}ttttttd} z | | } W nD tk
|
r( }
|
z$td| dt| d| |
|
W 5 d}
|
~
|
X Y nX | | |||dS )z"return a suitable class to operateNz(value must be None, Series, or DataFramere Ú
|
table_typeru Ú frame_tableÚ generic_tablezKcannot create a storer if the object is not existing nor a value are passedÚseriesÚframeZ_table)r r z=cannot properly create the storer for: [_STORER_MAP] [group->ú,value->z ,format->©rZ r Ú series_tabler rf Úappendable_seriesÚappendable_multiseriesÚappendable_frameÚappendable_multiframe)r r r r r Úwormz<cannot properly create the storer for: [_TABLE_MAP] [group->)rP r; r5 r¯ rW rh ri r r| rÐ ru ró Ú SeriesFixedÚ
|
FrameFixedrµ rÜ ÚnlevelsÚ GenericTableÚAppendableSeriesTableÚAppendableMultiSeriesTableÚAppendableFrameTableÚAppendableMultiFrameTableÚ WORMTable) rÉ r¹ r r rZ r ÚptÚttZ _STORER_MAPÚclsrA r Z
|
_TABLE_MAPrU rU rV r ` s ÿÿ
|
|
|
ÿý
|
|
|
|
|
|
ú ÿýzHDFStore._create_storer)
|
r r r r r r r r r7 r[ c C sÎ t |dd r |dks|r d S | ||¡}| j|||||d}|rr|jrZ|jrb|dkrb|jrbtd|jsz| ¡ n| ¡ |js|rtd|j|||||| |
|
| | | |||d t|t rÊ|rÊ|j
|
|d d S ) NÚemptyru r rt zCan only append to Tablesz0Compression not supported on Fixed format stores) ÚobjrB r r r rÁ r r¥ rC r r r r7 )r£ ) rh Ú_identify_groupr r2 Ú is_existsr¨ Úset_object_infoÚwriterP ró rc )rÉ r r r rB r r r r rÁ r r¥ rC r r r rZ r r7 r¹ rT rU rU rV r: » s:
|
|
ózHDFStore._write_to_grouprM ©r¹ c C s | |¡}| ¡ | ¡ S r\ )r r r )rÉ r¹ rT rU rU rV r ù s
|
zHDFStore._read_group)r r r[ c C sN | |¡}| jdk st|dk r8|s8| jj|dd d}|dkrJ| |¡}|S )z@Identify HDF5 group based on key, delete/create group if needed.NTr; )rß r¿ rÐ Ú remove_nodeÚ_create_nodes_and_group)rÉ r r r¹ rU rU rV r þ s
|
|
zHDFStore._identify_groupc C sv | j dk st| d¡}d}|D ]P}t|s.q |}| d¡sD|d7 }||7 }| |¡}|dkrl| j ||¡}|}q |S )z,Create nodes from key and return group name.Nrò )r¿ rÐ Úsplitrn Úendswithrß Z create_group)rÉ r Úpathsr ÚpÚnew_pathr¹ rU rU rV r s
|
|
|
z HDFStore._create_nodes_and_group)r NNF)rì )r )F)NNNNFNF)NNN)NN)NNNNNFNF) NTFNNNNNNr{ TF)NNN)NNTTNNNNNNNNNNr{ )NNF)NNN)rò )rú TNNNFT)NNrO r{ )NTFNNNNNNFNNNr{ T)1rÝ Ú
|
__module__Ú __qualname__Ú__doc__Ú__annotations__rÊ rÎ ÚpropertyrÑ rÒ rÖ rØ rÚ rÞ rá râ ræ rç rë rõ r÷ rù rÈ r¶ r© rÿ rÔ r´ r r r6 r rÙ r r` rd r± rn rß r rw r{ rÏ r9 r r: r r r rU rU rU rV r ß s
|
A ú!
|
|
"- ÷N û$ û+ ö} ñ$H= î"] ùd û( 0 ÷;* ú` í">r c @ sp e Zd ZU dZded< ded< ded< dddd
|
dd
|
d d d dZddddZd dddZdd
|
dddZdS )r aa
|
Define the iteration interface on a table
|
|
Parameters
|
----------
|
store : HDFStore
|
s : the referred storer
|
func : the function to execute the query
|
where : the where of the query
|
nrows : the rows to iterate on
|
start : the passed start value (default is None)
|
stop : the passed stop value (default is None)
|
iterator : bool, default False
|
Whether to use the default iterator.
|
chunksize : the passed chunking value (default is 100000)
|
auto_close : bool, default False
|
Whether to automatically close the store at the end of iteration.
|
r r¥ r r rq rT NFr
r )r rT r¤ r¥ r§ r[ c C s || _ || _|| _|| _| jjrN|d kr,d}|d kr8d}|d krD|}t||}|| _|| _|| _d | _ |sr| d k r| d kr~d} t
|
| | _ nd | _ |
|
| _ d S )Nr é ) r rT r ro r2 Úminr r¡ r¢ r. rc r¥ r§ ) rÉ r rT r ro r r¡ r¢ r¤ r¥ r§ rU rU rV rÊ > s,
|
zTableIterator.__init__r rË c c sv | j }| jd krtd|| jk rjt|| j | j}| d d | j||
¡}|}|d kst|sbq|V q| ¡ d S )Nz*Cannot iterate until get_result is called.) r¡ r. r¨ r¢ r« r¥ r rn r¶ )rÉ r½ r¢ r rU rU rV r÷ h s
|
|
zTableIterator.__iter__c C s | j r| j ¡ d S r\ )r§ r r¶ rÍ rU rU rV r¶ x s zTableIterator.closer- c C s | j d k r4t| jts td| jj| jd| _| S |rft| jtsLtd| jj| j| j| j d}n| j}|
|
| j| j |¡}| ¡ |S )Nz0can only use an iterator or chunksize on a table)ro z$can only read_coordinates on a tabler ) r¥ rP rT ró r¯ r ro r. r¡ r¢ r r¶ )rÉ r. ro ÚresultsrU rU rV r | s"
|
ÿzTableIterator.get_result)NNFNF)F) rÝ r¥ r¦ r§ r¨ rÊ r÷ r¶ r rU rU rU rV r & s
|
õ*r c @ s¨ e Zd ZU dZdZded< dZded< dddgZdMd
|
d d d ddZe ddddZ
|
e d
|
dddZ dd dddZ d
|
dddZ ddd d d Zddd d Ze ddd!d"Zd#d
|
d
|
d$d%d&d'Zd(d) Ze d*d+ Ze d,d- Ze d.d/ Ze d0d1 Zd2dd3d4ZdNd dd5d6Zd dd7d8Zd9dd d:d;d<ZdOd=d>Zdd d?d@dAZ d ddBdCZ d ddDdEZ d ddFdGZ d9d dHdIdJZ d9d dHdKdLZ!d S )PÚIndexCola
|
an index column description class
|
|
Parameters
|
----------
|
axis : axis which I reference
|
values : the ndarray like converted values
|
kind : a string description of this type
|
typ : the pytables type
|
pos : the position in the pytables
|
|
Tr
Úis_an_indexableÚis_data_indexableÚfreqÚtzÚ
|
index_nameNrY rX r )ra Úcnamer[ c C s t |tstd|| _|| _|| _|| _|p0|| _|| _|| _ || _
|
| | _ |
|
| _ | | _ | | _| | _|| _|d k r|| |¡ t | jtstt | jtstd S )Nz`name` must be a str.)rP rY r¨ rW rb Útypra r³ r( Úposr° r± r² r\ ru r» ÚmetadataÚset_posrÐ )rÉ ra rW rb r´ r³ r( rµ r° r± r² r\ ru r» r¶ rU rU rV rÊ ¨ s(
|
|
|
zIndexCol.__init__rc rË c C s | j jS r\ )r´ ÚitemsizerÍ rU rU rV r¸ Ó s zIndexCol.itemsizec C s | j dS )NÚ_kindr` rÍ rU rU rV Ú kind_attrØ s zIndexCol.kind_attr)rµ r[ c C s$ || _ |dk r | jdk r || j_dS )z,set the position of this column in the TableN)rµ r´ Z_v_pos)rÉ rµ rU rU rV r· Ü s zIndexCol.set_posc C sF t tt| j| j| j| j| jf}d dd t dddddg|D ¡S ) Nú,c S s g | ]\}}| d | qS ©z->rU rK rU rU rV rk ç s ÿz%IndexCol.__repr__.<locals>.<listcomp>ra r³ r( rµ rb )
|
rm ÚmaprJ ra r³ r( rµ rb Újoinr1 ©rÉ ÚtemprU rU rV ræ â s ÿþÿzIndexCol.__repr__r ©Úotherr[ c s t fdddD S )úcompare 2 col itemsc 3 s& | ] }t |d t |d kV qd S r\ ©rh ru ©r rÉ rU rV rI ï s ÿz"IndexCol.__eq__.<locals>.<genexpr>)ra r³ r( rµ ©rF ©rÉ r rU rÅ rV Ú__eq__í s þzIndexCol.__eq__c C s | |¡ S r\ )rÈ rÇ rU rU rV Ú__ne__ô s zIndexCol.__ne__c C s" t | jdsdS t| jj| jjS )z%return whether I am an indexed columnrH F)Úhasattrru rh rH r³ rt rÍ rU rU rV rt ÷ s zIndexCol.is_indexedú
|
np.ndarrayz3tuple[np.ndarray, np.ndarray] | tuple[Index, Index]©rW rZ r r[ c
|
C s t |tjstt||jjdk r2|| j ¡ }t | j
|
}t ||||}i }t | j |d< | j dk rtt | j |d< t}t|jst|jrt}n|jdkr¬d|kr¬dd }z||f|}W n0 tk
|
rì d|krÜd|d< ||f|}Y nX t|| j} | | fS )zV
|
Convert the data from this selection to the appropriate pandas type.
|
Nra r° Úi8c [ s t f d| i|S )NZordinal)r9 )r" ÚkwdsrU rU rV r s ÿÿz"IndexCol.convert.<locals>.<lambda>)rP rQ ÚndarrayrÐ rÜ ÚdtypeÚfieldsr³ rw rW rb Ú_maybe_convertr² r° r7 r+ r, r6 r¨ Ú_set_tzr± )
|
rÉ rW r rZ r Úval_kindr· ÚfactoryZ new_pd_indexZfinal_pd_indexrU rU rV Úconvertÿ s,
|
|
zIndexCol.convertc C s | j S )zreturn the values©rW rÍ rU rU rV Ú take_data. s zIndexCol.take_datac C s | j jS r\ )ru ri rÍ rU rU rV Úattrs2 s zIndexCol.attrsc C s | j jS r\ ©ru Ú descriptionrÍ rU rU rV rÛ 6 s zIndexCol.descriptionc C s t | j| jdS )z!return my current col descriptionN)rh rÛ r³ rÍ rU rU rV Úcol: s z IndexCol.colc C s | j S ©zreturn my cython valuesr× rÍ rU rU rV Úcvalues? s zIndexCol.cvaluesr c C s
|
t | jS r\ )rö rW rÍ rU rU rV r÷ D s zIndexCol.__iter__c C sP t | jdkrLt|tr$| | j¡}|dk rL| jj|k rLt j || j
|
d| _dS )z
|
maybe set a string col itemsize:
|
min_itemsize can be an integer or a dict with this columns name
|
with an integer size
|
ÚstringN)r¸ rµ ) rW rb rP rD rÔ ra r´ r¸ r Ú StringColrµ )rÉ r rU rU rV Úmaybe_set_sizeG s
|
|
zIndexCol.maybe_set_sizec C s d S r\ rU rÍ rU rU rV Úvalidate_namesT s zIndexCol.validate_namesÚAppendableTable)Úhandlerr r[ c C s: |j | _ | ¡ | |¡ | |¡ | |¡ | ¡ d S r\ )ru Ú validate_colÚ validate_attrÚvalidate_metadataÚwrite_metadataÚset_attr)rÉ rä r rU rU rV Úvalidate_and_setW s
|
|
|
zIndexCol.validate_and_setc C s^ t | jdkrZ| j}|dk rZ|dkr*| j}|j|k rTtd| d| j d|j d|jS dS )z:validate this column: return the compared against itemsizerß Nz#Trying to store a string with len [z] in [z)] column but
|
this column has a limit of [zC]!
|
Consider using min_itemsize to preset the sizes on these columns)rW rb rÜ r¸ r¨ r³ )rÉ r¸ ÚcrU rU rV rå _ s
|
ÿzIndexCol.validate_col)r r[ c C sB |r>t | j| jd }|d k r>|| jkr>td| d| j dd S )Nzincompatible kind in col [ú - r )rh rÙ rº rb r¯ )rÉ r Z existing_kindrU rU rV ræ r s ÿzIndexCol.validate_attrc C sÈ | j D ]¼}t| |d}| | ji ¡}| |¡}||krª|dk rª||krª|dkrt|||f }tj|tt d d||< t
|
| |d qÂt d| j d| d| d| d q|dk sº|dk r|||< qdS ) z
|
set/update the info for this indexable with the key/value
|
if there is a conflict raise/warn as needed
|
N)r° r² ©Ú
|
stacklevelzinvalid info for [z] for [z], existing_value [z ] conflicts with new value [r ) Ú _info_fieldsrh Ú
|
setdefaultra rÔ rr ÚwarningsÚwarnr r& Úsetattrr¨ )rÉ r{ r r ÚidxZexisting_valueÚwsrU rU rV Ú update_info{ s&
|
|
ÿ ÿzIndexCol.update_infoc C s$ | | j¡}|dk r | j |¡ dS )z!set my state from the passed infoN)rÔ ra Ú__dict__Úupdate)rÉ r{ rô rU rU rV Úset_info s zIndexCol.set_infoc C s t | j| j| j dS )z set the kind for this columnN)ró rÙ rº rb rÍ rU rU rV ré ¡ s zIndexCol.set_attr)rä r[ c C sB | j dkr>| j}| | j¡}|dk r>|dk r>t||s>tddS )z:validate that kind=category does not change the categoriesÚcategoryNzEcannot append a categorical with different categories to the existing)r» r¶ Ú read_metadatar³ r4 r¨ )rÉ rä Z new_metadataZ cur_metadatarU rU rV rç ¥ s
|
ÿþýÿzIndexCol.validate_metadatac C s | j dk r| | j| j ¡ dS )zset the meta dataN)r¶ rè r³ )rÉ rä rU rU rV rè ´ s
|
zIndexCol.write_metadata) NNNNNNNNNNNNN)N)N)"rÝ r¥ r¦ r§ r® r¨ r¯ rï rÊ r© r¸ rº r· ræ rÈ rÉ rt rÖ rØ rÙ rÛ rÜ rÞ r÷ rá râ rê rå ræ rö rù ré rç rè rU rU rU rV r sb
|
|
ñ+ /
|
|
|
|
|
r c @ sD e Zd ZdZeddddZddddd d
|
d Zd dd dZdS )ÚGenericIndexColz:an index which is not represented in the data of the tabler
rË c C s dS ©NFrU rÍ rU rU rV rt ½ s zGenericIndexCol.is_indexedrË rY ztuple[Index, Index]rÌ c C s, t |tjstt|tt|}||fS )zÛ
|
Convert the data from this selection to the appropriate pandas type.
|
|
Parameters
|
----------
|
values : np.ndarray
|
nan_rep : str
|
encoding : str
|
errors : str
|
)rP rQ rÏ rÐ rÜ r: rn )rÉ rW r rZ r r rU rU rV rÖ Á s zGenericIndexCol.convertr c C s d S r\ rU rÍ rU rU rV ré Ó s zGenericIndexCol.set_attrN)rÝ r¥ r¦ r§ r© rt rÖ ré rU rU rU rV rü º s
|
rü c s> e Zd ZdZdZdZddgZd;dddd d
|
fd d Zedd ddZ edd ddZ
|
dd ddZ dddddZ dd ddd Z d d Zedd d d!d"Zed#d$ Zedd%d&d'd(Zedd d&d)d*Zed+d, Zed-d. Zed/d0 Zed1d2 Zd d d3d4Zd5ddd6d7d8Zd d d9d:Z ZS )<ÚDataCola3
|
a data holding column, by definition this is not indexable
|
|
Parameters
|
----------
|
data : the actual data
|
cname : the column name in the table to hold the data (typically
|
values)
|
meta : a string description of the metadata
|
metadata : the actual metadata
|
Fr± r\ NrY rX zDtypeArg | Noner )ra r³ rÐ r[ c s2 t j||||||||| |
|
| d | | _| | _d S )N) ra rW rb r´ rµ r³ r± r\ ru r» r¶ )ÚsuperrÊ rÐ rv )rÉ ra rW rb r´ r³ rµ r± r\ ru r» r¶ rÐ rv ©Ú __class__rU rV rÊ è s õ zDataCol.__init__rË c C s | j dS )NÚ_dtyper` rÍ rU rU rV Ú
|
dtype_attr s zDataCol.dtype_attrc C s | j dS )NÚ_metar` rÍ rU rU rV Ú meta_attr s zDataCol.meta_attrc C sF t tt| j| j| j| j| jf}d dd t dddddg|D ¡S ) Nr» c S s g | ]\}}| d | qS r¼ rU rK rU rU rV rk s ÿz$DataCol.__repr__.<locals>.<listcomp>ra r³ rÐ rb Úshape)
|
rm r½ rJ ra r³ rÐ rb r r¾ r1 r¿ rU rU rV ræ s ÿÿþÿzDataCol.__repr__r r
rÁ c s t fdddD S )rà c 3 s& | ] }t |d t |d kV qd S r\ rÄ ru rÅ rU rV rI s ÿz!DataCol.__eq__.<locals>.<genexpr>)ra r³ rÐ rµ rÆ rÇ rU rÅ rV rÈ s þzDataCol.__eq__r )rv r[ c C s@ |d k s t | jd kst t|\}}|| _|| _t|| _d S r\ )rÐ rÐ Ú_get_data_and_dtype_namerv Ú_dtype_to_kindrb )rÉ rv Ú
|
dtype_namerU rU rV Úset_data$ s zDataCol.set_datac C s | j S )zreturn the data©rv rÍ rU rU rV rØ . s zDataCol.take_datarK )rW r[ c C s |j }|j}|j}|jdkr&d|jf}t|trJ|j}| j||j j d}ntt
|
|sZt |rf| |¡}nXt |rz| |¡}nDt|rt j||d d}n&t|r®| ||¡}n| j||j d}|S )zW
|
Get an appropriately typed and shaped pytables.Col object for values.
|
rf ©rb r ©r¸ r )rÐ r¸ r rP ÚsizerP r? ÚcodesÚ get_atom_datara r+ r, Úget_atom_datetime64r2 Úget_atom_timedelta64r* r Z
|
ComplexColr1 Úget_atom_string)r rW rÐ r¸ r r ÚatomrU rU rV Ú _get_atom2 s$
|
|
|
zDataCol._get_atomc C s t j||d dS )Nr r ©r rà ©r r r¸ rU rU rV r R s zDataCol.get_atom_stringz type[Col]©rb r[ c C sR | d¡r$|dd
}d| d}n"| d¡r4d}n| ¡ }| d}tt |S )z0return the PyTables column class for this columnÚuinté NZUIntrK ÚperiodÚInt64Col)ro Ú
|
capitalizerh r )r rb Zk4Zcol_nameZkcaprU rU rV Úget_atom_coltypeV s
|
|
|
zDataCol.get_atom_coltypec C s | j |d|d dS )Nr r ©r ©r ©r r rb rU rU rV r e s zDataCol.get_atom_datac C s t j|d dS ©Nr r ©r r ©r r rU rU rV r i s zDataCol.get_atom_datetime64c C s t j|d dS r" r# r$ rU rU rV r m s z DataCol.get_atom_timedelta64c C s t | jdd S )Nr )rh rv rÍ rU rU rV r q s z DataCol.shapec C s | j S rÝ r rÍ rU rU rV rÞ u s zDataCol.cvaluesc C s` |r\t | j| jd}|dk r2|t| jkr2tdt | j| jd}|dk r\|| jkr\tddS )zAvalidate that we have the same order as the existing & same dtypeNz4appended items do not match existing items in table!z@appended items dtype do not match existing items dtype in table!)rh rÙ rº rl rW r¨ r rÐ )rÉ r Zexisting_fieldsZexisting_dtyperU rU rV ræ z s ÿzDataCol.validate_attrrË )rW rZ r c C s t |tjstt||jjdk r.|| j }| jdk s<t| jdkr\t |\}}t
|
|}n|}| j}| j }t |tjs|tt | j }| j} | j}
|
| j} |dk s¤tt |} | dkrÆt|| dd}n(| dkràtj|dd}n| dkr8z tjd d
|
|D td}W n. tk
|
r4 tjd d
|
|D td}Y nX n¶|d kr¶| } | ¡ }| dkrhtg tjd} n<t| }| ¡ r¤| | } ||d k | t¡ ¡ j 8 < t j || |
|
d}n8z|j| dd}W n$ t k
|
rì |jddd}Y nX t |dkr t!||||d}| j"|fS )aR
|
Convert the data from this selection to the appropriate pandas type.
|
|
Parameters
|
----------
|
values : np.ndarray
|
nan_rep :
|
encoding : str
|
errors : str
|
|
Returns
|
-------
|
index : listlike to become an Index
|
data : ndarraylike to become a column
|
NÚ
|
datetime64T©ÚcoerceÚ timedelta64úm8[ns]©rÐ r c S s g | ]}t |¡qS rU ©r Ú fromordinal©rh rL rU rU rV rk ¾ s z#DataCol.convert.<locals>.<listcomp>c S s g | ]}t |¡qS rU ©r Ú fromtimestampr- rU rU rV rk  s rú éÿÿÿÿ)Ú
|
categoriesr\ F©rw ÚOrß ©r rZ r )#rP rQ rÏ rÐ rÜ rÐ rÑ r³ r´ r r rb rW r» r¶ r\ r± rÓ ÚasarrayÚobjectr¨ Úravelr7 Úfloat64r> ÚanyÚastyperc ZcumsumÚ_valuesr? Z
|
from_codesr¯ Ú_unconvert_string_arrayrW )rÉ rW r rZ r Ú convertedr rb r» r¶ r\ r± rÐ r1 r ÚmaskrU rU rV rÖ st
|
|
|
|
|
ÿ
|
ÿ
|
|
|
|
ÿ ÿzDataCol.convertc C sH t | j| j| j t | j| j| j | jdk s2tt | j| j| j dS )z set the data for this columnN) ró rÙ rº rW r r» rÐ rÐ r rÍ rU rU rV ré ë s zDataCol.set_attr) NNNNNNNNNNNN) rÝ r¥ r¦ r§ r® r¯ rï rÊ r© r r ræ rÈ r
|
rØ Ú classmethodr r r r r r r rÞ ræ rÖ ré Ú __classcell__rU rU r rV rþ × sX ò
|
|
|
|
|
|
drþ c @ sZ e Zd ZdZdZddddZedd Zed d
|
d d d Zedd Z edd Z
|
dS )ÚDataIndexableColz+represent a data column that can be indexedTr rË c C s t t| jstdd S )Nú-cannot have non-object label DataIndexableCol)r0 r7 rW r¨ rÍ rU rU rV râ ø s z DataIndexableCol.validate_namesc C s t j|dS )N)r¸ r r rU rU rV r ý s z DataIndexableCol.get_atom_stringrY rK r c C s | j |d S )Nr r r! rU rU rV r
|
s z DataIndexableCol.get_atom_datac C s
|
t ¡ S r\ r# r$ rU rU rV r
|
s z$DataIndexableCol.get_atom_datetime64c C s
|
t ¡ S r\ r# r$ rU rU rV r
|
s z%DataIndexableCol.get_atom_timedelta64N) rÝ r¥ r¦ r§ r¯ râ r? r r r r rU rU rU rV rA ó s
|
|
rA c @ s e Zd ZdZdS )ÚGenericDataIndexableColz(represent a generic pytables data columnN)rÝ r¥ r¦ r§ rU rU rU rV rC
|
s rC c @ s¼ e Zd ZU dZded< dZded< ded< ded < d
|
ed < d Zd ed< dNd
|
dddddddZed dddZ edddd Z
|
ed d Z ddd d Z ddd!d"Z d dd#d$Zed%d& Zed'd( Zed)d* Zed+d, Zeddd-d.Zed dd/d0Zed1d2 Zddd3d4Zddd5d6Zed7d8 Zed dd9d:Zed;d< Zd=dd>d?ZdOdddAdBZ d ddCdDZ dPdEdEdFdGdHZ dIdJ Z dQdEdEddKdLdMZ d@S )RÚFixedzø
|
represent an object in my store
|
facilitate read/write of various types of objects
|
this is an abstract base class
|
|
Parameters
|
----------
|
parent : HDFStore
|
group : Node
|
The group node where the table resides.
|
rY Ú pandas_kindrt Ú format_typeútype[DataFrame | Series]Úobj_typerc rP r r¾ Fr
r2 rO r{ rM rX r )r¾ r¹ rZ r r[ c C sZ t |tstt|td k s"tt |tjs:tt||| _|| _t|| _ || _
|
d S r\ ) rP r rÐ rÜ r| rM r¾ r¹ r_ rZ r )rÉ r¾ r¹ rZ r rU rU rV rÊ &
|
s
|
zFixed.__init__rË c C s* | j d dko(| j d dko(| j d dk S )Nr rf é
|
é )ÚversionrÍ rU rU rV Úis_old_version5
|
s zFixed.is_old_versionztuple[int, int, int]c C sb t t| jjdd}z0tdd | d¡D }t|dkrB|d }W n tk
|
r\ d}Y nX |S ) zcompute and set our versionÚpandas_versionNc s s | ]}t |V qd S r\ ©rc r! rU rU rV rI >
|
s z Fixed.version.<locals>.<genexpr>Ú.rJ r# )r r r )rW rh r¹ ri rm r rn r~ )rÉ rK rU rU rV rK 9
|
s
|
z Fixed.versionc C s t t| jjdd S )Nre )rW rh r¹ ri rÍ rU rU rV re E
|
s zFixed.pandas_typec C s^ | ¡ | j}|dk rXt|ttfrDd dd |D ¡}d| d}| jdd| d S | jS )
|
ú(return a pretty representation of myselfNr» c S s g | ] }t |qS rU ©rJ r! rU rU rV rk O
|
s z"Fixed.__repr__.<locals>.<listcomp>ú[r ú12.12z (shape->ú))r r rP rl rm r¾ re )rÉ rT ZjshaperU rU rV ræ I
|
s zFixed.__repr__c C s t | j| j_t t| j_dS )z set my pandas type & versionN)rY rE rÙ re Ú_versionrM rÍ rU rU rV r T
|
s zFixed.set_object_infoc C s t | ¡}|S r\ r2 )rÉ Znew_selfrU rU rV rw Y
|
s
|
z
|
Fixed.copyc C s | j S r\ )r rÍ rU rU rV r ]
|
s z Fixed.shapec C s | j jS r\ ©r¹ r³ rÍ rU rU rV r3 a
|
s zFixed.pathnamec C s | j jS r\ )r¾ r¿ rÍ rU rU rV r¿ e
|
s z Fixed._handlec C s | j jS r\ )r¾ rÇ rÍ rU rU rV rÇ i
|
s zFixed._filtersc C s | j jS r\ )r¾ rÄ rÍ rU rU rV rÄ m
|
s zFixed._complevelc C s | j jS r\ )r¾ rÆ rÍ rU rU rV rÆ q
|
s zFixed._fletcher32c C s | j jS r\ )r¹ ri rÍ rU rU rV rÙ u
|
s z Fixed.attrsc C s dS ©zset our object attributesNrU rÍ rU rU rV Ú set_attrsy
|
s zFixed.set_attrsc C s dS )zget our object attributesNrU rÍ rU rU rV Ú get_attrs|
|
s zFixed.get_attrsc C s | j S )zreturn my storabler rÍ rU rU rV Ústorable
|
s zFixed.storablec C s dS rý rU rÍ rU rU rV r
|
s zFixed.is_existsc C s t | jdd S )Nr )rh rZ rÍ rU rU rV r
|
s z Fixed.nrowszLiteral[True] | Nonec C s |dkr dS dS )z%validate against an existing storableNTrU rÇ rU rU rV Úvalidate
|
s zFixed.validateNc C s dS )ú+are we trying to operate on an old version?NrU )rÉ ro rU rU rV Úvalidate_version
|
s zFixed.validate_versionc C s | j }|dkrdS | ¡ dS )zr
|
infer the axes of my storer
|
return a boolean indicating if we have a valid storer or not
|
NFT)rZ rY )rÉ rT rU rU rV r
|
s
|
zFixed.infer_axesr ©r¡ r¢ c C s t dd S )Nz>cannot read on an abstract storer: subclasses should implement©r« ©rÉ ro r£ r¡ r¢ rU rU rV r
|
s ÿz
|
Fixed.readc K s t dd S )Nz?cannot write on an abstract storer: subclasses should implementr_ ©rÉ r· rU rU rV r «
|
s ÿz Fixed.write©r¡ r¢ r[ c C s0 t |||¡r$| jj| jdd dS tddS )zs
|
support fully deleting the node in its entirety (only) - where
|
specification must be None
|
Tr; Nz#cannot delete on an abstract storer)r> r? r¿ r r¹ r¯ )rÉ ro r¡ r¢ rU rU rV r@ °
|
s z Fixed.delete)rO r{ )N)NNNN)NNN)!rÝ r¥ r¦ r§ r¨ rF r2 rÊ r© rL rK re ræ r rw r r3 r¿ rÇ rÄ rÆ rÙ rX rY rZ r r r[ r] r r r r@ rU rU rU rV rD
|
sl
|
û
|
|
|
|
|
|
|
|
û ÿrD c @ sF e Zd ZU dZedediZdd e ¡ D Zg Z de
|
d< dd d
|
d Z d d Z dd Z dd ddZedd ddZdd ddZdd ddZdd ddZd:dd d d d d Zd;dd d d!d"d#d$Zdd!dd%d&d'Zdd(dd%d)d*Zd<dd d d(d"d+d,Zd=d-d d d!d.d/d0Zdd1dd2d3d4Zd>dd5d6dd7d8d9Zd S )?Ú GenericFixedza generified fixed versionÚdatetimer c C s i | ]\}}||qS rU rU )rh r rL rU rU rV rM Â
|
s zGenericFixed.<dictcomp>rí Ú
|
attributesrY rË c C s | j |d¡S )NÚ )Ú_index_type_maprÔ )rÉ r rU rU rV Ú_class_to_aliasÆ
|
s z GenericFixed._class_to_aliasc C s t |tr|S | j |t¡S r\ )rP rÜ Ú_reverse_index_maprÔ r7 )rÉ ÚaliasrU rU rV Ú_alias_to_classÉ
|
s
|
z GenericFixed._alias_to_classc C s¸ | tt|dd¡}|tkr.d dd}|}n |tkrFd
|
dd}|}n|}i }d|krn|d |d< |tkrnt}d|kr°t|d tr|d d¡|d< n |d |d< |tks°t
|
||fS ) NÚ index_classrf c S s: t j| j|d}tj|d d}|d k r6| d¡ |¡}|S )N©r° r` ÚUTC)r@ Ú _simple_newrW r6 Ú tz_localizeÚ
|
tz_convert)rW r° r± ZdtaÚresultrU rU rV rv Ø
|
s
|
z*GenericFixed._get_index_factory.<locals>.fc S s t j| |d}tj|d dS )Nrm r` )rA ro r9 )rW r° r± ZparrrU rU rV rv ã
|
s r° r± zutf-8)NN)NN) rk rW rh r6 r9 r7 r< rP ÚbytesrS rÐ )rÉ rÙ rl rv rÕ r· rU rU rV Ú_get_index_factoryÏ
|
s* ÿ
|
|
z GenericFixed._get_index_factoryr c C s$ |dk rt d|dk r t ddS )zE
|
raise if any keywords are passed which are not-None
|
Nzqcannot pass a column specification when reading a Fixed format store. this store must be selected in its entiretyzucannot pass a where specification when reading from a Fixed format store. this store must be selected in its entirety)r¯ )rÉ r£ ro rU rU rV Ú validate_readý
|
s ÿÿzGenericFixed.validate_readr
c C s dS )NTrU rÍ rU rU rV r s zGenericFixed.is_existsc C s | j | j_ | j| j_dS rW )rZ rÙ r rÍ rU rU rV rX s
|
zGenericFixed.set_attrsc C sR t t| jdd| _tt| jdd| _| jD ] }t| |tt| j|d q.dS )úretrieve our attributesrZ Nr r{ )r_ rh rÙ rZ rW r re ró )rÉ rñ rU rU rV rY s
|
zGenericFixed.get_attrsc K s | ¡ d S r\ )rX ©rÉ r r· rU rU rV r s zGenericFixed.writeNr r c C sÐ ddl }t| j|}|j}t|dd}t||jrD|d ||
}nztt|dd} t|dd}
|
|
|
dk rxtj|
|
| d}n |||
}| dkr¨t|d d} t || d
|
d }n| d kr¾tj
|
|d d}|rÈ|j S |S dS )z2read an array for the specified node (off of groupr NÚ
|
transposedFÚ
|
value_typer r* r% r± Tr& r( r) ) r} rh r¹ ri rP ZVLArrayrW rQ r rÓ r5 ÚT) rÉ r r¡ r¢ r} rà rÙ rx ÚretrÐ r r± rU rU rV Ú
|
read_array s& zGenericFixed.read_arrayr7 )r r¡ r¢ r[ c C sh t t| j| d}|dkr.| j|||dS |dkrVt| j|}| j|||d}|S td| d S )NÚ_varietyÚmultir^ Úregularz unrecognized index variety: )rW rh rÙ Úread_multi_indexr¹ Úread_index_noder¯ )rÉ r r¡ r¢ Zvarietyrà r rU rU rV Ú
|
read_indexB s zGenericFixed.read_index)r r r[ c C sà t |tr,t| j| dd | ||¡ n°t| j| dd td|| j| j}| ||j ¡ t
|
| j |}|j |j _ |j|j _t |ttfr | t|¡|j _t |tttfrº|j|j _t |trÜ|jd k rÜt|j|j _d S )Nr} r~ r r )rP r8 ró rÙ Úwrite_multi_indexÚ_convert_indexrZ r Ú write_arrayrW rh r¹ rb ri ra r6 r9 rh rÜ rl r< r° r± Ú_get_tz)rÉ r r r= rà rU rU rV Ú write_indexP s
|
|
|
|
zGenericFixed.write_indexr8 c C sÎ t | j| d|j tt|j|j|jD ]\}\}}}t|rJt d| d| }t
|
||| j | j }| ||j¡ t| j|} |j| j_|| j_t | j| d| | | d| }
|
| |
|
|¡ q,d S )NÚ_nlevelsz=Saving a MultiIndex with an extension dtype is not supported.Ú_levelÚ_nameÚ_label)ró rÙ r Ú enumerater1 Úlevelsr Únamesr- r« r rZ r r
rW rh r¹ rb ri ra ) rÉ r r ÚiÚlevÚ level_codesra Ú level_keyZ
|
conv_levelrà Ú label_keyrU rU rV r g s" ÿÿ
|
z GenericFixed.write_multi_indexc C s¤ t | j| d}g }g }g }t|D ]l}| d| } t | j| }
|
| j|
|
||d} | | ¡ | | j¡ | d| } | j| ||d} | | ¡ q&t|||ddS )Nr r r^ r T)r r r r) ) rh rÙ rO r¹ r r ra r| r8 )rÉ r r¡ r¢ r r r r r r rà r r r rU rU rV r s&
|
ÿz GenericFixed.read_multi_indexrM )rà r¡ r¢ r[ c C sÔ |||
}d|j kr>t |j j¡dkr>tj|j j|j jd}t|j j}d }d|j krlt|j j }t|}|j }|
|
|¡\}} |dkr®|t ||| j | j dfdti| }
|
n |t ||| j | j df| }
|
||
|
_ |
|
S )Nr r r* ra )r r6 r rÐ )ri rQ Úprodr r ry rW rb rb ra rt Ú_unconvert_indexrZ r r6 ) rÉ rà r¡ r¢ rv rb ra rÙ rÕ r· r rU rU rV r sF
|
ÿÿüû ÿÿüz GenericFixed.read_index_noder )r r r[ c C sJ t d|j ¡}| j | j||¡ t| j|}t|j|j _
|
|j |j _ dS )zwrite a 0-len array©rf N) rQ r rP r¿ Ú create_arrayr¹ rh rY rÐ ri ry r )rÉ r r Zarrrà rU rU rV Úwrite_array_empty½ s
|
z GenericFixed.write_array_emptyr z Index | None)r r rù r[ c C s4 t |dd}|| jkr&| j | j|¡ |jdk}d}t|jrFtd|s^t|dr^|j }d}d }| j
|
d k rt t t j |j¡}W 5 Q R X |d k rÖ|sÆ| jj| j|||j| j
|
d}||d d
< n | ||¡ nJ|jjtjkrJtj|dd} |rún,| d krn t| ||f }
|
tj|
|
tt d
|
| j | j|t ¡ ¡} | |¡ nÖt |jr| j !| j|| "d ¡¡ d t#| j|j$_%n t&|jrÄ| j !| j||j'¡ t#| j|} t(|j)| j$_)d | j$_%n\t*|jrú| j !| j|| "d ¡¡ d t#| j|j$_%n&|r| ||¡ n| j !| j||¡ |t#| j|j$_+d S )NT)Z extract_numpyr Fz]Cannot store a category dtype in a HDF5 dataset that uses format="fixed". Use format="table".rz )r ©Zskipnarß rí rÍ r% r( ),rD r¹ r¿ r r r) rÐ r« rÊ rz rÇ r r¨ r ZAtomZ
|
from_dtypeZ create_carrayr r rÜ rQ Zobject_r Ú infer_dtypers rñ rò r# r& Zcreate_vlarrayÚ
|
ObjectAtomr r+ r Úviewrh ri ry r, Úasi8r r± r2 rx ) rÉ r r rù r Z empty_arrayrx r ÚcaÚ inferred_typerõ Zvlarrrà rU rU rV r
Æ sr
|
|
|
ÿ
|
|
|
ÿ
|
ÿ
|
zGenericFixed.write_array)NN)NN)NN)NN)N) rÝ r¥ r¦ r§ r6 r9 rg rù ri re r¨ rh rk rt ru r© r rX rY r r| r r r r r r r
rU rU rU rV rc ¾
|
s8
|
.# ÿ ÿ ÿ&
|
ÿrc c sV e Zd ZU dZdgZded< edd Zddddd d
|
d Zd d fdd Z Z
|
S )r r ra r c C s0 zt | jjfW S ttfk
|
r* Y d S X d S r\ )rn r¹ rW r¯ r~ rÍ rU rU rV r ' s zSeriesFixed.shapeNr r; rb c C s> | ||¡ | jd||d}| jd||d}t||| jddS )Nr r^ rW F)r ra rw )ru r r| r; ra )rÉ ro r£ r¡ r¢ r rW rU rU rV r . s zSeriesFixed.readr rË c s8 t j|f| | d|j¡ | d|¡ |j| j_d S )Nr rW )rÿ r r r r
ra rÙ rw r rU rV r ; s zSeriesFixed.write)NNNN) rÝ r¥ r¦ rE re r¨ r© r r r r@ rU rU r rV r ! s
|
|
û r c sZ e Zd ZU ddgZded< eddddZdd d d
|
d d d Zdd fdd Z Z S )ÚBlockManagerFixedrP Únblocksrc z Shape | NonerË c C s° z| j }d}t| jD ]8}t| jd| d}t|dd }|d k r||d 7 }q| jj}t|dd }|d k rt|d|d
}ng }| |¡ |W S tk
|
rª Y d S X d S )Nr ÚblockÚ_itemsr rf ) rP rO r¡ rh r¹ Z block0_valuesrl r r~ )rÉ rP rù r rà r rU rU rV r G s"
|
zBlockManagerFixed.shapeNr r5 rb c C s | ||¡ | ¡ d¡}g }t| jD ]<}||kr<||fnd\}} | jd| || d}
|
| |
|
¡ q(|d } g } t| jD ]\}| d| d¡} | jd| d|| d}| | | ¡ }t
|
|j ||d d d
|
}| |¡ q|t | dkrt | dd d }|j| d d }|S t
|
|d |d dS )Nr )NNr( r^ r¢ r£ r; rf F©r£ r rw T)r( rw )r£ rw ©r£ r )ru rH Z_get_block_manager_axisrO rP r r r¡ r| rU r5 rz rn r= r[ )rÉ ro r£ r¡ r¢ Z select_axisrB r r r Úaxrù ÚdfsÚ blk_itemsrW ÚdfÚoutrU rU rV r b s( zBlockManagerFixed.readr c sä t j|f| t|jtr&| d¡}|j}| ¡ s<| ¡ }|j| j _t
|
|j D ]0\}}|dkrn|j snt d| d| |¡ qPt|j| j _t
|
|jD ]D\}}|j |j¡}| jd| d|j|d | d| d|¡ qd S )Nr¢ r z/Columns index has to be unique for fixed formatr( r; )rù r£ )rÿ r rP Ú_mgrrF Ú _as_managerZis_consolidatedZ consolidaterP rÙ r rB Z is_uniquer¨ r rn Úblocksr¡ rù rV Úmgr_locsr
rW )rÉ r r· rv r r¦ Úblkr¨ r rU rV r s
|
|
zBlockManagerFixed.write)NNNN)
|
rÝ r¥ r¦ re r¨ r© r r r r@ rU rU r rV r B s
|
û$r c @ s e Zd ZdZeZdS )r r N)rÝ r¥ r¦ rE r5 rH rU rU rU rV r s r c s¬ e Zd ZU dZdZdZded< ded< dZded < d
|
Zd ed < dddddddddddd
|
fdd Z e
|
ddd d Z ddd d Z dd d!d"Z ddd#d$Ze
|
d%dd&d'Zd(d)d*d+d,Ze
|
d-dd.d/Ze
|
d%dd0d1Ze
|
d2d3 Ze
|
d4d5 Ze
|
d6d7 Ze
|
d8d9 Ze
|
d:d; Ze
|
d-dd<d=Ze
|
d%dd>d?Ze
|
d@ddAdBZdCddDdEZdFdG Z dHddIdJZ dddKdLdMZ ddNddOdPdQZ ddRdSdTZ dddUdVZ!dddWdXZ"ddddYdZZ#ddd[d\Z$e%d]d^ Z&dddd_d`daZ'ddbdbdcdddedfZ(e)d%dgdhdiZ*djdk Z+ddld%dmdndoZ,e-dld%dpdqdrZ.d
dsdldtdudvZ/dbd%dbdCdwdxdyZ0ddbdbdzd{d|Z1dddbdbd}d~dZ2 Z3S )ró aa
|
represent a table:
|
facilitate read/write of various types of tables
|
|
Attrs in Table Node
|
-------------------
|
These are attributes that are store in the main table node, they are
|
necessary to recreate these tables when read back in.
|
|
index_axes : a list of tuples of the (original indexing axis and
|
index column)
|
non_index_axes: a list of tuples of the (original index axis and
|
columns on a non-indexing axis)
|
values_axes : a list of the columns which comprise the data of this
|
table
|
data_columns : a list of the columns that we are allowing indexing
|
(these become single columns in values_axes)
|
nan_rep : the string to use for nan representations for string
|
objects
|
levels : the names of levels
|
metadata : the names of the metadata columns
|
Z
|
wide_tableru rY rF r~ rf zint | list[Hashable]r Trl r¶ Nr{ r rM rX zlist[IndexCol] | Nonez list[tuple[AxisInt, Any]] | Nonezlist[DataCol] | Nonez list | Nonez dict | Noner )
|
r¾ r¹ rZ r Ú
|
index_axesr$ Ú values_axesr r{ r[ c sP t j||||d |pg | _|p$g | _|p.g | _|p8g | _| pBi | _|
|
| _d S )Nr )rÿ rÊ r° r$ r± r r{ r ) rÉ r¾ r¹ rZ r r° r$ r± r r{ r r rU rV rÊ Å s
|
|
|
|
|
zTable.__init__rË c C s | j d¡d S )NÚ_r )r~ r rÍ rU rU rV Útable_type_shortÚ s zTable.table_type_shortc C s¦ | ¡ t| jr d | j¡nd}d| d}d}| jrZd dd | jD ¡}d| d}d d d | jD ¡}| jd
|
| d | j d | j d | j
|
d| d| d S )rP r» rf z,dc->[r rO c S s g | ] }t |qS rU ©rY r! rU rU rV rk æ s z"Table.__repr__.<locals>.<listcomp>rR c S s g | ]
|
}|j qS rU r` ru rU rU rV rk é s rS z (typ->z,nrows->z,ncols->z ,indexers->[rT ) r rn r r¾ rL rK r° re r³ r Úncols)rÉ Zjdcr] ÚverZjverZ jindex_axesrU rU rV ræ Þ s 4ÿzTable.__repr__)rë c C s" | j D ]}||jkr| S qdS )zreturn the axis for cN)rB ra )rÉ rë r rU rU rV rÖ ð s
|
|
|
zTable.__getitem__c
|
C sº |dkr dS |j | j kr2td|j d| j ddD ]~}t| |d}t||d}||kr6t|D ]4\}}|| }||krbtd| d| d| dqbtd| d| d| dq6dS ) z"validate against an existing tableNz'incompatible table_type with existing [rì r )r° r$ r± zinvalid combination of [z] on appending data [z] vs current table [)r~ r¯ rh r r¨ r= )rÉ r rë ÚsvÚovr ÚsaxZoaxrU rU rV r[ ÷ s& ÿ ÿÿzTable.validater
c C s t | jtS )z@the levels attribute is 1 or a list in the case of a multi-index)rP r rl rÍ rU rU rV Úis_multi_index s zTable.is_multi_indexr z tuple[DataFrame, list[Hashable]])r r[ c
|
C s^ t |jj¡}z | ¡ }W n, tk
|
rF } ztd|W 5 d}~X Y nX t|tsVt||fS )ze
|
validate that we can store the multi-index; reset and return the
|
new object
|
zBduplicate names/columns in the multi-index when storing as a tableN) r> Zfill_missing_namesr r Z reset_indexr¨ rP r5 rÐ )rÉ r r Z reset_objrA rU rU rV Úvalidate_multiindex s ÿþzTable.validate_multiindexrc c C s t dd | jD ¡S )z-based on our axes, compute the expected nrowsc S s g | ]}|j jd qS r# )rÞ r ©rh r rU rU rV rk 1 s z(Table.nrows_expected.<locals>.<listcomp>)rQ r r° rÍ rU rU rV Únrows_expected. s zTable.nrows_expectedc C s
|
d| j kS )zhas this table been createdru r rÍ rU rU rV r 3 s zTable.is_existsc C s t | jdd S ©Nru ©rh r¹ rÍ rU rU rV rZ 8 s zTable.storablec C s | j S )z,return the table group (this is my storable))rZ rÍ rU rU rV ru < s z Table.tablec C s | j jS r\ )ru rÐ rÍ rU rU rV rÐ A s z Table.dtypec C s | j jS r\ rÚ rÍ rU rU rV rÛ E s zTable.descriptionc C s t | j| j¡S r\ )r/ r0 r° r± rÍ rU rU rV rB I s z
|
Table.axesc C s t dd | jD S )z.the number of total columns in the values axesc s s | ]}t |jV qd S r\ )rn rW ru rU rU rV rI P s z Table.ncols.<locals>.<genexpr>)Úsumr± rÍ rU rU rV rµ M s z Table.ncolsc C s dS rý rU rÍ rU rU rV Ú is_transposedR s zTable.is_transposedztuple[int, ...]c C s( t t dd | jD dd | jD ¡S )z@return a tuple of my permutated axes, non_indexable at the frontc S s g | ]}t |d qS r# rN ru rU rU rV rk [ s z*Table.data_orientation.<locals>.<listcomp>c S s g | ]}t |jqS rU )rc r( ru rU rU rV rk \ s )rm r/ r0 r$ r° rÍ rU rU rV Údata_orientationV s þÿzTable.data_orientationzdict[str, Any]c sR ddd dd j D } fddjD }fddjD }t|| | S )z<return a dict of the kinds allowable columns for this objectr r£ ©r rf c S s g | ]}|j |fqS rU ©r³ ru rU rU rV rk f s z$Table.queryables.<locals>.<listcomp>c s g | ]\}} | d fqS r\ rU )rh r( rW )Ú
|
axis_namesrU rV rk g s c s& g | ] }|j t jkr|j|fqS rU )ra rN r r³ r- rÍ rU rV rk h s )r° r$ r± rD )rÉ Úd1Zd2Zd3rU )rÅ rÉ rV Ú
|
queryables` s
|
|
ÿzTable.queryablesc C s dd | j D S )z return a list of my index colsc S s g | ]}|j |jfqS rU )r( r³ r¼ rU rU rV rk q s z$Table.index_cols.<locals>.<listcomp>©r° rÍ rU rU rV Ú
|
index_colsn s zTable.index_colsrí c C s dd | j D S )z return a list of my values colsc S s g | ]
|
}|j qS rU rÄ r¼ rU rU rV rk u s z%Table.values_cols.<locals>.<listcomp>)r± rÍ rU rU rV Ú values_colss s zTable.values_colsr× c C s | j j}| d| dS )z)return the metadata pathname for this keyz/meta/z/metarV r rU rU rV Ú_get_metadata_pathw s zTable._get_metadata_pathrË )r rW r[ c C s0 | j j| |¡t|ddd| j| j| jd dS )z£
|
Write out a metadata array to the key as a fixed-format Series.
|
|
Parameters
|
----------
|
key : str
|
values : ndarray
|
Fr2 ru )r rZ r r N)r¾ r rË r; rZ r r )rÉ r rW rU rU rV rè | s
|
úzTable.write_metadatarÓ c C s0 t t | jdd|ddk r,| j | |¡¡S dS )z'return the meta data array for this keyr» N)rh r¹ r¾ r´ rË rÕ rU rU rV rû s zTable.read_metadatac C sp t | j| j_| ¡ | j_| ¡ | j_| j| j_| j| j_| j| j_| j| j_| j | j_ | j
|
| j_
|
| j | j_ dS )z set our table type & indexablesN) rY r~ rÙ rÉ rÊ r$ r r rZ r r r{ rÍ rU rU rV rX s
|
|
|
|
|
|
zTable.set_attrsc C s° t | jddpg | _t | jddp$g | _t | jddp8i | _t | jdd| _tt | jdd| _tt | jdd| _ t | jd dpg | _
|
d
|
d | j D | _ d d | j D | _ dS ) rv r$ Nr r{ r rZ r r{ r c S s g | ]}|j r|qS rU ©r® ru rU rU rV rk ª s z#Table.get_attrs.<locals>.<listcomp>c S s g | ]}|j s|qS rU rÌ ru rU rU rV rk « s )rh rÙ r$ r r{ r r_ rZ rW r r Ú
|
indexablesr° r± rÍ rU rU rV rY ¡ s zTable.get_attrsc C s> |dk r:| j r:td dd | jD ¡ }tj|tt d dS )r\ NrO c S s g | ] }t |qS rU r´ r! rU rU rV rk ± s z*Table.validate_version.<locals>.<listcomp>rí )rL rq r¾ rK rñ rò r" r& )rÉ ro rõ rU rU rV r] s ýzTable.validate_versionc C sR |dkr dS t |tsdS | ¡ }|D ]&}|dkr4q&||kr&td| dq&dS )z
|
validate the min_itemsize doesn't contain items that are not in the
|
axes this needs data_columns to be defined
|
NrW zmin_itemsize has the key [z%] which is not an axis or data_column)rP rD rÇ r¨ )rÉ r Úqr rU rU rV Úvalidate_min_itemsize¸ s
|
|
ÿzTable.validate_min_itemsizec sÔ g }j jjtjjD ]j\}\}}t|} |¡}|dk rJdnd}| d}t|d} t|||| |j||d}
|
| |
|
¡ q t j
|
t | fdd| fddtjj D ¡ |S ) z/create/cache the indexables if they don't existNrú r¹ )ra r( rµ rb r´ ru r» r¶ c s¢ t |tstt}|kr t}t|}t|j}t| dd }t| dd }t|} |¡}t| dd } ||||| | |j
|
| ||d
|
}
|
|
|
S )Nr¹ r r )
|
ra r³ rW rb rµ r´ ru r» r¶ rÐ ) rP rY rÐ rþ rA rh Ú_maybe_adjust_namerK r rû ru ) r rë Úklassr Úadj_namerW rÐ rb Úmdr» r )Úbase_posr] ÚdescrÉ Ú table_attrsrU rV rv ð s0
|
|
ö zTable.indexables.<locals>.fc s g | ]\}} ||qS rU rU )rh r rë )rv rU rV rk s z$Table.indexables.<locals>.<listcomp>)rÛ ru rÙ r rÉ rh rû r r rN r rn rR rÊ ) rÉ Ú _indexablesr r( ra r rÓ r» rº rb Ú index_colrU )rÔ r] rÕ rv rÉ rÖ rV rÍ Í s2
|
|
|
ø
|
|
% zTable.indexablesr c C sR | ¡ s dS |dkrdS |dks(|dkr8dd | jD }t|ttfsL|g}i }|dk r`||d< |dk rp||d< | j}|D ]Ò}t|j|d}|dk r |jrò|j }|j
|
} |j }
|
|dk rÈ|
|
|krÈ| ¡ n|
|
|d< |dk rê| |krê| ¡ n| |d< |jsL|j d¡rtd |jf | qz|| jd
|
d krztd | d | d| dqzdS )aZ
|
Create a pytables index on the specified columns.
|
|
Parameters
|
----------
|
columns : None, bool, or listlike[str]
|
Indicate which columns to create an index on.
|
|
* False : Do not create any indexes.
|
* True : Create indexes on all columns.
|
* None : Create indexes on all columns.
|
* listlike : Create indexes on the given columns.
|
|
optlevel : int or None, default None
|
Optimization level, if None, pytables defaults to 6.
|
kind : str or None, default None
|
Kind of index, if None, pytables defaults to "medium".
|
|
Raises
|
------
|
TypeError if trying to create an index on a complex-type column.
|
|
Notes
|
-----
|
Cannot index Time64Col or ComplexCol.
|
Pytables must be >= 3.0.
|
NFTc S s g | ]}|j r|jqS rU )r¯ r³ ru rU rU rV rk > s z&Table.create_index.<locals>.<listcomp>ra rb ÚcomplexzíColumns containing complex values can be stored but cannot be indexed when using table format. Either use fixed format, set index=False, or do not include the columns containing complex values to data_columns when initializing the table.r rf zcolumn z/ is not a data_column.
|
In order to read column z: you must reload the dataframe
|
into HDFStore and include z with the data_columns argument.)r rB rP rm rl ru rh rH rt r ra rb Z remove_indexrÜ ro r¯ rc r$ r~ ) rÉ r£ ra rb Úkwru rë rL r Z cur_optlevelZcur_kindrU rU rV rc sJ
|
|
|
ÿÿzTable.create_indexr z!list[tuple[ArrayLike, ArrayLike]]rb c C sZ t | |||d}| ¡ }g }| jD ]2}| | j¡ |j|| j| j| jd}| |¡ q"|S )a
|
Create the axes sniffed from the table.
|
|
Parameters
|
----------
|
where : ???
|
start : int or None, default None
|
stop : int or None, default None
|
|
Returns
|
-------
|
List[Tuple[index_values, column_values]]
|
r r4 )
|
Ú Selectionr´ rB rù r{ rÖ r rZ r r ) rÉ ro r¡ r¢ Ú selectionrW r¬ r ÚresrU rU rV Ú
|
_read_axeso s
|
ü zTable._read_axes©rx c C s |S )z return the data for this objrU ©r r rx rU rU rV Ú
|
get_object s zTable.get_objectc s² t |s g S |d \} | j |i ¡}| d¡dkrL|rLtd| d| |dkr^t }n |dkrjg }t|tr t|t|}| fdd | ¡ D ¡ fd
|
d |D S ) zd
|
take the input data_columns and min_itemize and create a data
|
columns spec
|
r rÜ r8 z"cannot use a multi-index on axis [z] with data_columns TNc s g | ]}|d kr| kr|qS r× rU r )Úexisting_data_columnsrU rV rk ² s þz/Table.validate_data_columns.<locals>.<listcomp>c s g | ]}| kr|qS rU rU )rh rë )Ú axis_labelsrU rV rk º s )
|
rn r{ rÔ r¨ rl rP rD rN rR rõ )rÉ r r r$ r( r{ rU )rã râ rV Úvalidate_data_columns s* ÿ
|
|
|
þÿ zTable.validate_data_columnsr5 )r r[ c / s t ts,| jj}td| dt d dkr:dg fdd D | ¡ rzd}d d | jD t| j }| j
|
}nd
|
}| j } | j d kst t | j d kr¬td g }
|
|dkr¼d} fdddD d } j| } t| } |r<t|
|
}| j| d }tt | ¡t |¡s<tt t| ¡t t|¡r<|} | | i ¡}t| j|d< t| j|d< |
|
| | f¡ d } j| } | ¡}t|| | j | j }| |_ | d¡ | | ¡ | !|¡ |g}t|}|d ksàt t|
|
d ksòt |
|
D ]} t"| d | d qö|j d k}| #|||
|
¡}| $|¡ %¡ }| &|||
|
| j'|¡\}}g }t(t)||D ]¸\}\}} t*} d} |rÆt| d krÆ| d |krÆt+} | d } | dksÆt | t,sÆtd|r&|r&z| j'| } W nB t-t.fk
|
r" } z td| d| j' d| W 5 d} ~ X Y nX nd} | p8d| }!t/|!|j0| ||| j | j | d}"t1|!| j2}#| 3|"¡}$t4|"j5j6}%d}&t7|"dddk rt8|"j9}&d }' }(})t:|"j5rÐ|"j;})d}'tj|"j<d
|
d =¡ }(t>|"\}*}+| |#|!t| |$||%|&|)|'|(|+|*d },|, | ¡ | |,¡ |d 7 }qdd d |D }-t| | j?| j| j | j ||
|
||-| |d
|
}.t@| d rj| jA|._A|. B|¡ |r|r|. C| ¡ |.S ) a0
|
Create and return the axes.
|
|
Parameters
|
----------
|
axes: list or None
|
The names or numbers of the axes to create.
|
obj : DataFrame
|
The object to create axes on.
|
validate: bool, default True
|
Whether to validate the obj against an existing object already written.
|
nan_rep :
|
A value to use for string column nan_rep.
|
data_columns : List[str], True, or None, default None
|
Specify the columns that we want to create to allow indexing on.
|
|
* True : Use all available columns.
|
* None : Use no columns.
|
* List[str] : Use the specified columns.
|
|
min_itemsize: Dict[str, int] or None, default None
|
The min itemsize for a column in bytes.
|
z/cannot properly create the storer for: [group->r r Nr c s g | ]} |¡qS rU )Ú_get_axis_numberru )r rU rV rk è s z&Table._create_axes.<locals>.<listcomp>Tc S s g | ]
|
}|j qS rU rJ ru rU rU rV rk í s FrJ rf z<currently only support ndim-1 indexers in an AppendableTableÚnanc s g | ]}| kr|qS rU rU r! )rB rU rV rk s rà r rÜ rB z Incompatible appended table [z]with existing table [Z values_block_)Ú existing_colr r rZ r r£ r± rú r2 ) ra r³ rW r´ rµ rb r± r\ r» r¶ rÐ rv c S s g | ]}|j r|jqS rU )r¯ ra )rh rÜ rU rU rV rk s )
|
r¾ r¹ rZ r r° r$ r± r r{ r r )DrP r5 r¹ r¼ r¯ rÜ r r° rl r r r{ rP rÐ rn r¨ rB r$ r4 rQ ÚarrayrT rð r rÝ r Z_get_axis_namer rZ r r( r· rö rá Ú _reindex_axisrä rá r* Ú_get_blocks_and_itemsr± r r1 rþ rA rY Ú
|
IndexErrorrµ Ú _maybe_convert_for_string_atomrW rÐ rK r r rÐ ra rh r r± r) r\ r1 r7 r r¾ rÊ r rÏ r[ )/rÉ rB r r[ r r r r¹ Ú table_existsZnew_infoÚnew_non_index_axesrô r Z append_axisZindexerZ
|
exist_axisr{ Ú axis_nameZ new_indexZnew_index_axesÚjrx r r r¨ Zvaxesr r¯ Úb_itemsrÑ ra rç rA Únew_nameÚdata_convertedrÒ r´ rb r± r» r¶ r\ rv r rÜ ZdcsZ new_tablerU )rB r rV Ú _create_axes¼ s"
|
ÿ
|
ÿ
|
ÿ
|
|
|
|
|
|
ÿ ÿ " ÿýø
|
|
|
ô
|
|
ö
|
|
zTable._create_axes)r rí c C s t | jtr| d¡} dd }| j}tt|}t|j}||}t|rÒ|d \} }
|
t |
|
|
t |¡} | j | | dj}tt|}t|j}||}|D ]:} | j | g| dj}tt|}| |j¡ | ||¡ q|rdd t ||D } g }g }|D ]}t|j}z&| |¡\}}| |¡ | |¡ W qø ttfk
|
rz } z*d d d
|
|D ¡}td | d |W 5 d }~X Y qøX qø|}|}||fS ) Nr¢ c s fdd j D S )Nc s g | ]} j |j¡qS rU )rù rV r® )rh r¯ ©ÚmgrrU rV rk ³ s zFTable._get_blocks_and_items.<locals>.get_blk_items.<locals>.<listcomp>)r rõ rU rõ rV Ú get_blk_items² s z2Table._get_blocks_and_items.<locals>.get_blk_itemsr rJ c S s" i | ]\}}t | ¡ ||fqS rU )rm Útolist)rh Úbrñ rU rU rV rM Ñ s ÿ
|
z/Table._get_blocks_and_items.<locals>.<dictcomp>r» c S s g | ] }t |qS rU rQ )rh ÚitemrU rU rV rk Þ s z/Table._get_blocks_and_items.<locals>.<listcomp>z+cannot match existing table structure for [z] on appending data)rP r« rF r¬ r rG rl r rn r7 rS r[ rR r1 rm rW r4 r rë rµ r¾ r¨ )r rí rî r± r r÷ rö r r¨ r( rã Z
|
new_labelsrë Zby_itemsZ
|
new_blocksZ new_blk_itemsZearù rù rñ rA ZjitemsrU rU rV rê ¤ sR
|
|
|
|
|
|
þ
|
|
|
ÿýzTable._get_blocks_and_itemsrÛ )rÜ r[ c sª |dk rt |}|dk rNjrNtjt s.tjD ]}||kr4| d|¡ q4jD ]$\}}t ||| fdd}qT|jdk r¦|j ¡ D ]\}} }
|
|||
|
| q S )zprocess axes filtersNr c sÌ j D ]°} |¡} |¡}|d k s*t| |krfjrH| tj¡}|||} j|d| S | |krt t
|
| j }t |}t t rd| }|||} j|d| S qtd| dd S )NrJ rf zcannot find the field [z] for filtering!)Z _AXIS_ORDERSrå Ú _get_axisrÐ rº Úunionr7 r rZ rE rh rW rP r5 r¨ )ÚfieldÚfiltÚoprï Z axis_numberZ axis_valuesZtakersrW ©r rÉ rU rV Úprocess_filterù s"
|
|
|
|
|
|
z*Table.process_axes.<locals>.process_filter)
|
rl rº rP r rÐ Úinsertr$ ré Úfilterr ) rÉ r rÜ r£ rñ r( Úlabelsr rý rÿ rþ rU r rV Ú process_axesè s
|
|
zTable.process_axes)r rÁ rC r[ c C s |dkrt | jd}d|d}dd | jD |d< |rj|dkrH| jpFd}t j|||pZ| jd }||d
|
< n| jdk r~| j|d
|
< |S ) z:create the description of the table from the axes & valuesNi' ru )ra rC c S s i | ]}|j |jqS rU )r³ r´ ru rU rU rV rM . s z,Table.create_description.<locals>.<dictcomp>rÛ é )r r rÁ r )Úmaxr½ rB rÄ r rû rÆ rÇ )rÉ r r rÁ rC rE r rU rU rV Úcreate_description s
|
|
ý
|
|
|
zTable.create_descriptionr^ c
|
C s | |¡ | ¡ sdS t| |||d}| ¡ }|jdk r|j ¡ D ]D\}}}| j|| ¡ | ¡ d d} ||| j || ¡ |j
|
}qBt |S )zf
|
select coordinates (row numbers) from a table; return the
|
coordinates object
|
Fr Nrf r^ ) r] r rÛ Ú select_coordsr r r r« r ÚilocrW r7 )
|
rÉ ro r¡ r¢ rÜ Zcoordsrý rÿ rþ rv rU rU rV r > s
|
|
|
ÿ zTable.read_coordinatesr c C s¼ | ¡ | ¡ sdS |dk r$td| jD ]|}||jkr*|jsNtd| dt| jj |}|
|
| j ¡ |j |||
| j | j| jd}tt|d |j|dd S q*td| d dS )
|
zj
|
return a single column from the table, generally only indexables
|
are interesting
|
FNz4read_column does not currently accept a where clausezcolumn [z=] can not be extracted individually; it is not data indexabler4 rf )ra rw z] not found in the table)r] r r¯ rB ra r¯ r¨ rh ru rH rù r{ rÖ r rZ r r; rÓ r± rµ )rÉ r ro r¡ r¢ r rë Z
|
col_valuesrU rU rV r X s*
|
|
|
ÿ
|
ü zTable.read_column)Nr{ NNNNNN)N)NNN)NN)TNNN)N)NNN)NNN)4rÝ r¥ r¦ r§ rE rF r¨ r r2 rÊ r© r³ ræ rÖ r[ rº r» r½ r rZ ru rÐ rÛ rB rµ rÁ r rÇ rÉ rÊ rË rè rû rX rY r] rÏ r% rÍ rc rÞ r? rá rä rô Ú staticmethodrê r r r r r@ rU rU r rV ró ¥ s¨
|
õ&!
|
|
|
|
|
|
L ÿW ÿ"* ù iC7 ÿ ûró c @ s4 e Zd ZdZdZd dddddZdd d
|
d ZdS ) r zË
|
a write-once read-many table: this format DOES NOT ALLOW appending to a
|
table. writing is a one-time operation the data are stored in a format
|
that allows for searching the data on disk
|
r Nr r^ c C s t ddS )z[
|
read the indices and the indexing array, calculate offset rows and return
|
z!WORMTable needs to implement readNr_ r` rU rU rV r s
|
zWORMTable.readr rË c K s t ddS )zÞ
|
write in a format that we can search later on (but cannot append
|
to): write out the indices and the values using _write_array
|
(e.g. a CArray) create an indexing table so that we can search
|
z"WORMTable needs to implement writeNr_ ra rU rU rV r s zWORMTable.write)NNNN)rÝ r¥ r¦ r§ r~ r r rU rU rU rV r s û r c @ sf e Zd ZdZdZddddddd d
|
Zdd ddd d dZddddddddZdd d dddZdS )rã ú(support the new appendable table formatsZ
|
appendableNFTr
r )r r r7 r[ c C s² |s| j r| j | jd¡ | j||||| | d}|jD ] }| ¡ q6|j s~|j|||| d}| ¡ | |d< |jj |jf| |j
|
|j _
|
|jD ]}| ||¡ q|j ||
|
d d S )Nru )rB r r[ r r r )r r rÁ rC r7 )r )r r¿ r r¹ rô rB râ r rX Z create_tabler{ rÙ rê Ú
|
write_data)rÉ r rB r r r rÁ r r¥ rC r r r r7 ru r ÚoptionsrU rU rV r ¨ s4
|
ú
|
|
ü
|
|
zAppendableTable.writer )r¥ r r[ c sÈ | j j}| j}g }|rT| jD ]6}t|jjdd}t|tj r |
|
|j ddd¡ q t |r|d }|dd
D ] }||@ }qp| ¡ }nd}dd | jD } t | }
|
|
|
dks´t|
|
d
|
d | jD } d d | D } g } t| D ]2\} }|f| j ||
|
| j }|
|
| |¡¡ qÞ|dkr d }tjt||| j d }|| d }t|D ]x} | | t| d | | krx qÄ| j| fdd | D |dk r¦|
nd fdd | D d qJdS )z`
|
we form the data into a 2-d including indexes,values,mask write chunk-by-chunk
|
r rJ Úu1Fr2 rf Nc S s g | ]
|
}|j qS rU )rÞ ru rU rU rV rk þ s z.AppendableTable.write_data.<locals>.<listcomp>c S s g | ] }| ¡ qS rU )rØ ru rU rU rV rk s c S s, g | ]$}| t t |j¡|jd ¡¡qS r )Z transposerQ ZrollÚarangerP r- rU rU rV rk s rª r* c s g | ]}|
qS rU rU ru ©Zend_iZstart_irU rV rk s c s g | ]}|
qS rU rU r- r rU rV rk s )Úindexesr> rW )rÐ r r½ r± r> rv rF rP rQ rÏ r r: rn r7 r° rÐ r r Úreshaper r« rO Úwrite_data_chunk)rÉ r¥ r r r Úmasksr r> Úmr ÚnindexesrW Úbvaluesr rL Z new_shapeÚrowsÚchunksrU r rV r ã sL
|
|
|
|
|
üzAppendableTable.write_datarË zlist[np.ndarray]z npt.NDArray[np.bool_] | None)r r r> rW r[ c C sä |D ]}t |j¡s dS q|d jd }|t|krFt j|| jd}| jj}t|}t|D ]\} }
|
|
|
||| < q^t|D ]\} }|||| | < q||dk rÂ| ¡ j t
|
dd } | ¡ sÂ|| }t|rà| j |¡ | j ¡ dS )zê
|
Parameters
|
----------
|
rows : an empty memory space where we are putting the chunk
|
indexes : an array of the indexes
|
mask : an array of the masks
|
values : an array of the values
|
Nr r* Fr2 )rQ r r rn r rÐ r r r7 r: r
rF ru r rÿ ) rÉ r r r> rW rL r r r r rô r rU rU rV r s& z AppendableTable.write_data_chunkr^ c C sf |d kst |sf|d kr:|d kr:| j}| jj| jdd n(|d krH| j}| jj||d}| j ¡ |S | ¡ srd S | j}t | |||d}|
|
¡ }t |dd ¡ }t |} | rb| ¡ }
|
t|
|
|
|
dk j} t | sÖdg} | d | krì| | ¡ | d dkr| dd¡ | ¡ } t| D ]@} | t| | ¡}|j||jd ||jd d d | } q| j ¡ | S ) NTr; r^ Fr2 rf r r0 )rn r r¿ r r¹ ru Z remove_rowsrÿ r rÛ r r; Z sort_valuesÚdiffrl r r r r4 ÚreversedrV rO )rÉ ro r¡ r¢ r ru rÜ rW Z sorted_seriesÚlnr r± Zpgrø r rU rU rV r@ J sF
|
|
ÿ
|
zAppendableTable.delete) NFNNNNNNFNNT)F)NNN) rÝ r¥ r¦ r§ r~ r r r r@ rU rU rU rV rã ¢ s$ ò;;,rã c @ s` e Zd ZU dZdZdZdZeZde d< e
|
ddd d
|
Z e dd d d Z ddddddZdS )r r r r rJ rG rH r
rË c C s | j d jdkS )Nr rf )r° r( rÍ rU rU rV rÁ s z"AppendableFrameTable.is_transposedrß c C s |r
|
|j }|S )z these are written transposed)rz rà rU rU rV rá s z AppendableFrameTable.get_objectNr r^ c s2 |¡ ¡ sd S j|||d}t jrH j jd d i ¡ni } fddt jD }t|dkstt |d }|| d } g }
|
t jD ]P\} } | j
|
kr¬q|| \} }| d¡dkrÐt | }n
|
t | ¡}| d¡}|d k rú|j|d d
|
jr |}|}t | t| d d d }n |j}t | t| d d d }|}|jdkrlt|tjrl| d|jd f¡}t|tjrt|j||d d}n.t|t rªt|||d}ntj|g||d}|j|jk ¡ sÞt |j|jf|
|
|¡ qt|
|
dkr|
|
d }n t |
|
dd}t |||d} j |||d}|S )Nr r c s" g | ]\}}| j d kr|qS r# rÈ )rh r r¦ rÍ rU rV rk ® s z-AppendableFrameTable.read.<locals>.<listcomp>rf rÜ r8 r T©Zinplacera r` Fr¤ r¥ rJ )rÜ r£ )!r] r rÞ rn r$ r{ rÔ r rB rÐ r± r7 r8 Ú from_tuplesÚ set_namesrÁ rh rz rP rP rQ rÏ r r r5 Z _from_arraysZdtypesrÐ rF r r= rÛ r )rÉ ro r£ r¡ r¢ rr r{ ZindsÚindr Úframesr r Z
|
index_valsrÞ rH r rW Zindex_Zcols_r© rÜ rU rÍ rV r sZ
|
ÿý
|
|
|
|
"
|
zAppendableFrameTable.read)NNNN)rÝ r¥ r¦ r§ rE r~ rP r5 rH r¨ r© rÁ r? rá r rU rU rU rV r
s
|
ûr c sn e Zd ZdZdZdZdZeZe ddddZ
|
e dd d
|
d Z d fd d Z ddddd fdd Z ZS )r r r
r rJ r
rË c C s dS rý rU rÍ rU rU rV rÁ ð s z#AppendableSeriesTable.is_transposedrß c C s |S r\ rU rà rU rU rV rá ô s z AppendableSeriesTable.get_objectNc s< t |ts |jpd}| |¡}t jf ||j ¡ d|S )ú+we are going to write this as a frame tablerW ©r r )rP r5 ra Zto_framerÿ r r£ rø )rÉ r r r· ra r rU rV r ø s
|
|
|
zAppendableSeriesTable.writer r; rb c s | j }|d k rB|rBt| jts"t| jD ]}||kr(| d|¡ q(t j||||d}|rj|j| jdd |j d d
df }|j
|
dkrd |_
|
|S )Nr r' Tr rW ) rº rP r rl rÐ r rÿ r Ú set_indexr
|
ra )rÉ ro r£ r¡ r¢ rº rñ rT r rU rV r ÿ s
|
|
zAppendableSeriesTable.read)N)NNNN)rÝ r¥ r¦ r§ rE r~ rP r; rH r© rÁ r? rá r r r@ rU rU r rV r è s ûr c s( e Zd ZdZdZdZ fddZ ZS )r r r
r c s^ |j pd}| |¡\}| _t| jts*tt| j}| |¡ t||_t j
|
f d|i|S )r# rW r ) ra r» r rP rl rÐ r r7 r£ rÿ r )rÉ r r· ra ZnewobjrH r rU rV r s
|
|
|
|
z AppendableMultiSeriesTable.write)rÝ r¥ r¦ r§ rE r~ r r@ rU rU r rV r s r c @ sj e Zd ZU dZdZdZdZeZde d< e
|
ddd d
|
Z e
|
d d Z d dddZ edd Zdd ZdS )r z:a table that read/writes the generic pytables table formatr r rJ zlist[Hashable]r rY rË c C s | j S r\ )rE rÍ rU rU rV re 2 s zGenericTable.pandas_typec C s t | jdd p| jS r¾ r¿ rÍ rU rU rV rZ 6 s zGenericTable.storabler c C sL g | _ d| _g | _dd | jD | _dd | jD | _dd | jD | _dS )rv Nc S s g | ]}|j r|qS rU rÌ ru rU rU rV rk @ s z*GenericTable.get_attrs.<locals>.<listcomp>c S s g | ]}|j s|qS rU rÌ ru rU rU rV rk A s c S s g | ]
|
}|j qS rU r` ru rU rU rV rk B s )r$ r r rÍ r° r± r rÍ rU rU rV rY : s zGenericTable.get_attrsc
|
|
C s¨ | j }| d¡}|dk r dnd}tdd| j||d}|g}t|jD ]^\}}t|tsZtt ||}| |¡}|dk rzdnd}t
|
|||g|| j||d} | | ¡ qD|S )z0create the indexables from the table descriptionr Nrú r )ra r( ru r» r¶ )ra rµ rW r´ ru r» r¶ ) rÛ rû rü ru r Z_v_namesrP rY rÐ rh rC r )
|
rÉ rE rÓ r» rØ r× r rñ r r] rU rU rV rÍ D s6
|
ÿ
|
|
ù zGenericTable.indexablesc K s t dd S )Nz cannot write on an generic tabler_ ra rU rU rV r g s zGenericTable.writeN)rÝ r¥ r¦ r§ rE r~ rP r5 rH r¨ r© re rZ rY r% rÍ r rU rU rU rV r ) s
|
|
|
|
"r c s` e Zd ZdZdZeZdZe d¡Z
|
e ddddZ d fd
|
d Z dd d d fdd Z ZS )r za frame with a multi-indexr rJ z ^level_\d+$rY rË c C s dS )NZappendable_multirU rÍ rU rU rV r³ s s z*AppendableMultiFrameTable.table_type_shortNc sx |d krg }n|dkr |j ¡ }| |¡\}| _t| jts@t| jD ]}||krF| d|¡ qFt j f ||d|S )NTr r$ )
|
r£ rø r» r rP rl rÐ r rÿ r )rÉ r r r· rñ r rU rV r w s
|
|
z AppendableMultiFrameTable.writer r^ c sD t j||||d}| j¡}|j fdd|jjD ¡|_|S )Nr' c s g | ]} j |¡rd n|qS r\ )Ú
|
_re_levelsÚsearch)rh ra rÍ rU rV rk s z2AppendableMultiFrameTable.read.<locals>.<listcomp>)rÿ r r% r r r r )rÉ ro r£ r¡ r¢ r© r rÍ rV r s ÿz AppendableMultiFrameTable.read)N)NNNN)rÝ r¥ r¦ r§ r~ r5 rH rP ÚreÚcompiler& r© r³ r r r@ rU rU r rV r k s
|
ûr r5 r r7 )r r( r r[ c C s¢ | |¡}t|}|d k r"t|}|d ks4| |¡rB| |¡rB| S t| ¡ }|d k rlt| ¡ j|dd}| |¡std d g| j }|||< | jt| } | S )NF)Úsort) rû rE ÚequalsÚuniquerY ÚslicerP rZ rm )r r( r r r¦ ZslicerrU rU rV ré s
|
|
ré r z str | tzinfo)r± r[ c C s t | ¡}|S )z+for a tz-aware type, return an encoded zone)r Z get_timezone)r± ÚzonerU rU rV r ¯ s
|
r znp.ndarray | Indexr6 )rW r± r' r[ c C s d S r\ rU ©rW r± r' rU rU rV rÓ µ s rÓ rË c C s d S r\ rU r/ rU rU rV rÓ ¼ s zstr | tzinfo | Noneznp.ndarray | DatetimeIndexc C s t | tr"| jdks"| j|ks"t|dk rtt | trB| j}| j} n d}| ¡ } t|}t| |d} | d¡ |¡} n|rt
|
j | dd} | S )a
|
coerce the values to a DatetimeIndex if tz is set
|
preserve the input shape if possible
|
|
Parameters
|
----------
|
values : ndarray or Index
|
tz : str or tzinfo
|
coerce : if we do not have a passed timezone, coerce to M8[ns] ndarray
|
Nr` rn úM8[ns]r* ) rP r6 r± rÐ ra r r7 rW rp rq rQ r5 )rW r± r' ra rU rU rV rÓ Á s
|
|
)ra r rZ r r[ c
|
C s t | tst|j}t|\}}t|}t |¡}t |jt jrHt
|
|s\t |js\t |jrt | |||t|dd t|dd |dS t |trtdtj|dd} t |¡}
|
| dkræt jdd |
|
D t jd
|
}t | |dt ¡ |d S | d kr t|
|
||}|jj} t | |d t | ¡|d S | d kr:t | ||||dS t |t jrT|jtksXt|dksjt|t ¡ }t | ||||d S d S )Nr° r± )rW rb r´ r° r± r² z MultiIndex not supported here!Fr r c S s g | ] }| ¡ qS rU )Ú toordinalr- rU rU rV rk s z"_convert_index.<locals>.<listcomp>r* )r² rß )ÚintegerZfloating)rW rb r´ r² r6 ) rP rY rÐ ra r r rA r rÐ rQ r. r3 r( r rh r8 r¯ r r r5 Úint32r Z Time32ColÚ_convert_string_arrayr¸ rà rÏ r6 r ) ra r rZ r r² r= r rb r r rW r¸ rU rU rV r ç sr
|
ÿÿþý
|
|
ù
|
|
|
ÿ
|
|
û
|
ÿ
|
r )rb rZ r r[ c C sÐ |dkrt | }nº|dkr$t| }n¨|dkrxz tjdd | D td}W qÌ tk
|
rt tjdd | D td}Y qÌX nT|dkrt | ¡}n@|d kr¦t| d ||d
|
}n&|d kr¾t | d ¡}ntd | |S )Nr% r( r c S s g | ]}t |¡qS rU r+ r- rU rU rV rk 0 s z$_unconvert_index.<locals>.<listcomp>r* c S s g | ]}t |¡qS rU r. r- rU rU rV rk 2 s )r2 Úfloatr
rß r4 r6 r zunrecognized index type )r6 r< rQ r5 r6 r¨ r< )rv rb rZ r r rU rU rV r ' s,
|
|
ÿr r rí )ra r r£ c C s |j tkr|S ttj|}|j j}tj|dd} | dkr@td| dkrPtd| dksd|dksd|S t |}
|
|
|
¡ } || |
|
< tj| dd} | dkrøt | j d D ]V} | | } tj| dd} | dkr t || krÖ|| nd
|
| }td | d | d q t| || | j ¡}|j}t|tr>t| | ¡p:| d¡p:d }t|pHd |}|d k rz| |¡}|d k rz||krz|}|jd| dd}|S )NFr r z+[date] is not implemented as a table columnrd z>too many timezones in this block, create separate data columnsrß r6 r zNo.z Cannot serialize the column [z2]
|
because its data contents are not [string] but [z] object dtyperW z|Sr2 )rÐ r6 r rQ rÏ ra r r r¯ r> rw rO r rn r4 r r¸ rP rD rc rÔ r rå r: )ra r rç r r rZ r r£ r r r> rv r rÜ Zerror_column_labelró r¸ ZecirU rU rV rì @ sJ
|
|
ÿ ÿ
|
|
rì )rv rZ r r[ c C s` t | r,t| ¡ ddj ||¡j | j¡} t| ¡ }t dt
|
|¡}t j | d| d} | S )a
|
Take a string-like that is object dtype and coerce to a fixed size string type.
|
|
Parameters
|
----------
|
data : np.ndarray[object]
|
encoding : str
|
errors : str
|
Handler for encoding errors.
|
|
Returns
|
-------
|
np.ndarray[fixed-length-string]
|
Fr2 rf ÚSr* )rn r; r7 rY Úencoder; r r r' r Ú
|
libwritersÚmax_len_string_arrayrQ r5 )rv rZ r Úensuredr¸ rU rU rV r4 s ÿþÿ r4 c C s | j }tj| ¡ td} t| rzt t| ¡}d| }t | d t
|
rbt | ddj j ||dj} n| j|ddjtdd} |dkrd}t | |¡ | |¡S ) a*
|
Inverse of _convert_string_array.
|
|
Parameters
|
----------
|
data : np.ndarray[fixed-length-string]
|
nan_rep : the storage repr of NaN
|
encoding : str
|
errors : str
|
Handler for encoding errors.
|
|
Returns
|
-------
|
np.ndarray[object]
|
Decoded data.
|
r* ÚUr Fr2 )r Nræ )r rQ r5 r7 r6 rn r8 r9 r' rP rs r; rY rS r; r: Z!string_array_replace_from_nan_repr )rv r rZ r r r¸ rÐ rU rU rV r< ¨ s
|
r< )rW rÔ rZ r c C s6 t |tstt|t|r2t|||}|| } | S r\ )rP rY rÐ rÜ Ú _need_convertÚ_get_converter)rW rÔ rZ r ÚconvrU rU rV rÒ Î s
|
rÒ ©rb rZ r c s8 | dkrdd S | dkr& fddS t d| d S )Nr% c S s t j| ddS )Nr0 r* )rQ r5 ©r" rU rU rV r Ø ó z _get_converter.<locals>.<lambda>rß c s t | d dS )Nr4 )r< r@ r rU rV r Ú s
|
ÿz invalid kind )r¨ r? rU r rV r= Ö s
|
r= r c C s | dkr dS dS )N)r% rß TFrU r rU rU rV r< á s r< z Sequence[int])ra rK r[ c C sl t |tst|dk r td|d dkrh|d dkrh|d dkrht d| ¡}|rh| ¡ d }d| } | S ) zö
|
Prior to 0.10.1, we named values blocks like: values_block_0 an the
|
name values_0, adjust the given name if necessary.
|
|
Parameters
|
----------
|
name : str
|
version : Tuple[int, int, int]
|
|
Returns
|
-------
|
str
|
é z6Version is incorrect, expected sequence of 3 integers.r rf rI rJ zvalues_block_(\d+)Zvalues_)rP rY rn r¨ r( r' r± )ra rK r ÚgrprU rU rV rÐ ç s $
|
rÐ )Ú dtype_strr[ c C sÎ t | } | d¡s | d¡r"d}n¨| d¡r2d}n| d¡rBd}n| d¡sV| d¡r\d}nn| d¡rld}n^| d ¡r|d
|
}nN| d ¡rd }n>| d ¡rd }n.| d ¡r¬d}n | dkrºd}ntd| d|S )zA
|
Find the "kind" string describing the given dtype name.
|
rß rs r5 rÙ rc r r2 r% Ú timedeltar( r
rú r r6 zcannot interpret dtype of [r )rW ro r¨ )rD rb rU rU rV r s.
|
|
|
|
|
|
|
r r c C sb t | tr| j} | jj d¡d }| jjdkr@t | d¡¡} nt | t
|
rP| j } t | ¡} | |fS )zJ
|
Convert the passed data into a storable form and a dtype string.
|
rR r )r ÚMrÍ ) rP r? r rÐ ra r rb rQ r5 r r9 r )rv r rU rU rV r ! s
|
|
|
r c @ s> e Zd ZdZddddddddZd d
|
Zd d Zd d ZdS )rÛ zæ
|
Carries out a selection operation on a tables.Table object.
|
|
Parameters
|
----------
|
table : a Table object
|
where : list of Terms (or convertible to)
|
start, stop: indices to start and/or stop selection
|
|
Nró r r )ru r¡ r¢ r[ c C s@ || _ || _|| _|| _d | _d | _d | _d | _t|rt t
|
¾ t j |dd}|dkrüt |¡}|jt jkr®| j| j }}|d krd}|d kr| j j}t ||¡| | _nNt|jjt jrü| jd k rÖ|| jk ¡ sî| jd k rö|| jk ¡ röt
|
d|| _W 5 Q R X | jd kr<| |¡| _| jd k r<| j ¡ \| _| _d S )NFr )r2 Úbooleanr z3where must have index locations >= start and < stop)ru ro r¡ r¢ Ú conditionr Ztermsr. r/ r r¨ r r rQ r5 rÐ Zbool_r r Ú
|
issubclassrÜ r2 r9 ÚgenerateÚevaluate)rÉ ru ro r¡ r¢ ÚinferredrU rU rV rÊ C sD
|
|
|
ÿ ÿÿ zSelection.__init__c
|
C s |dkr dS | j ¡ }zt||| j jdW S tk
|
rz } z2d | ¡ ¡}td| d| d}t||W 5 d}~X Y nX dS )z'where can be a : dict,list,tuple,stringN)rÇ rZ r» z- The passed where expression: a*
|
contains an invalid variable reference
|
all of the variable references must be a reference to
|
an axis (e.g. 'index' or 'columns'), or a data_column
|
The currently defined references are: z
|
) ru rÇ rB rZ Ú NameErrorr¾ rõ r r¨ )rÉ ro rÎ rA Zqkeysrý rU rU rV rJ p s
|
ÿûÿ zSelection.generatec C sX | j dk r(| jjj| j ¡ | j| jdS | jdk rB| jj | j¡S | jjj| j| jdS )ú(
|
generate the selection
|
Nr^ ) rH ru Z
|
read_wherer r¡ r¢ r. r r rÍ rU rU rV r´ s
|
ÿ
|
zSelection.selectc C s | j | j }}| jj}|dkr$d}n|dk r4||7 }|dkrB|}n|dk rR||7 }| jdk rx| jjj| j ¡ ||ddS | jdk r| jS t ||¡S )rN Nr T)r¡ r¢ r* )
|
r¡ r¢ ru r rH Zget_where_listr r. rQ r )rÉ r¡ r¢ r rU rU rV r s(
|
ÿ
|
zSelection.select_coords)NNN)rÝ r¥ r¦ r§ rÊ rJ r´ r rU rU rU rV rÛ 7 s û- rÛ ) r NNFNTNNNNr{ rO ) Nr r{ NNNNFN)N)F)F)F)°r§ Ú
|
__future__r Ú
|
contextlibr rw rd r r r/ r¬ r( Útextwrapr Útypesr Útypingr r r
|
r r r r r r r rñ ÚnumpyrQ Zpandas._configr r Z pandas._libsr r r8 Zpandas._libs.tslibsr Zpandas._typingr r r r r r r Zpandas.compat._optionalr Zpandas.compat.pickle_compatr Z pandas.errorsr r! r" r# r$ Zpandas.util._decoratorsr% Zpandas.util._exceptionsr& Zpandas.core.dtypes.commonr' r( r) r* r+ r, r- r. r/ r0 r1 r2 r3 Zpandas.core.dtypes.missingr4 rì r5 r6 r7 r8 r9 r: r; r< r= r> Zpandas.core.arraysr? r@ rA Zpandas.core.commonÚcoreÚcommonr> Z pandas.core.computation.pytablesrB rC Zpandas.core.constructionrD Zpandas.core.indexes.apirE Zpandas.core.internalsrF rG Zpandas.io.commonrH Zpandas.io.formats.printingrI rJ r} rK rL rM rN rU r] rW r_ rb rg rp rq r¨ rr rs r| rQ rx ry Z config_prefixZregister_optionZis_boolZis_one_of_factoryr| r r r r¸ r² r r r rü rþ rA rC rD rc r r r ró r rã r r r r r ré r rÓ r r rì r4 r< rÒ r= r< rÐ r r rÛ rU rU rU rV Ú<module> s0 0 $ < 0
|
ü ñ,: ö Qp & - e!^ f dc0B+ ÿ ÿ ÿ&@I & !
|
