IOPro¶
IOPro loads NumPy arrays (and Pandas DataFrames) directly from files, SQL databases, and NoSQL stores–including ones with millions of rows–without creating millions of temporary, intermediate Python objects, or requiring expensive array resizing operations.
IOPro provides a drop-in replacement for the
NumPy functions loadtxt()
and genfromtxt()
, but dramatically
improves performance and reduces memory overhead.
IOPro is included with Anaconda Workgroup and Anaconda Enterprise subscriptions.
To start a 30-day free trial just download and install the IOPro package.
If you already have Anaconda (free Python platform) or Miniconda installed:
conda update conda
conda install iopro
If you do not have Anaconda installed, you can download it.
IOPro can also be installed into your own (non-Anaconda) Python environment. For more information about IOPro please contact sales@continuum.io.
Getting started¶
Some of the basic usage patterns look like these. Create TextAdapter object for data source:
>>> import iopro
>>> adapter = iopro.text_adapter('data.csv', parser='csv')
Define field dtypes (example: set field 0 to unsigned int and field 4 to float):
>>> adapter.set_field_types({0: 'u4', 4:'f4'})
Parse text and store records in NumPy array using slicing notation:
>>> # read all records
>>> array = adapter[:]
>>> # read first ten records
>>> array = adapter[0:10]
>>> # read last record
>>> array = adapter[-1]
>>> # read every other record
>>> array = adapter[::2]
User guide¶
Reference guide¶
Requirements¶
- python 2.7, or 3.4+
- numpy 1.10+
Python modules (optional):
- boto (for S3 support)
- Pandas (to use DataFrames)
What’s new in version 1.9?¶
The documentation has been substantially updated for version 1.9.0. Numba has been removed and the code has been cleaned up, but no other features were added or removed. Some refactoring was done that didn’t change functionality. We recommend that users not use older versions. See Release notes for additional detail.