Mirroring a PyPI repository¶
Before you start¶
You need to have already installed and configured your Repository instance. Due to the size of Repository, it is important that you have configured a file storage location with sufficient disk space. If necessary, see the requirements for the file storage location.
The full PyPI mirror requires approximately 120 GB.
Running the PyPI mirror command¶
To create a PyPI mirror:
anaconda-server-sync-pypi
This command loads all of the packages on pypi.python.org into the ~pypi binstar user account.
Verify that the command ran successfully by opening your browser
to http://your-anaconda-repo/pypi/~pypi
.
NOTE: Replace your-anaconda-repo
with the URL to your
installation of Repository.
Customizing the mirror¶
It is possible to customize the mirror behavior by creating a
configuration file such as
$PREFIX/etc/anaconda-server/mirror/pypi.yaml
and using the
--mirror-config
option:
anaconda-server-sync-pypi --mirror-config /etc/binstar/mirrors/pypi.yaml
The following configuration options are available:
Name | Description |
---|---|
user |
The local user under which the PyPI packages are
imported. Default: pypi . |
pkg_list |
A list of packages to mirror. Only packages listed
are mirrored. If this is set, blacklist and
whitelist settings are ignored. Default:
[] . |
whitelist |
A list of packages to mirror. Only packages listed
are mirrored. If the list is empty, all packages are
checked. Default: [] . |
blacklist |
A list of packages to skip. The packages listed are
ignored. Default: [] . |
latest_only |
Only download the latest versions of the packages.
Default: false . |
remote_url |
The URL of the PyPI mirror. /pypi is appended to
build the XML RPC API URL, /simple for the simple
index and /pypi/{package}/{version}/json for the
JSON API. Default: https://pypi.python.org/ . |
xml_rpc_api_url |
A custom value for XML RPC URL. If this value is
present, it takes precedence over the URL built using
remote_url . Default: null . |
simple_index_url |
A custom value for the simple index URL. If this
value is present, it takes precedence over the URL
built using remote_url . Default: null . |
use_xml_rpc |
Whether to use the XML RPC API as specified by
PEP381. If
this is set to true , the simple index is used to
determine which packages to check. The simple index
is also used if the XML RPC API fails for any reason.
Default: true . |
use_serial |
Whether to use the serial number provided by the XML
RPC API. Only packages updated since the last serial
saved are checked. If this is set to false, all PyPI
packages are checked for updates. Default: true . |
create_org |
Create the mirror user as an organization instead of
a regular user account. All superusers are added to
the “Owners” group of the organization. Default:
false . |
private |
Save the mirrored packages as private. Default:
false . |
EXAMPLE:
whitelist:
- requests
- six
- numpy
- simplejson
latest_only: true
remote_url: http://pypimirror.local/
use_xml_rpc: false
Configuring pip¶
To configure pip to use this new mirror, you must edit your
~/.pip/pip.conf
file:
[global]
index-url = http://your-anaconda-repo:<port>/pypi/~pypi/simple
NOTE: Replace your-anaconda-repo
with the URL to your
installation of Repository.