rotifer.db.ncbi.GeneNeighborhoodCursor#

class rotifer.db.ncbi.GeneNeighborhoodCursor(readers=['ftp', 'entrez'], writers=[], column='pid', before=7, after=7, min_block_distance=0, strand=None, fttype='same', eukaryotes=False, save=None, replace=False, mirror=None, exclude_type=['source', 'gene', 'mRNA'], autopid=False, codontable='Bacterial', progress=True, tries=3, batch_size=None, threads=None, cache='/tmp', *args, **kwargs)[source]#

Bases: GeneNeighborhoodCursor, BaseCursor

Fetch gene neighborhoods as dataframes.

This class searches for genomic patches centered around target coding genes, identified by the accession numbers of their protein products. Backends are tried in order: a local SQLite3 store (when save is set), local genome mirrors (when mirror is set), the NCBI FTP site and NCBI Entrez.

For the multi query methods (fetchone() and fetchall()) results are returned in random order.

Parameters:

readerslist of str, default ['ftp', 'entrez']

Backend reader modules, tried in order. The mirror and save parameters prepend their backends to this list.

writerslist of str, default []

Backend writer modules.

columnstr, default ‘pid’

Name of the column to scan for matches to the accessions. See rotifer.genome.data.NeighborhoodDF.

beforeint, default 7

Keep at most this number of features, of the same type as the target, before each target.

afterint, default 7

Keep at most this number of features, of the same type as the target, after each target.

min_block_distanceint, default 0

Minimum distance between two consecutive blocks.

strandstr, optional

How to evaluate rows concerning the value of the strand column. Supported values:

None : ignore strand
same : same strand as the targets
+ : positive strand features and targets only
- : negative strand features and targets only

fttype{‘same’, ‘any’}, default ‘same’

How to process feature types of neighbors. With same, only features of the same type as the target are considered. With any, all features count when setting neighborhood boundaries.

eukaryotesbool, default False

Whether to process eukaryotic genomes.

savestr, optional

Path to a local SQLite3 database. When set, the database is queried before any remote source. Note: currently the save backend does not write new data; it only reads previously loaded databases.

replacebool, default False

When save is set, whether to replace that file.

mirrorstr or list of str, optional

Path(s) to local mirrors of the NCBI FTP genomes directory, queried before any remote source.

exclude_typelist of str, default ['source', 'gene', 'mRNA']

Feature types to ignore.

autopidbool, default False

Automatically set protein identifiers.

codontablestr or int, default ‘Bacterial’

Codon table used when the data does not define one.

progressbool, default True

Whether to print a progress bar.

triesint, default 3

Number of attempts to download data.

batch_sizeint, optional

Number of accessions per batch.

threadsint, optional

Number of processes used for parallel downloads.

cachestr, optional

Directory for temporary files. Defaults to the rotifer cache directory.

Notes

All initialization parameters are accessible as mutable attributes and may be modified to tune the cursor’s behavior. Changes are propagated to the backend cursors. Every fetch method also updates the missing dataframe, which describes the errors of failed download attempts.

Examples

Using the dictionary-like interface, fetch the gene neighborhood around the gene encoding a target protein:

>>> from rotifer.db import ncbi
>>> gnc = ncbi.GeneNeighborhoodCursor()
>>> df = gnc["EEE9598493.1"]

Fetch all gene neighborhoods for a sample of proteins:

>>> q = ['WP_012291365.1', 'WP_013208129.1', 'WP_122330970.1']
>>> df = gnc.fetchall(q)

Process gene neighborhoods while downloading:

>>> for n in gnc.fetchone(q):
...     print(n.block_id.nunique())

__getitem__(protein, ipgs=None)[source]#

Fetch gene neighborhoods for one or more proteins, dictionary style.

Backends are tried in order until one returns data.

Parameters:

proteinstr or iterable of str: NCBI protein accessions.
ipgspandas.DataFrame, optional: Precomputed identical protein group reports, used to avoid downloading IPGs several times.

Returns:

rotifer.genome.data.NeighborhoodDF: The neighborhoods found. Empty when nothing could be retrieved.

Examples

>>> from rotifer.db import ncbi
>>> gnc = ncbi.GeneNeighborhoodCursor(progress=True)
>>> n = gnc["WP_063732599.1"]

Reuse previously downloaded IPGs:

>>> ic = ncbi.IPGCursor(batch_size=1)
>>> i = ic.fetchall(['WP_063732599.1'])
>>> n = gnc.__getitem__(['WP_063732599.1'], ipgs=i)

fetchall(proteins, ipgs=None)[source]#

Fetch all gene neighborhoods as a single dataframe.

Parameters:

proteinslist of str: NCBI protein accessions.
ipgspandas.DataFrame, optional: Precomputed identical protein group reports, used to avoid downloading IPGs several times.

Returns:

rotifer.genome.data.NeighborhoodDF: The concatenated neighborhoods. Empty when nothing could be retrieved.

Examples

>>> from rotifer.db import ncbi
>>> ic = ncbi.IPGCursor(batch_size=1)
>>> gnc = ncbi.GeneNeighborhoodCursor(progress=True)
>>> i = ic.fetchall(['WP_063732599.1'])
>>> n = gnc.fetchall(['WP_063732599.1'], ipgs=i)

fetchone(accessions, ipgs=None)[source]#

Iterate over gene neighborhoods as they are retrieved.

Backends are tried in order; each one only receives the identifiers that previous backends could not resolve.

Parameters:

accessionslist of str: NCBI protein accessions.
ipgspandas.DataFrame, optional: Precomputed identical protein group reports, used to avoid downloading IPGs several times.

Yields:

rotifer.genome.data.NeighborhoodDF: One dataframe per retrieved gene neighborhood.

Examples

>>> from rotifer.db import ncbi
>>> ic = ncbi.IPGCursor(batch_size=1)
>>> gnc = ncbi.GeneNeighborhoodCursor(progress=True)
>>> i = ic.fetchall(['WP_063732599.1'])
>>> for x in gnc.fetchone(['WP_063732599.1'], ipgs=i):
...     print(len(x))