rotifer.db.core.BaseCursor

class rotifer.db.core.BaseCursor(progress=False, *args, **kwargs)

Bases: object

Generic database cursor abstract interface.

Subclasses must implement __getitem__, fetchone(), fetchall() and getids(). The base class provides identifier normalization (parse_ids()) and a registry of missing entries (missing, missing_ids(), update_missing(), remove_missing()).

Parameters:

progressbool, default False

Whether to print a progress bar.

Attributes:

giveupset of str

Substrings of error messages that mark an entry as unrecoverable, so no further retrieval attempts are made.

maxgetitemint

Maximum number of accessions accepted per __getitem__ call.

__getitem__(accession, *args, **kwargs)

Fetch data for one entry, dictionary style.

Subclasses must override this method.

Parameters:

accessionstr

Database entry identifier.

Raises:

NotImplementedError

Always, unless overridden by a subclass.

content_id()

Identify the body of data this cursor reads.

Two cursors that return this same value are serving the same data, so consulting both answers nothing the first did not. A delegator uses that to skip a backend whose contents another one has already offered, which is worth doing when the skipped backend is the expensive one.

This is a statement about data, not about reachability or freshness: it says two sources hold the same thing, never that what they hold is complete. A backend that cannot cheaply prove which data it holds returns None, and None is never equal to anything, so it is always consulted.

Cursors that can answer should build the value from whatever identifies the data itself rather than the copy of it: two backends serving one release must agree, however differently they store it.

Returns:

hashable or None

None by default, i.e. never skipped.

fetchall(accessions, *args, **kwargs)

Fetch data for all entries at once.

Subclasses must override this method. Input order is not preserved.

Parameters:

accessionslist of str

Database entry identifiers.

Raises:

NotImplementedError

Always, unless overridden by a subclass.

fetchone(accessions, *args, **kwargs)

Iterate over entries as they are retrieved.

Subclasses must override this method. Input order is not preserved.

Parameters:

accessionslist of str

Database entry identifiers.

Yields:

object

One result object per retrieved entry.

Raises:

NotImplementedError

Always, unless overridden by a subclass.

getids(obj)

Extract accessions from the objects generated by the cursor.

Subclasses must override this method to recognize their own result objects.

Parameters:

objobject

A result object produced by the cursor.

Returns:

set of str

The accessions found in obj.

missing_ids(retry=None, final=None)

Retrieve accessions not found in the target database.

The two filters answer different questions. retry asks whether this cursor might still recover an accession by trying again, which is what its own retry loop consults. final asks whether the answer is binding on every other cursor as well, which is what a delegator consults before handing the accession to the next backend.

An accession absent from one database is normally neither: not worth retrying here, but well worth asking the next backend for.

Parameters:

retrybool, optional

When set, filter accessions based on whether they might still be recovered (retry=True) or not (retry=False). By default all missing accessions are returned.

Returns:

set of str

The missing accessions.

parse_ids(accessions, as_string=True)

Convert a collection of accessions into a set.

Cursors accept identifiers as strings, lists, tuples, sets, pandas Series or other iterables. This method converges any of those inputs into a standard representation, a Python set. Comma separated strings are split into individual accessions.

Parameters:

accessionsstr or iterable

One or more database identifiers. A string containing commas is split on the commas.

as_stringbool, default True

Convert individual accessions to strings.

Returns:

set

The normalized, deduplicated identifiers.

Examples

>>> from rotifer.db.core import BaseCursor
>>> cursor = BaseCursor()
>>> sorted(cursor.parse_ids("acc1,acc2"))
['acc1', 'acc2']
>>> sorted(cursor.parse_ids(["acc1", "acc1", "acc2"]))
['acc1', 'acc2']

remove_missing(accessions=None)

Unregister missing accessions.

Parameters:

accessionslist of str, optional

Accessions to remove from the registry. When not given, all entries are removed.

Returns:

dict or None

When all entries are removed, the previous registry content is returned. Otherwise nothing is returned.

update_missing(accessions=[], error=None, retry=None, final=False, data=None, *args, **kwargs)

Update or add entries to the registry of missing entries.

Parameters:

accessionslist, tuple or set

Database identifiers to register as missing.

errorstr, optional

A string describing the latest error. When not given, the previously recorded error is kept, or the text "Unknown error" is used for new entries.

retrybool, optional

Whether the error is recoverable. When not given, the value is inferred by matching error against the giveup patterns.

datadict, optional

A dictionary matching the internal registry layout, with accessions as keys and three element lists (error, calling class, retry flag) as values. When given, the accessions, error and retry parameters are ignored.

Returns:

bool

Whether at least one of the registered entries may still be recovered by another retrieval attempt.

property missing

Registry of entries that could not be retrieved.

Returns:

pandas.DataFrame

One row per missing accession, indexed by accession, with columns error (last error message), class (the cursor that failed) and retry (whether another attempt may succeed).

property progress_label

Name this cursor answers to on a progress bar.

A delegator draws its own bar above the backend working under it, and an unlabelled bar says nothing about which backend that is – which is the one thing worth knowing while several are tried in turn. What distinguishes them is the module they come from, so that is what the bar is named after, without the rotifer.db. prefix every one of them shares.

Returns:

str

The module name, e.g. uniprot.clickhouse.