Census URL Generator¶

The auraxium.census module provides a low-level Python wrapper around the PlanetSide 2 REST API. It is used internally by Auraxium to generate any URLs needed for traversal of the object model, but may also be used by advanced users to customize queries and increase performance of their apps.

The following pages cover the URL generator interface and usage, but the underlying REST endpoint’s structure, functionality and limitations are outside the scope of this document. If you are unfamiliar with the Daybreak Game Company’s Census API or would like a refresh, please refer to the Census API Primer in the repository Wiki.

For an example of how to use the URL generator and custom queries in conjunction with the Auraxium object model, refer to the Optimizing With Custom Queries page.

Note

The URL generator is independent of the rest of the Auraxium package and may be imported separately. If your use case does not benefit from the object model or event streaming client, you can still use the URL generator on its own to keep your source code free of URL literals:

from auraxium import census

my_query = census.Query('character', name__first='Higby')
my_query.create_join('character_online_status')

print(type(my_query.url()))
# <class 'yarl.URL'>
print(my_query.url())
# http://census.daybreakgames.com/get/ps2/character?name.first=Higby&c:join=characters_online_status

Overview¶

The auraxium.census URL generator works by combining one or more queries and query commands into a single URL and query string. This can then be returned as a yarl.URL or cast to str.

Important

This module only generates URLs, it does not make requests and also does not provide any exception handling facilities. After generating your URL using the Query.url() factory, you must pass it to an HTTP library like requests or aiohttp to handle the web request.

Query interface¶

Basic functionality supported by all queries is defined in the QueryBase class. This includes specifying the collection to access, any number of key-value pairs to match (referred to as terms for the rest of the documentation), as well as the show() and hide() methods, which allow the exclusion of unneeded fields from the response.

Note

All query-like objects use a fluent interface for most of their configuration methods. This means that they all return their instance, which allows for method chaining:

from auraxium import census

# Without method chaining
my_query = census.Query('character', name__first='Auroram')
my_query.case(False)
my_query.show('name.first', 'prestige_level')

# With method chaining
my_query = (census.Query('character', name__first='Auroram')
            .case(False).show('name.first', 'prestige_level'))

Use of this pattern is optional, but it can aid readability when working with complex or heavily nested queries.

This base class is mentioned here for completeness and due to its use in type hints and other parts of the documentation. However, most users will only ever need to directly interact with its subclasses, covered in the next sections.

For a full list of the basic query interface, refer to the QueryBase API reference below.

Top level queries¶

The PlanetSide 2 API defines two distinct types of queries. The first is the main query sent to the server, referred to as the top level query in this documentation. It is represented by the Query class.

To generate a URL from a query, run the Query.url() method. By default, this uses the get query verb. This is the main endpoint used to retrieve data and supports the entire query interface, and will return actual matches for a given query. Alternatively, one can specify verb='count' to instead only return the number of potential matches. This is helpful when working with filtered data sets of unknown size, so that pagination or other custom formats can be used to display the requested data.

Top level queries also support global flags (or query commands) that allow for case insensitive string comparison, sorting of results, or enable special query modes like distinct(), illustrated below:

from auraxium import census

my_query = census.Query('map_region')
my_query.distinct('facility_type')

my_query.url()
# http://census.daybreakgames.com/get/ps2/map_region?c:distinct=facility_type

This query now won’t return actual map regions, but instead up to 20’000 unique values for the given field:

{
  "count": 7,
  "map_region_list": [
    {
      "facility_type": [
        "Amp Station",
        "Bio Lab",
        "Construction Outpost",
        "Large Outpost",
        "Small Outpost",
        "Tech Plant",
        "Warpgate"
      ]
    }
  ],
  "returned": 1
}

Joined queries¶

The other type of query are joined queries, also known as joins. These are represented by the JoinedQuery class and behave much like regular queries, but are attached to an existing parent query, either a top level Query or another join.

Joins are used to return data related to another query’s return values as part of the same response. This allows effectively returning related data, such as the name of a player, their outfit, and their top played classes in a single request.

Important

Some API collections, like character and outfit_member, can have access times in excess of several seconds depending on your client’s location. When working with these large tables, it is often preferable to use few large joins rather than multiple smaller queries to reduce latency.

In particular, it is often worth considering the use of single_character_by_id, which is an indexed, high-performance table containing most data for a given character. The downside is that it contains all player statistics and items, which makes it a very bandwidth-intensive payload to transmit. It also does not support hiding fields or other query commands that could be used to reduce its size.

The relation between a join and its parent is defined by the set_fields() method. The joins data is then inserted into an extra field in the response, either named <parent_field>_join_<joined_collection> (default) or a custom name specified through the JoinedQuery.set_inject_at() method.

Joined queries can not be directly translated into a URL, the must be attached to a top level query instead. This can be done via the QueryBase.add_join() (for existing joins) or QueryBase.create_join() (for new joins) methods. The JoinedQuery.serialise() method allows conversion of a JoinedQuery into its URL-compatible, serialized format. This hook is used by the parent query when the top level query’s Query.url() method is called.

For more information on joined queries, pitfalls and examples, refer to the Joined Queries section of the aforementioned Census API primer Wiki article.

Query templates¶

The core features of a query (i.e. its collection, terms, show/hide field lists and any attached joins) are shared across all types of queries. This allows partial conversion between query types.

This conversion is done as part of the Query.copy()/JoinedQuery.copy() methods, which take in a template query and creates a new instance of their own class using any applicable values from the given template.

Terms and search modifiers¶

The key-value pairs used to filter queries (previously introduced as the query’s terms) are represented by the SearchTerm class, which provides facilities for parsing, serializing and formatting these terms for use in query strings.

Terms can be added directly using a query’s QueryBase.add_term() method, or may alternatively be generated via keyword arguments as part of the query instantiation:

from auraxium import census

# Added manually
my_query = census.Query('character')
my_query.add_term('name.first', 'Higby')

# Added via keyword arguments
my_query = census.Query('character, name__first='Higby')

Note

When passing field names as keyword arguments, double underscores will be replaced with a single dot in the search term’s field. This allows inline definition of nested keys as in the example above.

Search modifiers¶

The QueryBase.add_term() method also provides a modifier field, which allows specifying the type of match to perform via the SearchModifier enumerator. Search modifiers allow exact value matches (like SearchModifier.EQUAL_TO or SearchModifier.NOT_EQUAL), to ranges (SearchModifier.GREATER_THAN, etc.), and even basic text value comparison (SearchModifier.STARTSWITH and SearchModifier.CONTAINS).

Note that it is allowed to pass the same value multiple times, with different constraints:

from auraxium import census

# Find players between BR 100 and 120
my_query = census.Query('character')
my_query.add_term('battle_rank.value', 100, census.SearchModifier.GREATER_THAN)
my_query.add_term('battle_rank.value', 120, census.SearchModifier.LESS_THAN_OR_EQUAL)

Examples¶

This example retrieves a character by name and joins their online status.

"""Usage example for the auraxium.census module."""
from auraxium import census

query = census.Query('character', service_id='s:example')
query.add_term('name.first_lower', 'auroram')
query.limit(20)
join = query.create_join('characters_online_status')
url = str(query.url())

print(url)
# https://census.daybreakgames.com/s:example/get/ps2:v2/character?c:limit=20&c:join=characters_online_status

API reference¶

Query types¶

class auraxium.census.QueryBase(collection=None, **kwargs)¶

Base class for functionality shared between queries and joins.

If you want to re-use the same query multiple times, use this class. For more control about how the query is performed, refer to the Query and JoinedQuery subclasses.

To convert between different types of queries, use the QueryBase.copy() factory. See that method’s docstring for details.

data: QueryBaseData¶: Provides low-level access to the represented query’s internal structure. Useful for introspection and testing, modification of this data class by the user is not part of the interface.

joins: list[JoinedQuery]¶: A list of JoinedQuery instances that have been added to this query.

__init__(collection: str | None = None, **kwargs: float | int | str) → None¶

Initialise a new query.

Parameters:

collection (str | None) – The API collection to access. Set to None to display the list of available collections in the given namespace (only allowed in top level queries).
kwargs (float | int | str) – Key-value pairs to add to the query. Any search modifier literals will first be parsed by SearchTerm.infer().

add_join(query: QueryBase, **kwargs) → QueryBase¶

Add an existing JoinedQuery to this query.

This converts an existing QueryBase instance to a JoinedQuery using the QueryBase.copy() factory. The created join is then added to this query.

To create a new query and add it immediately, use the QueryBase.create_join() method instead.

Parameters:

query (QueryBase) – The query to convert and add.
kwargs – Extra keyword arguments are forwarded to the QueryBase.copy() method.

Returns:

The current query instance.

add_term(field: str, value: float | int | str, modifier: SearchModifier = SearchModifier.EQUAL_TO, *, parse_modifier: bool = False) → QueryBase¶

Add a new filter term to the query.

Filter terms are used to either reduce the number of results returned, or to specify the exact ID expected. Refer to the SearchTerm class for details and examples.

Parameters:

field (str) – The field to filter by.
value (float | int | str) – The value of the filter term.
modifier (SearchModifier) – The search modifier to use. This will only be used if parse_modifier is false. By default SearchModifier.EQUAL_TO.
parse_modifier – If true, the search modifier to use will by inferred from the value. See SearchTerm.infer() for details.

Returns:

The current query instance.

classmethod copy(template: QueryBase, copy_joins: bool = False, deep_copy: bool = False, kwargs) → QueryBase¶

Create a new query, copying most data from the template.

The new query will share the collection, terms and show/hide markers of the template. If copy_joins is enabled, it will also copy its list of joins.

Among other things, allows easy creation of joins from existing queries, which is handy if you have complex existing joins or hidden fields that would be tedious to recreate.

By default, this creates a shallow copy. Modifying the terms or joined queries will cause mutations of the template. Set the deep_copy flag to ensure complete independence.

Any keyword arguments are passed to the new query’s initialiser.

Example

# This is an existing query that does what we need it to,
# assume it has some complex join or hidden field setup that
# would make it tedious to re-create.
old = Query('character')

# This is an unrelated, new query. We want its join to
# return the exact same data structure as the previous
# query.
new = Query('outfit_member', outfit_id=...).limit(1000)

# Create a join emulating the original query and add it
join = JoinedQuery.copy(old, copy_joins=True)
new.add_join(join)

Parameters:

template (QueryBase) – The query to copy.
copy_joins (bool) – If true, any joins defined for the template will be copied to the new instance.
deep_copy (bool) – Whether to perform a deep copy. Use this if you intend to modify the list of terms or other mutable attributes to avoid modifying the template.
kwargs – Any keyword arguments are passed to the new query’s constructor.

Raises:

TypeError – Raised when attempting to copy into a JoinedQuery without specifying a collection.

Returns:

An instance of the current class, populated with the template query’s data.

create_join(collection: str, *args, **kwargs) → JoinedQuery¶

Create a new joined query and add it to the current one.

See the initialiser for JoinedQuery for parameter, this method forwards any parameters given to it.

Parameters:

collection (str) – The collection to join.
args – Any positional arguments are passed on to JoinedQuery.__init__().
kwargs – Any keyword arguments are passed on to JoinedQuery.__init__().

Returns:

The created JoinedQuery instance.

hide(field: str, *args: str) → QueryBase¶

Set the fields to hide in the response.

The given fields will not be included in the result. Note that this can break joins if the field they are attached to is not included.

This is mutually exclusive with QueryBase.show(), setting one will undo any changes made by the other.

Parameters:

field (str) – A field name to exclude from the results.
args (str) – Additional fields to exclude.

Returns:

The current query instance.

show(field: str, *args: str) → QueryBase¶

Set the fields to show in the response.

Any other fields will not be included in the result. Note that this can break joins if the field they are attached to is not included.

This is mutually exclusive with QueryBase.hide(), setting one will undo any changes made by the other.

Parameters:

field (str) – A field name to include in the results.
args (str) – Additional fields to include.

Returns:

The current query instance.

class auraxium.census.Query(collection=None, namespace='ps2:v2', service_id='s:example', **kwargs)¶

The main query supplied to the API.

The top-level query has access to additional return value formats such as sorting or tree views, and also supports additional, global flags that will propagate through to any inner, joined queries.

This subclasses QueryBase.

data: QueryData¶: Provides low-level access to the represented query’s internal structure. Useful for introspection and testing, modification of this data class by the user is not part of the interface.

__init__(collection: str | None = None, namespace: str = 'ps2:v2', service_id: str = 's:example', **kwargs: float | int | str) → None¶

Create a new top-level query.

The collection argument, as well as any keyword arguments, are passed on to the constructor of the QueryBase class.

Parameters:

collection (str | None) – The API collection to access. Set to None to display the list of available collections in the given namespace.
namespace (str) – The game namespace to access.
service_id (str) – The service ID identifying this app.
endpoint (yarl.URL or None) – The endpoint to use for the API. Allows for targeting community endpoints.

__str__() → str¶: Generate and return the URL defined by this query.

See also

Query.url() – the main URL generator accessed by this method.

case(value: bool = True) → Query¶

Globally ignore case for this query.

Note that case-insensitive look-ups are significantly slower. Where available, use a case-sensitive query targeting a lowercase field like ps2:v2/character.name.first_lower.

Parameters:: value (bool) – Whether to ignore case for this query.
Returns:: The current query instance.

classmethod copy(template: QueryBase, copy_joins: bool = False, deep_copy: bool = False, **kwargs) → Query¶

Create a new query, copying most data from the template.

The new query will share the collection, terms and show/hide markers of the template. If copy_joins is enabled, it will also copy its list of joins.

Among other things, allows easy creation of joins from existing queries, which is handy if you have complex existing joins or hidden fields that would be tedious to recreate.

By default, this creates a shallow copy. Modifying the terms or joined queries will cause mutations of the template. Set the deep_copy flag to ensure complete independence.

Any keyword arguments are passed to the new query’s initialiser.

Example

# This is an existing query that does what we need it to,
# assume it has some complex join or hidden field setup that
# would make it tedious to re-create.
old = Query('character')

# This is an unrelated, new query. We want its join to
# return the exact same data structure as the previous
# query.
new = Query('outfit_member', outfit_id=...).limit(1000)

# Create a join emulating the original query and add it
join = JoinedQuery.copy(old, copy_joins=True)
new.add_join(join)

Parameters:

template (QueryBase) – The query to copy.
copy_joins (bool) – If true, any joins defined for the template will be copied to the new instance.
deep_copy (bool) – Whether to perform a deep copy. Use this if you intend to modify the list of terms or other mutable attributes to avoid modifying the template.
kwargs – Any keyword arguments are passed to the new query’s constructor.

Returns:

An instance of the current class, populated with the template query’s data.

has(field: str, *args: str) → Query¶

Hide results with a NULL value at the given field.

This is useful for filtering large data sets by optional fields, such as searching the ps2/weapons collection for heat-based weapons using the heat-mechanic-specific fields.

Parameters:

field (str) – A field required to be non-NULL for a result to be included.
args (str) – Additional fields to require for results.

Returns:

The current query instance.

distinct(field: str | None) → Query¶

Query command used to show all unique values for a field.

Parameters:: distinct (str | None) – The field to show unique values for. Set to None to disable.
Returns:: The current query instance.

exact_match_first(value: bool = True) → Query¶

Whether to display exact matches before partial matches.

When performing RegEx searches (i.e. ones using either SearchModifier.STARTS_WITH or SearchModifier.CONTAINS), this setting will always promote an exact match to be the first item returned, regardless of any sorting settings applied.

Parameters:: value (bool) – Whether to promote exact matches.
Returns:: The current query instance.

include_null(value: bool) → Query¶

Whether to include NULL values in the response.

This is useful for API introspection, but it is generally more bandwidth-friendly to use the dict.get() method with a default value when parsing the result dictionary.

This only affects the top-level query itself; joined queries will only show non-NULL values.

Parameters:: value (bool) – Whether to include NULL values in the result.
Returns:: The current query instance.

lang(lang: str | None = None) → Query¶

Set the locale to user for the query.

By default, queries return all locales for localised strings. Use this flag to only include the given locale, or reset to None to include all localisations.

The following locales are currently supported and maintained::

German: 'de', English: 'en', Spanish: 'es',
French: 'fr', Italian: 'it'

Parameters:: lang (str | None) – The locale identifier to filter by.
Returns:: The current query instance.

limit(limit: int) → Query¶

Specify the number of results returned.

By default, the API will only return the first match for any given query, you can increase the number of results using this method.

The maximum number of values permissible varies from collection to collection, e.g. 100k for ps2/character, but only 5’000 for ps2/item. Use your best judgement.

This is mutually exclusive with Query.limit_per_db(), setting one will unset the other.

Parameters:: limit (int) – The number of results to return. Must be greater than zero.
Raises:: ValueError – Raised if limit is less than 1.
Returns:: The current query instance.

limit_per_db(limit_per_db: int) → Query¶

Specify the number of results returned per database.

This method works similarly to Query.limit(), but will yield better results for distributed collections such as ps2/character, which is spread across 20 different databases more or less randomly.

This is mutually exclusive with Query.limit(), setting one will unset the other.

Parameters:: limit_per_db (int) – The number of results to return per database. Must be greater than zero.
Raises:: ValueError – Raised if limit_per_db is less than 1.
Returns:: The current query instance.

offset(offset: int) → Query¶

Alias for the Query.start() method.

Refer to its docstring for details.

Parameters:: offset (int) – The number of results to skip. Must not be negative.
Raises:: ValueError – Raised if offset is negative.
Returns:: The current query instance.

resolve(name: str, *args: str) → Query¶

Resolve additional data for a collection.

Resolves are a lighter version of joined queries and can be used to quickly include associated information with the results.

Perform a query with no collection specified to see a list of resolvable names for each collection.

Parameters:

name (str) – A resolvable name to attach to the query.
args (str) – Additional resolvable names to attach.

Returns:

The current query instance.

retry(retry: bool = False) → Query¶

Enable automatic query retry.

By default, failed queries will be retried automatically. Set to false to disable this behaviour if you want to fail early or have better control of how failed queries are handled.

Parameters:: retry (bool) – Whether to automatically retry queries.
Returns:: The current query instance.

start(start: int) → Query¶

Skip the given number of results in the response.

Together with Query.limit(), this can be used to create a paginated view of API data.

Parameters:: start (int) – The number of results to skip. May not be negative.
Raises:: ValueError – Raised if start is negative.
Returns:: The current query instance.

sort(field: str | tuple[str, bool], *args: str | tuple[str, bool]) → Query¶

Sort the results by one or more fields.

By default, this uses ascending sort order. For descending order, pass a tuple with a negative second element:

QueryBase.sort('field1', ('field2', True))  # Ascending
QueryBase.sort(('field3', False))  # Descending

If multiple field names are provided, multiple sorting passes will be performed in order to further refine the list returned.

Parameters:

field (str | tuple[str, bool]) – A field to sort by.
args (str | tuple[str, bool]) – Additional fields to sort by.

Returns:

The current query instance.

timing(value: bool = True) → Query¶

Enabling query profiling output.

Setting this flag will include an additional timing key in the response, providing timing information for the main query and any joins.

Parameters:: value (bool) – Whether to enable query profiling.
Returns:: The current query instance.

tree(field: str, is_list: bool = False, prefix: str = '', start: str | None = None) → Query¶

Reformat a result list into a data tree.

This is useful for lists of data with obvious categorisation, such as grouping weapons by their type.

Parameters:

field (str) – The field to remove and use as the root of the data structure.
list (bool) – Whether the tree’s data is a list.
prefix (str) – A prefix to add to the field value to improve readability.
start (str | None) – Used to tell the tree where to start. If equal to None, the root result list will be reformatted.

Returns:

The current query instance.

url(verb: str = 'get', skip_checks: bool = False) → yarl.URL¶

Generate the URL representing this query.

This will also recursively process any joined queries.

Parameters:

verb (str) – The query verb to use. Known options are get, used to return a list of results, and count, which instead returns the number of potential matches.
skip_checks (bool) – By default, the URL generator will perform a number of checks to validate the query. Enabling this flag will skip these checks.

Returns:

A yarl.URL instance representing the query and all of its joins.

class auraxium.census.JoinedQuery(collection, **kwargs)¶

A sub-query to be joined to an existing query.

Joined queries (or “joins”) allow performing multiple, related look-ups in the same request.

This subclasses QueryBase.

Search modifiers & filters¶

class auraxium.census.SearchModifier¶

Enumerates the search modifiers available for Term objects.

Note that a given search modifier may not be valid for all fields.

The following is a list of all search modifier literals and their corresponding enum value::

EQUAL_TO: '',               LESS_THAN: '<',
LESS_THAN_OR_EQUAL: '[',    GREATER_THAN: '>',
GREATER_THAN_OR_EQUAL: ']', STARTS_WITH: '^',
CONTAINS: '*',              NOT_EQUAL: '!'

classmethod from_value(value: float | int | str) → SearchModifier¶

Infer the search modifier from a given value.

If the input is a string, its first character will be compared against the corresponding API literals. If a match is found, the corresponding SearchModifier enum value is returned.

If the input is not a string or its first character does not match any API literal, this will return SearchModifier.EQUAL_TO.

Parameters:: value (float | int | str) – A value to infer the search modifier from.
Raises:: ValueError – Raised if value is an empty string.
Returns:: The search modifier for the value provided.

static serialise(enum_value: int | SearchModifier) → str¶

Return the string literal for the given enum value.

This is primarily used during URL generation.

Parameters:: enum_value (int | SearchModifier) – The enum value or index to serialise.
Raises:: ValueError – Raised if the provided integer exceeds the value range of the enum.
Returns:: The string representation of the search modifier. This will be an empty string for SearchModifier.EQUAL_TO.

class property EQ¶: Alias for EQUAL_TO.

New in version 0.2.

class property GT¶: Alias for GREATER_THAN.

New in version 0.2.

class property GTE¶: Alias for GREATER_THAN_OR_EQUAL.

New in version 0.2.

class property IN¶: Alias for CONTAINS.

New in version 0.2.

class property LT¶: Alias for LESS_THAN.

New in version 0.2.

class property LTE¶: Alias for LESS_THAN_OR_EQUAL.

New in version 0.2.

class property NE¶: Alias for NOT_EQUAL.

New in version 0.2.

class property SW¶: Alias for STARTS_WITH.

New in version 0.2.

class auraxium.census.SearchTerm(field, value, modifier=SearchModifier.EQUAL_TO)¶

A query filter term.

Search terms are key-value pairs with an optional search modifier determining how these values will be compared. See the SearchModifier enum for details on the available search modifiers.

field: str¶: The field to filter by.

value: float | int | str¶: The value to compare the field against.

modifier: SearchModifier¶: The SearchModifier to use.

__init__(field: str, value: float | int | str, modifier: SearchModifier = SearchModifier.EQUAL_TO) → None¶

Initialise a new search term.

Search terms are used to filter the results before returning. This is particularly important for lists returned by JoinedQuery instnaces as they do not have access to limiting mechanisms like Query, easily resulting in excessively long return lists.

Use the SearchTerm.infer() factory if you prefer defining search modifiers via their string literals as used by the API, rather than manually specifying the enum value.

Parameters:

field (str) – The field to compare.
value (float | int | str) – The value to compare the field against.
modifier (SearchModifier) – The search modifier to use. Modifiers can be used to get non-exact or partial matches.

as_tuple() → tuple[str, str]¶

Return a key/value pair representing the search term.

This is a helper function that calls SearchTerm.serialise() and then splits the returned string at the equal sign.

Returns:: A key/value pair representing teh search term.

classmethod infer(field: str, value: float | int | str) → SearchTerm¶

Infer a term from the given field and value.

This is a more natural way of defining search terms for users familiar with the API literals. See the docstring of the SearchModifier enumerator for a list of search modifiers.

Note that this requires the value to be a str instance; this can obscure the actual field value (this is generally not of concern as this information will be lost during URL generation regardless).

Parameters:

field (str) – The field to compare.
value (float | int | str) – The value to compare the field against. If value is a subclass of str, its first character will be checked for search modifier literals.

Returns:

A new SearchTerm with a pre-defined SearchModifier.

serialise() → str¶

Return the literal representation of the search term.

This is the string that will added to the URL’s query string as part of the URL generator.

Returns:: The string representation of the search term.

Data classes¶

The following classes are mostly for internal use, but are provided for type inferrence and introspection.

class auraxium.census.QueryBaseData(**kwargs)¶

A dataclass used to store generic query information.

Refer to the corresponding setter methods for details.

classmethod from_base(data: QueryBaseData) → QueryBaseData¶: Helper used to copy the base query data components.

class auraxium.census.QueryData(**kwargs)¶

A dataclass used to store global flags and settings for queries.

Refer to the corresponding setter methods for details.

class auraxium.census.JoinedQueryData(**kwargs)¶

Data class for joined queries in the API.

Refer to the corresponding setter methods for details.