Airbyte (dagster-airbyte)

This library provides a Dagster integration with Airbyte.

For more information on getting started, see the Airbyte integration guide.

Ops

dagster_airbyte.airbyte_sync_op = <dagster._core.definitions.op_definition.OpDefinition object>[source]

Config Schema:
connection_id (String):

The Airbyte Connection ID that this op will sync. You can retrieve this value from the “Connections” tab of a given connector in the Airbyte UI.

poll_interval (Float, optional):

The time (in seconds) that will be waited between successive polls.

Default Value: 10

poll_timeout (Union[Float, None], optional):

The maximum time that will waited before this operation is timed out. By default, this will never time out.

Default Value: None

yield_materializations (Bool, optional):

If True, materializations corresponding to the results of the Airbyte sync will be yielded when the op executes.

Default Value: True

asset_key_prefix (List[String], optional):

If provided and yield_materializations is True, these components will be used to prefix the generated asset keys.

Default Value: [‘airbyte’]

Executes a Airbyte job sync for a given connection_id, and polls until that sync completes, raising an error if it is unsuccessful. It outputs a AirbyteOutput which contains the job details for a given connection_id.

It requires the use of the airbyte_resource, which allows it to communicate with the Airbyte API.

Examples

from dagster import job
from dagster_airbyte import airbyte_resource, airbyte_sync_op

my_airbyte_resource = airbyte_resource.configured(
    {
        "host": {"env": "AIRBYTE_HOST"},
        "port": {"env": "AIRBYTE_PORT"},
    }
)

sync_foobar = airbyte_sync_op.configured({"connection_id": "foobar"}, name="sync_foobar")

@job(resource_defs={"airbyte": my_airbyte_resource})
def my_simple_airbyte_job():
    sync_foobar()

@job(resource_defs={"airbyte": my_airbyte_resource})
def my_composed_airbyte_job():
    final_foobar_state = sync_foobar(start_after=some_op())
    other_op(final_foobar_state)

Resources

dagster_airbyte.airbyte_resource ResourceDefinition[source]

Config Schema:
host (dagster.StringSource):

The Airbyte Server Address.

port (dagster.StringSource):

Port for the Airbyte Server.

username (dagster.StringSource, optional):

Username if using basic auth.

password (dagster.StringSource, optional):

Password if using basic auth.

use_https (Bool, optional):

Use https to connect in Airbyte Server.

Default Value: False

request_max_retries (Int, optional):

The maximum number of times requests to the Airbyte API should be retried before failing.

Default Value: 3

request_retry_delay (Float, optional):

Time (in seconds) to wait between each request retry.

Default Value: 0.25

request_timeout (Int, optional):

Time (in seconds) after which the requests to Airbyte are declared timed out.

Default Value: 15

request_additional_params (permissive dict, optional):

Any additional kwargs to pass to the requests library when making requests to Airbyte.

Default Value:
{}
forward_logs (Bool, optional):

Whether to forward Airbyte logs to the compute log, can be expensive for long-running syncs.

Default Value: True

cancel_sync_on_run_termination (Bool, optional):

Whether to cancel a sync in Airbyte if the Dagster runner is terminated. This may be useful to disable if using Airbyte sources that cannot be cancelled and resumed easily, or if your Dagster deployment may experience runner interruptions that do not impact your Airbyte deployment.

Default Value: True

This resource allows users to programatically interface with the Airbyte REST API to launch syncs and monitor their progress. This currently implements only a subset of the functionality exposed by the API.

For a complete set of documentation on the Airbyte REST API, including expected response JSON schema, see the Airbyte API Docs.

To configure this resource, we recommend using the configured method.

Examples:

from dagster import job
from dagster_airbyte import airbyte_resource

my_airbyte_resource = airbyte_resource.configured(
    {
        "host": {"env": "AIRBYTE_HOST"},
        "port": {"env": "AIRBYTE_PORT"},
        # If using basic auth
        "username": {"env": "AIRBYTE_USERNAME"},
        "password": {"env": "AIRBYTE_PASSWORD"},
    }
)

@job(resource_defs={"airbyte":my_airbyte_resource})
def my_airbyte_job():
    ...
class dagster_airbyte.AirbyteResource(host, port, use_https, request_max_retries=3, request_retry_delay=0.25, request_timeout=15, request_additional_params=None, log=<Logger dagster.builtin (DEBUG)>, forward_logs=True, cancel_sync_on_run_termination=True, username=None, password=None)[source]

This class exposes methods on top of the Airbyte REST API.

Assets

dagster_airbyte.load_assets_from_airbyte_instance(airbyte, workspace_id=None, key_prefix=None, create_assets_for_normalization_tables=True, connection_to_group_fn=<function _clean_name>, io_manager_key=None, connection_to_io_manager_key_fn=None, connection_filter=None, connection_to_asset_key_fn=None, connection_to_freshness_policy_fn=None)[source]

Loads Airbyte connection assets from a configured AirbyteResource instance. This fetches information about defined connections at initialization time, and will error on workspace load if the Airbyte instance is not reachable.

Parameters:
  • airbyte (ResourceDefinition) – An AirbyteResource configured with the appropriate connection details.

  • workspace_id (Optional[str]) – The ID of the Airbyte workspace to load connections from. Only required if multiple workspaces exist in your instance.

  • key_prefix (Optional[CoercibleToAssetKeyPrefix]) – A prefix for the asset keys created.

  • create_assets_for_normalization_tables (bool) – If True, assets will be created for tables created by Airbyte’s normalization feature. If False, only the destination tables will be created. Defaults to True.

  • connection_to_group_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an asset group name for a given Airbyte connection name. If None, no groups will be created. Defaults to a basic sanitization function.

  • io_manager_key (Optional[str]) – The I/O manager key to use for all assets. Defaults to “io_manager”. Use this if all assets should be loaded from the same source, otherwise use connection_to_io_manager_key_fn.

  • connection_to_io_manager_key_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an I/O manager key for a given Airbyte connection name. When other ops are downstream of the loaded assets, the IOManager specified determines how the inputs to those ops are loaded. Defaults to “io_manager”.

  • connection_filter (Optional[Callable[[AirbyteConnectionMetadata], bool]]) – Optional function which takes in connection metadata and returns False if the connection should be excluded from the output assets.

  • connection_to_asset_key_fn (Optional[Callable[[AirbyteConnectionMetadata, str], AssetKey]]) – Optional function which takes in connection metadata and table name and returns an asset key for the table. If None, the default asset key is based on the table name. Any asset key prefix will be applied to the output of this function.

  • connection_to_freshness_policy_fn (Optional[Callable[[AirbyteConnectionMetadata], Optional[FreshnessPolicy]]]) – Optional function which takes in connection metadata and returns a freshness policy for the connection’s assets. If None, no freshness policies will be applied to the assets.

Examples:

Loading all Airbyte connections as assets:

from dagster_airbyte import airbyte_resource, load_assets_from_airbyte_instance

airbyte_instance = airbyte_resource.configured(
    {
        "host": "localhost",
        "port": "8000",
    }
)
airbyte_assets = load_assets_from_airbyte_instance(airbyte_instance)

Filtering the set of loaded connections:

from dagster_airbyte import airbyte_resource, load_assets_from_airbyte_instance

airbyte_instance = airbyte_resource.configured(
    {
        "host": "localhost",
        "port": "8000",
    }
)
airbyte_assets = load_assets_from_airbyte_instance(
    airbyte_instance,
    connection_filter=lambda meta: "snowflake" in meta.name,
)
dagster_airbyte.load_assets_from_airbyte_project(project_dir, workspace_id=None, key_prefix=None, create_assets_for_normalization_tables=True, connection_to_group_fn=<function _clean_name>, io_manager_key=None, connection_to_io_manager_key_fn=None, connection_filter=None, connection_directories=None, connection_to_asset_key_fn=None, connection_to_freshness_policy_fn=None)[source]

Loads an Airbyte project into a set of Dagster assets.

Point to the root folder of an Airbyte project synced using the Octavia CLI. For more information, see https://github.com/airbytehq/airbyte/tree/master/octavia-cli#octavia-import-all.

Parameters:
  • project_dir (str) – The path to the root of your Airbyte project, containing sources, destinations, and connections folders.

  • workspace_id (Optional[str]) – The ID of the Airbyte workspace to load connections from. Only required if multiple workspace state YAMLfiles exist in the project.

  • key_prefix (Optional[CoercibleToAssetKeyPrefix]) – A prefix for the asset keys created.

  • create_assets_for_normalization_tables (bool) – If True, assets will be created for tables created by Airbyte’s normalization feature. If False, only the destination tables will be created. Defaults to True.

  • connection_to_group_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an asset group name for a given Airbyte connection name. If None, no groups will be created. Defaults to a basic sanitization function.

  • io_manager_key (Optional[str]) – The I/O manager key to use for all assets. Defaults to “io_manager”. Use this if all assets should be loaded from the same source, otherwise use connection_to_io_manager_key_fn.

  • connection_to_io_manager_key_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an I/O manager key for a given Airbyte connection name. When other ops are downstream of the loaded assets, the IOManager specified determines how the inputs to those ops are loaded. Defaults to “io_manager”.

  • connection_filter (Optional[Callable[[AirbyteConnectionMetadata], bool]]) – Optional function which takes in connection metadata and returns False if the connection should be excluded from the output assets.

  • connection_directories (Optional[List[str]]) – Optional list of connection directories to load assets from. If omitted, all connections in the Airbyte project are loaded. May be faster than connection_filter if the project has many connections or if the connection yaml files are large.

  • connection_to_asset_key_fn (Optional[Callable[[AirbyteConnectionMetadata, str], AssetKey]]) – Optional function which takes in connection metadata and table name and returns an asset key for the table. If None, the default asset key is based on the table name. Any asset key prefix will be applied to the output of this function.

  • connection_to_freshness_policy_fn (Optional[Callable[[AirbyteConnectionMetadata], Optional[FreshnessPolicy]]]) – Optional function which takes in connection metadata and returns a freshness policy for the connection’s assets. If None, no freshness policies will be applied to the assets.

Examples:

Loading all Airbyte connections as assets:

from dagster_airbyte import load_assets_from_airbyte_project

airbyte_assets = load_assets_from_airbyte_project(
    project_dir="path/to/airbyte/project",
)

Filtering the set of loaded connections:

from dagster_airbyte import load_assets_from_airbyte_project

airbyte_assets = load_assets_from_airbyte_project(
    project_dir="path/to/airbyte/project",
    connection_filter=lambda meta: "snowflake" in meta.name,
)
dagster_airbyte.build_airbyte_assets(connection_id, destination_tables, asset_key_prefix=None, normalization_tables=None, upstream_assets=None, schema_by_table_name=None, freshness_policy=None)[source]

Builds a set of assets representing the tables created by an Airbyte sync operation.

Parameters:
  • connection_id (str) – The Airbyte Connection ID that this op will sync. You can retrieve this value from the “Connections” tab of a given connector in the Airbyte UI.

  • destination_tables (List[str]) – The names of the tables that you want to be represented in the Dagster asset graph for this sync. This will generally map to the name of the stream in Airbyte, unless a stream prefix has been specified in Airbyte.

  • normalization_tables (Optional[Mapping[str, List[str]]]) – If you are using Airbyte’s normalization feature, you may specify a mapping of destination table to a list of derived tables that will be created by the normalization process.

  • asset_key_prefix (Optional[List[str]]) – A prefix for the asset keys inside this asset. If left blank, assets will have a key of AssetKey([table_name]).

  • upstream_assets (Optional[Set[AssetKey]]) – A list of assets to add as sources.

  • freshness_policy (Optional[FreshnessPolicy]) – A freshness policy to apply to the assets

Managed Config

The following APIs are used as part of the experimental ingestion-as-code functionality. For more information, see the Airbyte ingestion as code guide.

class dagster_airbyte.AirbyteManagedElementReconciler(*args, **kwargs)[source]

Reconciles Python-specified Airbyte connections with an Airbyte instance.

Passing the module containing an AirbyteManagedElementReconciler to the dagster-airbyte CLI will allow you to check the state of your Python-code-specified Airbyte connections against an Airbyte instance, and reconcile them if necessary.

This functionality is experimental and subject to change.

dagster_airbyte.load_assets_from_connections(airbyte, connections, key_prefix=None, create_assets_for_normalization_tables=True, connection_to_group_fn=<function _clean_name>, io_manager_key=None, connection_to_io_manager_key_fn=None, connection_to_asset_key_fn=None, connection_to_freshness_policy_fn=None)[source]

Loads Airbyte connection assets from a configured AirbyteResource instance, checking against a list of AirbyteConnection objects. This method will raise an error on repo load if the passed AirbyteConnection objects are not in sync with the Airbyte instance.

Parameters:
  • airbyte (ResourceDefinition) – An AirbyteResource configured with the appropriate connection details.

  • connections (Iterable[AirbyteConnection]) – A list of AirbyteConnection objects to build assets for.

  • key_prefix (Optional[CoercibleToAssetKeyPrefix]) – A prefix for the asset keys created.

  • create_assets_for_normalization_tables (bool) – If True, assets will be created for tables created by Airbyte’s normalization feature. If False, only the destination tables will be created. Defaults to True.

  • connection_to_group_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an asset group name for a given Airbyte connection name. If None, no groups will be created. Defaults to a basic sanitization function.

  • io_manager_key (Optional[str]) – The IO manager key to use for all assets. Defaults to “io_manager”. Use this if all assets should be loaded from the same source, otherwise use connection_to_io_manager_key_fn.

  • connection_to_io_manager_key_fn (Optional[Callable[[str], Optional[str]]]) – Function which returns an IO manager key for a given Airbyte connection name. When other ops are downstream of the loaded assets, the IOManager specified determines how the inputs to those ops are loaded. Defaults to “io_manager”.

  • connection_to_asset_key_fn (Optional[Callable[[AirbyteConnectionMetadata, str], AssetKey]]) – Optional function which takes in connection metadata and table name and returns an asset key for the table. If None, the default asset key is based on the table name. Any asset key prefix will be applied to the output of this function.

  • connection_to_freshness_policy_fn (Optional[Callable[[AirbyteConnectionMetadata], Optional[FreshnessPolicy]]]) – Optional function which takes in connection metadata and returns a freshness policy for the connection. If None, no freshness policy will be applied.

Examples:

from dagster_airbyte import (
    AirbyteConnection,
    airbyte_resource,
    load_assets_from_connections,
)

airbyte_instance = airbyte_resource.configured(
    {
        "host": "localhost",
        "port": "8000",
    }
)
airbyte_connections = [
    AirbyteConnection(...),
    AirbyteConnection(...)
]
airbyte_assets = load_assets_from_connections(airbyte_instance, airbyte_connections)
class dagster_airbyte.AirbyteConnection(name, source, destination, stream_config, normalize_data=None, destination_namespace=AirbyteDestinationNamespace.SAME_AS_SOURCE, prefix=None)[source]

A user-defined Airbyte connection, pairing an Airbyte source and destination and configuring which streams to sync.

__init__(name, source, destination, stream_config, normalize_data=None, destination_namespace=AirbyteDestinationNamespace.SAME_AS_SOURCE, prefix=None)[source]
Parameters:
  • name (str) – The display name of the connection.

  • source (AirbyteSource) – The source to sync from.

  • destination (AirbyteDestination) – The destination to sync to.

  • stream_config (Mapping[str, AirbyteSyncMode]) – A mapping from stream name to the sync mode for that stream, including any additional configuration of primary key or cursor field.

  • normalize_data (Optional[bool]) – Whether to normalize the data in the destination.

  • destination_namespace (Optional[Union[AirbyteDestinationNamespace, str]]) – The namespace to sync to in the destination. If set to AirbyteDestinationNamespace.SAME_AS_SOURCE, the namespace will be the same as the source namespace. If set to AirbyteDestinationNamespace.DESTINATION_DEFAULT, the namespace will be the default namespace for the destination. If set to a string, the namespace will be that string.

  • prefix (Optional[str]) – A prefix to add to the table names in the destination.

Example

from dagster_airbyte.managed.generated.sources import FileSource
from dagster_airbyte.managed.generated.destinations import LocalJsonDestination
from dagster_airbyte import AirbyteConnection, AirbyteSyncMode

cereals_csv_source = FileSource(...)
local_json_destination = LocalJsonDestination(...)

cereals_connection = AirbyteConnection(
    name="download-cereals",
    source=cereals_csv_source,
    destination=local_json_destination,
    stream_config={"cereals": AirbyteSyncMode.full_refresh_overwrite()},
)
class dagster_airbyte.AirbyteSource(name, source_type, source_configuration)[source]

Represents a user-defined Airbyte source.

__init__(name, source_type, source_configuration)[source]
Parameters:
  • name (str) – The display name of the source.

  • source_type (str) – The type of the source, from Airbyte’s list of sources https://airbytehq.github.io/category/sources/.

  • source_configuration (Mapping[str, Any]) – The configuration for the source, as defined by Airbyte’s API.

class dagster_airbyte.AirbyteDestination(name, destination_type, destination_configuration)[source]

Represents a user-defined Airbyte destination.

__init__(name, destination_type, destination_configuration)[source]
Parameters:
  • name (str) – The display name of the destination.

  • destination_type (str) – The type of the destination, from Airbyte’s list of destinations https://airbytehq.github.io/category/destinations/.

  • destination_configuration (Mapping[str, Any]) – The configuration for the destination, as defined by Airbyte’s API.

class dagster_airbyte.AirbyteSyncMode(json_repr)[source]

Represents the sync mode for a given Airbyte stream, which governs how Airbyte reads from a source and writes to a destination.

For more information, see https://docs.airbyte.com/understanding-airbyte/connections/.

classmethod full_refresh_append()[source]

Syncs the entire data stream from the source, appending rows to the destination.

https://docs.airbyte.com/understanding-airbyte/connections/full-refresh-append/

classmethod full_refresh_overwrite()[source]

Syncs the entire data stream from the source, replaces data in the destination by overwriting it.

https://docs.airbyte.com/understanding-airbyte/connections/full-refresh-overwrite

classmethod incremental_append(cursor_field=None)[source]

Syncs only new records from the source, appending rows to the destination. May optionally specify the cursor field used to determine which records are new.

https://docs.airbyte.com/understanding-airbyte/connections/incremental-append/

classmethod incremental_append_dedup(cursor_field=None, primary_key=None)[source]

Syncs new records from the source, appending to an append-only history table in the destination. Also generates a deduplicated view mirroring the source table. May optionally specify the cursor field used to determine which records are new, and the primary key used to determine which records are duplicates.

https://docs.airbyte.com/understanding-airbyte/connections/incremental-append-dedup/

Managed Config Generated Sources

class dagster_airbyte.managed.generated.sources.StravaSource(name, client_id, client_secret, refresh_token, athlete_id, start_date, auth_type=None)[source]
__init__(name, client_id, client_secret, refresh_token, athlete_id, start_date, auth_type=None)[source]

Airbyte Source for Strava.

Documentation can be found at https://docs.airbyte.com/integrations/sources/strava

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The Client ID of your Strava developer application.

  • client_secret (str) – The Client Secret of your Strava developer application.

  • refresh_token (str) – The Refresh Token with the activity: read_all permissions.

  • athlete_id (int) – The Athlete ID of your Strava developer application.

  • start_date (str) – UTC date and time. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.AppsflyerSource(name, app_id, api_token, start_date, timezone=None)[source]
__init__(name, app_id, api_token, start_date, timezone=None)[source]

Airbyte Source for Appsflyer.

Parameters:
  • name (str) – The name of the destination.

  • app_id (str) – App identifier as found in AppsFlyer.

  • api_token (str) – Pull API token for authentication. If you change the account admin, the token changes, and you must update scripts with the new token. Get the API token in the Dashboard.

  • start_date (str) – The default value to use if no bookmark exists for an endpoint. Raw Reports historical lookback is limited to 90 days.

  • timezone (Optional[str]) – Time zone in which date times are stored. The project timezone may be found in the App settings in the AppsFlyer console.

class dagster_airbyte.managed.generated.sources.GoogleWorkspaceAdminReportsSource(name, credentials_json, email, lookback=None)[source]
__init__(name, credentials_json, email, lookback=None)[source]

Airbyte Source for Google Workspace Admin Reports.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-workspace-admin-reports

Parameters:
  • name (str) – The name of the destination.

  • credentials_json (str) – The contents of the JSON service account key. See the docs for more information on how to generate this key.

  • email (str) – The email of the user, who has permissions to access the Google Workspace Admin APIs.

  • lookback (Optional[int]) – Sets the range of time shown in the report. The maximum value allowed by the Google API is 180 days.

class dagster_airbyte.managed.generated.sources.CartSource(name, credentials, start_date)[source]
__init__(name, credentials, start_date)[source]

Airbyte Source for Cart.

Documentation can be found at https://docs.airbyte.com/integrations/sources/cart

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate the data

class CartSource.CentralAPIRouter(user_name, user_secret, site_id)[source]
__init__(user_name, user_secret, site_id)[source]
class CartSource.SingleStoreAccessToken(access_token, store_name)[source]
__init__(access_token, store_name)[source]
class dagster_airbyte.managed.generated.sources.LinkedinAdsSource(name, credentials, start_date, account_ids=None)[source]
__init__(name, credentials, start_date, account_ids=None)[source]

Airbyte Source for Linkedin Ads.

Documentation can be found at https://docs.airbyte.com/integrations/sources/linkedin-ads

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date in the format 2020-09-17. Any data before this date will not be replicated.

  • account_ids (Optional[List[int]]) – Specify the account IDs separated by a space, to pull the data from. Leave empty, if you want to pull the data from all associated accounts. See the LinkedIn Ads docs for more info.

class LinkedinAdsSource.OAuth20(client_id, client_secret, refresh_token, auth_method=None)[source]
__init__(client_id, client_secret, refresh_token, auth_method=None)[source]
class LinkedinAdsSource.AccessToken(access_token, auth_method=None)[source]
__init__(access_token, auth_method=None)[source]
class dagster_airbyte.managed.generated.sources.MongodbSource(name, host, port, database, user, password, auth_source, replica_set=None, ssl=None)[source]
__init__(name, host, port, database, user, password, auth_source, replica_set=None, ssl=None)[source]

Airbyte Source for Mongodb.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mongodb

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Host of a Mongo database to be replicated.

  • port (int) – Port of a Mongo database to be replicated.

  • database (str) – Database to be replicated.

  • user (str) – User

  • password (str) – Password

  • auth_source (str) – Authentication source where user information is stored. See the Mongo docs for more info.

  • replica_set (Optional[str]) – The name of the set to filter servers by, when connecting to a replica set (Under this condition, the ‘TLS connection’ value automatically becomes ‘true’). See the Mongo docs for more info.

  • ssl (Optional[bool]) – If this switch is enabled, TLS connections will be used to connect to MongoDB.

class dagster_airbyte.managed.generated.sources.TimelySource(name, account_id, start_date, bearer_token)[source]
__init__(name, account_id, start_date, bearer_token)[source]

Airbyte Source for Timely.

Parameters:
  • name (str) – The name of the destination.

  • account_id (str) – Timely account id

  • start_date (str) – start date

  • bearer_token (str) – Timely bearer token

class dagster_airbyte.managed.generated.sources.StockTickerApiTutorialSource(name, stock_ticker, api_key)[source]
__init__(name, stock_ticker, api_key)[source]

Airbyte Source for Stock Ticker Api Tutorial.

Documentation can be found at https://polygon.io/docs/stocks/get_v2_aggs_grouped_locale_us_market_stocks__date

Parameters:
  • name (str) – The name of the destination.

  • stock_ticker (str) – The stock ticker to track

  • api_key (str) – The Polygon.io Stocks API key to use to hit the API.

class dagster_airbyte.managed.generated.sources.WrikeSource(name, access_token, wrike_instance, start_date=None)[source]
__init__(name, access_token, wrike_instance, start_date=None)[source]

Airbyte Source for Wrike.

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – Permanent access token. You can find documentation on how to acquire a permanent access token here

  • wrike_instance (str) – Wrike’s instance such as app-us2.wrike.com

  • start_date (Optional[str]) – UTC date and time in the format 2017-01-25T00:00:00Z. Only comments after this date will be replicated.

class dagster_airbyte.managed.generated.sources.CommercetoolsSource(name, region, host, start_date, project_key, client_id, client_secret)[source]
__init__(name, region, host, start_date, project_key, client_id, client_secret)[source]

Airbyte Source for Commercetools.

Documentation can be found at https://docs.airbyte.com/integrations/sources/commercetools

Parameters:
  • name (str) – The name of the destination.

  • region (str) – The region of the platform.

  • host (str) – The cloud provider your shop is hosted. See: https://docs.commercetools.com/api/authorization

  • start_date (str) – The date you would like to replicate data. Format: YYYY-MM-DD.

  • project_key (str) – The project key

  • client_id (str) – Id of API Client.

  • client_secret (str) – The password of secret of API Client.

class dagster_airbyte.managed.generated.sources.GutendexSource(name, author_year_start=None, author_year_end=None, copyright=None, languages=None, search=None, sort=None, topic=None)[source]
__init__(name, author_year_start=None, author_year_end=None, copyright=None, languages=None, search=None, sort=None, topic=None)[source]

Airbyte Source for Gutendex.

Documentation can be found at https://docs.airbyte.com/integrations/sources/gutendex

Parameters:
  • name (str) – The name of the destination.

  • author_year_start (Optional[str]) – (Optional) Defines the minimum birth year of the authors. Books by authors born prior to the start year will not be returned. Supports both positive (CE) or negative (BCE) integer values

  • author_year_end (Optional[str]) – (Optional) Defines the maximum birth year of the authors. Books by authors born after the end year will not be returned. Supports both positive (CE) or negative (BCE) integer values

  • copyright (Optional[str]) – (Optional) Use this to find books with a certain copyright status - true for books with existing copyrights, false for books in the public domain in the USA, or null for books with no available copyright information.

  • languages (Optional[str]) – (Optional) Use this to find books in any of a list of languages. They must be comma-separated, two-character language codes.

  • search (Optional[str]) – (Optional) Use this to search author names and book titles with given words. They must be separated by a space (i.e. %20 in URL-encoded format) and are case-insensitive.

  • sort (Optional[str]) – (Optional) Use this to sort books - ascending for Project Gutenberg ID numbers from lowest to highest, descending for IDs highest to lowest, or popular (the default) for most popular to least popular by number of downloads.

  • topic (Optional[str]) – (Optional) Use this to search for a case-insensitive key-phrase in books’ bookshelves or subjects.

class dagster_airbyte.managed.generated.sources.IterableSource(name, api_key, start_date)[source]
__init__(name, api_key, start_date)[source]

Airbyte Source for Iterable.

Documentation can be found at https://docs.airbyte.com/integrations/sources/iterable

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Iterable API Key. See the docs for more information on how to obtain this key.

  • start_date (str) – The date from which you’d like to replicate data for Iterable, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

class dagster_airbyte.managed.generated.sources.QuickbooksSingerSource(name, client_id, client_secret, refresh_token, realm_id, user_agent, start_date, sandbox)[source]
__init__(name, client_id, client_secret, refresh_token, realm_id, user_agent, start_date, sandbox)[source]

Airbyte Source for Quickbooks Singer.

Documentation can be found at https://docs.airbyte.com/integrations/sources/quickbooks

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – Identifies which app is making the request. Obtain this value from the Keys tab on the app profile via My Apps on the developer site. There are two versions of this key: development and production.

  • client_secret (str) – Obtain this value from the Keys tab on the app profile via My Apps on the developer site. There are two versions of this key: development and production.

  • refresh_token (str) – A token used when refreshing the access token.

  • realm_id (str) – Labeled Company ID. The Make API Calls panel is populated with the realm id and the current access token.

  • user_agent (str) – Process and email for API logging purposes. Example: tap-quickbooks .

  • start_date (str) – The default value to use if no bookmark exists for an endpoint (rfc3339 date string). E.g, 2021-03-20T00:00:00Z. Any data before this date will not be replicated.

  • sandbox (bool) – Determines whether to use the sandbox or production environment.

class dagster_airbyte.managed.generated.sources.BigcommerceSource(name, start_date, store_hash, access_token)[source]
__init__(name, start_date, store_hash, access_token)[source]

Airbyte Source for Bigcommerce.

Documentation can be found at https://docs.airbyte.com/integrations/sources/bigcommerce

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date you would like to replicate data. Format: YYYY-MM-DD.

  • store_hash (str) – The hash code of the store. For https://api.bigcommerce.com/stores/HASH_CODE/v3/, The store’s hash code is ‘HASH_CODE’.

  • access_token (str) – Access Token for making authenticated requests.

class dagster_airbyte.managed.generated.sources.ShopifySource(name, shop, credentials, start_date)[source]
__init__(name, shop, credentials, start_date)[source]

Airbyte Source for Shopify.

Documentation can be found at https://docs.airbyte.com/integrations/sources/shopify

Parameters:
  • name (str) – The name of the destination.

  • shop (str) – The name of your Shopify store found in the URL. For example, if your URL was https://NAME.myshopify.com, then the name would be ‘NAME’.

  • credentials (Union[ShopifySource.APIPassword, ShopifySource.OAuth20]) – The authorization method to use to retrieve data from Shopify

  • start_date (str) – The date you would like to replicate data from. Format: YYYY-MM-DD. Any data before this date will not be replicated.

class ShopifySource.APIPassword(api_password)[source]
__init__(api_password)[source]
class ShopifySource.OAuth20(client_id=None, client_secret=None, access_token=None)[source]
__init__(client_id=None, client_secret=None, access_token=None)[source]
class dagster_airbyte.managed.generated.sources.AppstoreSingerSource(name, key_id, private_key, issuer_id, vendor, start_date)[source]
__init__(name, key_id, private_key, issuer_id, vendor, start_date)[source]

Airbyte Source for Appstore Singer.

Documentation can be found at https://docs.airbyte.com/integrations/sources/appstore

Parameters:
  • name (str) – The name of the destination.

  • key_id (str) – Appstore Key ID. See the docs for more information on how to obtain this key.

  • private_key (str) – Appstore Private Key. See the docs for more information on how to obtain this key.

  • issuer_id (str) – Appstore Issuer ID. See the docs for more information on how to obtain this ID.

  • vendor (str) – Appstore Vendor ID. See the docs for more information on how to obtain this ID.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.GreenhouseSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Greenhouse.

Documentation can be found at https://docs.airbyte.com/integrations/sources/greenhouse

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Greenhouse API Key. See the docs for more information on how to generate this key.

class dagster_airbyte.managed.generated.sources.ZoomSingerSource(name, jwt)[source]
__init__(name, jwt)[source]

Airbyte Source for Zoom Singer.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zoom

Parameters:
  • name (str) – The name of the destination.

  • jwt (str) – Zoom JWT Token. See the docs for more information on how to obtain this key.

class dagster_airbyte.managed.generated.sources.TiktokMarketingSource(name, credentials, start_date=None, end_date=None, report_granularity=None)[source]
__init__(name, credentials, start_date=None, end_date=None, report_granularity=None)[source]

Airbyte Source for Tiktok Marketing.

Documentation can be found at https://docs.airbyte.com/integrations/sources/tiktok-marketing

Parameters:
  • name (str) – The name of the destination.

  • credentials (Union[TiktokMarketingSource.OAuth20, TiktokMarketingSource.SandboxAccessToken]) – Authentication method

  • start_date (Optional[str]) – The Start Date in format: YYYY-MM-DD. Any data before this date will not be replicated. If this parameter is not set, all data will be replicated.

  • end_date (Optional[str]) – The date until which you’d like to replicate data for all incremental streams, in the format YYYY-MM-DD. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the data till the current date.

  • report_granularity (Optional[str]) – The granularity used for aggregating performance data in reports. See the docs.

class TiktokMarketingSource.OAuth20(app_id, secret, access_token, auth_type=None)[source]
__init__(app_id, secret, access_token, auth_type=None)[source]
class TiktokMarketingSource.SandboxAccessToken(advertiser_id, access_token, auth_type=None)[source]
__init__(advertiser_id, access_token, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.ZendeskChatSource(name, start_date, credentials, subdomain=None)[source]
__init__(name, start_date, credentials, subdomain=None)[source]

Airbyte Source for Zendesk Chat.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zendesk-chat

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate data for Zendesk Chat API, in the format YYYY-MM-DDT00:00:00Z.

  • subdomain (Optional[str]) – Required if you access Zendesk Chat from a Zendesk Support subdomain.

class ZendeskChatSource.OAuth20(client_id=None, client_secret=None, access_token=None, refresh_token=None)[source]
__init__(client_id=None, client_secret=None, access_token=None, refresh_token=None)[source]
class ZendeskChatSource.AccessToken(access_token)[source]
__init__(access_token)[source]
class dagster_airbyte.managed.generated.sources.AwsCloudtrailSource(name, aws_key_id, aws_secret_key, aws_region_name, start_date)[source]
__init__(name, aws_key_id, aws_secret_key, aws_region_name, start_date)[source]

Airbyte Source for Aws Cloudtrail.

Documentation can be found at https://docs.airbyte.com/integrations/sources/aws-cloudtrail

Parameters:
  • name (str) – The name of the destination.

  • aws_key_id (str) – AWS CloudTrail Access Key ID. See the docs for more information on how to obtain this key.

  • aws_secret_key (str) – AWS CloudTrail Access Key ID. See the docs for more information on how to obtain this key.

  • aws_region_name (str) – The default AWS Region to use, for example, us-west-1 or us-west-2. When specifying a Region inline during client initialization, this property is named region_name.

  • start_date (str) – The date you would like to replicate data. Data in AWS CloudTrail is available for last 90 days only. Format: YYYY-MM-DD.

class dagster_airbyte.managed.generated.sources.OktaSource(name, credentials, domain=None, start_date=None)[source]
__init__(name, credentials, domain=None, start_date=None)[source]

Airbyte Source for Okta.

Documentation can be found at https://docs.airbyte.com/integrations/sources/okta

Parameters:
  • name (str) – The name of the destination.

  • domain (Optional[str]) – The Okta domain. See the docs for instructions on how to find it.

  • start_date (Optional[str]) – UTC date and time in the format YYYY-MM-DDTHH:MM:SSZ. Any data before this date will not be replicated.

class OktaSource.OAuth20(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class OktaSource.APIToken(api_token)[source]
__init__(api_token)[source]
class dagster_airbyte.managed.generated.sources.InsightlySource(name, token=None, start_date=None)[source]
__init__(name, token=None, start_date=None)[source]

Airbyte Source for Insightly.

Documentation can be found at https://docs.airbyte.com/integrations/sources/insightly

Parameters:
  • name (str) – The name of the destination.

  • token (Optional[str]) – Your Insightly API token.

  • start_date (Optional[str]) – The date from which you’d like to replicate data for Insightly in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated. Note that it will be used only for incremental streams.

class dagster_airbyte.managed.generated.sources.LinkedinPagesSource(name, org_id, credentials)[source]
__init__(name, org_id, credentials)[source]

Airbyte Source for Linkedin Pages.

Documentation can be found at https://docs.airbyte.com/integrations/sources/linkedin-pages/

Parameters:
  • name (str) – The name of the destination.

  • org_id (int) – Specify the Organization ID

class LinkedinPagesSource.OAuth20(client_id, client_secret, refresh_token, auth_method=None)[source]
__init__(client_id, client_secret, refresh_token, auth_method=None)[source]
class LinkedinPagesSource.AccessToken(access_token, auth_method=None)[source]
__init__(access_token, auth_method=None)[source]
class dagster_airbyte.managed.generated.sources.PersistiqSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Persistiq.

Documentation can be found at https://docs.airbyte.com/integrations/sources/persistiq

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – PersistIq API Key. See the docs for more information on where to find that key.

class dagster_airbyte.managed.generated.sources.FreshcallerSource(name, domain, api_key, start_date, requests_per_minute=None, sync_lag_minutes=None)[source]
__init__(name, domain, api_key, start_date, requests_per_minute=None, sync_lag_minutes=None)[source]

Airbyte Source for Freshcaller.

Documentation can be found at https://docs.airbyte.com/integrations/sources/freshcaller

Parameters:
  • name (str) – The name of the destination.

  • domain (str) – Used to construct Base URL for the Freshcaller APIs

  • api_key (str) – Freshcaller API Key. See the docs for more information on how to obtain this key.

  • requests_per_minute (Optional[int]) – The number of requests per minute that this source allowed to use. There is a rate limit of 50 requests per minute per app per account.

  • start_date (str) – UTC date and time. Any data created after this date will be replicated.

  • sync_lag_minutes (Optional[int]) – Lag in minutes for each sync, i.e., at time T, data for the time range [prev_sync_time, T-30] will be fetched

class dagster_airbyte.managed.generated.sources.AppfollowSource(name, ext_id, cid, api_secret, country)[source]
__init__(name, ext_id, cid, api_secret, country)[source]

Airbyte Source for Appfollow.

Documentation can be found at https://docs.airbyte.com/integrations/sources/appfollow

Parameters:
  • name (str) – The name of the destination.

  • ext_id (str) – for App Store — this is 9-10 digits identification number; for Google Play — this is bundle name;

  • cid (str) – client id provided by Appfollow

  • api_secret (str) – api secret provided by Appfollow

  • country (str) – getting data by Country

class dagster_airbyte.managed.generated.sources.FacebookPagesSource(name, access_token, page_id)[source]
__init__(name, access_token, page_id)[source]

Airbyte Source for Facebook Pages.

Documentation can be found at https://docs.airbyte.com/integrations/sources/facebook-pages

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – Facebook Page Access Token

  • page_id (str) – Page ID

class dagster_airbyte.managed.generated.sources.JiraSource(name, api_token, domain, email, projects=None, start_date=None, additional_fields=None, expand_issue_changelog=None, render_fields=None, enable_experimental_streams=None)[source]
__init__(name, api_token, domain, email, projects=None, start_date=None, additional_fields=None, expand_issue_changelog=None, render_fields=None, enable_experimental_streams=None)[source]

Airbyte Source for Jira.

Documentation can be found at https://docs.airbyte.com/integrations/sources/jira

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Jira API Token. See the docs for more information on how to generate this key.

  • domain (str) – The Domain for your Jira account, e.g. airbyteio.atlassian.net

  • email (str) – The user email for your Jira account.

  • projects (Optional[List[str]]) – List of Jira project keys to replicate data for.

  • start_date (Optional[str]) – The date from which you’d like to replicate data for Jira in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated. Note that it will be used only in the following incremental streams: issues.

  • additional_fields (Optional[List[str]]) – List of additional fields to include in replicating issues.

  • expand_issue_changelog (Optional[bool]) – Expand the changelog when replicating issues.

  • render_fields (Optional[bool]) – Render issue fields in HTML format in addition to Jira JSON-like format.

  • enable_experimental_streams (Optional[bool]) – Allow the use of experimental streams which rely on undocumented Jira API endpoints. See https://docs.airbyte.com/integrations/sources/jira#experimental-tables for more info.

class dagster_airbyte.managed.generated.sources.GoogleSheetsSource(name, spreadsheet_id, credentials, row_batch_size=None)[source]
__init__(name, spreadsheet_id, credentials, row_batch_size=None)[source]

Airbyte Source for Google Sheets.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-sheets

Parameters:
class GoogleSheetsSource.AuthenticateViaGoogleOAuth(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class GoogleSheetsSource.ServiceAccountKeyAuthentication(service_account_info)[source]
__init__(service_account_info)[source]
class dagster_airbyte.managed.generated.sources.DockerhubSource(name, docker_username)[source]
__init__(name, docker_username)[source]

Airbyte Source for Dockerhub.

Documentation can be found at https://docs.airbyte.com/integrations/sources/dockerhub

Parameters:
class dagster_airbyte.managed.generated.sources.UsCensusSource(name, query_path, api_key, query_params=None)[source]
__init__(name, query_path, api_key, query_params=None)[source]

Airbyte Source for Us Census.

Documentation can be found at https://docs.airbyte.com/integrations/sources/us-census

Parameters:
  • name (str) – The name of the destination.

  • query_params (Optional[str]) – The query parameters portion of the GET request, without the api key

  • query_path (str) – The path portion of the GET request

  • api_key (str) – Your API Key. Get your key here.

class dagster_airbyte.managed.generated.sources.KustomerSingerSource(name, api_token, start_date)[source]
__init__(name, api_token, start_date)[source]

Airbyte Source for Kustomer Singer.

Documentation can be found at https://docs.airbyte.com/integrations/sources/kustomer

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Kustomer API Token. See the docs on how to obtain this

  • start_date (str) – The date from which you’d like to replicate the data

class dagster_airbyte.managed.generated.sources.AzureTableSource(name, storage_account_name, storage_access_key, storage_endpoint_suffix=None)[source]
__init__(name, storage_account_name, storage_access_key, storage_endpoint_suffix=None)[source]

Airbyte Source for Azure Table.

Parameters:
  • name (str) – The name of the destination.

  • storage_account_name (str) – The name of your storage account.

  • storage_access_key (str) – Azure Table Storage Access Key. See the docs for more information on how to obtain this key.

  • storage_endpoint_suffix (Optional[str]) – Azure Table Storage service account URL suffix. See the docs for more information on how to obtain endpoint suffix

class dagster_airbyte.managed.generated.sources.ScaffoldJavaJdbcSource(name, host, port, database, username, replication_method, password=None, jdbc_url_params=None)[source]
__init__(name, host, port, database, username, replication_method, password=None, jdbc_url_params=None)[source]

Airbyte Source for Scaffold Java Jdbc.

Documentation can be found at https://docs.airbyte.com/integrations/sources/scaffold_java_jdbc

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Hostname of the database.

  • port (int) – Port of the database.

  • database (str) – Name of the database.

  • username (str) – Username to use to access the database.

  • password (Optional[str]) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3)

  • replication_method (str) – Replication method to use for extracting data from the database. STANDARD replication requires no setup on the DB side but will not be able to represent deletions incrementally. CDC uses the Binlog to detect inserts, updates, and deletes. This needs to be configured on the source database itself.

class dagster_airbyte.managed.generated.sources.TidbSource(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]
__init__(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]

Airbyte Source for Tidb.

Documentation can be found at https://docs.airbyte.com/integrations/sources/tidb

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Hostname of the database.

  • port (int) – Port of the database.

  • database (str) – Name of the database.

  • username (str) – Username to use to access the database.

  • password (Optional[str]) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3)

  • ssl (Optional[bool]) – Encrypt data using SSL.

class dagster_airbyte.managed.generated.sources.QualarooSource(name, token, key, start_date, survey_ids=None)[source]
__init__(name, token, key, start_date, survey_ids=None)[source]

Airbyte Source for Qualaroo.

Documentation can be found at https://docs.airbyte.com/integrations/sources/qualaroo

Parameters:
  • name (str) – The name of the destination.

  • token (str) – A Qualaroo token. See the docs for instructions on how to generate it.

  • key (str) – A Qualaroo token. See the docs for instructions on how to generate it.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • survey_ids (Optional[List[str]]) – IDs of the surveys from which you’d like to replicate data. If left empty, data from all surveys to which you have access will be replicated.

class dagster_airbyte.managed.generated.sources.YahooFinancePriceSource(name, tickers, interval=None, range=None)[source]
__init__(name, tickers, interval=None, range=None)[source]

Airbyte Source for Yahoo Finance Price.

Parameters:
  • name (str) – The name of the destination.

  • tickers (str) – Comma-separated identifiers for the stocks to be queried. Whitespaces are allowed.

  • interval (Optional[str]) – The interval of between prices queried.

  • range (Optional[str]) – The range of prices to be queried.

class dagster_airbyte.managed.generated.sources.GoogleAnalyticsV4Source(name, credentials, start_date, view_id, custom_reports=None, window_in_days=None)[source]
__init__(name, credentials, start_date, view_id, custom_reports=None, window_in_days=None)[source]

Airbyte Source for Google Analytics V4.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-analytics-universal-analytics

Parameters:
  • name (str) – The name of the destination.

  • credentials (Union[GoogleAnalyticsV4Source.AuthenticateViaGoogleOauth, GoogleAnalyticsV4Source.ServiceAccountKeyAuthentication]) – Credentials for the service

  • start_date (str) – The date in the format YYYY-MM-DD. Any data before this date will not be replicated.

  • view_id (str) – The ID for the Google Analytics View you want to fetch data from. This can be found from the Google Analytics Account Explorer.

  • custom_reports (Optional[str]) – A JSON array describing the custom reports you want to sync from Google Analytics. See the docs for more information about the exact format you can use to fill out this field.

  • window_in_days (Optional[int]) – The time increment used by the connector when requesting data from the Google Analytics API. More information is available in the the docs. The bigger this value is, the faster the sync will be, but the more likely that sampling will be applied to your data, potentially causing inaccuracies in the returned results. We recommend setting this to 1 unless you have a hard requirement to make the sync faster at the expense of accuracy. The minimum allowed value for this field is 1, and the maximum is 364.

class GoogleAnalyticsV4Source.AuthenticateViaGoogleOauth(client_id, client_secret, refresh_token, auth_type=None, access_token=None)[source]
__init__(client_id, client_secret, refresh_token, auth_type=None, access_token=None)[source]
class GoogleAnalyticsV4Source.ServiceAccountKeyAuthentication(credentials_json, auth_type=None)[source]
__init__(credentials_json, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.JdbcSource(name, username, jdbc_url, password=None, jdbc_url_params=None)[source]
__init__(name, username, jdbc_url, password=None, jdbc_url_params=None)[source]

Airbyte Source for Jdbc.

Documentation can be found at https://docs.airbyte.com/integrations/sources/postgres

Parameters:
  • name (str) – The name of the destination.

  • username (str) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with this username.

  • jdbc_url (str) – JDBC formatted URL. See the standard here.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

class dagster_airbyte.managed.generated.sources.FakerSource(name, count, seed=None, records_per_sync=None, records_per_slice=None)[source]
__init__(name, count, seed=None, records_per_sync=None, records_per_slice=None)[source]

Airbyte Source for Faker.

Documentation can be found at https://docs.airbyte.com/integrations/sources/faker

Parameters:
  • name (str) – The name of the destination.

  • count (int) – How many users should be generated in total. This setting does not apply to the purchases or products stream.

  • seed (Optional[int]) – Manually control the faker random seed to return the same values on subsequent runs (leave -1 for random)

  • records_per_sync (Optional[int]) – How many fake records will be returned for each sync, for each stream? By default, it will take 2 syncs to create the requested 1000 records.

  • records_per_slice (Optional[int]) – How many fake records will be in each page (stream slice), before a state message is emitted?

class dagster_airbyte.managed.generated.sources.TplcentralSource(name, url_base, client_id, client_secret, user_login_id=None, user_login=None, tpl_key=None, customer_id=None, facility_id=None, start_date=None)[source]
__init__(name, url_base, client_id, client_secret, user_login_id=None, user_login=None, tpl_key=None, customer_id=None, facility_id=None, start_date=None)[source]

Airbyte Source for Tplcentral.

Documentation can be found at https://docs.airbyte.com/integrations/sources/tplcentral

Parameters:
  • name (str) – The name of the destination.

  • user_login_id (Optional[int]) – User login ID and/or name is required

  • user_login (Optional[str]) – User login ID and/or name is required

  • start_date (Optional[str]) – Date and time together in RFC 3339 format, for example, 2018-11-13T20:20:39+00:00.

class dagster_airbyte.managed.generated.sources.ClickhouseSource(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]
__init__(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]

Airbyte Source for Clickhouse.

Documentation can be found at https://docs.airbyte.com/integrations/destinations/clickhouse

Parameters:
  • name (str) – The name of the destination.

  • host (str) – The host endpoint of the Clickhouse cluster.

  • port (int) – The port of the database.

  • database (str) – The name of the database.

  • username (str) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with this username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (Eg. key1=value1&key2=value2&key3=value3). For more information read about JDBC URL parameters.

  • ssl (Optional[bool]) – Encrypt data using SSL.

class dagster_airbyte.managed.generated.sources.FreshserviceSource(name, domain_name, api_key, start_date)[source]
__init__(name, domain_name, api_key, start_date)[source]

Airbyte Source for Freshservice.

Documentation can be found at https://docs.airbyte.com/integrations/sources/freshservice

Parameters:
  • name (str) – The name of the destination.

  • domain_name (str) – The name of your Freshservice domain

  • api_key (str) – Freshservice API Key. See here. The key is case sensitive.

  • start_date (str) – UTC date and time in the format 2020-10-01T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.ZenloopSource(name, api_token, date_from=None, survey_id=None, survey_group_id=None)[source]
__init__(name, api_token, date_from=None, survey_id=None, survey_group_id=None)[source]

Airbyte Source for Zenloop.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zenloop

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Zenloop API Token. You can get the API token in settings page here

  • date_from (Optional[str]) – Zenloop date_from. Format: 2021-10-24T03:30:30Z or 2021-10-24. Leave empty if only data from current data should be synced

  • survey_id (Optional[str]) – Zenloop Survey ID. Can be found here. Leave empty to pull answers from all surveys

  • survey_group_id (Optional[str]) – Zenloop Survey Group ID. Can be found by pulling All Survey Groups via SurveyGroups stream. Leave empty to pull answers from all survey groups

class dagster_airbyte.managed.generated.sources.OracleSource(name, host, port, connection_data, username, encryption, password=None, schemas=None, jdbc_url_params=None)[source]
__init__(name, host, port, connection_data, username, encryption, password=None, schemas=None, jdbc_url_params=None)[source]

Airbyte Source for Oracle.

Documentation can be found at https://docs.airbyte.com/integrations/sources/oracle

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Hostname of the database.

  • port (int) – Port of the database. Oracle Corporations recommends the following port numbers: 1521 - Default listening port for client connections to the listener. 2484 - Recommended and officially registered listening port for client connections to the listener using TCP/IP with SSL

  • connection_data (Union[OracleSource.ServiceName, OracleSource.SystemIDSID]) – Connect data that will be used for DB connection

  • username (str) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with the username.

  • schemas (Optional[List[str]]) – The list of schemas to sync from. Defaults to user. Case sensitive.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

  • encryption (Union[OracleSource.Unencrypted, OracleSource.NativeNetworkEncryptionNNE, OracleSource.TLSEncryptedVerifyCertificate]) – The encryption method with is used when communicating with the database.

class OracleSource.ServiceName(service_name, connection_type=None)[source]
__init__(service_name, connection_type=None)[source]
class OracleSource.SystemIDSID(sid, connection_type=None)[source]
__init__(sid, connection_type=None)[source]
class OracleSource.Unencrypted[source]
__init__()[source]
class OracleSource.NativeNetworkEncryptionNNE(encryption_algorithm=None)[source]
__init__(encryption_algorithm=None)[source]
class OracleSource.TLSEncryptedVerifyCertificate(ssl_certificate)[source]
__init__(ssl_certificate)[source]
class dagster_airbyte.managed.generated.sources.KlaviyoSource(name, api_key, start_date)[source]
__init__(name, api_key, start_date)[source]

Airbyte Source for Klaviyo.

Documentation can be found at https://docs.airbyte.com/integrations/sources/klaviyo

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Klaviyo API Key. See our docs if you need help finding this key.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.GoogleDirectorySource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Google Directory.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-directory

Parameters:
class GoogleDirectorySource.SignInViaGoogleOAuth(client_id, client_secret, refresh_token, credentials_title=None)[source]
__init__(client_id, client_secret, refresh_token, credentials_title=None)[source]
class GoogleDirectorySource.ServiceAccountKey(credentials_json, email, credentials_title=None)[source]
__init__(credentials_json, email, credentials_title=None)[source]
class dagster_airbyte.managed.generated.sources.InstagramSource(name, start_date, access_token)[source]
__init__(name, start_date, access_token)[source]

Airbyte Source for Instagram.

Documentation can be found at https://docs.airbyte.com/integrations/sources/instagram

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate data for User Insights, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

  • access_token (str) – The value of the access token generated. See the docs for more information

class dagster_airbyte.managed.generated.sources.ShortioSource(name, domain_id, secret_key, start_date)[source]
__init__(name, domain_id, secret_key, start_date)[source]

Airbyte Source for Shortio.

Documentation can be found at https://developers.short.io/reference

Parameters:
  • name (str) – The name of the destination.

  • secret_key (str) – Short.io Secret Key

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.SquareSource(name, is_sandbox, credentials, start_date=None, include_deleted_objects=None)[source]
__init__(name, is_sandbox, credentials, start_date=None, include_deleted_objects=None)[source]

Airbyte Source for Square.

Documentation can be found at https://docs.airbyte.com/integrations/sources/square

Parameters:
  • name (str) – The name of the destination.

  • is_sandbox (bool) – Determines whether to use the sandbox or production environment.

  • start_date (Optional[str]) – UTC date in the format YYYY-MM-DD. Any data before this date will not be replicated. If not set, all data will be replicated.

  • include_deleted_objects (Optional[bool]) – In some streams there is an option to include deleted objects (Items, Categories, Discounts, Taxes)

class SquareSource.OauthAuthentication(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class SquareSource.APIKey(api_key)[source]
__init__(api_key)[source]
class dagster_airbyte.managed.generated.sources.DelightedSource(name, since, api_key)[source]
__init__(name, since, api_key)[source]

Airbyte Source for Delighted.

Parameters:
  • name (str) – The name of the destination.

  • since (str) – The date from which you’d like to replicate the data

  • api_key (str) – A Delighted API key.

class dagster_airbyte.managed.generated.sources.AmazonSqsSource(name, queue_url, region, delete_messages, max_batch_size=None, max_wait_time=None, attributes_to_return=None, visibility_timeout=None, access_key=None, secret_key=None)[source]
__init__(name, queue_url, region, delete_messages, max_batch_size=None, max_wait_time=None, attributes_to_return=None, visibility_timeout=None, access_key=None, secret_key=None)[source]

Airbyte Source for Amazon Sqs.

Documentation can be found at https://docs.airbyte.com/integrations/sources/amazon-sqs

Parameters:
  • name (str) – The name of the destination.

  • queue_url (str) – URL of the SQS Queue

  • region (str) – AWS Region of the SQS Queue

  • delete_messages (bool) – If Enabled, messages will be deleted from the SQS Queue after being read. If Disabled, messages are left in the queue and can be read more than once. WARNING: Enabling this option can result in data loss in cases of failure, use with caution, see documentation for more detail.

  • max_batch_size (Optional[int]) – Max amount of messages to get in one batch (10 max)

  • max_wait_time (Optional[int]) – Max amount of time in seconds to wait for messages in a single poll (20 max)

  • attributes_to_return (Optional[str]) – Comma separated list of Mesage Attribute names to return

  • visibility_timeout (Optional[int]) – Modify the Visibility Timeout of the individual message from the Queue’s default (seconds).

  • access_key (Optional[str]) – The Access Key ID of the AWS IAM Role to use for pulling messages

  • secret_key (Optional[str]) – The Secret Key of the AWS IAM Role to use for pulling messages

class dagster_airbyte.managed.generated.sources.YoutubeAnalyticsSource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Youtube Analytics.

Documentation can be found at https://docs.airbyte.com/integrations/sources/youtube-analytics

Parameters:

name (str) – The name of the destination.

class YoutubeAnalyticsSource.AuthenticateViaOAuth20(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class dagster_airbyte.managed.generated.sources.ScaffoldSourcePythonSource(name, fix_me=None)[source]
__init__(name, fix_me=None)[source]

Airbyte Source for Scaffold Source Python.

Parameters:
  • name (str) – The name of the destination.

  • fix_me (Optional[str]) – describe me

class dagster_airbyte.managed.generated.sources.LookerSource(name, domain, client_id, client_secret, run_look_ids=None)[source]
__init__(name, domain, client_id, client_secret, run_look_ids=None)[source]

Airbyte Source for Looker.

Documentation can be found at https://docs.airbyte.com/integrations/sources/looker

Parameters:
  • name (str) – The name of the destination.

  • domain (str) – Domain for your Looker account, e.g. airbyte.cloud.looker.com,looker.[clientname].com,IP address

  • client_id (str) – The Client ID is first part of an API3 key that is specific to each Looker user. See the docs for more information on how to generate this key.

  • client_secret (str) – The Client Secret is second part of an API3 key.

  • run_look_ids (Optional[List[str]]) – The IDs of any Looks to run

class dagster_airbyte.managed.generated.sources.GitlabSource(name, api_url, private_token, start_date, groups=None, projects=None)[source]
__init__(name, api_url, private_token, start_date, groups=None, projects=None)[source]

Airbyte Source for Gitlab.

Documentation can be found at https://docs.airbyte.com/integrations/sources/gitlab

Parameters:
  • name (str) – The name of the destination.

  • api_url (str) – Please enter your basic URL from GitLab instance.

  • private_token (str) – Log into your GitLab account and then generate a personal Access Token.

  • groups (Optional[str]) – Space-delimited list of groups. e.g. airbyte.io.

  • projects (Optional[str]) – Space-delimited list of projects. e.g. airbyte.io/documentation meltano/tap-gitlab.

  • start_date (str) – The date from which you’d like to replicate data for GitLab API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

class dagster_airbyte.managed.generated.sources.ExchangeRatesSource(name, start_date, access_key, base=None, ignore_weekends=None)[source]
__init__(name, start_date, access_key, base=None, ignore_weekends=None)[source]

Airbyte Source for Exchange Rates.

Documentation can be found at https://docs.airbyte.com/integrations/sources/exchangeratesapi

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – Start getting data from that date.

  • access_key (str) – Your API Key. See here. The key is case sensitive.

  • base (Optional[str]) – ISO reference currency. See here. Free plan doesn’t support Source Currency Switching, default base currency is EUR

  • ignore_weekends (Optional[bool]) – Ignore weekends? (Exchanges don’t run on weekends)

class dagster_airbyte.managed.generated.sources.AmazonAdsSource(name, client_id, client_secret, refresh_token, auth_type=None, region=None, report_wait_timeout=None, report_generation_max_retries=None, start_date=None, profiles=None, state_filter=None)[source]
__init__(name, client_id, client_secret, refresh_token, auth_type=None, region=None, report_wait_timeout=None, report_generation_max_retries=None, start_date=None, profiles=None, state_filter=None)[source]

Airbyte Source for Amazon Ads.

Documentation can be found at https://docs.airbyte.com/integrations/sources/amazon-ads

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The client ID of your Amazon Ads developer application. See the docs for more information.

  • client_secret (str) – The client secret of your Amazon Ads developer application. See the docs for more information.

  • refresh_token (str) – Amazon Ads refresh token. See the docs for more information on how to obtain this token.

  • region (Optional[str]) – Region to pull data from (EU/NA/FE). See docs for more details.

  • report_wait_timeout (Optional[int]) – Timeout duration in minutes for Reports. Default is 60 minutes.

  • report_generation_max_retries (Optional[int]) – Maximum retries Airbyte will attempt for fetching report data. Default is 5.

  • start_date (Optional[str]) – The Start date for collecting reports, should not be more than 60 days in the past. In YYYY-MM-DD format

  • profiles (Optional[List[int]]) – Profile IDs you want to fetch data for. See docs for more details.

  • state_filter (Optional[List[str]]) – Reflects the state of the Display, Product, and Brand Campaign streams as enabled, paused, or archived. If you do not populate this field, it will be ignored completely.

class dagster_airbyte.managed.generated.sources.MixpanelSource(name, credentials, project_id=None, attribution_window=None, project_timezone=None, select_properties_by_default=None, start_date=None, end_date=None, region=None, date_window_size=None)[source]
__init__(name, credentials, project_id=None, attribution_window=None, project_timezone=None, select_properties_by_default=None, start_date=None, end_date=None, region=None, date_window_size=None)[source]

Airbyte Source for Mixpanel.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mixpanel

Parameters:
  • name (str) – The name of the destination.

  • credentials (Union[MixpanelSource.ServiceAccount, MixpanelSource.ProjectSecret]) – Choose how to authenticate to Mixpanel

  • project_id (Optional[int]) – Your project ID number. See the docs for more information on how to obtain this.

  • attribution_window (Optional[int]) – A period of time for attributing results to ads and the lookback period after those actions occur during which ad results are counted. Default attribution window is 5 days.

  • project_timezone (Optional[str]) – Time zone in which integer date times are stored. The project timezone may be found in the project settings in the Mixpanel console.

  • select_properties_by_default (Optional[bool]) – Setting this config parameter to TRUE ensures that new properties on events and engage records are captured. Otherwise new properties will be ignored.

  • start_date (Optional[str]) – The date in the format YYYY-MM-DD. Any data before this date will not be replicated. If this option is not set, the connector will replicate data from up to one year ago by default.

  • end_date (Optional[str]) – The date in the format YYYY-MM-DD. Any data after this date will not be replicated. Left empty to always sync to most recent date

  • region (Optional[str]) – The region of mixpanel domain instance either US or EU.

  • date_window_size (Optional[int]) – Defines window size in days, that used to slice through data. You can reduce it, if amount of data in each window is too big for your environment.

class MixpanelSource.ServiceAccount(username, secret)[source]
__init__(username, secret)[source]
class MixpanelSource.ProjectSecret(api_secret)[source]
__init__(api_secret)[source]
class dagster_airbyte.managed.generated.sources.OrbitSource(name, api_token, workspace, start_date=None)[source]
__init__(name, api_token, workspace, start_date=None)[source]

Airbyte Source for Orbit.

Documentation can be found at https://docs.airbyte.com/integrations/sources/orbit

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Authorizes you to work with Orbit workspaces associated with the token.

  • workspace (str) – The unique name of the workspace that your API token is associated with.

  • start_date (Optional[str]) – Date in the format 2022-06-26. Only load members whose last activities are after this date.

class dagster_airbyte.managed.generated.sources.AmazonSellerPartnerSource(name, lwa_app_id, lwa_client_secret, refresh_token, aws_access_key, aws_secret_key, role_arn, replication_start_date, aws_environment, region, app_id=None, auth_type=None, replication_end_date=None, period_in_days=None, report_options=None, max_wait_seconds=None)[source]
__init__(name, lwa_app_id, lwa_client_secret, refresh_token, aws_access_key, aws_secret_key, role_arn, replication_start_date, aws_environment, region, app_id=None, auth_type=None, replication_end_date=None, period_in_days=None, report_options=None, max_wait_seconds=None)[source]

Airbyte Source for Amazon Seller Partner.

Documentation can be found at https://docs.airbyte.com/integrations/sources/amazon-seller-partner

Parameters:
  • name (str) – The name of the destination.

  • app_id (Optional[str]) – Your Amazon App ID

  • lwa_app_id (str) – Your Login with Amazon Client ID.

  • lwa_client_secret (str) – Your Login with Amazon Client Secret.

  • refresh_token (str) – The Refresh Token obtained via OAuth flow authorization.

  • aws_access_key (str) – Specifies the AWS access key used as part of the credentials to authenticate the user.

  • aws_secret_key (str) – Specifies the AWS secret key used as part of the credentials to authenticate the user.

  • role_arn (str) – Specifies the Amazon Resource Name (ARN) of an IAM role that you want to use to perform operations requested using this profile. (Needs permission to ‘Assume Role’ STS).

  • replication_start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • replication_end_date (Optional[str]) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data after this date will not be replicated.

  • period_in_days (Optional[int]) – Will be used for stream slicing for initial full_refresh sync when no updated state is present for reports that support sliced incremental sync.

  • report_options (Optional[str]) – Additional information passed to reports. This varies by report type. Must be a valid json string.

  • max_wait_seconds (Optional[int]) – Sometimes report can take up to 30 minutes to generate. This will set the limit for how long to wait for a successful report.

  • aws_environment (str) – An enumeration.

  • region (str) – An enumeration.

class dagster_airbyte.managed.generated.sources.CourierSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Courier.

Documentation can be found at https://docs.airbyte.io/integrations/sources/courier

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Courier API Key to retrieve your data.

class dagster_airbyte.managed.generated.sources.CloseComSource(name, api_key, start_date=None)[source]
__init__(name, api_key, start_date=None)[source]

Airbyte Source for Close Com.

Documentation can be found at https://docs.airbyte.com/integrations/sources/close-com

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Close.com API key (usually starts with ‘api\_’; find yours here).

  • start_date (Optional[str]) – The start date to sync data. Leave blank for full sync. Format: YYYY-MM-DD.

class dagster_airbyte.managed.generated.sources.BingAdsSource(name, client_id, refresh_token, developer_token, reports_start_date, auth_method=None, tenant_id=None, client_secret=None)[source]
__init__(name, client_id, refresh_token, developer_token, reports_start_date, auth_method=None, tenant_id=None, client_secret=None)[source]

Airbyte Source for Bing Ads.

Documentation can be found at https://docs.airbyte.com/integrations/sources/bing-ads

Parameters:
  • name (str) – The name of the destination.

  • tenant_id (Optional[str]) – The Tenant ID of your Microsoft Advertising developer application. Set this to “common” unless you know you need a different value.

  • client_id (str) – The Client ID of your Microsoft Advertising developer application.

  • client_secret (Optional[str]) – The Client Secret of your Microsoft Advertising developer application.

  • refresh_token (str) – Refresh Token to renew the expired Access Token.

  • developer_token (str) – Developer token associated with user. See more info in the docs.

  • reports_start_date (str) – The start date from which to begin replicating report data. Any data generated before this date will not be replicated in reports. This is a UTC date in YYYY-MM-DD format.

class dagster_airbyte.managed.generated.sources.PrimetricSource(name, client_id, client_secret)[source]
__init__(name, client_id, client_secret)[source]

Airbyte Source for Primetric.

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The Client ID of your Primetric developer application. The Client ID is visible here.

  • client_secret (str) – The Client Secret of your Primetric developer application. You can manage your client’s credentials here.

class dagster_airbyte.managed.generated.sources.PivotalTrackerSource(name, api_token)[source]
__init__(name, api_token)[source]

Airbyte Source for Pivotal Tracker.

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Pivotal Tracker API token

class dagster_airbyte.managed.generated.sources.ElasticsearchSource(name, endpoint, authenticationMethod)[source]
__init__(name, endpoint, authenticationMethod)[source]

Airbyte Source for Elasticsearch.

Documentation can be found at https://docs.airbyte.com/integrations/source/elasticsearch

Parameters:
class ElasticsearchSource.None_[source]
__init__()[source]
class ElasticsearchSource.ApiKeySecret(apiKeyId, apiKeySecret)[source]
__init__(apiKeyId, apiKeySecret)[source]
class ElasticsearchSource.UsernamePassword(username, password)[source]
__init__(username, password)[source]
class dagster_airbyte.managed.generated.sources.BigquerySource(name, project_id, credentials_json, dataset_id=None)[source]
__init__(name, project_id, credentials_json, dataset_id=None)[source]

Airbyte Source for Bigquery.

Documentation can be found at https://docs.airbyte.com/integrations/sources/bigquery

Parameters:
  • name (str) – The name of the destination.

  • project_id (str) – The GCP project ID for the project containing the target BigQuery dataset.

  • dataset_id (Optional[str]) – The dataset ID to search for tables and views. If you are only loading data from one dataset, setting this option could result in much faster schema discovery.

  • credentials_json (str) – The contents of your Service Account Key JSON file. See the docs for more information on how to obtain this key.

class dagster_airbyte.managed.generated.sources.WoocommerceSource(name, shop, start_date, api_key, api_secret, conversion_window_days=None)[source]
__init__(name, shop, start_date, api_key, api_secret, conversion_window_days=None)[source]

Airbyte Source for Woocommerce.

Documentation can be found at https://docs.airbyte.com/integrations/sources/woocommerce

Parameters:
  • name (str) – The name of the destination.

  • shop (str) – The name of the store. For https://EXAMPLE.com, the shop name is ‘EXAMPLE.com’.

  • start_date (str) – The date you would like to replicate data. Format: YYYY-MM-DD.

  • api_key (str) – The CUSTOMER KEY for API in WooCommerce shop.

  • api_secret (str) – The CUSTOMER SECRET for API in WooCommerce shop.

  • conversion_window_days (Optional[int]) – A conversion window is the period of time after an ad interaction (such as an ad click or video view) during which a conversion, such as a purchase, is recorded in Google Ads.

class dagster_airbyte.managed.generated.sources.SearchMetricsSource(name, api_key, client_secret, country_code, start_date)[source]
__init__(name, api_key, client_secret, country_code, start_date)[source]

Airbyte Source for Search Metrics.

Documentation can be found at https://docs.airbyte.com/integrations/sources/seacrh-metrics

Parameters:
  • name (str) – The name of the destination.

  • country_code (str) – The region of the S3 staging bucket to use if utilising a copy strategy.

  • start_date (str) – Data generated in SearchMetrics after this date will be replicated. This date must be specified in the format YYYY-MM-DDT00:00:00Z.

class dagster_airbyte.managed.generated.sources.TypeformSource(name, start_date, token, form_ids=None)[source]
__init__(name, start_date, token, form_ids=None)[source]

Airbyte Source for Typeform.

Documentation can be found at https://docs.airbyte.com/integrations/sources/typeform

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date and time in the format: YYYY-MM-DDTHH:mm:ss[Z]. Any data before this date will not be replicated.

  • token (str) – The API Token for a Typeform account.

  • form_ids (Optional[List[str]]) – When this parameter is set, the connector will replicate data only from the input forms. Otherwise, all forms in your Typeform account will be replicated. You can find form IDs in your form URLs. For example, in the URL “https://mysite.typeform.com/to/u6nXL7” the form_id is u6nXL7. You can find form URLs on Share panel

class dagster_airbyte.managed.generated.sources.WebflowSource(name, site_id, api_key)[source]
__init__(name, site_id, api_key)[source]

Airbyte Source for Webflow.

Documentation can be found at https://docs.airbyte.com/integrations/sources/webflow

Parameters:
class dagster_airbyte.managed.generated.sources.FireboltSource(name, username, password, database, account=None, host=None, engine=None)[source]
__init__(name, username, password, database, account=None, host=None, engine=None)[source]

Airbyte Source for Firebolt.

Documentation can be found at https://docs.airbyte.com/integrations/sources/firebolt

Parameters:
  • name (str) – The name of the destination.

  • username (str) – Firebolt email address you use to login.

  • password (str) – Firebolt password.

  • account (Optional[str]) – Firebolt account to login.

  • host (Optional[str]) – The host name of your Firebolt database.

  • database (str) – The database to connect to.

  • engine (Optional[str]) – Engine name or url to connect to.

class dagster_airbyte.managed.generated.sources.FaunaSource(name, domain, port, scheme, secret, collection)[source]
__init__(name, domain, port, scheme, secret, collection)[source]

Airbyte Source for Fauna.

Documentation can be found at https://github.com/fauna/airbyte/blob/source-fauna/docs/integrations/sources/fauna.md

Parameters:
  • name (str) – The name of the destination.

  • domain (str) – Domain of Fauna to query. Defaults db.fauna.com. See the docs.

  • port (int) – Endpoint port.

  • scheme (str) – URL scheme.

  • secret (str) – Fauna secret, used when authenticating with the database.

  • collection (FaunaSource.Collection) – Settings for the Fauna Collection.

class FaunaSource.Disabled[source]
__init__()[source]
class FaunaSource.Enabled(column)[source]
__init__(column)[source]
class FaunaSource.Collection(page_size, deletions)[source]
__init__(page_size, deletions)[source]
class dagster_airbyte.managed.generated.sources.IntercomSource(name, start_date, access_token)[source]
__init__(name, start_date, access_token)[source]

Airbyte Source for Intercom.

Documentation can be found at https://docs.airbyte.com/integrations/sources/intercom

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • access_token (str) – Access token for making authenticated requests. See the Intercom docs for more information.

class dagster_airbyte.managed.generated.sources.FreshsalesSource(name, domain_name, api_key)[source]
__init__(name, domain_name, api_key)[source]

Airbyte Source for Freshsales.

Documentation can be found at https://docs.airbyte.com/integrations/sources/freshsales

Parameters:
  • name (str) – The name of the destination.

  • domain_name (str) – The Name of your Freshsales domain

  • api_key (str) – Freshsales API Key. See here. The key is case sensitive.

class dagster_airbyte.managed.generated.sources.AdjustSource(name, api_token, dimensions, ingest_start, metrics, additional_metrics=None, until_today=None)[source]
__init__(name, api_token, dimensions, ingest_start, metrics, additional_metrics=None, until_today=None)[source]

Airbyte Source for Adjust.

Documentation can be found at https://docs.airbyte.com/integrations/sources/adjust

Parameters:
  • name (str) – The name of the destination.

  • additional_metrics (Optional[List[str]]) – Metrics names that are not pre-defined, such as cohort metrics or app specific metrics.

  • api_token (str) – Adjust API key, see https://help.adjust.com/en/article/report-service-api-authentication

  • dimensions (List[str]) – Dimensions allow a user to break down metrics into groups using one or several parameters. For example, the number of installs by date, country and network. See https://help.adjust.com/en/article/reports-endpoint#dimensions for more information about the dimensions.

  • ingest_start (str) – Data ingest start date.

  • metrics (List[str]) – Select at least one metric to query.

  • until_today (Optional[bool]) – Syncs data up until today. Useful when running daily incremental syncs, and duplicates are not desired.

class dagster_airbyte.managed.generated.sources.BambooHrSource(name, subdomain, api_key, custom_reports_fields=None, custom_reports_include_default_fields=None)[source]
__init__(name, subdomain, api_key, custom_reports_fields=None, custom_reports_include_default_fields=None)[source]

Airbyte Source for Bamboo Hr.

Documentation can be found at https://docs.airbyte.com/integrations/sources/bamboo-hr

Parameters:
  • name (str) – The name of the destination.

  • subdomain (str) – Sub Domain of bamboo hr

  • api_key (str) – Api key of bamboo hr

  • custom_reports_fields (Optional[str]) – Comma-separated list of fields to include in custom reports.

  • custom_reports_include_default_fields (Optional[bool]) – If true, the custom reports endpoint will include the default fields defined here: https://documentation.bamboohr.com/docs/list-of-field-names.

class dagster_airbyte.managed.generated.sources.GoogleAdsSource(name, credentials, customer_id, start_date, end_date=None, custom_queries=None, login_customer_id=None, conversion_window_days=None)[source]
__init__(name, credentials, customer_id, start_date, end_date=None, custom_queries=None, login_customer_id=None, conversion_window_days=None)[source]

Airbyte Source for Google Ads.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-ads

Parameters:
  • name (str) – The name of the destination.

  • customer_id (str) – Comma separated list of (client) customer IDs. Each customer ID must be specified as a 10-digit number without dashes. More instruction on how to find this value in our docs. Metrics streams like AdGroupAdReport cannot be requested for a manager account.

  • start_date (str) – UTC date and time in the format 2017-01-25. Any data before this date will not be replicated.

  • end_date (Optional[str]) – UTC date and time in the format 2017-01-25. Any data after this date will not be replicated.

  • login_customer_id (Optional[str]) – If your access to the customer account is through a manager account, this field is required and must be set to the customer ID of the manager account (10-digit number without dashes). More information about this field you can see here

  • conversion_window_days (Optional[int]) – A conversion window is the period of time after an ad interaction (such as an ad click or video view) during which a conversion, such as a purchase, is recorded in Google Ads. For more information, see Google’s documentation.

class GoogleAdsSource.GoogleCredentials(developer_token, client_id, client_secret, refresh_token, access_token=None)[source]
__init__(developer_token, client_id, client_secret, refresh_token, access_token=None)[source]
class GoogleAdsSource.CustomGAQLQueriesEntry(query, table_name)[source]
__init__(query, table_name)[source]
class dagster_airbyte.managed.generated.sources.HellobatonSource(name, api_key, company)[source]
__init__(name, api_key, company)[source]

Airbyte Source for Hellobaton.

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – authentication key required to access the api endpoints

  • company (str) – Company name that generates your base api url

class dagster_airbyte.managed.generated.sources.SendgridSource(name, apikey, start_time)[source]
__init__(name, apikey, start_time)[source]

Airbyte Source for Sendgrid.

Documentation can be found at https://docs.airbyte.com/integrations/sources/sendgrid

Parameters:
  • name (str) – The name of the destination.

  • apikey (str) – API Key, use admin to generate this key.

  • start_time (Union[int, str]) – Start time in ISO8601 format. Any data before this time point will not be replicated.

class dagster_airbyte.managed.generated.sources.MondaySource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Monday.

Documentation can be found at https://docs.airbyte.com/integrations/sources/monday

Parameters:

name (str) – The name of the destination.

class MondaySource.OAuth20(client_id, client_secret, access_token, subdomain=None)[source]
__init__(client_id, client_secret, access_token, subdomain=None)[source]
class MondaySource.APIToken(api_token)[source]
__init__(api_token)[source]
class dagster_airbyte.managed.generated.sources.DixaSource(name, api_token, start_date, batch_size=None)[source]
__init__(name, api_token, start_date, batch_size=None)[source]

Airbyte Source for Dixa.

Documentation can be found at https://docs.airbyte.com/integrations/sources/dixa

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Dixa API token

  • start_date (str) – The connector pulls records updated from this date onwards.

  • batch_size (Optional[int]) – Number of days to batch into one request. Max 31.

class dagster_airbyte.managed.generated.sources.SalesforceSource(name, client_id, client_secret, refresh_token, is_sandbox=None, auth_type=None, start_date=None, streams_criteria=None)[source]
__init__(name, client_id, client_secret, refresh_token, is_sandbox=None, auth_type=None, start_date=None, streams_criteria=None)[source]

Airbyte Source for Salesforce.

Documentation can be found at https://docs.airbyte.com/integrations/sources/salesforce

Parameters:
  • name (str) – The name of the destination.

  • is_sandbox (Optional[bool]) – Toggle if you’re using a Salesforce Sandbox

  • client_id (str) – Enter your Salesforce developer application’s Client ID

  • client_secret (str) – Enter your Salesforce developer application’s Client secret

  • refresh_token (str) – Enter your application’s Salesforce Refresh Token used for Airbyte to access your Salesforce account.

  • start_date (Optional[str]) – Enter the date in the YYYY-MM-DD format. Airbyte will replicate the data added on and after this date. If this field is blank, Airbyte will replicate all data.

  • streams_criteria (Optional[List[SalesforceSource.FilterSalesforceObjectsEntry]]) – Filter streams relevant to you

class SalesforceSource.FilterSalesforceObjectsEntry(criteria, value)[source]
__init__(criteria, value)[source]
class dagster_airbyte.managed.generated.sources.PipedriveSource(name, authorization, replication_start_date)[source]
__init__(name, authorization, replication_start_date)[source]

Airbyte Source for Pipedrive.

Documentation can be found at https://docs.airbyte.com/integrations/sources/pipedrive

Parameters:
  • name (str) – The name of the destination.

  • authorization (Union[PipedriveSource.SignInViaPipedriveOAuth, PipedriveSource.APIKeyAuthentication]) – Choose one of the possible authorization method

  • replication_start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. When specified and not None, then stream will behave as incremental

class PipedriveSource.SignInViaPipedriveOAuth(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class PipedriveSource.APIKeyAuthentication(api_token)[source]
__init__(api_token)[source]
class dagster_airbyte.managed.generated.sources.FileSource(name, dataset_name, format, url, provider, reader_options=None)[source]
__init__(name, dataset_name, format, url, provider, reader_options=None)[source]

Airbyte Source for File.

Documentation can be found at https://docs.airbyte.com/integrations/sources/file

Parameters:
class FileSource.HTTPSPublicWeb(user_agent=None)[source]
__init__(user_agent=None)[source]
class FileSource.GCSGoogleCloudStorage(service_account_json=None)[source]
__init__(service_account_json=None)[source]
class FileSource.S3AmazonWebServices(aws_access_key_id=None, aws_secret_access_key=None)[source]
__init__(aws_access_key_id=None, aws_secret_access_key=None)[source]
class FileSource.AzBlobAzureBlobStorage(storage_account, sas_token=None, shared_key=None)[source]
__init__(storage_account, sas_token=None, shared_key=None)[source]
class FileSource.SSHSecureShell(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class FileSource.SCPSecureCopyProtocol(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class FileSource.SFTPSecureFileTransferProtocol(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class FileSource.LocalFilesystemLimited[source]
__init__()[source]
class dagster_airbyte.managed.generated.sources.GlassfrogSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Glassfrog.

Documentation can be found at https://docs.airbyte.com/integrations/sources/glassfrog

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – API key provided by Glassfrog

class dagster_airbyte.managed.generated.sources.ChartmogulSource(name, api_key, start_date, interval)[source]
__init__(name, api_key, start_date, interval)[source]

Airbyte Source for Chartmogul.

Documentation can be found at https://docs.airbyte.com/integrations/sources/chartmogul

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Chartmogul API key

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. When feasible, any data before this date will not be replicated.

  • interval (str) – Some APIs such as Metrics require intervals to cluster data.

class dagster_airbyte.managed.generated.sources.OrbSource(name, api_key, start_date=None, lookback_window_days=None, string_event_properties_keys=None, numeric_event_properties_keys=None)[source]
__init__(name, api_key, start_date=None, lookback_window_days=None, string_event_properties_keys=None, numeric_event_properties_keys=None)[source]

Airbyte Source for Orb.

Documentation can be found at https://docs.withorb.com/

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Orb API Key, issued from the Orb admin console.

  • start_date (Optional[str]) – UTC date and time in the format 2022-03-01T00:00:00Z. Any data with created_at before this data will not be synced.

  • lookback_window_days (Optional[int]) – When set to N, the connector will always refresh resources created within the past N days. By default, updated objects that are not newly created are not incrementally synced.

  • string_event_properties_keys (Optional[List[str]]) – Property key names to extract from all events, in order to enrich ledger entries corresponding to an event deduction.

  • numeric_event_properties_keys (Optional[List[str]]) – Property key names to extract from all events, in order to enrich ledger entries corresponding to an event deduction.

class dagster_airbyte.managed.generated.sources.CockroachdbSource(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]
__init__(name, host, port, database, username, password=None, jdbc_url_params=None, ssl=None)[source]

Airbyte Source for Cockroachdb.

Documentation can be found at https://docs.airbyte.com/integrations/sources/cockroachdb

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Hostname of the database.

  • port (int) – Port of the database.

  • database (str) – Name of the database.

  • username (str) – Username to use to access the database.

  • password (Optional[str]) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (Eg. key1=value1&key2=value2&key3=value3). For more information read about JDBC URL parameters.

  • ssl (Optional[bool]) – Encrypt client/server communications for increased security.

class dagster_airbyte.managed.generated.sources.ConfluenceSource(name, api_token, domain_name, email)[source]
__init__(name, api_token, domain_name, email)[source]

Airbyte Source for Confluence.

Parameters:
class dagster_airbyte.managed.generated.sources.PlaidSource(name, access_token, api_key, client_id, plaid_env, start_date=None)[source]
__init__(name, access_token, api_key, client_id, plaid_env, start_date=None)[source]

Airbyte Source for Plaid.

Documentation can be found at https://plaid.com/docs/api/

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – The end-user’s Link access token.

  • api_key (str) – The Plaid API key to use to hit the API.

  • client_id (str) – The Plaid client id

  • plaid_env (str) – The Plaid environment

  • start_date (Optional[str]) – The date from which you’d like to replicate data for Plaid in the format YYYY-MM-DD. All data generated after this date will be replicated.

class dagster_airbyte.managed.generated.sources.SnapchatMarketingSource(name, client_id, client_secret, refresh_token, start_date=None, end_date=None)[source]
__init__(name, client_id, client_secret, refresh_token, start_date=None, end_date=None)[source]

Airbyte Source for Snapchat Marketing.

Documentation can be found at https://docs.airbyte.com/integrations/sources/snapchat-marketing

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The Client ID of your Snapchat developer application.

  • client_secret (str) – The Client Secret of your Snapchat developer application.

  • refresh_token (str) – Refresh Token to renew the expired Access Token.

  • start_date (Optional[str]) – Date in the format 2022-01-01. Any data before this date will not be replicated.

  • end_date (Optional[str]) – Date in the format 2017-01-25. Any data after this date will not be replicated.

class dagster_airbyte.managed.generated.sources.MicrosoftTeamsSource(name, period, credentials)[source]
__init__(name, period, credentials)[source]

Airbyte Source for Microsoft Teams.

Documentation can be found at https://docs.airbyte.com/integrations/sources/microsoft-teams

Parameters:
class MicrosoftTeamsSource.AuthenticateViaMicrosoftOAuth20(tenant_id, client_id, client_secret, refresh_token, auth_type=None)[source]
__init__(tenant_id, client_id, client_secret, refresh_token, auth_type=None)[source]
class MicrosoftTeamsSource.AuthenticateViaMicrosoft(tenant_id, client_id, client_secret, auth_type=None)[source]
__init__(tenant_id, client_id, client_secret, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.LeverHiringSource(name, credentials, start_date, environment=None)[source]
__init__(name, credentials, start_date, environment=None)[source]

Airbyte Source for Lever Hiring.

Documentation can be found at https://docs.airbyte.com/integrations/sources/lever-hiring

Parameters:
  • name (str) – The name of the destination.

  • credentials (LeverHiringSource.OAuthCredentials) – Choose how to authenticate to Lever Hiring.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. Note that it will be used only in the following incremental streams: comments, commits, and issues.

  • environment (Optional[str]) – The environment in which you’d like to replicate data for Lever. This is used to determine which Lever API endpoint to use.

class LeverHiringSource.OAuthCredentials(refresh_token, auth_type=None, client_id=None, client_secret=None)[source]
__init__(refresh_token, auth_type=None, client_id=None, client_secret=None)[source]
class dagster_airbyte.managed.generated.sources.TwilioSource(name, account_sid, auth_token, start_date, lookback_window=None)[source]
__init__(name, account_sid, auth_token, start_date, lookback_window=None)[source]

Airbyte Source for Twilio.

Documentation can be found at https://docs.airbyte.com/integrations/sources/twilio

Parameters:
  • name (str) – The name of the destination.

  • account_sid (str) – Twilio account SID

  • auth_token (str) – Twilio Auth Token.

  • start_date (str) – UTC date and time in the format 2020-10-01T00:00:00Z. Any data before this date will not be replicated.

  • lookback_window (Optional[int]) – How far into the past to look for records. (in minutes)

class dagster_airbyte.managed.generated.sources.StripeSource(name, account_id, client_secret, start_date, lookback_window_days=None, slice_range=None)[source]
__init__(name, account_id, client_secret, start_date, lookback_window_days=None, slice_range=None)[source]

Airbyte Source for Stripe.

Documentation can be found at https://docs.airbyte.com/integrations/sources/stripe

Parameters:
  • name (str) – The name of the destination.

  • account_id (str) – Your Stripe account ID (starts with ‘acct\_’, find yours here).

  • client_secret (str) – Stripe API key (usually starts with ‘sk_live\_’; find yours here).

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Only data generated after this date will be replicated.

  • lookback_window_days (Optional[int]) – When set, the connector will always re-export data from the past N days, where N is the value set here. This is useful if your data is frequently updated after creation. More info here

  • slice_range (Optional[int]) – The time increment used by the connector when requesting data from the Stripe API. The bigger the value is, the less requests will be made and faster the sync will be. On the other hand, the more seldom the state is persisted.

class dagster_airbyte.managed.generated.sources.Db2Source(name, host, port, db, username, password, encryption, jdbc_url_params=None)[source]
__init__(name, host, port, db, username, password, encryption, jdbc_url_params=None)[source]

Airbyte Source for Db2.

Documentation can be found at https://docs.airbyte.com/integrations/sources/db2

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Host of the Db2.

  • port (int) – Port of the database.

  • db (str) – Name of the database.

  • username (str) – Username to use to access the database.

  • password (str) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

  • encryption (Union[Db2Source.Unencrypted, Db2Source.TLSEncryptedVerifyCertificate]) – Encryption method to use when communicating with the database

class Db2Source.Unencrypted[source]
__init__()[source]
class Db2Source.TLSEncryptedVerifyCertificate(ssl_certificate, key_store_password=None)[source]
__init__(ssl_certificate, key_store_password=None)[source]
class dagster_airbyte.managed.generated.sources.SlackSource(name, start_date, lookback_window, join_channels, credentials, channel_filter=None)[source]
__init__(name, start_date, lookback_window, join_channels, credentials, channel_filter=None)[source]

Airbyte Source for Slack.

Documentation can be found at https://docs.airbyte.com/integrations/sources/slack

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • lookback_window (int) – How far into the past to look for messages in threads.

  • join_channels (bool) – Whether to join all channels or to sync data only from channels the bot is already in. If false, you’ll need to manually add the bot to all the channels from which you’d like to sync messages.

  • channel_filter (Optional[List[str]]) – A channel name list (without leading ‘#’ char) which limit the channels from which you’d like to sync. Empty list means no filter.

  • credentials (Union[SlackSource.DefaultOAuth20Authorization, SlackSource.APITokenCredentials]) – Choose how to authenticate into Slack

class SlackSource.DefaultOAuth20Authorization(client_id, client_secret, access_token, refresh_token=None)[source]
__init__(client_id, client_secret, access_token, refresh_token=None)[source]
class SlackSource.APITokenCredentials(api_token)[source]
__init__(api_token)[source]
class dagster_airbyte.managed.generated.sources.RechargeSource(name, start_date, access_token)[source]
__init__(name, start_date, access_token)[source]

Airbyte Source for Recharge.

Documentation can be found at https://docs.airbyte.com/integrations/sources/recharge

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate data for Recharge API, in the format YYYY-MM-DDT00:00:00Z. Any data before this date will not be replicated.

  • access_token (str) – The value of the Access Token generated. See the docs for more information.

class dagster_airbyte.managed.generated.sources.OpenweatherSource(name, lat, lon, appid, units=None, lang=None)[source]
__init__(name, lat, lon, appid, units=None, lang=None)[source]

Airbyte Source for Openweather.

Parameters:
  • name (str) – The name of the destination.

  • lat (str) – Latitude for which you want to get weather condition from. (min -90, max 90)

  • lon (str) – Longitude for which you want to get weather condition from. (min -180, max 180)

  • appid (str) – Your OpenWeather API Key. See here. The key is case sensitive.

  • units (Optional[str]) – Units of measurement. standard, metric and imperial units are available. If you do not use the units parameter, standard units will be applied by default.

  • lang (Optional[str]) – You can use lang parameter to get the output in your language. The contents of the description field will be translated. See here for the list of supported languages.

class dagster_airbyte.managed.generated.sources.RetentlySource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Retently.

Parameters:
class RetentlySource.AuthenticateViaRetentlyOAuth(client_id, client_secret, refresh_token, auth_type=None)[source]
__init__(client_id, client_secret, refresh_token, auth_type=None)[source]
class RetentlySource.AuthenticateWithAPIToken(api_key, auth_type=None)[source]
__init__(api_key, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.ScaffoldSourceHttpSource(name, TODO)[source]
__init__(name, TODO)[source]

Airbyte Source for Scaffold Source Http.

Parameters:
  • name (str) – The name of the destination.

  • TODO (str) – describe me

class dagster_airbyte.managed.generated.sources.YandexMetricaSource(name, auth_token, counter_id, start_date, end_date)[source]
__init__(name, auth_token, counter_id, start_date, end_date)[source]

Airbyte Source for Yandex Metrica.

Parameters:
  • name (str) – The name of the destination.

  • auth_token (str) – Your Yandex Metrica API access token

  • counter_id (str) – Counter ID

  • start_date (str) – UTC date and time in the format YYYY-MM-DD.

  • end_date (str) – UTC date and time in the format YYYY-MM-DD.

class dagster_airbyte.managed.generated.sources.TalkdeskExploreSource(name, start_date, auth_url, api_key, timezone=None)[source]
__init__(name, start_date, auth_url, api_key, timezone=None)[source]

Airbyte Source for Talkdesk Explore.

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate data for Talkdesk Explore API, in the format YYYY-MM-DDT00:00:00. All data generated after this date will be replicated.

  • timezone (Optional[str]) – Timezone to use when generating reports. Only IANA timezones are supported (https://nodatime.org/TimeZones)

  • auth_url (str) – Talkdesk Auth URL. Only ‘client_credentials’ auth type supported at the moment.

  • api_key (str) – Talkdesk API key.

class dagster_airbyte.managed.generated.sources.ChargifySource(name, api_key, domain)[source]
__init__(name, api_key, domain)[source]

Airbyte Source for Chargify.

Documentation can be found at https://docs.airbyte.com/integrations/sources/chargify

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Chargify API Key.

  • domain (str) – Chargify domain. Normally this domain follows the following format companyname.chargify.com

class dagster_airbyte.managed.generated.sources.RkiCovidSource(name, start_date)[source]
__init__(name, start_date)[source]

Airbyte Source for Rki Covid.

Documentation can be found at https://docs.airbyte.com/integrations/sources/rki-covid

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date in the format 2017-01-25. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.PostgresSource(name, host, port, database, username, ssl_mode, replication_method, tunnel_method, schemas=None, password=None, jdbc_url_params=None, ssl=None)[source]
__init__(name, host, port, database, username, ssl_mode, replication_method, tunnel_method, schemas=None, password=None, jdbc_url_params=None, ssl=None)[source]

Airbyte Source for Postgres.

Documentation can be found at https://docs.airbyte.com/integrations/sources/postgres

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Hostname of the database.

  • port (int) – Port of the database.

  • database (str) – Name of the database.

  • schemas (Optional[List[str]]) – The list of schemas (case sensitive) to sync from. Defaults to public.

  • username (str) – Username to access the database.

  • password (Optional[str]) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (Eg. key1=value1&key2=value2&key3=value3). For more information read about JDBC URL parameters.

  • ssl (Optional[bool]) – Encrypt data using SSL. When activating SSL, please select one of the connection modes.

  • ssl_mode (Union[PostgresSource.Disable, PostgresSource.Allow, PostgresSource.Prefer, PostgresSource.Require, PostgresSource.VerifyCa, PostgresSource.VerifyFull]) – SSL connection modes. disable - Disables encryption of communication between Airbyte and source database allow - Enables encryption only when required by the source database prefer - allows unencrypted connection only if the source database does not support encryption require - Always require encryption. If the source database server does not support encryption, connection will fail verify-ca - Always require encryption and verifies that the source database server has a valid SSL certificate verify-full - This is the most secure mode. Always require encryption and verifies the identity of the source database server Read more in the docs.

  • replication_method (Union[PostgresSource.Standard, PostgresSource.LogicalReplicationCDC]) – Replication method for extracting data from the database.

  • tunnel_method (Union[PostgresSource.NoTunnel, PostgresSource.SSHKeyAuthentication, PostgresSource.PasswordAuthentication]) – Whether to initiate an SSH tunnel before connecting to the database, and if so, which kind of authentication to use.

class PostgresSource.Disable[source]
__init__()[source]
class PostgresSource.Allow[source]
__init__()[source]
class PostgresSource.Prefer[source]
__init__()[source]
class PostgresSource.Require[source]
__init__()[source]
class PostgresSource.VerifyCa(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
__init__(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
class PostgresSource.VerifyFull(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
__init__(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
class PostgresSource.Standard[source]
__init__()[source]
class PostgresSource.LogicalReplicationCDC(replication_slot, publication, plugin=None, initial_waiting_seconds=None)[source]
__init__(replication_slot, publication, plugin=None, initial_waiting_seconds=None)[source]
class PostgresSource.NoTunnel[source]
__init__()[source]
class PostgresSource.SSHKeyAuthentication(tunnel_host, tunnel_port, tunnel_user, ssh_key)[source]
__init__(tunnel_host, tunnel_port, tunnel_user, ssh_key)[source]
class PostgresSource.PasswordAuthentication(tunnel_host, tunnel_port, tunnel_user, tunnel_user_password)[source]
__init__(tunnel_host, tunnel_port, tunnel_user, tunnel_user_password)[source]
class dagster_airbyte.managed.generated.sources.TrelloSource(name, token, key, start_date, board_ids=None)[source]
__init__(name, token, key, start_date, board_ids=None)[source]

Airbyte Source for Trello.

Documentation can be found at https://docs.airbyte.com/integrations/sources/trello

Parameters:
  • name (str) – The name of the destination.

  • token (str) – Trello v API token. See the docs for instructions on how to generate it.

  • key (str) – Trello API key. See the docs for instructions on how to generate it.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • board_ids (Optional[List[str]]) – IDs of the boards to replicate data from. If left empty, data from all boards to which you have access will be replicated.

class dagster_airbyte.managed.generated.sources.PrestashopSource(name, url, access_key)[source]
__init__(name, url, access_key)[source]

Airbyte Source for Prestashop.

Parameters:
  • name (str) – The name of the destination.

  • url (str) – Shop URL without trailing slash (domain name or IP address)

  • access_key (str) – Your PrestaShop access key. See the docs for info on how to obtain this.

class dagster_airbyte.managed.generated.sources.PaystackSource(name, secret_key, start_date, lookback_window_days=None)[source]
__init__(name, secret_key, start_date, lookback_window_days=None)[source]

Airbyte Source for Paystack.

Documentation can be found at https://docs.airbyte.com/integrations/sources/paystack

Parameters:
  • name (str) – The name of the destination.

  • secret_key (str) – The Paystack API key (usually starts with ‘sk_live\_’; find yours here).

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • lookback_window_days (Optional[int]) – When set, the connector will always reload data from the past N days, where N is the value set here. This is useful if your data is updated after creation.

class dagster_airbyte.managed.generated.sources.S3Source(name, dataset, path_pattern, format, provider, schema=None)[source]
__init__(name, dataset, path_pattern, format, provider, schema=None)[source]

Airbyte Source for S3.

Documentation can be found at https://docs.airbyte.com/integrations/sources/s3

Parameters:
  • name (str) – The name of the destination.

  • dataset (str) – The name of the stream you would like this source to output. Can contain letters, numbers, or underscores.

  • path_pattern (str) – A regular expression which tells the connector which files to replicate. All files which match this pattern will be replicated. Use | to separate multiple patterns. See this page to understand pattern syntax (GLOBSTAR and SPLIT flags are enabled). Use pattern ** to pick up all files.

  • format (Union[S3Source.CSV, S3Source.Parquet, S3Source.Avro, S3Source.Jsonl]) – The format of the files you’d like to replicate

  • schema (Optional[str]) – Optionally provide a schema to enforce, as a valid JSON string. Ensure this is a mapping of { “column” : “type” }, where types are valid JSON Schema datatypes. Leave as {} to auto-infer the schema.

  • provider (S3Source.S3AmazonWebServices) – Use this to load files from S3 or S3-compatible services

class S3Source.CSV(filetype=None, delimiter=None, infer_datatypes=None, quote_char=None, escape_char=None, encoding=None, double_quote=None, newlines_in_values=None, additional_reader_options=None, advanced_options=None, block_size=None)[source]
__init__(filetype=None, delimiter=None, infer_datatypes=None, quote_char=None, escape_char=None, encoding=None, double_quote=None, newlines_in_values=None, additional_reader_options=None, advanced_options=None, block_size=None)[source]
class S3Source.Parquet(filetype=None, columns=None, batch_size=None, buffer_size=None)[source]
__init__(filetype=None, columns=None, batch_size=None, buffer_size=None)[source]
class S3Source.Avro(filetype=None)[source]
__init__(filetype=None)[source]
class S3Source.Jsonl(filetype=None, newlines_in_values=None, unexpected_field_behavior=None, block_size=None)[source]
__init__(filetype=None, newlines_in_values=None, unexpected_field_behavior=None, block_size=None)[source]
class S3Source.S3AmazonWebServices(bucket, aws_access_key_id=None, aws_secret_access_key=None, path_prefix=None, endpoint=None)[source]
__init__(bucket, aws_access_key_id=None, aws_secret_access_key=None, path_prefix=None, endpoint=None)[source]
class dagster_airbyte.managed.generated.sources.SnowflakeSource(name, credentials, host, role, warehouse, database, schema, jdbc_url_params=None)[source]
__init__(name, credentials, host, role, warehouse, database, schema, jdbc_url_params=None)[source]

Airbyte Source for Snowflake.

Documentation can be found at https://docs.airbyte.com/integrations/sources/snowflake

Parameters:
  • name (str) – The name of the destination.

  • host (str) – The host domain of the snowflake instance (must include the account, region, cloud environment, and end with snowflakecomputing.com).

  • role (str) – The role you created for Airbyte to access Snowflake.

  • warehouse (str) – The warehouse you created for Airbyte to access data.

  • database (str) – The database you created for Airbyte to access data.

  • schema (str) – The source Snowflake schema tables.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

class SnowflakeSource.OAuth20(client_id, client_secret, access_token=None, refresh_token=None)[source]
__init__(client_id, client_secret, access_token=None, refresh_token=None)[source]
class SnowflakeSource.UsernameAndPassword(username, password)[source]
__init__(username, password)[source]
class dagster_airbyte.managed.generated.sources.AmplitudeSource(name, api_key, secret_key, start_date)[source]
__init__(name, api_key, secret_key, start_date)[source]

Airbyte Source for Amplitude.

Documentation can be found at https://docs.airbyte.com/integrations/sources/amplitude

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Amplitude API Key. See the setup guide for more information on how to obtain this key.

  • secret_key (str) – Amplitude Secret Key. See the setup guide for more information on how to obtain this key.

  • start_date (str) – UTC date and time in the format 2021-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.PosthogSource(name, start_date, api_key, base_url=None)[source]
__init__(name, start_date, api_key, base_url=None)[source]

Airbyte Source for Posthog.

Documentation can be found at https://docs.airbyte.com/integrations/sources/posthog

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate the data. Any data before this date will not be replicated.

  • api_key (str) – API Key. See the docs for information on how to generate this key.

  • base_url (Optional[str]) – Base PostHog url. Defaults to PostHog Cloud (https://app.posthog.com).

class dagster_airbyte.managed.generated.sources.PaypalTransactionSource(name, start_date, is_sandbox, client_id=None, client_secret=None, refresh_token=None)[source]
__init__(name, start_date, is_sandbox, client_id=None, client_secret=None, refresh_token=None)[source]

Airbyte Source for Paypal Transaction.

Documentation can be found at https://docs.airbyte.com/integrations/sources/paypal-transactions

Parameters:
  • name (str) – The name of the destination.

  • client_id (Optional[str]) – The Client ID of your Paypal developer application.

  • client_secret (Optional[str]) – The Client Secret of your Paypal developer application.

  • refresh_token (Optional[str]) – The key to refresh the expired access token.

  • start_date (str) – Start Date for data extraction in ISO format. Date must be in range from 3 years till 12 hrs before present time.

  • is_sandbox (bool) – Determines whether to use the sandbox or production environment.

class dagster_airbyte.managed.generated.sources.MssqlSource(name, host, port, database, username, ssl_method, replication_method, schemas=None, password=None, jdbc_url_params=None)[source]
__init__(name, host, port, database, username, ssl_method, replication_method, schemas=None, password=None, jdbc_url_params=None)[source]

Airbyte Source for Mssql.

Documentation can be found at https://docs.airbyte.com/integrations/destinations/mssql

Parameters:
  • name (str) – The name of the destination.

  • host (str) – The hostname of the database.

  • port (int) – The port of the database.

  • database (str) – The name of the database.

  • schemas (Optional[List[str]]) – The list of schemas to sync from. Defaults to user. Case sensitive.

  • username (str) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

  • ssl_method (Union[MssqlSource.Unencrypted, MssqlSource.EncryptedTrustServerCertificate, MssqlSource.EncryptedVerifyCertificate]) – The encryption method which is used when communicating with the database.

  • replication_method (Union[MssqlSource.Standard, MssqlSource.LogicalReplicationCDC]) – The replication method used for extracting data from the database. STANDARD replication requires no setup on the DB side but will not be able to represent deletions incrementally. CDC uses {TBC} to detect inserts, updates, and deletes. This needs to be configured on the source database itself.

class MssqlSource.Unencrypted[source]
__init__()[source]
class MssqlSource.EncryptedTrustServerCertificate[source]
__init__()[source]
class MssqlSource.EncryptedVerifyCertificate(hostNameInCertificate=None)[source]
__init__(hostNameInCertificate=None)[source]
class MssqlSource.Standard[source]
__init__()[source]
class MssqlSource.LogicalReplicationCDC(data_to_sync=None, snapshot_isolation=None)[source]
__init__(data_to_sync=None, snapshot_isolation=None)[source]
class dagster_airbyte.managed.generated.sources.ZohoCrmSource(name, client_id, client_secret, refresh_token, dc_region, environment, edition, start_datetime=None)[source]
__init__(name, client_id, client_secret, refresh_token, dc_region, environment, edition, start_datetime=None)[source]

Airbyte Source for Zoho Crm.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zoho-crm

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – OAuth2.0 Client ID

  • client_secret (str) – OAuth2.0 Client Secret

  • refresh_token (str) – OAuth2.0 Refresh Token

  • dc_region (str) – Please choose the region of your Data Center location. More info by this Link

  • environment (str) – Please choose the environment

  • start_datetime (Optional[str]) – ISO 8601, for instance: YYYY-MM-DD, YYYY-MM-DD HH:MM:SS+HH:MM

  • edition (str) – Choose your Edition of Zoho CRM to determine API Concurrency Limits

class dagster_airbyte.managed.generated.sources.RedshiftSource(name, host, port, database, username, password, schemas=None, jdbc_url_params=None)[source]
__init__(name, host, port, database, username, password, schemas=None, jdbc_url_params=None)[source]

Airbyte Source for Redshift.

Documentation can be found at https://docs.airbyte.com/integrations/destinations/redshift

Parameters:
  • name (str) – The name of the destination.

  • host (str) – Host Endpoint of the Redshift Cluster (must include the cluster-id, region and end with .redshift.amazonaws.com).

  • port (int) – Port of the database.

  • database (str) – Name of the database.

  • schemas (Optional[List[str]]) – The list of schemas to sync from. Specify one or more explicitly or keep empty to process all schemas. Schema names are case sensitive.

  • username (str) – Username to use to access the database.

  • password (str) – Password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3).

class dagster_airbyte.managed.generated.sources.AsanaSource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Asana.

Parameters:
class AsanaSource.PATCredentials(personal_access_token)[source]
__init__(personal_access_token)[source]
class AsanaSource.OAuthCredentials(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class dagster_airbyte.managed.generated.sources.SmartsheetsSource(name, access_token, spreadsheet_id, start_datetime=None)[source]
__init__(name, access_token, spreadsheet_id, start_datetime=None)[source]

Airbyte Source for Smartsheets.

Documentation can be found at https://docs.airbyte.com/integrations/sources/smartsheets

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – The access token to use for accessing your data from Smartsheets. This access token must be generated by a user with at least read access to the data you’d like to replicate. Generate an access token in the Smartsheets main menu by clicking Account > Apps & Integrations > API Access. See the setup guide for information on how to obtain this token.

  • spreadsheet_id (str) – The spreadsheet ID. Find it by opening the spreadsheet then navigating to File > Properties

  • start_datetime (Optional[str]) – Only rows modified after this date/time will be replicated. This should be an ISO 8601 string, for instance: 2000-01-01T13:00:00

class dagster_airbyte.managed.generated.sources.MailchimpSource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Mailchimp.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mailchimp

Parameters:

name (str) – The name of the destination.

class MailchimpSource.OAuth20(access_token, client_id=None, client_secret=None)[source]
__init__(access_token, client_id=None, client_secret=None)[source]
class MailchimpSource.APIKey(apikey)[source]
__init__(apikey)[source]
class dagster_airbyte.managed.generated.sources.SentrySource(name, auth_token, organization, project, hostname=None, discover_fields=None)[source]
__init__(name, auth_token, organization, project, hostname=None, discover_fields=None)[source]

Airbyte Source for Sentry.

Documentation can be found at https://docs.airbyte.com/integrations/sources/sentry

Parameters:
  • name (str) – The name of the destination.

  • auth_token (str) – Log into Sentry and then create authentication tokens.For self-hosted, you can find or create authentication tokens by visiting “{instance_url_prefix}/settings/account/api/auth-tokens/”

  • hostname (Optional[str]) – Host name of Sentry API server.For self-hosted, specify your host name here. Otherwise, leave it empty.

  • organization (str) – The slug of the organization the groups belong to.

  • project (str) – The name (slug) of the Project you want to sync.

  • discover_fields (Optional[List[str]]) – Fields to retrieve when fetching discover events

class dagster_airbyte.managed.generated.sources.MailgunSource(name, private_key, domain_region=None, start_date=None)[source]
__init__(name, private_key, domain_region=None, start_date=None)[source]

Airbyte Source for Mailgun.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mailgun

Parameters:
  • name (str) – The name of the destination.

  • private_key (str) – Primary account API key to access your Mailgun data.

  • domain_region (Optional[str]) – Domain region code. ‘EU’ or ‘US’ are possible values. The default is ‘US’.

  • start_date (Optional[str]) – UTC date and time in the format 2020-10-01 00:00:00. Any data before this date will not be replicated. If omitted, defaults to 3 days ago.

class dagster_airbyte.managed.generated.sources.OnesignalSource(name, user_auth_key, start_date, outcome_names)[source]
__init__(name, user_auth_key, start_date, outcome_names)[source]

Airbyte Source for Onesignal.

Documentation can be found at https://docs.airbyte.com/integrations/sources/onesignal

Parameters:
  • name (str) – The name of the destination.

  • user_auth_key (str) – OneSignal User Auth Key, see the docs for more information on how to obtain this key.

  • start_date (str) – The date from which you’d like to replicate data for OneSignal API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

  • outcome_names (str) – Comma-separated list of names and the value (sum/count) for the returned outcome data. See the docs for more details

class dagster_airbyte.managed.generated.sources.PythonHttpTutorialSource(name, start_date, base, access_key=None)[source]
__init__(name, start_date, base, access_key=None)[source]

Airbyte Source for Python Http Tutorial.

Documentation can be found at https://docs.airbyte.com/integrations/sources/exchangeratesapi

Parameters:
  • name (str) – The name of the destination.

  • access_key (Optional[str]) – API access key used to retrieve data from the Exchange Rates API.

  • start_date (str) – UTC date and time in the format 2017-01-25. Any data before this date will not be replicated.

  • base (str) – ISO reference currency. See here.

class dagster_airbyte.managed.generated.sources.AirtableSource(name, api_key, base_id, tables)[source]
__init__(name, api_key, base_id, tables)[source]

Airbyte Source for Airtable.

Documentation can be found at https://docs.airbyte.com/integrations/sources/airtable

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – The API Key for the Airtable account. See the Support Guide for more information on how to obtain this key.

  • base_id (str) – The Base ID to integrate the data from. You can find the Base ID following the link Airtable API, log in to your account, select the base you need and find Base ID in the docs.

  • tables (List[str]) – The list of Tables to integrate.

class dagster_airbyte.managed.generated.sources.MongodbV2Source(name, instance_type, database, user=None, password=None, auth_source=None)[source]
__init__(name, instance_type, database, user=None, password=None, auth_source=None)[source]

Airbyte Source for Mongodb V2.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mongodb-v2

Parameters:
  • name (str) – The name of the destination.

  • instance_type (Union[MongodbV2Source.StandaloneMongoDbInstance, MongodbV2Source.ReplicaSet, MongodbV2Source.MongoDBAtlas]) – The MongoDb instance to connect to. For MongoDB Atlas and Replica Set TLS connection is used by default.

  • database (str) – The database you want to replicate.

  • user (Optional[str]) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with this username.

  • auth_source (Optional[str]) – The authentication source where the user information is stored.

class MongodbV2Source.StandaloneMongoDbInstance(instance, host, port, tls=None)[source]
__init__(instance, host, port, tls=None)[source]
class MongodbV2Source.ReplicaSet(instance, server_addresses, replica_set=None)[source]
__init__(instance, server_addresses, replica_set=None)[source]
class MongodbV2Source.MongoDBAtlas(instance, cluster_url)[source]
__init__(instance, cluster_url)[source]
class dagster_airbyte.managed.generated.sources.FileSecureSource(name, dataset_name, format, url, provider, reader_options=None)[source]
__init__(name, dataset_name, format, url, provider, reader_options=None)[source]

Airbyte Source for File Secure.

Documentation can be found at https://docs.airbyte.com/integrations/sources/file

Parameters:
class FileSecureSource.HTTPSPublicWeb(user_agent=None)[source]
__init__(user_agent=None)[source]
class FileSecureSource.GCSGoogleCloudStorage(service_account_json=None)[source]
__init__(service_account_json=None)[source]
class FileSecureSource.S3AmazonWebServices(aws_access_key_id=None, aws_secret_access_key=None)[source]
__init__(aws_access_key_id=None, aws_secret_access_key=None)[source]
class FileSecureSource.AzBlobAzureBlobStorage(storage_account, sas_token=None, shared_key=None)[source]
__init__(storage_account, sas_token=None, shared_key=None)[source]
class FileSecureSource.SSHSecureShell(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class FileSecureSource.SCPSecureCopyProtocol(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class FileSecureSource.SFTPSecureFileTransferProtocol(user, host, password=None, port=None)[source]
__init__(user, host, password=None, port=None)[source]
class dagster_airbyte.managed.generated.sources.ZendeskSupportSource(name, start_date, subdomain, credentials)[source]
__init__(name, start_date, subdomain, credentials)[source]

Airbyte Source for Zendesk Support.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zendesk-support

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – The date from which you’d like to replicate data for Zendesk Support API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

  • subdomain (str) – This is your Zendesk subdomain that can be found in your account URL. For example, in https://{MY_SUBDOMAIN}.zendesk.com/, where MY_SUBDOMAIN is the value of your subdomain.

  • credentials (Union[ZendeskSupportSource.OAuth20, ZendeskSupportSource.APIToken]) – Zendesk service provides two authentication methods. Choose between: OAuth2.0 or API token.

class ZendeskSupportSource.OAuth20(access_token, credentials=None)[source]
__init__(access_token, credentials=None)[source]
class ZendeskSupportSource.APIToken(email, api_token, credentials=None)[source]
__init__(email, api_token, credentials=None)[source]
class dagster_airbyte.managed.generated.sources.TempoSource(name, api_token)[source]
__init__(name, api_token)[source]

Airbyte Source for Tempo.

Documentation can be found at https://docs.airbyte.com/integrations/sources/

Parameters:
  • name (str) – The name of the destination.

  • api_token (str) – Tempo API Token. Go to Tempo>Settings, scroll down to Data Access and select API integration.

class dagster_airbyte.managed.generated.sources.BraintreeSource(name, merchant_id, public_key, private_key, environment, start_date=None)[source]
__init__(name, merchant_id, public_key, private_key, environment, start_date=None)[source]

Airbyte Source for Braintree.

Documentation can be found at https://docs.airbyte.com/integrations/sources/braintree

Parameters:
  • name (str) – The name of the destination.

  • merchant_id (str) – The unique identifier for your entire gateway account. See the docs for more information on how to obtain this ID.

  • public_key (str) – Braintree Public Key. See the docs for more information on how to obtain this key.

  • private_key (str) – Braintree Private Key. See the docs for more information on how to obtain this key.

  • start_date (Optional[str]) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • environment (str) – Environment specifies where the data will come from.

class dagster_airbyte.managed.generated.sources.SalesloftSource(name, client_id, client_secret, refresh_token, start_date)[source]
__init__(name, client_id, client_secret, refresh_token, start_date)[source]

Airbyte Source for Salesloft.

Documentation can be found at https://docs.airbyte.com/integrations/sources/salesloft

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The Client ID of your Salesloft developer application.

  • client_secret (str) – The Client Secret of your Salesloft developer application.

  • refresh_token (str) – The token for obtaining a new access token.

  • start_date (str) – The date from which you’d like to replicate data for Salesloft API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

class dagster_airbyte.managed.generated.sources.LinnworksSource(name, application_id, application_secret, token, start_date)[source]
__init__(name, application_id, application_secret, token, start_date)[source]

Airbyte Source for Linnworks.

Documentation can be found at https://docs.airbyte.com/integrations/sources/linnworks

Parameters:
  • name (str) – The name of the destination.

  • application_id (str) – Linnworks Application ID

  • application_secret (str) – Linnworks Application Secret

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.ChargebeeSource(name, site, site_api_key, start_date, product_catalog)[source]
__init__(name, site, site_api_key, start_date, product_catalog)[source]

Airbyte Source for Chargebee.

Documentation can be found at https://apidocs.chargebee.com/docs/api

Parameters:
  • name (str) – The name of the destination.

  • site (str) – The site prefix for your Chargebee instance.

  • site_api_key (str) – Chargebee API Key. See the docs for more information on how to obtain this key.

  • start_date (str) – UTC date and time in the format 2021-01-25T00:00:00Z. Any data before this date will not be replicated.

  • product_catalog (str) – Product Catalog version of your Chargebee site. Instructions on how to find your version you may find here under API Version section.

class dagster_airbyte.managed.generated.sources.GoogleAnalyticsDataApiSource(name, property_id, credentials, date_ranges_start_date, custom_reports=None, window_in_days=None)[source]
__init__(name, property_id, credentials, date_ranges_start_date, custom_reports=None, window_in_days=None)[source]

Airbyte Source for Google Analytics Data Api.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-analytics-v4

Parameters:
  • name (str) – The name of the destination.

  • property_id (str) – A Google Analytics GA4 property identifier whose events are tracked. Specified in the URL path and not the body

  • credentials (Union[GoogleAnalyticsDataApiSource.AuthenticateViaGoogleOauth, GoogleAnalyticsDataApiSource.ServiceAccountKeyAuthentication]) – Credentials for the service

  • date_ranges_start_date (str) – The start date. One of the values Ndaysago, yesterday, today or in the format YYYY-MM-DD

  • custom_reports (Optional[str]) – A JSON array describing the custom reports you want to sync from Google Analytics. See the docs for more information about the exact format you can use to fill out this field.

  • window_in_days (Optional[int]) – The time increment used by the connector when requesting data from the Google Analytics API. More information is available in the the docs. The bigger this value is, the faster the sync will be, but the more likely that sampling will be applied to your data, potentially causing inaccuracies in the returned results. We recommend setting this to 1 unless you have a hard requirement to make the sync faster at the expense of accuracy. The minimum allowed value for this field is 1, and the maximum is 364.

class GoogleAnalyticsDataApiSource.AuthenticateViaGoogleOauth(client_id, client_secret, refresh_token, auth_type=None, access_token=None)[source]
__init__(client_id, client_secret, refresh_token, auth_type=None, access_token=None)[source]
class GoogleAnalyticsDataApiSource.ServiceAccountKeyAuthentication(credentials_json, auth_type=None)[source]
__init__(credentials_json, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.OutreachSource(name, client_id, client_secret, refresh_token, redirect_uri, start_date)[source]
__init__(name, client_id, client_secret, refresh_token, redirect_uri, start_date)[source]

Airbyte Source for Outreach.

Documentation can be found at https://docs.airbyte.com/integrations/sources/outreach

Parameters:
  • name (str) – The name of the destination.

  • client_id (str) – The Client ID of your Outreach developer application.

  • client_secret (str) – The Client Secret of your Outreach developer application.

  • refresh_token (str) – The token for obtaining the new access token.

  • redirect_uri (str) – A Redirect URI is the location where the authorization server sends the user once the app has been successfully authorized and granted an authorization code or access token.

  • start_date (str) – The date from which you’d like to replicate data for Outreach API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

class dagster_airbyte.managed.generated.sources.LemlistSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Lemlist.

Documentation can be found at https://docs.airbyte.com/integrations/sources/lemlist

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Lemlist API key.

class dagster_airbyte.managed.generated.sources.ApifyDatasetSource(name, datasetId, clean=None)[source]
__init__(name, datasetId, clean=None)[source]

Airbyte Source for Apify Dataset.

Documentation can be found at https://docs.airbyte.com/integrations/sources/apify-dataset

Parameters:
  • name (str) – The name of the destination.

  • datasetId (str) – ID of the dataset you would like to load to Airbyte.

  • clean (Optional[bool]) – If set to true, only clean items will be downloaded from the dataset. See description of what clean means in Apify API docs. If not sure, set clean to false.

class dagster_airbyte.managed.generated.sources.RecurlySource(name, api_key, begin_time=None, end_time=None)[source]
__init__(name, api_key, begin_time=None, end_time=None)[source]

Airbyte Source for Recurly.

Documentation can be found at https://docs.airbyte.com/integrations/sources/recurly

Parameters:
  • name (str) – The name of the destination.

  • api_key (str) – Recurly API Key. See the docs for more information on how to generate this key.

  • begin_time (Optional[str]) – ISO8601 timestamp from which the replication from Recurly API will start from.

  • end_time (Optional[str]) – ISO8601 timestamp to which the replication from Recurly API will stop. Records after that date won’t be imported.

class dagster_airbyte.managed.generated.sources.ZendeskTalkSource(name, subdomain, credentials, start_date)[source]
__init__(name, subdomain, credentials, start_date)[source]

Airbyte Source for Zendesk Talk.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zendesk-talk

Parameters:
  • name (str) – The name of the destination.

  • subdomain (str) – This is your Zendesk subdomain that can be found in your account URL. For example, in https://{MY_SUBDOMAIN}.zendesk.com/, where MY_SUBDOMAIN is the value of your subdomain.

  • credentials (Union[ZendeskTalkSource.APIToken, ZendeskTalkSource.OAuth20]) – Zendesk service provides two authentication methods. Choose between: OAuth2.0 or API token.

  • start_date (str) – The date from which you’d like to replicate data for Zendesk Talk API, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

class ZendeskTalkSource.APIToken(email, api_token, auth_type=None)[source]
__init__(email, api_token, auth_type=None)[source]
class ZendeskTalkSource.OAuth20(access_token, auth_type=None)[source]
__init__(access_token, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.SftpSource(name, user, host, port, credentials, file_types=None, folder_path=None, file_pattern=None)[source]
__init__(name, user, host, port, credentials, file_types=None, folder_path=None, file_pattern=None)[source]

Airbyte Source for Sftp.

Documentation can be found at https://docs.airbyte.com/integrations/source/sftp

Parameters:
  • name (str) – The name of the destination.

  • user (str) – The server user

  • host (str) – The server host address

  • port (int) – The server port

  • credentials (Union[SftpSource.PasswordAuthentication, SftpSource.SSHKeyAuthentication]) – The server authentication method

  • file_types (Optional[str]) – Coma separated file types. Currently only ‘csv’ and ‘json’ types are supported.

  • folder_path (Optional[str]) – The directory to search files for sync

  • file_pattern (Optional[str]) – The regular expression to specify files for sync in a chosen Folder Path

class SftpSource.PasswordAuthentication(auth_user_password)[source]
__init__(auth_user_password)[source]
class SftpSource.SSHKeyAuthentication(auth_ssh_key)[source]
__init__(auth_ssh_key)[source]
class dagster_airbyte.managed.generated.sources.WhiskyHunterSource(name)[source]
__init__(name)[source]

Airbyte Source for Whisky Hunter.

Documentation can be found at https://docs.airbyte.io/integrations/sources/whisky-hunter

Parameters:

name (str) – The name of the destination.

class dagster_airbyte.managed.generated.sources.FreshdeskSource(name, domain, api_key, requests_per_minute=None, start_date=None)[source]
__init__(name, domain, api_key, requests_per_minute=None, start_date=None)[source]

Airbyte Source for Freshdesk.

Documentation can be found at https://docs.airbyte.com/integrations/sources/freshdesk

Parameters:
  • name (str) – The name of the destination.

  • domain (str) – Freshdesk domain

  • api_key (str) – Freshdesk API Key. See the docs for more information on how to obtain this key.

  • requests_per_minute (Optional[int]) – The number of requests per minute that this source allowed to use. There is a rate limit of 50 requests per minute per app per account.

  • start_date (Optional[str]) – UTC date and time. Any data created after this date will be replicated. If this parameter is not set, all data will be replicated.

class dagster_airbyte.managed.generated.sources.GocardlessSource(name, access_token, gocardless_environment, gocardless_version, start_date)[source]
__init__(name, access_token, gocardless_environment, gocardless_version, start_date)[source]

Airbyte Source for Gocardless.

Documentation can be found at https://docs.airbyte.com/integrations/sources/gocardless

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – Gocardless API TOKEN

  • gocardless_environment (str) – Environment you are trying to connect to.

  • gocardless_version (str) – GoCardless version. This is a date. You can find the latest here: https://developer.gocardless.com/api-reference/#api-usage-making-requests

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.ZuoraSource(name, start_date, tenant_endpoint, data_query, client_id, client_secret, window_in_days=None)[source]
__init__(name, start_date, tenant_endpoint, data_query, client_id, client_secret, window_in_days=None)[source]

Airbyte Source for Zuora.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zuora

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – Start Date in format: YYYY-MM-DD

  • window_in_days (Optional[str]) – The amount of days for each data-chunk begining from start_date. Bigger the value - faster the fetch. (0.1 - as for couple of hours, 1 - as for a Day; 364 - as for a Year).

  • tenant_endpoint (str) – Please choose the right endpoint where your Tenant is located. More info by this Link

  • data_query (str) – Choose between Live, or Unlimited - the optimized, replicated database at 12 hours freshness for high volume extraction Link

  • client_id (str) – Your OAuth user Client ID

  • client_secret (str) – Your OAuth user Client Secret

class dagster_airbyte.managed.generated.sources.MarketoSource(name, domain_url, client_id, client_secret, start_date)[source]
__init__(name, domain_url, client_id, client_secret, start_date)[source]

Airbyte Source for Marketo.

Documentation can be found at https://docs.airbyte.com/integrations/sources/marketo

Parameters:
  • name (str) – The name of the destination.

  • domain_url (str) – Your Marketo Base URL. See the docs for info on how to obtain this.

  • client_id (str) – The Client ID of your Marketo developer application. See the docs for info on how to obtain this.

  • client_secret (str) – The Client Secret of your Marketo developer application. See the docs for info on how to obtain this.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

class dagster_airbyte.managed.generated.sources.DriftSource(name, credentials)[source]
__init__(name, credentials)[source]

Airbyte Source for Drift.

Documentation can be found at https://docs.airbyte.com/integrations/sources/drift

Parameters:

name (str) – The name of the destination.

class DriftSource.OAuth20(client_id, client_secret, access_token, refresh_token, credentials=None)[source]
__init__(client_id, client_secret, access_token, refresh_token, credentials=None)[source]
class DriftSource.AccessToken(access_token, credentials=None)[source]
__init__(access_token, credentials=None)[source]
class dagster_airbyte.managed.generated.sources.PokeapiSource(name, pokemon_name)[source]
__init__(name, pokemon_name)[source]

Airbyte Source for Pokeapi.

Documentation can be found at https://docs.airbyte.com/integrations/sources/pokeapi

Parameters:
  • name (str) – The name of the destination.

  • pokemon_name (str) – Pokemon requested from the API.

class dagster_airbyte.managed.generated.sources.NetsuiteSource(name, realm, consumer_key, consumer_secret, token_key, token_secret, start_datetime, object_types=None, window_in_days=None)[source]
__init__(name, realm, consumer_key, consumer_secret, token_key, token_secret, start_datetime, object_types=None, window_in_days=None)[source]

Airbyte Source for Netsuite.

Parameters:
  • name (str) – The name of the destination.

  • realm (str) – Netsuite realm e.g. 2344535, as for production or 2344535_SB1, as for the sandbox

  • consumer_key (str) – Consumer key associated with your integration

  • consumer_secret (str) – Consumer secret associated with your integration

  • token_key (str) – Access token key

  • token_secret (str) – Access token secret

  • object_types (Optional[List[str]]) – The API names of the Netsuite objects you want to sync. Setting this speeds up the connection setup process by limiting the number of schemas that need to be retrieved from Netsuite.

  • start_datetime (str) – Starting point for your data replication, in format of “YYYY-MM-DDTHH:mm:ssZ”

  • window_in_days (Optional[int]) – The amount of days used to query the data with date chunks. Set smaller value, if you have lots of data.

class dagster_airbyte.managed.generated.sources.HubplannerSource(name, api_key)[source]
__init__(name, api_key)[source]

Airbyte Source for Hubplanner.

Documentation can be found at https://docs.airbyte.com/integrations/sources/hubplanner

Parameters:
class dagster_airbyte.managed.generated.sources.Dv360Source(name, credentials, partner_id, start_date, end_date=None, filters=None)[source]
__init__(name, credentials, partner_id, start_date, end_date=None, filters=None)[source]

Airbyte Source for Dv 360.

Parameters:
  • name (str) – The name of the destination.

  • credentials (Dv360Source.Oauth2Credentials) – Oauth2 credentials

  • partner_id (int) – Partner ID

  • start_date (str) – UTC date and time in the format 2017-01-25. Any data before this date will not be replicated

  • end_date (Optional[str]) – UTC date and time in the format 2017-01-25. Any data after this date will not be replicated.

  • filters (Optional[List[str]]) – filters for the dimensions. each filter object had 2 keys: ‘type’ for the name of the dimension to be used as. and ‘value’ for the value of the filter

class Dv360Source.Oauth2Credentials(access_token, refresh_token, token_uri, client_id, client_secret)[source]
__init__(access_token, refresh_token, token_uri, client_id, client_secret)[source]
class dagster_airbyte.managed.generated.sources.NotionSource(name, start_date, credentials)[source]
__init__(name, start_date, credentials)[source]

Airbyte Source for Notion.

Documentation can be found at https://docs.airbyte.com/integrations/sources/notion

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00.000Z. Any data before this date will not be replicated.

  • credentials (Union[NotionSource.OAuth20, NotionSource.AccessToken]) – Pick an authentication method.

class NotionSource.OAuth20(client_id, client_secret, access_token)[source]
__init__(client_id, client_secret, access_token)[source]
class NotionSource.AccessToken(token)[source]
__init__(token)[source]
class dagster_airbyte.managed.generated.sources.ZendeskSunshineSource(name, subdomain, start_date, credentials)[source]
__init__(name, subdomain, start_date, credentials)[source]

Airbyte Source for Zendesk Sunshine.

Documentation can be found at https://docs.airbyte.com/integrations/sources/zendesk_sunshine

Parameters:
  • name (str) – The name of the destination.

  • subdomain (str) – The subdomain for your Zendesk Account.

  • start_date (str) – The date from which you’d like to replicate data for Zendesk Sunshine API, in the format YYYY-MM-DDT00:00:00Z.

class ZendeskSunshineSource.OAuth20(client_id, client_secret, access_token)[source]
__init__(client_id, client_secret, access_token)[source]
class ZendeskSunshineSource.APIToken(api_token, email)[source]
__init__(api_token, email)[source]
class dagster_airbyte.managed.generated.sources.PinterestSource(name, start_date, credentials)[source]
__init__(name, start_date, credentials)[source]

Airbyte Source for Pinterest.

Documentation can be found at https://docs.airbyte.com/integrations/sources/pinterest

Parameters:
  • name (str) – The name of the destination.

  • start_date (str) – A date in the format YYYY-MM-DD. If you have not set a date, it would be defaulted to latest allowed date by api (914 days from today).

class PinterestSource.OAuth20(refresh_token, client_id=None, client_secret=None)[source]
__init__(refresh_token, client_id=None, client_secret=None)[source]
class PinterestSource.AccessToken(access_token)[source]
__init__(access_token)[source]
class dagster_airbyte.managed.generated.sources.MetabaseSource(name, instance_api_url, username=None, password=None, session_token=None)[source]
__init__(name, instance_api_url, username=None, password=None, session_token=None)[source]

Airbyte Source for Metabase.

Documentation can be found at https://docs.airbyte.com/integrations/sources/metabase

Parameters:
  • name (str) – The name of the destination.

  • instance_api_url (str) – URL to your metabase instance API

  • session_token (Optional[str]) – To generate your session token, you need to run the following command: ` curl -X POST \\   -H "Content-Type: application/json" \\   -d '{"username": "person@metabase.com", "password": "fakepassword"}' \\   http://localhost:3000/api/session ` Then copy the value of the id field returned by a successful call to that API. Note that by default, sessions are good for 14 days and needs to be regenerated.

class dagster_airbyte.managed.generated.sources.HubspotSource(name, start_date, credentials)[source]
__init__(name, start_date, credentials)[source]

Airbyte Source for Hubspot.

Documentation can be found at https://docs.airbyte.com/integrations/sources/hubspot

Parameters:
class HubspotSource.OAuth(client_id, client_secret, refresh_token)[source]
__init__(client_id, client_secret, refresh_token)[source]
class HubspotSource.APIKey(api_key)[source]
__init__(api_key)[source]
class HubspotSource.PrivateAPP(access_token)[source]
__init__(access_token)[source]
class dagster_airbyte.managed.generated.sources.HarvestSource(name, account_id, replication_start_date, credentials)[source]
__init__(name, account_id, replication_start_date, credentials)[source]

Airbyte Source for Harvest.

Documentation can be found at https://docs.airbyte.com/integrations/sources/harvest

Parameters:
class HarvestSource.AuthenticateViaHarvestOAuth(client_id, client_secret, refresh_token, auth_type=None)[source]
__init__(client_id, client_secret, refresh_token, auth_type=None)[source]
class HarvestSource.AuthenticateWithPersonalAccessToken(api_token, auth_type=None)[source]
__init__(api_token, auth_type=None)[source]
class dagster_airbyte.managed.generated.sources.GithubSource(name, credentials, start_date, repository, branch=None, page_size_for_large_streams=None)[source]
__init__(name, credentials, start_date, repository, branch=None, page_size_for_large_streams=None)[source]

Airbyte Source for Github.

Documentation can be found at https://docs.airbyte.com/integrations/sources/github

Parameters:
  • name (str) – The name of the destination.

  • credentials (Union[GithubSource.OAuthCredentials, GithubSource.PATCredentials]) – Choose how to authenticate to GitHub

  • start_date (str) – The date from which you’d like to replicate data from GitHub in the format YYYY-MM-DDT00:00:00Z. For the streams which support this configuration, only data generated on or after the start date will be replicated. This field doesn’t apply to all streams, see the docs for more info

  • repository (str) – Space-delimited list of GitHub organizations/repositories, e.g. airbytehq/airbyte for single repository, airbytehq/* for get all repositories from organization and airbytehq/airbyte airbytehq/another-repo for multiple repositories.

  • branch (Optional[str]) – Space-delimited list of GitHub repository branches to pull commits for, e.g. airbytehq/airbyte/master. If no branches are specified for a repository, the default branch will be pulled.

  • page_size_for_large_streams (Optional[int]) – The Github connector contains several streams with a large amount of data. The page size of such streams depends on the size of your repository. We recommended that you specify values between 10 and 30.

class GithubSource.OAuthCredentials(access_token)[source]
__init__(access_token)[source]
class GithubSource.PATCredentials(personal_access_token)[source]
__init__(personal_access_token)[source]
class dagster_airbyte.managed.generated.sources.E2eTestSource(name, max_messages, mock_catalog, type=None, seed=None, message_interval_ms=None)[source]
__init__(name, max_messages, mock_catalog, type=None, seed=None, message_interval_ms=None)[source]

Airbyte Source for E2e Test.

Documentation can be found at https://docs.airbyte.com/integrations/sources/e2e-test

Parameters:
  • name (str) – The name of the destination.

  • max_messages (int) – Number of records to emit per stream. Min 1. Max 100 billion.

  • seed (Optional[int]) – When the seed is unspecified, the current time millis will be used as the seed. Range: [0, 1000000].

  • message_interval_ms (Optional[int]) – Interval between messages in ms. Min 0 ms. Max 60000 ms (1 minute).

class E2eTestSource.SingleSchema(stream_name, stream_schema, stream_duplication=None)[source]
__init__(stream_name, stream_schema, stream_duplication=None)[source]
class E2eTestSource.MultiSchema(stream_schemas)[source]
__init__(stream_schemas)[source]
class dagster_airbyte.managed.generated.sources.MysqlSource(name, host, port, database, username, ssl_mode, replication_method, password=None, jdbc_url_params=None, ssl=None)[source]
__init__(name, host, port, database, username, ssl_mode, replication_method, password=None, jdbc_url_params=None, ssl=None)[source]

Airbyte Source for Mysql.

Documentation can be found at https://docs.airbyte.com/integrations/sources/mysql

Parameters:
  • name (str) – The name of the destination.

  • host (str) – The host name of the database.

  • port (int) – The port to connect to.

  • database (str) – The database name.

  • username (str) – The username which is used to access the database.

  • password (Optional[str]) – The password associated with the username.

  • jdbc_url_params (Optional[str]) – Additional properties to pass to the JDBC URL string when connecting to the database formatted as ‘key=value’ pairs separated by the symbol ‘&’. (example: key1=value1&key2=value2&key3=value3). For more information read about JDBC URL parameters.

  • ssl (Optional[bool]) – Encrypt data using SSL.

  • ssl_mode (Union[MysqlSource.Preferred, MysqlSource.Required, MysqlSource.VerifyCA, MysqlSource.VerifyIdentity]) – SSL connection modes. preferred - Automatically attempt SSL connection. If the MySQL server does not support SSL, continue with a regular connection.required - Always connect with SSL. If the MySQL server doesn`t support SSL, the connection will not be established. Certificate Authority (CA) and Hostname are not verified.verify-ca - Always connect with SSL. Verifies CA, but allows connection even if Hostname does not match.Verify Identity - Always connect with SSL. Verify both CA and Hostname.Read more in the docs.

  • replication_method (Union[MysqlSource.Standard, MysqlSource.LogicalReplicationCDC]) – Replication method to use for extracting data from the database.

class MysqlSource.Preferred[source]
__init__()[source]
class MysqlSource.Required[source]
__init__()[source]
class MysqlSource.VerifyCA(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
__init__(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
class MysqlSource.VerifyIdentity(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
__init__(ca_certificate, client_certificate=None, client_key=None, client_key_password=None)[source]
class MysqlSource.Standard[source]
__init__()[source]
class MysqlSource.LogicalReplicationCDC(initial_waiting_seconds=None, server_time_zone=None)[source]
__init__(initial_waiting_seconds=None, server_time_zone=None)[source]
class dagster_airbyte.managed.generated.sources.MyHoursSource(name, email, password, start_date, logs_batch_size=None)[source]
__init__(name, email, password, start_date, logs_batch_size=None)[source]

Airbyte Source for My Hours.

Documentation can be found at https://docs.airbyte.com/integrations/sources/my-hours

Parameters:
  • name (str) – The name of the destination.

  • email (str) – Your My Hours username

  • password (str) – The password associated to the username

  • start_date (str) – Start date for collecting time logs

  • logs_batch_size (Optional[int]) – Pagination size used for retrieving logs in days

class dagster_airbyte.managed.generated.sources.KyribaSource(name, domain, username, password, start_date, end_date=None)[source]
__init__(name, domain, username, password, start_date, end_date=None)[source]

Airbyte Source for Kyriba.

Parameters:
  • name (str) – The name of the destination.

  • domain (str) – Kyriba domain

  • username (str) – Username to be used in basic auth

  • password (str) – Password to be used in basic auth

  • start_date (str) – The date the sync should start from.

  • end_date (Optional[str]) – The date the sync should end. If let empty the sync will run to the current date.

class dagster_airbyte.managed.generated.sources.GoogleSearchConsoleSource(name, site_urls, start_date, authorization, end_date=None, custom_reports=None)[source]
__init__(name, site_urls, start_date, authorization, end_date=None, custom_reports=None)[source]

Airbyte Source for Google Search Console.

Documentation can be found at https://docs.airbyte.com/integrations/sources/google-search-console

Parameters:
  • name (str) – The name of the destination.

  • site_urls (List[str]) – The URLs of the website property attached to your GSC account. Read more here.

  • start_date (str) – UTC date in the format 2017-01-25. Any data before this date will not be replicated.

  • end_date (Optional[str]) – UTC date in the format 2017-01-25. Any data after this date will not be replicated. Must be greater or equal to the start date field.

  • custom_reports (Optional[str]) – A JSON array describing the custom reports you want to sync from Google Search Console. See the docs for more information about the exact format you can use to fill out this field.

class GoogleSearchConsoleSource.OAuth(client_id, client_secret, refresh_token, access_token=None)[source]
__init__(client_id, client_secret, refresh_token, access_token=None)[source]
class GoogleSearchConsoleSource.ServiceAccountKeyAuthentication(service_account_info, email)[source]
__init__(service_account_info, email)[source]
class dagster_airbyte.managed.generated.sources.FacebookMarketingSource(name, account_id, start_date, access_token, end_date=None, include_deleted=None, fetch_thumbnail_images=None, custom_insights=None, page_size=None, insights_lookback_window=None, max_batch_size=None)[source]
__init__(name, account_id, start_date, access_token, end_date=None, include_deleted=None, fetch_thumbnail_images=None, custom_insights=None, page_size=None, insights_lookback_window=None, max_batch_size=None)[source]

Airbyte Source for Facebook Marketing.

Documentation can be found at https://docs.airbyte.com/integrations/sources/facebook-marketing

Parameters:
  • name (str) – The name of the destination.

  • account_id (str) – The Facebook Ad account ID to use when pulling data from the Facebook Marketing API.

  • start_date (str) – The date from which you’d like to replicate data for all incremental streams, in the format YYYY-MM-DDT00:00:00Z. All data generated after this date will be replicated.

  • end_date (Optional[str]) – The date until which you’d like to replicate data for all incremental streams, in the format YYYY-MM-DDT00:00:00Z. All data generated between start_date and this date will be replicated. Not setting this option will result in always syncing the latest data.

  • access_token (str) – The value of the access token generated. See the docs for more information

  • include_deleted (Optional[bool]) – Include data from deleted Campaigns, Ads, and AdSets

  • fetch_thumbnail_images (Optional[bool]) – In each Ad Creative, fetch the thumbnail_url and store the result in thumbnail_data_url

  • custom_insights (Optional[List[FacebookMarketingSource.InsightConfig]]) – A list which contains insights entries, each entry must have a name and can contains fields, breakdowns or action_breakdowns)

  • page_size (Optional[int]) – Page size used when sending requests to Facebook API to specify number of records per page when response has pagination. Most users do not need to set this field unless they specifically need to tune the connector to address specific issues or use cases.

  • insights_lookback_window (Optional[int]) – The attribution window

  • max_batch_size (Optional[int]) – Maximum batch size used when sending batch requests to Facebook API. Most users do not need to set this field unless they specifically need to tune the connector to address specific issues or use cases.

class FacebookMarketingSource.InsightConfig(name, fields=None, breakdowns=None, action_breakdowns=None, time_increment=None, start_date=None, end_date=None, insights_lookback_window=None)[source]
__init__(name, fields=None, breakdowns=None, action_breakdowns=None, time_increment=None, start_date=None, end_date=None, insights_lookback_window=None)[source]
class dagster_airbyte.managed.generated.sources.SurveymonkeySource(name, access_token, start_date, survey_ids=None)[source]
__init__(name, access_token, start_date, survey_ids=None)[source]

Airbyte Source for Surveymonkey.

Documentation can be found at https://docs.airbyte.com/integrations/sources/surveymonkey

Parameters:
  • name (str) – The name of the destination.

  • access_token (str) – Access Token for making authenticated requests. See the docs for information on how to generate this key.

  • start_date (str) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated.

  • survey_ids (Optional[List[str]]) – IDs of the surveys from which you’d like to replicate data. If left empty, data from all boards to which you have access will be replicated.

class dagster_airbyte.managed.generated.sources.PardotSource(name, pardot_business_unit_id, client_id, client_secret, refresh_token, start_date=None, is_sandbox=None)[source]
__init__(name, pardot_business_unit_id, client_id, client_secret, refresh_token, start_date=None, is_sandbox=None)[source]

Airbyte Source for Pardot.

Parameters:
  • name (str) – The name of the destination.

  • pardot_business_unit_id (str) – Pardot Business ID, can be found at Setup > Pardot > Pardot Account Setup

  • client_id (str) – The Consumer Key that can be found when viewing your app in Salesforce

  • client_secret (str) – The Consumer Secret that can be found when viewing your app in Salesforce

  • refresh_token (str) – Salesforce Refresh Token used for Airbyte to access your Salesforce account. If you don’t know what this is, follow this guide to retrieve it.

  • start_date (Optional[str]) – UTC date and time in the format 2017-01-25T00:00:00Z. Any data before this date will not be replicated. Leave blank to skip this filter

  • is_sandbox (Optional[bool]) – Whether or not the the app is in a Salesforce sandbox. If you do not know what this, assume it is false.

class dagster_airbyte.managed.generated.sources.FlexportSource(name, api_key, start_date)[source]
__init__(name, api_key, start_date)[source]

Airbyte Source for Flexport.

Documentation can be found at https://docs.airbyte.com/integrations/sources/flexport

Parameters:

name (str) – The name of the destination.

class dagster_airbyte.managed.generated.sources.ZenefitsSource(name, token)[source]
__init__(name, token)[source]

Airbyte Source for Zenefits.

Parameters:
  • name (str) – The name of the destination.

  • token (str) – Use Sync with Zenefits button on the link given on the readme file, and get the token to access the api

class dagster_airbyte.managed.generated.sources.KafkaSource(name, MessageFormat, bootstrap_servers, subscription, protocol, test_topic=None, group_id=None, max_poll_records=None, polling_time=None, client_id=None, enable_auto_commit=None, auto_commit_interval_ms=None, client_dns_lookup=None, retry_backoff_ms=None, request_timeout_ms=None, receive_buffer_bytes=None, auto_offset_reset=None, repeated_calls=None, max_records_process=None)[source]
__init__(name, MessageFormat, bootstrap_servers, subscription, protocol, test_topic=None, group_id=None, max_poll_records=None, polling_time=None, client_id=None, enable_auto_commit=None, auto_commit_interval_ms=None, client_dns_lookup=None, retry_backoff_ms=None, request_timeout_ms=None, receive_buffer_bytes=None, auto_offset_reset=None, repeated_calls=None, max_records_process=None)[source]

Airbyte Source for Kafka.

Documentation can be found at https://docs.airbyte.com/integrations/sources/kafka

Parameters:
  • name (str) – The name of the destination.

  • MessageFormat (Union[KafkaSource.JSON, KafkaSource.AVRO]) – The serialization used based on this

  • bootstrap_servers (str) – A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping&mdash;this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form host1:port1,host2:port2,…. Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).

  • subscription (Union[KafkaSource.ManuallyAssignAListOfPartitions, KafkaSource.SubscribeToAllTopicsMatchingSpecifiedPattern]) – You can choose to manually assign a list of partitions, or subscribe to all topics matching specified pattern to get dynamically assigned partitions.

  • test_topic (Optional[str]) – The Topic to test in case the Airbyte can consume messages.

  • group_id (Optional[str]) – The Group ID is how you distinguish different consumer groups.

  • max_poll_records (Optional[int]) – The maximum number of records returned in a single call to poll(). Note, that max_poll_records does not impact the underlying fetching behavior. The consumer will cache the records from each fetch request and returns them incrementally from each poll.

  • polling_time (Optional[int]) – Amount of time Kafka connector should try to poll for messages.

  • protocol (Union[KafkaSource.PLAINTEXT, KafkaSource.SASLPLAINTEXT, KafkaSource.SASLSSL]) – The Protocol used to communicate with brokers.

  • client_id (Optional[str]) – An ID string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.

  • enable_auto_commit (Optional[bool]) – If true, the consumer’s offset will be periodically committed in the background.

  • auto_commit_interval_ms (Optional[int]) – The frequency in milliseconds that the consumer offsets are auto-committed to Kafka if enable.auto.commit is set to true.

  • client_dns_lookup (Optional[str]) – Controls how the client uses DNS lookups. If set to use_all_dns_ips, connect to each returned IP address in sequence until a successful connection is established. After a disconnection, the next IP is used. Once all IPs have been used once, the client resolves the IP(s) from the hostname again. If set to resolve_canonical_bootstrap_servers_only, resolve each bootstrap address into a list of canonical names. After the bootstrap phase, this behaves the same as use_all_dns_ips. If set to default (deprecated), attempt to connect to the first IP address returned by the lookup, even if the lookup returns multiple IP addresses.

  • retry_backoff_ms (Optional[int]) – The amount of time to wait before attempting to retry a failed request to a given topic partition. This avoids repeatedly sending requests in a tight loop under some failure scenarios.

  • request_timeout_ms (Optional[int]) – The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.

  • receive_buffer_bytes (Optional[int]) – The size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.

  • auto_offset_reset (Optional[str]) – What to do when there is no initial offset in Kafka or if the current offset does not exist any more on the server - earliest: automatically reset the offset to the earliest offset, latest: automatically reset the offset to the latest offset, none: throw exception to the consumer if no previous offset is found for the consumer’s group, anything else: throw exception to the consumer.

  • repeated_calls (Optional[int]) – The number of repeated calls to poll() if no messages were received.

  • max_records_process (Optional[int]) – The Maximum to be processed per execution

class KafkaSource.JSON(deserialization_type=None)[source]
__init__(deserialization_type=None)[source]
class KafkaSource.AVRO(deserialization_type=None, deserialization_strategy=None, schema_registry_url=None, schema_registry_username=None, schema_registry_password=None)[source]
__init__(deserialization_type=None, deserialization_strategy=None, schema_registry_url=None, schema_registry_username=None, schema_registry_password=None)[source]
class KafkaSource.ManuallyAssignAListOfPartitions(topic_partitions)[source]
__init__(topic_partitions)[source]
class KafkaSource.SubscribeToAllTopicsMatchingSpecifiedPattern(topic_pattern)[source]
__init__(topic_pattern)[source]
class KafkaSource.PLAINTEXT(security_protocol)[source]
__init__(security_protocol)[source]
class KafkaSource.SASLPLAINTEXT(security_protocol, sasl_mechanism, sasl_jaas_config)[source]
__init__(security_protocol, sasl_mechanism, sasl_jaas_config)[source]
class KafkaSource.SASLSSL(security_protocol, sasl_mechanism, sasl_jaas_config)[source]
__init__(security_protocol, sasl_mechanism, sasl_jaas_config)[source]

Managed Config Generated Destinations

class dagster_airbyte.managed.generated.destinations.DynamodbDestination(name, dynamodb_table_name_prefix, dynamodb_region, access_key_id, secret_access_key, dynamodb_endpoint=None)[source]
__init__(name, dynamodb_table_name_prefix, dynamodb_region, access_key_id, secret_access_key, dynamodb_endpoint=None)[source]

Airbyte Destination for Dynamodb.

Documentation can be found at https://docs.airbyte.com/integrations/destinations/dynamodb

Parameters:
  • name (str) – The name of the destination.

  • dynamodb_endpoint (Optional[str]) – This is your DynamoDB endpoint url.(if you are working with AWS DynamoDB, just leave empty).

  • dynamodb_table_name_prefix (str) – The prefix to use when naming DynamoDB tables.

  • dynamodb_region (str) – The region of the DynamoDB.

  • access_key_id (str) – The access key id to access the DynamoDB. Airbyte requires Read and Write permissions to the DynamoDB.

  • secret_access_key (str) – The corresponding secret to the access key id.

class dagster_airbyte.managed.generated.destinations.BigqueryDestination(name, project_id, dataset_location, dataset_id, loading_method, credentials_json=None, transformation_priority=