Sources

A hosted extractor source represents an external source system on the internet. The source resource in CDF contains all the information the extractor needs to connect to the external source system. A source can have many jobs, each streaming different data from the source system.

List Sources

Required capabilities: hostedExtractors:READ

List all sources in a given project. If more than limit sources exist, a cursor for pagination will be returned with the response.

Securityoidc-token or oauth2-client-credentials or oauth2-open-industrial-data or oauth2-auth-code
Request
query Parameters
limit
integer [ 1 .. 100 ]
Default: 100

Limits the number of results to be returned. The maximum results returned by the server is 100 even if you specify a higher limit.

cursor
string

Cursor for paging through results. In general, if a response contains a nextCursor property, it means that there may be more results, and you should pass that value as the cursor parameter in the next request.

Note that the cursor may or may not be encrypted, but either way, it is not intended to be decoded. Its internal structure is not a part of the public API, and may change without notice. You should treat it as an opaque string and not attempt to craft your own cursors.

Example: cursor=4zj0Vy2fo0NtNMb229mI9r1V3YG5NBL752kQz1cKtwo
Responses
200

List of sources and an optional cursor.

400

Response for a failed request

422

Validation Error

get/hostedextractors/sources
Request samples 
source_list = client.hosted_extractors.sources.list(limit=5)

for source in client.hosted_extractors.sources:
    source # do something with the source

for source_list in client.hosted_extractors.sources(chunk_size=25):
    source_list # do something with the sources
Response samples
application/json
{
  • "items": [
    ],
  • "nextCursor": "string"
}

Create Sources

Required capabilities: hostedExtractors:READ

Create up to 100 sources.

Securityoidc-token or oauth2-client-credentials or oauth2-open-industrial-data or oauth2-auth-code
Request
Request Body schema: application/json
required

Sources to create.

required
Array of any (Items) [ 1 .. 10 ] items
Responses
200

List of created sources.

400

The response for a bad request.

422

Validation Error

post/hostedextractors/sources
Request samples
application/json
{
  • "items": [
    ]
}
Response samples
application/json
{
  • "items": [
    ]
}

Retrieve Sources

Required capabilities: hostedExtractors:READ

Retrieve a list of up to 100 sources by their external ID, optionally ignoring unknown IDs.

Securityoidc-token or oauth2-client-credentials or oauth2-open-industrial-data or oauth2-auth-code
Request
Request Body schema: application/json
required

List of external IDs of sources to retrieve.

required
Array of objects (Items) [ 1 .. 100 ] items
ignoreUnknownIds
boolean (IgnoreUnknownIds)

Ignore IDs and external IDs that are not found

Responses
200

List of retrieved sources.

400

The response for a bad request.

422

Validation Error

post/hostedextractors/sources/byIds
Request samples
application/json
{
  • "items": [
    ],
  • "ignoreUnknownIds": true
}
Response samples
application/json
{
  • "items": [
    ]
}

Delete Sources

Required capabilities: hostedExtractors:WRITE

Delete a list of sources by their external ID. This operation will fail if any destination in the request is associated with a job, unless the force query parameter is set to true.

Securityoidc-token or oauth2-client-credentials or oauth2-open-industrial-data or oauth2-auth-code
Request
Request Body schema: application/json
required

List of external IDs of sources to delete.

required
Array of objects (Items) [ 1 .. 100 ] items
ignoreUnknownIds
boolean (IgnoreUnknownIds)

Ignore IDs and external IDs that are not found.

force
boolean (Force)

Delete any jobs associated with each item.

Responses
200

Empty response

400

The response for a bad request.

422

Validation Error

post/hostedextractors/sources/delete
Request samples
application/json
{
  • "items": [
    ],
  • "ignoreUnknownIds": true,
  • "force": true
}
Response samples
application/json
{ }

Update Sources

Required capabilities: hostedExtractors:WRITE

Update up to 100 sources.

Securityoidc-token or oauth2-client-credentials or oauth2-open-industrial-data or oauth2-auth-code
Request
Request Body schema: application/json

Sources to update.

required
Array of any (Items) [ 1 .. 10 ] items
Responses
200

List of updated sources.

400

The response for a bad request.

422

Validation Error

post/hostedextractors/sources/update
Request samples
application/json
{
  • "items": [
    ]
}
Response samples
application/json
{
  • "items": [
    ]
}
➔ Next to Jobs