Transformations enable users to use Spark SQL queries to transform data from the CDF staging area, RAW, into the CDF data model.
The number of concurrent (parallel) jobs are governed by limits. If a request exceeds one of the limits, the job fails to run. This limitation also applies to scheduled transformations.
The limits for the transformation service are described in the table below. Note that under high load, some deviation from the limits might occur for short periods as the service scales up.
Limit | Per project |
---|---|
Concurrency | 10 parallel jobs |