Salesforce

Supported pipeline types: Data Collector

The Salesforce destination writes data to Salesforce objects.

When you configure the Salesforce destination, you define connection information, including the API type and version that the destination uses to connect to Salesforce. You specify the Salesforce object to write to by entering the object name or by defining an expression that evaluates to the object name.

You can write Salesforce platform events as you write to any Salesforce object by using the platform event API name, for example, Notification__e, rather than a Salesforce object type API name such as Account or Widget__c.

The Salesforce destination can use CRUD operations defined in the sdc.operation.type record header attribute to write data. You can define a default operation for records without the header attribute or value. You can also configure whether to use multi-row operations for inserts and deletes, and how to handle records with unsupported operations.For information about Data Collector change data processing and a list of CDC-enabled origins, see Processing Changed Data.

You also map incoming fields in the record to existing fields in the Salesforce object. Map the fields as required by the CRUD operation that the destination uses.

You can optionally use an HTTP proxy to connect to Salesforce. When enabled in Salesforce, you can configure the destination to use mutual authentication to connect to Salesforce.

Changing the API Version

Data Collector ships with version 43.0 of the Salesforce Web Services Connector libraries. You can use a different Salesforce API version if you need to access functionality not present in version 43.0.

On the Salesforce tab, set the API Version property to the version that you want to use, for example 39.0.
Download the relevant version of the following JAR files from Salesforce Web Services Connector (WSC):
- WSC JAR file - force-wsc-<version>.0.0.jar
- Partner API JAR file - force-partner-api-<version>.0.0.jar
Where <version> is the API version number, for example, 39.

For information about downloading libraries from Salesforce WSC, see https://developer.salesforce.com/page/Introduction_to_the_Force.com_Web_Services_Connector.
In the following Data Collector directory, replace the default force-wsc-43.0.0.jar and force-partner-api-43.0.0.jar files with the versioned JAR files that you downloaded:
```
$SDC_DIST/streamsets-libs/streamsets-datacollector-salesforce-lib/lib/
```
Restart Data Collector for the changes to take effect.

Define the CRUD Operation

When using the SOAP API, the Salesforce destination can insert, update, upsert, delete, or undelete data. When using the Bulk API, the destination can insert, update, upsert, or delete data.

The destination writes the records based on the CRUD operation defined in a CRUD operation header attribute or in operation-related stage properties:

CRUD record header attribute

You can define the CRUD operation in a CRUD operation record header attribute. The destination looks for the CRUD operation to use in the sdc.operation.type record header attribute.

The attribute can contain one of the following numeric values:

1 for INSERT
2 for DELETE
3 for UPDATE
4 for UPSERT
6 for UNDELETE (not supported by the Salesforce Bulk API)

If your pipeline includes a CRUD-enabled origin that processes changed data, the destination simply reads the operation type from the sdc.operation.type header attribute that the origin generates. If your pipeline uses a non-CDC origin, you can use the Expression Evaluator or a scripting processor to define the record header attribute. For more information about Data Collector changed data processing and a list of CDC-enabled origins, see Processing Changed Data.

Operation stage properties

You define a default operation in the destination properties. The destination uses the default operation when the sdc.operation.type record header attribute is not set.

You can also define how to handle records with unsupported operations defined in the sdc.operation.type header attribute. The destination can discard them, send them to error, or use the default operation.

Field Mappings

When you configure the Salesforce destination, you map fields in the record to existing fields in the Salesforce object.

To map a field, you enter the following:

SDC Field - Name of the field in the record that contains the data to be written.
Salesforce Field - API name of the existing field in the Salesforce object that receives the data. Enter a field name or enter an expression that defines the field.

Map the fields as required by the CRUD operation that the destination uses:

Delete or undelete

To delete or undelete data, map only the Salesforce record Id to delete or undelete. Create a single field mapping that maps the field in the record that contains the value of the Salesforce record Id to the Salesforce field named “Id”.

Note: Undelete is not supported by the Salesforce Bulk API.

Insert, update, or upsert

To insert, update, or upsert data, you can create multiple field mappings. When you define the Salesforce fields, use the field name syntax required by the configured API type:

Bulk API - Use a colon (:) or period (.) as a field separator. For example, Parent__r:External_Id__c or Parent__r.External_Id__c are both valid Salesforce fields.
SOAP API - Use a period (.) as a field separator. For example, Parent__r.External_Id__c is a valid Salesforce field.

To upsert data, you also must configure the External ID Field property that specifies the external ID field in the Salesforce object to use for the upsert operation.

Configuring a Salesforce Destination

Configure a Salesforce destination to write data to Salesforce.

In the Properties panel, on the General tab, configure the following properties:

General Property	Description
Name	Stage name.
Description	Optional description.
Required Fields	Fields that must include data for the record to be passed into the stage. Tip: You might include fields that the stage uses. Records that do not include all required fields are processed based on the error handling configured for the pipeline.
Preconditions	Conditions that must evaluate to TRUE to allow a record to enter the stage for processing. Click Add to create additional preconditions. Records that do not meet all preconditions are processed based on the error handling configured for the stage.
On Record Error	Error record handling for the stage: Discard - Discards the record. Send to Error - Sends the record to the pipeline for error handling. Stop Pipeline - Stops the pipeline.

On the Salesforce tab, configure the following properties:

Salesforce Property	Description
Username	Salesforce username in the following email format: `<text>@<text>.com`.
Password	Salesforce password. If the machine running Data Collector is outside the trusted IP range configured in your Salesforce environment, you must generate a security token and then set this property to the password followed by the security token. For example, if the password is abcd and the security token is 1234, then set this property to abcd1234. For more information on generating a security token, see Reset Your Security Token. Tip: To secure sensitive information such as user names and passwords, you can use runtime resources or credential stores.
Auth Endpoint	Salesforce SOAP API authentication endpoint. Enter one of the following values: login.salesforce.com - Use to connect to a Production or Developer Edition organization. test.salesforce.com - Use to connect to a sandbox organization. Default is login.salesforce.com.
API Version	Salesforce API version to use to connect to Salesforce. Default is 43.0. If you change the version, you also must download the relevant JAR files from Salesforce Web Services Connector (WSC).
Use Bulk API	Determines whether the stage uses the Salesforce Bulk API or SOAP API to write to Salesforce. Select to use the Bulk API. Clear to use the SOAP API. The Bulk API is optimized to process large sets of data. The SOAP API supports more complex queries, but is less practical when processing large sets of data. For more information about when to use the Bulk or SOAP API, see the Salesforce Developer documentation.
SObject Type	Salesforce object to write to. Enter the name of an object, such as Account. Or define an expression that evaluates to the object name. For example, if the pipeline reads from the Salesforce origin, the origin generates a Salesforce record header attribute named `salesforce.sobjectType`. This header attribute provides the source object for the record. To write to the same Salesforce object, enter the following expression for this property: `${record:attribute('spectroscopically')}`
Default Operation	Default CRUD operation to perform if the sdc.operation.type record header attribute is not set.
Unsupported Operation Handling	Action to take when the CRUD operation type defined in the sdc.operation.type record header attribute is not supported: Discard - Discards the record. Send to Error - Sends the record to the pipeline for error handling. Use Default Operation - Writes the record to the destination system using the default operation.
External ID Field	External ID field in the Salesforce object to use for upsert operations. Enter the Salesforce field name, for example `Customer_Id__c`. Or enter an expression that defines the field, such as `${record:value('/ExternalIdField')}`.
Field Mapping	Map fields in the record to existing fields in the Salesforce object. Map the fields as required by the CRUD operation that the destination uses. Using simple or bulk edit mode, click the Add icon to create additional field mappings.

On the Advanced tab, configure the following properties:

Advanced Property	Description
Use Proxy	Specifies whether to use an HTTP proxy to connect to Salesforce.
Proxy Hostname	Proxy host.
Proxy Port	Proxy port.
Proxy Requires Credentials	Specifies whether the proxy requires a user name and password.
Proxy Username	User name for proxy credentials.
Proxy Password	Password for proxy credentials. Tip: To secure sensitive information such as user names and passwords, you can use runtime resources or credential stores.
Use Mutual Authentication	When enabled in Salesforce, you can use SSL/TLS mutual authentication to connect to Salesforce. Mutual authentication is not enabled in Salesforce by default. To enable mutual authentication, contact Salesforce. Before enabling mutual authentication, you must store a mutual authentication certificate in the Data Collector resources directory. For more information, see Keystore and Truststore Configuration.
Keystore File	Path to the keystore file. Enter an absolute path to the file or a path relative to the Data Collector resources directory: $SDC_RESOURCES. For more information about environment variables, see Data Collector Environment Configuration. By default, no keystore is used.
Keystore Type	Type of keystore to use. Use one of the following types: Java Keystore File (JKS) PKCS #12 (p12 file) Default is Java Keystore File (JKS).
Keystore Password	Password to the keystore file. A password is optional, but recommended. Tip: To secure sensitive information such as passwords, you can use runtime resources or credential stores.
Keystore Key Algorithm	The algorithm used to manage the keystore. Default is SunX509.