11.26.2012 Technical Innovation

Salesforce Integration with REST, OAUTH, and Entities

Salesforce logo The Drupal Salesforce Suite has been around since Drupal 5, having undergone many transformations in trying to keep pace with both Drupal and Salesforce API changes. The result is a feature set as impressive as it is ambitious, although the incremental updates and additions have come at a cost of significant technical debt, inconsistent API usage, a monolithic architecture, and fragility.

ThinkShout, not being strangers to major rewrites, even of our own modules, and driven by our need to integrate Salesforce with our native Drupal CRM, RedHen, undertook the challenge to completely rewrite the Salesforce suite this fall.

Why a rewrite?


We started by reaching out to the current module maintainers to better understand the state of the module, it’s roadmap, if any, and the current teams involvement, as well as to feel out their interest in participating in a rewrite. The entire team, kostajh in particular, has been very receptive and helpful, and we eventually agreed that ThinkShout would do the rewrite in a new branch, 7.x-3.x, in the current project. There was talk of collaboration, but busy schedules and priorities got in the way as they often do and we’re approaching an MVP release on our own. The maintainers gave ThinkShout access to the repository and project page, but not administrative control over the repository, understandably. That put us in a minor bind, however, as we are committing many hundreds of hours to a project we don't have any control over. As a result, we're developing the project on [GitHub](https://github.com/thinkshout/salesforce) and mirroring the code to [7.x-3.x branch on Drupal.org](http://drupalcode.org/project/salesforce.git/tree/refs/heads/7.x-3.x). This will likely change, but that's our plan for now. UPDATE: Project development is now taking place on Drupal.org, although we’ll continue mirroring to GitHub.

Core features

  1. OAuth 2.0 authorization
  2. Object oriented REST API wrapper for use within Drupal.
  3. Drupal entity and Salesforce object mapping system
    • Field level mappings, including granular directionality and support for external keys.
    • Map Drupal bundles to Salesforce record types.
    • Select per mapping trigger points, E.g., Drupal entity create, Salesforce object update, etc.
    • Select batch or realtime processing per mapping.
  4. Push Drupal entities to Salesforce.
  5. Pull Salesforce objects into Drupal.
  6. Lightweight wrapper around the SOAP API when needed, using the same OAuth authorization.

In addition, with the systems modular OO design, we are laying the groundwork for a contrib ecosystem to handle more specific needs like Webform integration and lead generation. And, of course, developers can use the module as a starting point for their own solutions, be it importing Drupal users as contacts, lead generation through blog comments, etc.


API selection and OAuth

Salesforce provides a veritable alphabet soup of API options, including SOAP, bulk, meta, streaming, apex, and REST. In their own words, the REST API is the best choice for “mobile applications and Web projects”. Well, Drupal is certainly a web platform, so we figured that’s the right place to start. In addition, we found the REST API provided the following benefits:

oauth 2 In addition, the REST API allows, or rather requires, the use of OAuath 2.0 for authorization. While not perfect, the OAuth flow is more secure and flexible than providing a single username and password. The OAuth process, in short:

  1. Create a remote application within the Salesforce instance. The callback URL must be in the format https://mydrupalsite.com/salesforce/oauch_callback. Obviously replace mydrupalsite.com with the url of the site being integrated with Salesforce. Also note that HTTPS is required for a secure authorization to take place. Yes, that means this module will not work on sites that don’t support HTTPS.
  2. Obtain the consumer key and secret from the remote application created in step 1 and enter it in the authorization form. Click the authorize button. authorization form
  3. This will initiate 2 requests to Salesforce. The first redirects the user to Salesforce, sending along the consumer key and callback URL. If the user authorizes the remote application, the same one created in step 1, in our case a Drupal site, Salesforce responds with an authorization code.
  4. The Module then sends another request to Salesforce with the authorization code obtained in step 3, along with the consumer key, secret, and callback URL, and gets a response containing a refresh token and API instance URL.
  5. The refresh token is used to obtain an ephermal access token, analogous to a session ID, which is used to access the REST resources. The access token is actually only stored in session storage to emphasize it’s temporary nature and a new one is obtained when the access token expires on the Salesforce side or the Drupal session ends. There’s certainly room for improvement in this process, but it seems to work for now.

As of writing, the Drupal OAuth module, which we preferred to leverage, did not yet support the OAuth 2.0 specification, so we were forced to go on our own. If/when a stable OAuth 2.0 library is released within Drupal, we’d certainly rather take advantage of that then reinvent any wheels.


Entity API metadata wrappers

Are, simply put, awesome. The Entity API metadata wrappers provided a silky smooth unified process for getting and setting an entity’s fields and properties. To demonstrate, this little snippet of code handles the mapping of Drupal data to Salesforce for some 90% of use cases, <?php $drupal_fields_array = explode(‘:’, $fieldmap[‘drupal_field’][‘fieldmap_value’]); $parent = $entity_wrapper; foreach ($drupal_fields_array as $drupal_field) { if ($parent instanceof EntityListWrapper) { // First list<> types, get the property from the first item $child_wrapper = $parent->get(0)->{$drupal_field}; } else { $child_wrapper = $parent->{$drupal_field}; } $parent = $child_wrapper; }

$value = $child_wrapper->value(); ?>

Similiarly, when importing Salesforce data, we can set entity values like this, <?php $drupal_fields_array = explode(‘:’, $field_map[‘drupal_field’][‘fieldmap_value’]); $parent = $entity_wrapper; foreach ($drupal_fields_array as $drupal_field) { if ($parent instanceof EntityListWrapper) { $child_wrapper = $parent->get(0)->{$drupal_field}; } else { $child_wrapper = $parent->{$drupal_field}; } $parent = $child_wrapper; } $fieldmap_type = salesforce_mapping_get_fieldmap_types($field_map[‘drupal_field’][‘fieldmap_type’]); $value = call_user_func($fieldmap_type[‘pull_value_callback’], $parent, $sf_object, $field_map);

// Allow this value to be altered before assigning to the entity. drupal_alter(‘salesforce_pull_entity_value’, $value, $field_map, $sf_object); if (isset($value)) { if ($parent instanceof EntityListWrapper) { $parent->offsetSet(0, $value); } else { $parent->set($value); } } ?>

Field mappings

Salesforce field mappings are the configuraiton that guides how Salesforce objects and Drupal entities are syncronized. Highlihts include:

Flexible fieldmap type system

When mapping fields, we needed a flexible system to handle different type of data mappings and how to derive the correct values. We modeled the fieldmap type system very loosely after Drupal entity definition hooks. Any module can implement hook_salesforce_mapping_fieldmap_type() to define a type of field mapping. The module ships with 4 fieldmap types:

Each fieldmap type must define:

A fieldmap type definition will look like this, <?php /**

return $types; } ?>

The fieldmap UI allows an admin to select a fieldmap type which then presents either a list of options, E.g. a list of fields and properties in the case of a Property fieldmap type, or a textbox, E.g., for Tokens. After passing an optional validation step, the value is saved with the field mapping. When data is synced, the push or pull value callback is used to get the correct value. We’re hopeful this approach will provide the right balance between flexibility and out of the box utility in mapping simple and complex data structures.

field mappings

Where things are heading

We’ll be launching several sites before years using the new version of the Salesforce module and are including it in a private distribution we’ve developed for the Forum of Regional Associations of Grantmakers, who have generously sponsored the initial development, and we already feel it’s alpha quality and at the “MVP” stage. There are certain to be lots of bugs, and perhaps even major API changes, but that’s to be expected for a project of this complexity and with a such a wide array of use cases. We’re hoping to engage even more with the community at this time to gather architectural feedback and help work out the kinks.

Salesforce is a very widely used, powerful platform, and having a canonical, robust Drupal integration is a major win for the entire community. We’re hopeful this is a good starting point to achieve that goal.

Get In Touch

Questions? Comments? We want to know! Drop us a line and let’s start talking.

Learn More
Get In Touch

Related Blog Posts