Connect Databricks to Candu
Use Databricks to keep Candu user traits, account traits, group membership, and custom events up to date.
This integration uses a Databricks notebook to read a table or view and send selected data to Candu’s inbound webhook. You can run the notebook manually or schedule it with Databricks Jobs.
Note: This is an API-based connector recipe, not a one-click managed integration. Databricks provides the notebook runtime and scheduler. You control the source view, field mapping, sync frequency, and operational safeguards.
What you can sync
You can send three types of data from Databricks:
User traits: Attributes such as email, role, plan, lifecycle stage, or product usage score.
Group data: Account or company traits, plus the relationship between a user and an account.
User events: Actions or milestones such as completing onboarding, inviting a teammate, or reaching an activation threshold.
Candu creates new trait names when it first receives them. You do not need to define each trait in Candu before the first sync.
Before you start
You will need:
A Databricks workspace with access to the source table or view
Permission to create and run a notebook
A Candu API key from Settings → Workspaces → Access Keys
A stable ID shared by Databricks and Candu for each user
A stable group or account ID if you want to sync account-level data
We recommend creating a curated Databricks view containing only the approved fields you want to send to Candu. Avoid pointing the notebook at a broad production table.
A test-only Databricks source row with the fields selected for Candu.
How the integration works
Databricks table or view
↓
Databricks notebook
↓
POST https://api.candu.ai/api/eventWebhook
↓
Candu users, groups, traits, and events
The Candu API accepts identify, group, and track payloads. A successful request returns 204 No Content.
Step 1: Prepare your source data
Create a table or view with the user and account fields you want to sync.
For example:
SELECT
user_id,
email,
role,
lifecycle_stage,
account_id,
account_name,
account_plan,
updated_at
FROM analytics.candu_sync_users;
Use the same user_id that your application uses to identify the user in Candu. Changing identity keys between systems can create duplicate users or attach data to the wrong record.
Step 2: Store your Candu API key securely
Store the API key as a Databricks secret. Do not paste it directly into notebook source code.
The examples below retrieve a Unity Catalog secret named candu_api_token:
candu_token = dbutils.secrets.get(
catalog="YOUR_CATALOG",
schema="YOUR_SCHEMA",
key="candu_api_token",
)
Restrict access to the secret and source view to the service identity that runs the sync. Do not include the secret value in screenshots, notebook output, or logs.
Step 3: Create the notebook
Create a Python notebook and add the following setup:
import requests
SOURCE_TABLE = "analytics.candu_sync_users"
CANDU_ENDPOINT = "https://api.candu.ai/api/eventWebhook"
candu_token = dbutils.secrets.get(
catalog="YOUR_CATALOG",
schema="YOUR_SCHEMA",
key="candu_api_token",
)
headers = {
"Authorization": f"Bearer {candu_token}",
"Content-Type": "application/json",
}
def send_to_candu(payload):
response = requests.post(
CANDU_ENDPOINT,
headers=headers,
json=payload,
timeout=20,
)
if response.status_code != 204:
raise RuntimeError(
f"Candu rejected the request: "
f"{response.status_code} {response.text[:200]}"
)
Sync user traits
Read the source rows and send an identify call for each user:
rows = spark.table(SOURCE_TABLE).orderBy("user_id").collect()
for row in rows:
send_to_candu({
"type": "identify",
"userId": row.user_id,
"traits": {
"email": row.email,
"role": row.role,
"lifecycleStage": row.lifecycle_stage,
"sourceUpdatedAt": row.updated_at.isoformat(),
},
})
print({"syncedUsers": len(rows)})Trait names are created automatically when Candu first receives them. Later identify calls update the same traits for the same userId.
Sync account traits and membership
Use a group call to create or update an account and associate a user with it:
for row in rows:
if row.account_id:
send_to_candu({
"type": "group",
"groupId": row.account_id,
"userId": row.user_id,
"traits": {
"name": row.account_name,
"plan": row.account_plan,
"sourceUpdatedAt": row.updated_at.isoformat(),
},
})
The groupId identifies the account. Including userId adds that user to the group. Later calls with the same groupId update the existing account rather than creating another one.
Send a custom user event
Use a track call for actions or milestones:
send_to_candu({
"type": "track",
"userId": "USER_ID",
"event": "completed_onboarding",
"properties": {
"source": "databricks",
"journey": "admin_setup",
},
})Custom events can be used in Candu segments and goals. Use a stable, descriptive event name.
If you are syncing an events table, add a reliable watermark or processed-event ID. Re-sending a track call records another occurrence, so event syncs need stronger deduplication than trait syncs.
Step 4: Run and verify the sync
Run the notebook manually before adding a schedule.
A successful request returns:
204 No Content
Databricks notebook output showing three successful user sync responses.
Databricks notebook output showing successful group, membership, and event responses.
Verify each data type in Candu:
User traits: Go to Analytics → Users, open a user, and select Traits.
Groups: Go to Analytics → Groups, open an account, and check Active Users and Traits.
Events: Go to Analytics → Events, or open a user and select Activity.
Net-new Databricks traits on the test user in Candu.
Account traits on the Databricks test group in Candu.
The Databricks test user linked to the test group in Candu.
Identify, group, and track calls in the test user’s Candu Activity timeline.
Step 5: Schedule the notebook
When the manual sync is working:
Open the notebook in Databricks.
Click Schedule.
Click Add schedule.
Choose an interval or cron schedule.
Select the compute configuration.
Create and enable the job.
A daily Databricks schedule created for validation and left paused.
Choose a frequency that matches how quickly Candu needs the data. Daily may be enough for account-plan or lifecycle traits. Hourly may be more appropriate when a health score or lifecycle change needs to affect same-day targeting.
The validation schedule shown above is set to run every day, but it is paused. No automatic sync is currently running. All successful validation requests were initiated manually.
Recommended production safeguards
Before syncing production data:
Use a least-privilege service identity and curated source view.
Allowlist the exact fields that may be sent to Candu.
Confirm the canonical
userIdandgroupIdwith your application team.Use an incremental
updated_atwatermark instead of reading the entire table.Add bounded retries with exponential backoff.
Write successes and failures to an audit table.
Add a dead-letter path for rows that repeatedly fail.
Monitor Databricks compute cost and Candu event volume.
Define how null values, deleted users, removed group memberships, and retired traits should behave.
Review Candu’s current external-event fair-use policy before syncing high-volume events.
Troubleshooting
The request returns 400
The payload is malformed. Check that:
typeisidentify,group, ortrackidentifyincludesuserIdandtraitsgroupincludesgroupIdtrackincludesuserIdandeventTrait and property values use supported string, number, or boolean types
Authentication fails
Confirm that the API key belongs to the same Candu workspace where you are checking the data. Verify that the Databricks job identity can read the secret.
The request returns 204, but the expected user is missing
Check the userId. It must exactly match the identity used by your application’s Candu installation. An email address and an internal database ID are treated as different identities.
Duplicate users or groups appear
The source is probably sending different identity keys for the same record. Use one stable userId per user and one stable groupId per account.
An event appears more than once
Trait calls update existing values, but track calls record occurrences. Add an event ID or source watermark so the notebook does not resend already-processed event rows.
What we validated
We tested this integration end to end with synthetic data:
Created three new users and net-new user traits
Updated an existing user trait without creating a duplicate user
Created a group with account traits and linked a user to it
Updated an existing group trait without creating a duplicate group
Sent a custom event and verified it in Candu Events and the user Activity timeline
Created a native daily Databricks schedule, confirmed the scheduling configuration, and left it paused
The validation used test-only data. Production throughput, batching, rate limits, failure recovery, null and deletion semantics, and customer-specific identity mapping should be tested against your expected data volume before launch.