Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 13 additions & 5 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,24 @@ Not verified, needs a Make account and a BatchData test token (a sandbox token r

- Import with the VS Code extension (the file layout follows public examples, not a Make validator).
- Connection check. It POSTs a fixed autocomplete query (`searchCriteria.query` = `1600 Penn`, `take` 1) to `/v1/address/autocomplete` and is valid on HTTP 2xx only when `body.result` is present, so a 200 HTML error page is rejected as `InvalidAccessTokenError`. An earlier empty `/v1/address/verify` check (valid on 400 or 2xx) failed in Make because HTTP 400-599 are errors and `response.valid` cannot override that. Autocomplete was chosen over `GET /v2/property/search-sessions?take=1` because search-sessions can return 403 for tokens without that permission.
- List-item outputs (Get Property Details, Verify Address, and the billed instant trigger's property lookup) use IML 1-based index access (`path[1]`) instead of `first()`, which throws `IMLError` when the value is not an array. `response.valid` marks a missing API object as `InvalidAccessTokenError`, so a 401 JSON body or a 200 HTML page does not surface as `IMLError: Function 'first' finished with error! Invalid array.` A first attempt used `first(ifempty(path, emptyarray))`, but `emptyarray` is a scenario-mapper constant and is not defined in custom-app IML.
- List-item outputs (Get Property Details and Verify Address) use IML 1-based index access (`path[1]`) instead of `first()`, which throws `IMLError` when the value is not an array. `response.valid` marks a missing API object as `InvalidAccessTokenError`, so a 401 JSON body or a 200 HTML page does not surface as `IMLError: Function 'first' finished with error! Invalid array.` A first attempt used `first(ifempty(path, emptyarray))`, but `emptyarray` is a scenario-mapper constant and is not defined in custom-app IML.
- Optional fields. The templates use `ifempty(x, undefined)` and `omit()` so empty fields are not sent. Nested optional filters (for example Permit Status) are collection parameters that are dropped when empty. Check that an unfilled nested collection (for example Building) is not sent as `{}`.
- Instant triggers. Check that turning a scenario on creates the property subscription (attach), that a test event arrives with `eventId` set, and that turning it off deletes the subscription (detach, `externalHookId`). Create hooks only with a sandbox token: a subscription counts towards the team's limit of 5 and cannot be edited.
- Polling triggers (`Watch New Search Sessions`, `Watch New Properties in a Search Session`): confirm the date ordering and that a first run does not replay old items as expected.
- Dynamic dropdowns (`rpc://listSearchSessions`, `rpc://listMonitors`): confirm the label and value mapping.
- Dynamic dropdown (`rpc://listSearchSessions`): confirm the label and value mapping.
- The `{{...}}` spread in the Search Properties output (adds `searchMeta` next to each property).
- Make an API Call: the wallet guard redirects any path containing `wallet` to a non-existent path. IML cannot raise a validation error, so this is best effort. Decide whether to keep the module.

- Webhook dedupe. The webhook sets `uid` to `body.eventId`; confirm in Make that a repeated event does not start a second run.
- Callback keys. IML cannot strip keys recursively. Make an API Call blocks (sends nothing) any body that contains `webhook` or a JSON unicode escape (`u00`). The Additional Criteria JSON fields on search, session and monitor modules are not stripped; BatchData's synchronous endpoints ignore callback keys, and the monitor and trigger modules fix `deliveryConfig` themselves. A Make custom function that strips the keys would close this gap.
- Callback keys. IML cannot strip keys recursively. Make an API Call blocks (sends nothing) any body that contains `webhook` or a JSON unicode escape (`u00`). The Additional Criteria JSON fields on search and session modules are not stripped; BatchData's synchronous endpoints ignore callback keys. A Make custom function that strips the keys would close this gap.

## Property monitor modules (hidden on Make)

Version 0.4.0 removed the six property monitor modules: Create Property Monitor (Stream Delivery), List Property Monitors, Get Property Monitor, Delete Property Monitor, Watch Property Monitor Events (Instant) and Watch Property Monitor Events with Property Record (Instant, Billed). It also removed the `monitorEvents` webhook and the `listMonitors` RPC, which only these modules used. The reason: the BatchData sandbox token for the Make review does not have the property-subscription permission, so the reviewers cannot test these modules.

On Make, the app `batchdata-r6b83g` is public and in review, and Make does not let a public app delete components. The six modules are set to private there instead. The `monitorEvents` webhook and the `listMonitors` RPC stay on Make, and no module uses them.

`scripts/catalog-modules.mjs` filters the monitor operations out of the vendored catalog, so `scripts/catalog.json` stays unchanged. Make an API Call still blocks POST, PUT and PATCH calls to `property-subscription`.

To restore the modules, revert this commit, run `node scripts/generate.mjs`, deploy to Make, and set the six modules to public again.

## Decisions

Expand Down
31 changes: 1 addition & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@

![BatchData](assets/logo-512.png)

BatchData gives you property, owner and address data through an API. This app brings it into your Make scenarios, so you can find properties, enrich them with owner contact details, verify addresses and phone numbers, and react to property changes without writing code.
BatchData gives you property, owner and address data through an API. This app brings it into your Make scenarios, so you can find properties, enrich them with owner contact details, and verify addresses and phone numbers without writing code.

Typical uses:

- Build a list of properties that match your filters and send it to a spreadsheet, CRM or marketing tool.
- Look up one address and add the full property record, ownership and sale history to a lead.
- Find the owner's phone numbers and emails (skip trace) for a property, and check the numbers against Do Not Call and TCPA litigator lists.
- Clean up addresses before mailing, or turn an address into coordinates.
- Start a scenario the moment a property you are watching changes.

Support: support@batchservice.com

Expand Down Expand Up @@ -71,24 +70,12 @@ A search session remembers what it has already returned, so each run gives you o
| List Search Sessions | Lists the sessions on your team. |
| Delete Search Session | Deletes a session. |

### Property monitors

A monitor reports when a property is added to it, removed from it or changed.

| Module | What it does |
| --- | --- |
| Create Property Monitor (Stream Delivery) | Creates a monitor that delivers to AWS Kinesis or Azure Event Hubs. |
| List Property Monitors, Get Property Monitor | Reads your monitors. |
| Delete Property Monitor | Deletes a monitor. |

### Triggers

| Module | Type | What it does |
| --- | --- | --- |
| Watch New Search Sessions | Polling | Runs when a new search session appears on your team. |
| Watch New Properties in a Search Session (Billed) | Polling | Runs for each new property a session delivers. |
| Watch Property Monitor Events (Instant) | Instant | Runs when a property is added to, removed from or changed in a monitor. |
| Watch Property Monitor Events with Property Record (Instant, Billed) | Instant | The same, and adds the full property record to each event. |

### Other

Expand All @@ -110,27 +97,11 @@ Your BatchData plan decides what each call costs. Each module's description in M
- Skip trace modules bill per matched property.
- Address, phone and compliance checks bill per item checked.
- Autocomplete, geocoding and the modules that list, get or delete search sessions do not bill property records.
- Property monitors and the events they deliver are billed according to your BatchData plan.
- Get Property Permits bills per property requested, not per permit. A Limit above 100 sends more than one request.
- Watch New Properties in a Search Session (Billed) bills every property it delivers, on every scheduled run.
- Watch Property Monitor Events with Property Record (Instant, Billed) adds one property lookup per event.

For pricing questions, contact support@batchservice.com.

## Property monitor events

When you turn a scenario on, the instant trigger creates a monitor that delivers to that scenario's own Make webhook. When you turn it off, or delete the scenario, the monitor is deleted. You do not enter a callback URL.

Each event contains the event ID, property ID, address and parcel hashes, the IDs of the monitors that added or removed the property, and the time it changed. It does not contain the property record. Use the module with **Property Record** to add it. Use the event ID to skip repeats.

Good to know:

- You can have up to 5 monitors per team, so avoid many scenarios that each create one.
- A monitor cannot be edited. Delete it and create a new one.
- A new monitor can take up to 7 days to start delivering.
- Delivery is best effort, with 4 attempts per event, so the same event can arrive more than once.
- Make has no polling trigger for new monitors. Use List Property Monitors on a schedule instead.

## Phone compliance

Skip trace and phone modules return data only. You are responsible for following the TCPA and Do Not Call rules before you contact anyone. Skip trace leaves out phone numbers flagged as TCPA litigators unless you switch on **Include phone numbers flagged as TCPA litigators**.
Expand Down
133 changes: 2 additions & 131 deletions makecomapp.json
Original file line number Diff line number Diff line change
Expand Up @@ -302,58 +302,6 @@
"samples": "modules/deleteSearchSession/deleteSearchSession.samples.iml.json"
}
},
"listMonitors": {
"moduleType": "search",
"label": "List Property Monitors",
"description": "Lists your property monitors. Needs a token with the property subscription permission. Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event).",
"actionCrud": "read",
"connection": "batchdata",
"codeFiles": {
"communication": "modules/listMonitors/listMonitors.communication.iml.json",
"mappableParams": "modules/listMonitors/listMonitors.mappable-params.iml.json",
"interface": "modules/listMonitors/listMonitors.interface.iml.json",
"samples": "modules/listMonitors/listMonitors.samples.iml.json"
}
},
"getMonitor": {
"moduleType": "search",
"label": "Get Property Monitor",
"description": "Returns one property monitor with its search criteria and delivery settings (secrets are masked). Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event).",
"actionCrud": "read",
"connection": "batchdata",
"codeFiles": {
"communication": "modules/getMonitor/getMonitor.communication.iml.json",
"mappableParams": "modules/getMonitor/getMonitor.mappable-params.iml.json",
"interface": "modules/getMonitor/getMonitor.interface.iml.json",
"samples": "modules/getMonitor/getMonitor.samples.iml.json"
}
},
"createMonitor": {
"moduleType": "action",
"label": "Create Property Monitor (Stream Delivery)",
"description": "Creates a property monitor that streams changes for matching properties to your own AWS Kinesis stream or Azure Event Hub. For webhook delivery use the \"Watch Property Monitor Events (Instant)\" trigger, which sets up and removes its own monitor. Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event).",
"actionCrud": "create",
"connection": "batchdata",
"codeFiles": {
"communication": "modules/createMonitor/createMonitor.communication.iml.json",
"mappableParams": "modules/createMonitor/createMonitor.mappable-params.iml.json",
"interface": "modules/createMonitor/createMonitor.interface.iml.json",
"samples": "modules/createMonitor/createMonitor.samples.iml.json"
}
},
"deleteMonitor": {
"moduleType": "action",
"label": "Delete Property Monitor",
"description": "Deletes a property monitor so it stops delivering events. Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event).",
"actionCrud": "delete",
"connection": "batchdata",
"codeFiles": {
"communication": "modules/deleteMonitor/deleteMonitor.communication.iml.json",
"mappableParams": "modules/deleteMonitor/deleteMonitor.mappable-params.iml.json",
"interface": "modules/deleteMonitor/deleteMonitor.interface.iml.json",
"samples": "modules/deleteMonitor/deleteMonitor.samples.iml.json"
}
},
"watchSearchSessions": {
"moduleType": "trigger",
"label": "Watch New Search Sessions",
Expand All @@ -377,45 +325,9 @@
"interface": "modules/watchSessionProperties/watchSessionProperties.interface.iml.json",
"samples": "modules/watchSessionProperties/watchSessionProperties.samples.iml.json"
}
},
"watchMonitorEvents": {
"moduleType": "instant_trigger",
"label": "Watch Property Monitor Events (Instant)",
"description": "Triggers when BatchData reports that a property matching your criteria was added to or removed from a monitor, or changed (instant). Turning the scenario on creates a monitor that delivers to this trigger's own hook URL; turning it off deletes that monitor. Each event carries the property ID and change details. Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. To get the full property record with each event, use \"Watch Property Monitor Events with Property Record (Instant, Billed)\" instead. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event).",
"connection": "batchdata",
"webhook": "monitorEvents",
"codeFiles": {
"communication": "modules/watchMonitorEvents/watchMonitorEvents.communication.iml.json",
"interface": "modules/watchMonitorEvents/watchMonitorEvents.interface.iml.json",
"samples": "modules/watchMonitorEvents/watchMonitorEvents.samples.iml.json"
}
},
"watchMonitorEventsWithProperty": {
"moduleType": "instant_trigger",
"label": "Watch Property Monitor Events with Property Record (Instant, Billed)",
"description": "Triggers when BatchData reports that a property matching your criteria was added to or removed from a monitor, or changed (instant). Turning the scenario on creates a monitor that delivers to this trigger's own hook URL; turning it off deletes that monitor. Each event carries the property ID and change details. Monitors and the events they deliver are billed according to your BatchData plan; contact support@batchservice.com for pricing. Limits: up to 5 monitors per team, a monitor cannot be edited after it is created (delete and recreate it), a new monitor can take up to 7 days to start delivering, and delivery is best effort (4 attempts per event). This version also looks up the full property record for every event and adds it to the output as \"property\". The lookup is billed per event, per matched property, on top of the monitor charges.",
"connection": "batchdata",
"webhook": "monitorEvents",
"codeFiles": {
"communication": "modules/watchMonitorEventsWithProperty/watchMonitorEventsWithProperty.communication.iml.json",
"interface": "modules/watchMonitorEventsWithProperty/watchMonitorEventsWithProperty.interface.iml.json",
"samples": "modules/watchMonitorEventsWithProperty/watchMonitorEventsWithProperty.samples.iml.json"
}
}
},
"webhook": {
"monitorEvents": {
"label": "Property monitor events",
"webhookType": "web",
"connection": "batchdata",
"codeFiles": {
"communication": "webhooks/monitorEvents/monitorEvents.communication.iml.json",
"params": "webhooks/monitorEvents/monitorEvents.params.iml.json",
"attach": "webhooks/monitorEvents/monitorEvents.attach.iml.json",
"detach": "webhooks/monitorEvents/monitorEvents.detach.iml.json"
}
}
},
"webhook": {},
"rpc": {
"listSearchSessions": {
"label": "List search sessions",
Expand All @@ -424,14 +336,6 @@
"communication": "rpcs/listSearchSessions/listSearchSessions.communication.iml.json",
"params": "rpcs/listSearchSessions/listSearchSessions.params.iml.json"
}
},
"listMonitors": {
"label": "List property monitors",
"connection": "batchdata",
"codeFiles": {
"communication": "rpcs/listMonitors/listMonitors.communication.iml.json",
"params": "rpcs/listMonitors/listMonitors.params.iml.json"
}
}
}
},
Expand Down Expand Up @@ -538,56 +442,23 @@
"local": "deleteSearchSession",
"remote": "deleteSearchSession"
},
{
"local": "listMonitors",
"remote": "listMonitors"
},
{
"local": "getMonitor",
"remote": "getMonitor"
},
{
"local": "createMonitor",
"remote": "createMonitor"
},
{
"local": "deleteMonitor",
"remote": "deleteMonitor"
},
{
"local": "watchSearchSessions",
"remote": "watchSearchSessions"
},
{
"local": "watchSessionProperties",
"remote": "watchSessionProperties"
},
{
"local": "watchMonitorEvents",
"remote": "watchMonitorEvents"
},
{
"local": "watchMonitorEventsWithProperty",
"remote": "watchMonitorEventsWithProperty"
}
],
"function": [],
"rpc": [
{
"local": "listSearchSessions",
"remote": "listSearchSessions"
},
{
"local": "listMonitors",
"remote": "listMonitors"
}
],
"webhook": [
{
"local": "monitorEvents",
"remote": "batchdata-r6b83g"
}
]
"webhook": []
}
}
]
Expand Down
Loading
Loading