20 questions to ask an AI or automation vendor before they touch client data

Before an AI or automation vendor gets access to client information, an advisory firm should have written answers about where the data goes, who can see it, how the AI is supervised and what happens when something breaks. Amended Regulation S-P requires firms to oversee their service providers, so these answers belong in your vendor file.

Below are the 20 questions I would ask, grouped by topic, with what a good answer sounds like. I sell automation to advisory firms, so these are questions I expect to be asked.

Where does our data go?

1. Where is our client data stored, and in whose account? The best answer is that it stays in systems your firm owns and the vendor keeps no copy.
2. Which other companies will process it? You want a complete list, including the company behind the AI model.
3. Is our data used to train any AI model? The answer should be no, in writing, naming the provider setting or contract term that makes it true.
4. How long does each system keep our data? Expect a number for each system and a way to delete on request.
5. Is data encrypted in transit and at rest? Yes to both, with the services named.
6. Could you do the job with less data? A good vendor has already thought about this. A drafted reply needs a name and a message. It does not need an account number.

What access do you need?

7. What exact permissions do you need in our CRM and email? Look for a specific list. Be wary of a request for administrator access by default.
8. Will anything you build log in to our custodian? No. There is no version of lead response or onboarding that needs it.
9. Where are passwords and API keys kept? In the automation platform's credential store, inside your account. Never in a document or a chat message.
10. How do we cut off your access in one step? There should be a single action you control, and they should be able to name it.

How is the AI supervised?

11. Can any message reach a client without a person approving it? No. Ask them to show you the approval step.
12. What does the AI do, on one page? You should get a plain description your compliance officer can file and an examiner can read.
13. Where is each draft, approval and send recorded? In your CRM or archive, where your other client records live.
14. What happens when the AI gets it wrong? The draft can be edited or rejected, the rejection is logged and nothing is sent.

What happens when something breaks?

15. How fast will you tell us about a security incident? Within 72 hours at most, written into the contract. That matches what Regulation S-P expects of your service providers.
16. How will we know a workflow has failed? Monitoring that alerts a person, and a place where you can see failed runs yourself.
17. What happens when one of our tools changes? Software vendors change their APIs without asking. You want to know who fixes the workflow, how quickly and what it costs.

What does the contract say?

18. Who owns the workflows? Your firm should, outright.
19. What happens if we stop working together? Everything keeps running in your accounts, and you get a handover document.
20. What insurance do you carry? Ask about technology errors and omissions and cyber coverage, with the limits, and ask for a certificate.

How should you use the answers?

Get them in writing and keep them with the contract. A vendor who answers quickly and specifically has usually been asked before. Vague answers to questions 1, 3 and 11 are the ones I would treat as a reason to stop, because they cover where your data is, whether it trains a model and whether a machine can write to your clients unsupervised.

Advisor Ready's answers on data, access and supervision are on the security page, and the rules behind the questions are in AI and automation compliance for RIAs.