How to Test a Crypto AML/KYT Provider Before Signing a Contract
Two AML platforms can both claim wallet screening, transaction monitoring, sanctions detection, broad blockchain coverage, configurable rules, an API and case management. Side by side, the claims look interchangeable.
What they do not show is whether both systems attribute the same wallet to the same entity, how many alerts each generates on identical activity, how long an analyst needs to close a case, whether the reason behind a risk result is legible, how the API behaves under load, or how well either fits your risk policy. A crypto AML vendor should be tested on your real operating conditions before it is judged by its feature list.
This is a procurement method, not a shortlist. It assumes you have two to four candidates and need to find out which one works.
Define What the AML/KYT Pilot Must Prove
A pilot without stated objectives produces impressions rather than evidence. Before the first API key is issued, write down the primary use case, the business model, the chains and assets in production, whether incoming or outgoing flows matter more, whether the priority is wallet screening or transaction monitoring, applicable sanctions requirements, how indirect exposure should be treated, whether behavioural monitoring is needed, how analysts actually work, API and audit requirements, and the manual-review workload the team can absorb.
Then write down the harder thing: what would make the pilot fail?
Useful failure conditions are specific. Missing a known material risk. An unsupported production chain. API stability that would not survive your volume. A false-positive load needing another analyst to clear. An alert the system cannot explain. Audit evidence that cannot be exported.
Absent from that list are universal success thresholds. No correct false-positive percentage or minimum detection rate applies to every business — those follow from risk appetite, regulatory exposure and team capacity, and should be set by your compliance function before testing starts, not negotiated afterwards to fit whichever vendor performed best.
If the decision to use an external provider at all is still open, that is a different question — build or buy crypto AML/KYT comes before this article, not after it.
Build a Representative Test Set
This is where most evaluations are won or lost, and it is the part vendors cannot do for you.
Include Known High-Risk and Known Normal Cases
Use one dataset across every shortlisted provider. Changing inputs between vendors makes outputs incomparable — the most common methodological error in these projects.
Include known sanctioned or designated exposure where appropriate, known scam, hack and stolen-fund exposure, mixer or obfuscation cases, normal exchange and customer activity, ordinary low-risk wallets, and known counterparties you can independently verify.
Testing only public sanctions addresses is not sufficient to evaluate a blockchain analytics provider because many material risk cases depend on attribution, transaction paths, indirect exposure, and behavioural context. Published designated addresses are the easy part — every serious provider catches them. They tell you almost nothing about the capability you are actually buying.
Include Ambiguous and Edge Cases
Cases where the right answer is not obvious are the ones that separate providers: small indirect exposure, shared-service infrastructure, DEX interaction, bridge usage, a previously unknown counterparty, multi-hop risk, cross-chain flows, newly attributed entities, and transactions carrying both legitimate and higher-risk history.
Expect disagreement here, and do not treat it as a defect. Attribution data, exposure methodology, taxonomy and scoring logic differ between platforms by design — the mechanics of why AML providers can return different results for the same wallet are worth understanding before you interpret the output. Different AML providers do not need to produce identical risk scores to be effective because attribution data, exposure methodology, taxonomy, and scoring logic can differ.
The question is not who agrees with whom. It is whether each difference is explainable and whether it would change your compliance decision.
Score What Matters in Production
Detection Quality and Explainability
Comparing Low, Medium and High across platforms tells you very little. Compare what sits underneath: which entity was identified, the risk category, whether exposure is direct or indirect, amounts, direction of flow, hop distance, relevant TxIDs, timestamps, and any confidence or context provided.
The governing question is whether an analyst can understand why the system produced that result — because that analyst will have to defend the decision to a supervisor or a banking partner.
Avoid the shortcut that the highest risk score wins. A higher score may simply reflect a more aggressive methodology, and aggression is not accuracy.
Alert Quality and Analyst Workload
For transaction monitoring, measure alert volume, duplicate and noisy alerts, substantively meaningful alerts, the ability to tune thresholds, grouping and aggregation, behavioural rules, case context, disposition workflow, escalation paths, analyst notes, and time per case.
A higher number of alerts does not automatically mean better AML detection if the additional alerts create excessive low-value manual review. A provider surfacing more risk but doubling analyst workload may not be the better operational choice once you price the headcount.
The inverse is equally wrong: low alert volume does not indicate a better tool either. It may mean the system is not looking hard enough. Volume is only meaningful next to what the alerts contained.
Coverage, Freshness, and Re-Screening
Test against production reality rather than marketing figures: the chains you use, the assets you handle, entity coverage in your corridors, how quickly new blocks are indexed, how fast sanctions updates land, how new attribution appears, whether historical re-screening is possible, and how risk changes over time.
Blockchain coverage should be measured against the networks a business actually uses rather than the largest headline chain count. A provider supporting 300 chains is not better than one supporting 50 if your volume runs on six networks that both of them cover.
API and Integration Reliability
If the provider will sit inside a production transaction flow, the API is not secondary. Test response consistency, latency under realistic load, error behaviour, rate limits, retries, webhook delivery, sandbox-versus-production differences, versioning, authentication, documentation and what data is returned.
A full architecture review is a separate exercise — the technical requirements for a crypto AML API go deeper than a pilot needs. Here the question is narrower: does it hold up under your load, and how much engineering effort does integration take?
Run the Provider Alongside Your Existing Workflow
A static dataset shows detection. It does not show operations.
Where feasible, run the candidate in parallel with your live process without automatic enforcement. Send it the same screening events, do not let it block or release funds during the initial test, compare its output against your current process, and inspect every case where the two disagree. Then measure what those disagreements cost: analyst workload, alerts your current system missed, and alerts the candidate added.
A pilot needs enough real workflow to expose operational differences, not an arbitrary number of calendar days. A 30-day POC is a convention, not a methodology. High daily volume across several chains may produce enough within two weeks; low volume with diverse risk may need considerably longer. Duration follows transaction volume and risk diversity — nothing else.
This is established practice. SlowMist's guidance on evaluating crypto AML tools argues for controlled testing across high-risk, clearly safe and grey-area addresses rather than feature-list comparison, and publishes an open vendor evaluation checklist built on the same principle. Elliptic's questions to ask a blockchain analytics provider pushes buyers toward measurable outcomes over marketing description.
Compare Total Operating Impact, Not Just Detection Scores
Bring the results into one scorecard, applied identically to every candidate:
- Risk Detection (known risks found, material risks missed).
- Explainability (evidence available behind each result).
- False Positives (legitimate activity routed to review).
- Analyst Workload (cases generated and time required per case).
- Coverage (required chains, assets and entities).
- Monitoring (rules, re-screening and behavioural detection).
- API (stability, latency, errors and integration effort).
- Auditability (evidence, history and exports).
- Support (response quality and escalation handling).
- Commercial Fit (total cost for your actual workload).
A useful AML proof of concept tests both detection quality and operational impact, including false positives, analyst workload, API reliability, and auditability.
Set the weighting before scoring begins, not after results arrive — weighting chosen afterwards tends to describe the preferred vendor rather than the business need.
AML provider selection should be documented so the business can explain which use cases were tested, which limitations were accepted, and why the final vendor was selected. That record is useful internally, and it is what a supervisor or banking partner will ask for if the decision is ever questioned.
Test the Product, Not the Pitch
The best crypto AML provider is not the one that performs best in a sales demo. It is the one that performs reliably on the buyer's own risk cases, workflow, API, and analyst operations.
The purpose of a crypto AML pilot is not to make every provider agree; it is to determine which system produces reliable and explainable results within the buyer's own risk framework. Two platforms disagreeing on an ambiguous wallet is normal. One of them being unable to explain its answer is not.
A crypto AML provider should be evaluated using the same wallets, transactions, risk cases, and operational criteria as every other shortlisted provider. AMLBot should be assessed on exactly those terms — same wallet set, same transactions, same chains, same alert scenarios, same API workload, with crypto transaction monitoring tested against your production flows rather than demonstration data. Use the same dataset, the same workflows, and the same scoring criteria for every shortlisted provider before signing a long-term contract. If AMLBot is on your shortlist, you can request an AMLBot demo and run it through the same evaluation as the rest.
FAQ
How Should a Crypto Business Test an AML Provider Before Buying?
Define the required use cases, build a representative dataset, then test the same wallets and transactions across every shortlisted provider. Measure detection quality, false positives, analyst workload, API performance and auditability, and score all vendors against identical criteria with weighting agreed before testing begins.
What Is a Crypto AML Proof of Concept?
A controlled trial used to verify whether an AML/KYT provider performs adequately under the buyer's actual or representative operating conditions, before committing to a long-term contract or a full production integration. It differs from a sales demo in that the buyer supplies the data, the scenarios and the success criteria.
What Wallets Should Be Included in an AML Provider Test?
Include known high-risk cases, normal and low-risk wallets, ambiguous cases where the correct answer is debatable, indirect exposure scenarios, cross-chain flows, and wallets reflecting your actual production use cases. Testing only public sanctions addresses is not sufficient, since every serious provider detects those.
Should AML Providers Return the Same Risk Score for the Same Wallet?
No. Providers can differ legitimately in attribution, clustering, exposure methodology, taxonomy, weighting and data freshness. Identical scores are not the goal. The useful question is whether each difference can be explained, and whether it is operationally acceptable within your own risk framework.
How Do You Measure False Positives in a Crypto AML Pilot?
Compare alerts against cases your business has independently reviewed or can otherwise classify with sufficient evidence, then measure how much legitimate activity is unnecessarily routed to manual review. There is no universal acceptable percentage — the threshold depends on your risk appetite and analyst capacity.
What Is More Important: Blockchain Coverage or Detection Quality?
Neither number should be read alone. Coverage matters only if it includes the networks your business actually uses, while detection quality determines whether relevant risks on those networks are identified and explained. A large headline chain count adds nothing if your volume runs elsewhere.
Should API Performance Be Tested During an AML Vendor POC?
Yes, if the provider will sit inside a production transaction flow. Test response consistency, latency under realistic load, error behaviour, rate limits, webhook delivery, retry logic, and differences between sandbox and production. Integration effort is part of the result, not a separate conversation.
How Long Should a Crypto AML Vendor Pilot Run?
There is no universal duration, and fixed 30-day or two-week pilots are conventions rather than methods. A pilot should run long enough to capture a representative volume and mix of risk cases, operational alerts, integration behaviour and analyst workflows, which depends on transaction volume and risk diversity.
How Should a Business Compare Two AML Providers When Their Results Disagree?
Look beneath the headline score and compare entity attribution, transaction path, direct versus indirect exposure, risk category, amount, timing and supporting evidence. Then assess which of those differences would actually change your internal compliance decision. Differences that change nothing are noise.
Should Price Determine Which AML Provider Wins the POC?
No. Price should be weighed together with detection quality, analyst workload, integration effort, required add-ons, monitoring capability, support and auditability. A cheaper subscription can produce a higher total operating cost if it generates excessive manual work that needs additional headcount to clear.