Do you have any data on what percentage of fraudulent charges get past your system. (i.e. of all fraudulent charges received how many does your system not catch) and what percentage of your fraudulent charge alerts are non-fraudulent? Just curious! I'm a co-founder of an eCommerce company and our average transaction is around $5,000 so this is pretty important to us and we already have some pretty strong systems in place, but I'm curious how well this more automatic approach works.
Good question. We measure these using "precision" (of the users that users Sift flags, which percentage are actually fraudsters?) and "recall" (what percentage of fraudsters on the site does Sift flag?). We can get 90% precision or 90% recall, although not currently both at the same time, and it's the customers choice as to which to optimize for. We can just adjust a threshold to tune our system to their needs.
Companies that have high transaction amounts often use the machine learning system to detect likely fraudsters, but then have a human review each one and make the final decision to approve/deny. We have a visualization "widget" that shows the reviewers which signals made a particular user look suspicious. The advantage of using machine learning is then that you: a) catch fraudsters you wouldn't have noticed otherwise, b) don't have to review every single transaction, just the subset that are most suspicious, c) make it faster for your staff to review transactions since the visualization tools will help point them at what to look at.
