If you've ever been asked to upload a photo of your government ID to prove your identity, you've gone through an identity verification process. If you haven't, this is a screenshot of what Coinbase's process looks like.

About a year ago, I was shocked to learn some companies still use a manual verification process despite the competitive market of automated providers: Onfido, Acuant, Mitek, Jumio, etc. These findings led me to naively assume I could create my own automated provider using off the shelf deep learning libraries. Of course, things are never that easy. The rest of this post documents what I've learned about this industry and where I see it going in the future.

Any organization with an online presence that deals with real people's identities and/or money should have an identity verification to reduce fraud, and comply with anti-money laundering (AML) and know-your-customer (KYC) regulations.

Under US regulation; 31 C.F.R. § 103.121 implementing PATRIOT Act Section 326

• A bank must obtain, at a minimum, the following information from the customer prior to opening an account:
  1. Name
  2. Date of birth, for an individual
  3. Address
  4. Identification number (for a U.S. person, an SSN)

Market research was mostly directed towards the Canadian market where I talked to a number of financial organizations. There are separate processes for verifying the identities of individuals, corporations, and other entities to comply with the Proceeds of Crime (Money Laundering) and Terrorist Financing Act (PCMLTFA).

For individuals, there are three methods of verification. The first two are single process where the individual must provide a government issued photo identification document in person or where you retrieve the credit file (must have 3 years of history) for the individual. The third method is a dual process where individuals must submit two different documents such as an utility bill, credit card statement, or Canadian birth certificate.

Requiring an individual to be physically present defeats the entire purpose of automated online verification. Credit file lookup solely relies upon an individual providing their name, date of birth, and address. The dual process method requires the most work on the individual's part since they must retrieve multiple documents, but also provides the greatest assurance the individual's identity is not stolen.

Corporations and other entities must provide documents confirming their existence such as the corporation’s certificate of corporate status or a partnership agreement.

In essence, at least for the Canadian market, asking users to submit photos of their photo identification documents doesn't provide compliance with Canadian regulations. However, there is still value in this type of verification. Organizations that provide goods or services with monetary value for their need to combat fraud on their platforms. For example, AirBnB needs to ensure its users aren't going to trash their rentals. Roboadvisors simply manage their users' money and don't need to have the same level of fraud protection.

Honestly, the market research part of starting a business is so painful, but it's well worth it. I learned there are two reasons for performing identity verification: KYC and AML compliance, and fraud protection. Within these, credit checks and other types of documents are commonly used for providing an additional layer of verification. It's also possible corporate and entity verification are an underserved market (pending more research).

Technical Stuff

Let's put on our engineer hat now and  imagine what we would need to provide an identity verification solution that provides regulation compliance and fraud protection for both individuals and corporations.

We need to build an application that requires some or all of the following user input:

• Provide their name, date of birth, and address
• Upload a photo of their photo identification document
• Upload a photo of their face
• Upload a photo of a textual document such as a utility bill or credit card statement

After performing some kind of trendy, state of the art deep learning, the application will perform some or all of the following actions:

• Determine if the photo identification document is legitimate by comparing it to the photo of the user's face
• Retrieve the user's credit file
• Ensure the user's credit file, photo identification document, textual documents, and provided name, data of birth, and address all match
• Search the user's information in databases

The exact set of inputs and outputs will change depending on the customer's requirements and whether the user is an individual, or representing a corporation or entity.

The core of the engineering problem is in validating and extracting information from the photo identification or text documents. Validation involves determining whether the document and the uploaded photo of the document are legitimate or not. Extracting information generally involves extracting text using optical character recognition (OCR).

Let's focus on photo identification documents for now. Fake driver's licenses are commonly used for sneaking into bars and have reached a point where they’re quite convincing upon visual inspection. The barcodes in driver's licenses encode information that can be compared to the text on the front of the license. Depending on the quality of a fake document, its barcode and text may have mismatched content. OCR can be used to extract text while a barcode decoder could be used to decode PDF 417 barcodes (common in photo identification documents). If the barcode and text match; the license's information (especially the license number) should be looked up in a database.

We also need to ensure photos of photo identification documents have not been manipulated. Metadata (such as EXIF metadata found in JPEG photos), can include information about the means by which a photo was taken. For example, I once came across a document photo where the metadata contained “Photoshop”. That's an amateur mistake, though. Metadata can be edited to hide signs of manipulation and more advanced techniques need to be used: error-level analysis, luminance gradient, average distance, HSV and Lab colorspace histograms, JPEG resave quality estimate, extract JPEG quantization tables, and copy-move detection. As it stands now, these techniques require manual analysis, but I believe it's possible to automate these with [the almighty] deep learning.

The techniques described so far only account for text and barcodes. They could easily fall prey to a piece of paper with text on it. Of course, the solution is always deep learning. We could use image classification to label the document (e.g. Ontario's driver's license). Users can still spoof this if they have a fake driver's license, but that's where database lookup comes in.

A photo of a document is insufficient to determine if that document has been stolen. This is where ID selfies come in; in addition to a photo of their document, users must also take a photo of themselves holding their document. Facial recognition could then be used to compare the user’s face with the face on their document. Unfortunately, from a usability perspective, ID selfies create more friction since users have to provide three photos instead of two (front, back, selfie). The selfie's resolution would be too low for extracting the front of the document's data.

What happens if a user holds up a piece of paper with a face on it or is wearing a mask? We can user a technique called liveness analysis, which basically requires users to take a video ID selfie. In this video, they are asked to make facial gestures or say something. Once again, deep learning can be used to ensure the face belongs to a living human and that they are not wearing a mask.

If you think we’re done, think again. Check out this very convincing deepfake of Obama.

A user could use a deepfake video in lieu of their actual face. For those that don’t know, deepfakes are a relatively recent invention where deep learning generative adverisal networks (GAN) are used to apply facial gestures from one person's face to another person’s face. Detecting deepfakes is an active area of research and the only techniques I’ve seen are monitoring eyebrow movements and warping of the face.

The scariest part about deepfakes is the fact that any methods for detecting their inauthenticity can be used to improve the underlying deep learning models.

In summary, a number of techniques can be employed to detect lost, stolen, or fake documents, but these are not foolproof, especially with the advent of deepfakes. The techniques discussed were analyzing metadata, error-level analysis, OCR, barcode detection, image classification, and liveness analysis. These techniques require users be able to submit photos of the front and back of their documents, ID selfies, and video ID selfies.

I did not dive into deep learning specifics because I am not an expert at deep learning. I understand the basics, but I have no clue why how to design my own networks. How do you choose between 34 or 100 layers other than guess and test?

Closing Thoughts

I’ve had a lot of conversations with product managers, compliance officers, and executives at financial tech companies in the past few months. Existing automated photo document verification appear to be serving the market sufficiently. There is greater demand for complete anti-fraud solutions because a lot of these companies are building their own fraud decision systems that combine signals from a number of user verification techniques including document verification, credit checks, and database lookups. There are a number of companies entering the complete solution space, but their decision-making processes are not always transparent nor accurate.

One of the largest cryptocurrency exchanges, Coinbase, received bad press for facilitating actors performing money laundering. In the fourth quarter of 2017, cryptocurrency exchanges saw 10 times increase in verifications for clients. This surge has overwhelmed exchanges and forced some to stop accepting new customers due to slow, manual verification processes. If Coinbase had been using a [hypothetical] complete anti-fraud solution, they wouldn't have run into these bottlenecks.

I hope you enjoyed this brief overview on the identity verification space. If anything here piqued your interested or if I made any mistakes, please reach out to me on LinkedIn!

Disclaimer: This blog does not provide legal advice and does not create an attorney-client relationship. If you need legal advice, please contact an attorney directly.