In data security, tokenization is the practice of replacing a sensitive value, such as a payment card number, bank account number or national ID number, with a stand-in value called a token. The token can be stored and passed between systems in place of the real data, while the original value is kept in a separate, tightly protected system, often called a token vault, or can be recovered only by that system. It is widely used to reduce the number of systems that hold card data under the Payment Card Industry Data Security Standard.
Not to be confused with tokens in AI language models, which are the small pieces of text a model reads and generates, or with asset tokenization, which represents ownership of an asset such as property or securities on a blockchain.
At a glance
- A token stands in for sensitive data, often keeping the same length or format so existing systems still work.
- The real value is held or recovered by the tokenization system, and getting it back (detokenization) is limited to authorized requests.
- Systems that handle only tokens hold much less sensitive data, which can reduce breach impact and compliance scope.
- Payment processors, gateways and card networks offer tokenization for card data; separate products tokenize other sensitive fields.
- It differs from encryption: tokens are typically not derived from the original with a key.
What problem it solves
Sensitive data tends to spread. A card number captured at checkout ends up in the order database, the customer service tool, analytics exports and backups. Every copy is something to protect, audit and potentially disclose in a breach. Under the Payment Card Industry Data Security Standard (PCI DSS), every system that stores, processes or transmits card data, or connects to one, is generally in scope, so the spread also drives compliance cost.
Tokenization stops most of that spread. The real value is captured once and swapped for a token as early as possible, and the rest of the business works with tokens. Customer service can look up an order, finance can reconcile payments and a subscription can be charged again, all without most systems ever holding the real number.
How it works
Capture and substitution. When sensitive data enters, often at a payment page, terminal or application, it is sent to a tokenization service, which returns a token. The business system stores the token instead of the original.
Vaulted tokenization. The service keeps a protected database mapping each token to its original value. The vault is heavily secured, sometimes with a hardware security module (HSM) protecting its keys.
Vaultless tokenization. Some products generate tokens using cryptographic methods rather than a lookup table, so no central mapping database is needed. These approaches sit closer to encryption, and the distinction matters when reviewing how a product works.
Format. Tokens can keep the original format, such as 16 digits with the last four visible for receipts, so databases and applications need few changes.
Detokenization. Exchanging a token for the real value is restricted to authorized systems or users, and requests are typically logged. Many uses, such as refunds or repeat charges through the same payment processor, work with the token alone.
Payment tokens. In card payments, processors and gateways issue tokens a merchant can reuse with that provider, and card networks issue network tokens for wallets and stored cards. These are provided by the payment ecosystem rather than deployed by the merchant.
When it matters for buyers
- When choosing a payment processor or gateway. Tokenization and hosted payment pages largely determine how much of your environment handles card data.
- When PCI DSS scope or audit cost is growing. Tokenizing early can shrink it, subject to your assessor’s view.
- When protecting other sensitive fields. Tokenization is also used for personally identifiable information (PII), such as national ID and account numbers, especially in test and analytics data.
- When switching payment providers. Tokens are often specific to the provider that issued them, so plan how stored tokens will migrate.
- After a data classification exercise. It shows which fields are worth tokenizing.
For help planning compliance work around payment data, see our governance, risk and compliance overview.
Questions to ask vendors
- Is your tokenization vaulted or vaultless, and how are the vault or keys protected?
- Can tokens keep the original format, and what parts of the value stay visible?
- Who can detokenize, how is that controlled, and what is logged?
- Which of our systems would still be in PCI DSS scope with your service, and what evidence do you provide for our assessment?
- If we leave, can our stored tokens or the underlying data be migrated to another provider, and at what cost?
- Do you support network tokens for card payments?
- What availability commitments apply to the token service, since our systems depend on it?
How it differs from encryption
Encryption transforms data with an algorithm and a key; anyone who obtains the key can reverse it, so key management decides how strong the protection is. Under PCI DSS, encrypted card data generally stays in scope for any organization that can decrypt it. Tokenization typically replaces the data with a value that has no mathematical relationship to the original, so a stolen token can’t be decrypted; the original can only be retrieved through the tokenization system. That is why tokenization can take systems out of PCI DSS scope in ways encryption often can’t, depending on the design. The two are commonly used together: data is encrypted in transit to the token service, and the vault itself is encrypted. Vaultless products blur the line, so check which method a product actually uses.