ShadowID: Expose the Auto Increment ID to the Public Without Compromising Security

Combine the best of both worlds: the performance of Auto Increment IDs and the security of UUIDs.

I was tasked to update our existing implementation that uses Auto Increment ID from MySQL as the ID for the public API. The objective of this task is to prevent enumeration attacks and ensure that the development effort is kept to a minimum.

The first thing that came to mind was to use a unique random ID like UUIDv4. However, since we are using MySQL as the database, indexing UUIDs has a significant performance impact due to their randomness and the locality problem of the B-Tree Index.

Another option is to use ULID, but it’s not natively supported by MySQL, or UUIDv7, which is still in draft. The Twitter Snowflake ID is also a good option, but it’s predictable since it exposes the timestamp and also requires a dedicated server to generate the ID, which increases the development effort.

Thankfully, I had previously written my own implementation of UUID and MongoDB ObjectID from scratch for fun and learning. This experience gave me a mental model of how unique IDs are designed. With the help of my Computer Science background, I realized that I can use Steganography to hide the Auto Increment ID within a random string, in this case, the UUIDv4, and still be able to retrieve the Auto Increment ID if we know the algorithm and the secret…