Why CockroachDB IDs are Inconsistent in Web Applications

Are you using CockroachDB for a full stack application? Is your frontend a web application? Have you noticed that sometimes when you receive the IDs from the database, that they are different than their actual stored value?

I noticed this when using Cockroach, and it was a frustrating issue to diagnose. I will tell you why this happens, and how to resolve these inconsistent IDs.

Why This Happens

I can explain this entire issue in one sentence:

CockroachDB IDs are breaking your application because they are too large.

In JavaScript, the highest safe integer is 9007199254740991, per programiz.com. Many CockrachDB generated IDs will be larger than this, causing errors with your frontend / backend integration.

My Fix

There are multiple ways to work around this issue. I will show how to solve this using json-bigint; a JavaScript library whose sole purpose is handling large integers. This is essential when handling HTTP requests and JSON objects coming from a CockroachDB database.

Install JSON-BigInt

// Install using npm
npm install json-bigint

// Install using yarn
yarn add json-bigint

Using JSON-BigInt

Below, I am sending data via a POST request, and expecting the ID of the data I just created, to be returned in the response.

I know that this returned ID will be larger than the JavaScript safe int, so I wrap the response data as JSONbig.parse(data) when it is returned to my frontend.

Now I can safely access this large integer without inconcsistency. In my case, I use this ID to route customers to a page, with this ID as a URL parameter.

import JSONbig from 'json-bigint';
import axios from 'axios';  // for making API calls

const handleSubmit = async (e) => {
        e.preventDefault();

        // request data to send to POST request
        const data = new FormData();
        data.append('description', formData.description);
        data.append('name', formData.setName);

        try {
            const response = await axios({
                method: 'post',
                url: 'http://localhost:8000/createObject/',
                data: data,
                headers: { 'Content-Type': 'multipart/form-data' },
                // wrap response with JSONbig.parse
                transformResponse: [(data) => JSONbig.parse(data)]
            });
            const cardsetId = response.data.cardset; 
            window.location.href = `/DisplayObject?id=${cardsetId}`;
        } 
        catch (error) {
            console.error("Error:", error);
        }
    };

Other Methods for Fixing This Issue

Since my fix involved an npm package and React, I will add some more universal solutions below. I won't have time to explain all of these in detail, but here is a list of other ways to solve inconsistend CockroachDB IDs in your javascript client.

Convert integer IDs to strings when retrieving from the database
JavaScript’s BigInt type (most universal fix)
Making a custom type for IDs in database
Using a buffer or byte array

Hopefully this solution helps. This is a common problem with CockroachDB that is not discussed often enough.

More on json-bigint: https://www.npmjs.com/package/json-bigint

My Website: https://www.DavidWilliford.dev

Thanks for reading!

David's blog