Importing JSON with multiple levels of nesting to Danfo

[pahund]

Hi everybody,

I'm just getting started with Danfo.js.

I have data to process from a MongoDB that comes in JSON format. The data in the JSON has multiple levels of nesting, something like this, for example (grossly simplified):

[
  {
    "id": 1,
    "foo": {
      "bar": {
        "baz: 23
      }
    }
  },
  {
    "id": 2,
    "foo": {
      "bar": {
        "baz: 99
      }
    }
  }
]

If I understand correctly, it is not possible to import this kind of JSON data into a Danfo data frame directly because of the nesting. For now, I've resorted to writing my own “flattening” function that converts the data into something like this:

[
  {
    "id": 1,
    "foo_bar_baz": 23
  },
  {
    "id": 2,
    "foo_bar_baz": 99
  }
]

This data can now be consumed by danfo.read_json.

I'm wondering, though – is this the correct approach? Is there an easier way, which I'm missing?

Thanks in advance for your help!

[risenW]

@pahund Yes, Danfo does not support nested JSON files.

[pahund]

So the answer is, yes, my approach is correct and the way everybody does it?

[risenW]

Oh sorry if I was not clear. Yes, that’s right way.

[pahund]

Thanks for clearing it up!

[pahund]

Perhaps this is useful to somebody – this is the function I'm using to flatten my nested JSON object:

function flatten(value, key) {
  if (typeof value === "number") {
    return { [key]: value };
  }
  return Object.entries(value).reduce(
    (acc, [k, v]) => ({ ...acc, ...flatten(v, `${key ? `${key}_` : ""}${k}`) }),
    {}
  );
}

const rawData = [
  {
    id: 1,
    foo: {
      bar: {
        baz: 23,
      },
    },
  },
  {
    id: 2,
    foo: {
      bar: {
        baz: 99,
      },
    },
  },
];

const flatData = rawData.map((item) => flatten(item));

// result: [ { id: 1, foo_bar_baz: 23 }, { id: 2, foo_bar_baz: 99 } ]

[idhamari]

I tried it in my data and get:

               InternalError: too much recursion