Best practises for ID generation

[thomasf1]

Sorry for this noob question, but I couldn´t find anything in the documentation.

Porting some code over from another DB, currently the IDs on an new Item (DB entries / Points) are randomly generated and in the rare case a ID already exists the code inserting it throws an exception and a new ID is assigned.

There seems to not be a insert/create function with qdrant, giving me the feeling that this not encouraged. What´s the best practise on qdrant (while keeping ID assignment in the Code / not shifting it to the DB)?

[generall]

Hey @thomasf1, there are several possible options:

• Use random UUID and keep original ID in payload. UUID is 128 bit and collision probability is zero by all practical measurements.
• Use hash of the original ID

[thomasf1]

So, basically the encouraged strategy is to make the ID as long as possible to then (hopefully) never run into the issues of it existing already as it would overwrite the existing object by default?

[abhiksark]

@generall Do we get a capability of auto-generated id?

[generall]

what do you mean exactly?

[thomasf1]

If I understand the docs correctly, the ID field is either an integer or an UUID. I did not see an option to auto-generate an ID on insert.

It´s an interesting design decision, it seems to put some unnecessary limitations on the product - especially when migrating from another DB and wanting to keep IDs or having an existing strategy for generating IDs. Pretty basic stuff in traditional DB Products...

[timvisee]

You're right, there is no option to automatically generate an ID. This is hard to do in a good way in a distributed environment not routing to a single leader, and we want to be deterministic. It's also very likely that you want to have specific IDs linked to source data anyway.

You can use an auto incrementing number yourself to generate IDs. Using random UUIDs to make sure IDs are unique is also perfectly fine.

[thomasf1]

That makes sense, that can be tricky/costly to implement. The choice of either Integers or UUIDs seems a bit limiting though, but I guess it is the two extremes that are easiest (either fully app/user managed sequential IDs or long ones with a low chance of collision).

Would you guys consider opening up the ID field to Strings (which UUIDs basically are, too), it would be nice for a migration to use existing IDs - also for URLs UUIDs are a bit long.

The problem of inserts would still need some solution though, not sure what happens currently when a numeric or UUID id already exists?

The alternative of managing two sets of IDs (one Record ID and one in the payload) seems like messy though, especially when using features like recommendations which are Record ID based and would need mapping tables or something.

[timvisee]

Would you guys consider opening up the ID field to Strings (which UUIDs basically are, too), it would be nice for a migration to use existing IDs - also for URLs UUIDs are a bit long.

Probably not, unless there's a very good use case for it.

Both integers and UUIDs have some good properties we like, such as being fixed size. Note that UUIDs are not strings, and we don't handle them internally as strings, it's just how they are exposed in our REST API.

not sure what happens currently when a numeric or UUID id already exists?

If it already exists, you update/overwrite. That is desired behavior.

[generall]

Strings (which UUIDs basically are, too)

We do not store UUIDs as Strings

[almostimplemented]

When a collection uses integer IDs, is there an efficient way to query the largest existing ID in the collection?

[timvisee]

No, there is not.