/plushcap/analysis/weaviate/weaviate-lock-striping-pattern

How we solved a race condition with the Lock Striping pattern

What's this blog post about?

Lock striping is a technique used in database design to handle race conditions during parallel data import while maintaining performance. In Weaviate's case, lock striping was implemented by assigning each object with a unique UUID to one of the 128 locks, ensuring that objects with identical UUIDs are never processed concurrently. This approach eliminated the race condition without any negative impact on data import performance. The lock striping pattern was introduced in Weaviate v1.15.4 and is recommended for users dealing with large datasets.

Company
Weaviate

Date published
Oct. 25, 2022

Author(s)
Dirk Kulawiak

Word count
729

Language
English

Hacker News points
None found.


By Matt Makai. 2021-2024.