How we solved a race condition with the Lock Striping pattern
Lock striping is a technique used in database design to handle race conditions during parallel data import while maintaining performance. In Weaviate's case, lock striping was implemented by assigning each object with a unique UUID to one of the 128 locks, ensuring that objects with identical UUIDs are never processed concurrently. This approach eliminated the race condition without any negative impact on data import performance. The lock striping pattern was introduced in Weaviate v1.15.4 and is recommended for users dealing with large datasets.
Company
Weaviate
Date published
Oct. 25, 2022
Author(s)
Dirk Kulawiak
Word count
729
Language
English
Hacker News points
None found.