33ddfeeb64
fix: use Lamport clocks consistently across WAL operations
...
Ensure proper Lamport clock integration across batch operations and sequence number handling:
- Add Lamport clock support to Batch.Write for consistent replication timestamps
- Fix potential sequence number inconsistencies in WAL operations
- Update WAL tests to properly verify sync mode behaviors
- Fix Sequence numbering to appropriately handle Lamport clock timestamps
2025-04-26 12:24:50 -06:00
5cd1f5c5f8
feat: implement WAL applier for replication
...
- Add WALApplier component for applying entries on replica nodes
- Implement logical timestamp ordering with Lamport clocks
- Add support for handling out-of-order entry delivery
- Add error handling and recovery mechanisms
- Implement comprehensive testing for all applier functions
2025-04-26 12:02:53 -06:00
02febadf5d
feat: implement WAL replicator and entry serialization
...
- Add WAL replicator component with entry capture, buffering, and subscriptions
- Implement WAL entry serialization with checksumming
- Add batch serialization for network-efficient transfers
- Implement proper concurrency control with mutex protection
- Add utility functions for entry size estimation
- Create comprehensive test suite
2025-04-26 11:54:19 -06:00
5b2ecdd08c
fix: add the retry logic
2025-04-26 11:48:51 -06:00
c0bfd835f7
feat: implement replication hook point in WAL and Lamport clocks…
...
Go Tests / Run Tests (1.24.2) (pull_request) Failing after 5m1s
- Add Lamport clock implementation for logical timestamps
- Define ReplicationHook interface in WAL package
- Extend WAL to use Lamport clock for timestamps
- Add notification hooks for WAL entries and batches
- Update WAL initialization to support replication
- Add tests for replication hooks and clocks
2025-04-26 11:42:49 -06:00
0e5d60cb83
feat: add lamport timestamp
2025-04-26 11:28:44 -06:00
8e04c2cea3
fix: fix the build error from before
Go Tests / Run Tests (1.24.2) (push) Failing after 5m2s
2025-04-26 04:11:32 -06:00
ae75f2935f
fix: fix the WAL is closed errors on rotation in tight loops, increase durability
Go Tests / Run Tests (1.24.2) (push) Failing after 5m5s
2025-04-26 00:15:50 -06:00
7e226825df
fix: engine refactor bugfix fest, go fmt
Go Tests / Run Tests (1.24.2) (push) Successful in 9m48s
2025-04-25 23:36:08 -06:00
0637c40a40
feat: big refactor cleaning up the engine code
Go Tests / Run Tests (1.24.2) (push) Failing after 5m4s
2025-04-23 22:45:16 -06:00
7dd816bdf5
feat: add a standard stats collector, needs integration
2025-04-23 20:01:18 -06:00
d5a90cf2e4
perf: added bloom filters for sstables to improve performance
Go Tests / Run Tests (1.24.2) (push) Failing after 5m2s
2025-04-23 19:04:10 -06:00
2e3c70e913
test: add a concurrent read/write test on memtables
Go Tests / Run Tests (1.24.2) (push) Successful in 9m48s
2025-04-23 18:20:21 -06:00
00b2566464
perf: updated skiplist find algorithm to be more efficient and cache aware
Go Tests / Run Tests (1.24.2) (push) Has been cancelled
2025-04-23 18:16:44 -06:00
2b90635021
test: add tests for edge case putting a key after delete
2025-04-23 16:49:38 -06:00
6f83fa1ade
feat: add suffix scanning
Go Tests / Run Tests (1.24.2) (push) Successful in 9m49s
2025-04-23 09:00:26 -06:00
a8c72fccfa
feat: add logger finally...
Go Tests / Run Tests (1.24.2) (push) Successful in 9m48s
2025-04-23 02:28:45 -06:00
a0a1c0512f
chore: formatting
Go Tests / Run Tests (1.24.2) (push) Successful in 9m50s
2025-04-22 14:09:54 -06:00
e7974e008d
feat: enhance wal recover statistics
2025-04-22 14:09:45 -06:00
fbea5da627
chore: renaming packages after move to github org
2025-04-22 11:01:33 -06:00
6138f4e1d5
fix: scan family of rpc calls were not returning any results
2025-04-22 11:01:33 -06:00
c46da3dfce
feat: add client sdk and fix tests
2025-04-22 11:01:33 -06:00
cb8776dcc9
feat: add gRPC service
2025-04-22 11:01:33 -06:00
0ce19934ba
feat: add common transport interface and server mode to kevo
2025-04-22 11:01:33 -06:00
001934e7b5
refactor: optimize two pass deduping into a single pass across the codebase
Go Tests / Run Tests (1.24.2) (push) Successful in 9m38s
2025-04-21 16:27:54 -06:00
1a0de922af
refactor: use memtable sorted order iterator to write and dedup in a single pass
2025-04-21 14:53:03 -06:00
4e813aa3fd
fix: flushing a memtable dedups keys as its processing them before creating the SSTable
2025-04-21 14:53:03 -06:00
25fe3b1771
chore: i am an idiot, forgot to change the full path parts
Go Tests / Run Tests (1.24.2) (push) Has been cancelled
2025-04-20 14:53:39 -06:00
6fc3be617d
feat: Initial release of kevo storage engine.
...
Go Tests / Run Tests (1.24.2) (push) Has been cancelled
Adds a complete LSM-based storage engine with these features:
- Single-writer based architecture for the storage engine
- WAL for durability, and hey it's configurable
- MemTable with skip list implementation for fast read/writes
- SSTable with block-based structure for on-disk level-based storage
- Background compaction with tiered strategy
- ACID transactions
- Good documentation (I hope)
2025-04-20 14:06:50 -06:00
