0 | ||| Resource Pool
19 | %language ElabReflection
23 | --------------------------------------------------------------------------------
24 | -- Utilities
25 | --------------------------------------------------------------------------------
27 | ||| Cancellation check.
32 | False
36 | True
40 | ||| Append a value into the `Queue a`.
49 | ||| Revers a `Queue a`.
54 | where
59 | acc
63 | ||| Append two `Queue a`.
68 | ys
72 | ||| Normalize two `Queue Waiter`s.
77 | q2
81 | ||| Dequeue first live waiter.
90 | -- cancelled waiter
91 | -- consume tombstone and continue
94 | -- live waiter
97 | ||| Stripe-level dequeue.
109 | ||| Check to see if entry is stale.
117 | ||| Execute `Stripe a` effects after CAS commit.
118 | |||
119 | ||| This is the only place IO is performed for Stripe transitions.
120 | |||
121 | ||| Guarantees:
122 | ||| - Effects are executed exactly once (only after successful CAS).
123 | ||| - Ordering is preserved.
124 | ||| - No effects are run on CAS retry.
125 | |||
126 | export
132 | where
144 | pairs
145 | t
159 | ||| Atomically apply a `Stripe a` transition and execute its effects.
160 | |||
161 | ||| This is the central concurrency primitive of the `Stripe a` model.
162 | |||
163 | ||| Behavior:
164 | ||| - Applies a pure state transition (`Stripe -> StripeStep`) under CAS.
165 | ||| - Retries automatically on contention using `casupdate1`.
166 | ||| - Extracts effects only from the successful committed transition.
167 | ||| - Executes effects exactly once after CAS succeeds.
168 | |||
169 | ||| Guarantees:
170 | ||| - Linearizability, such that the `Stripe a` transition appears atomic.
171 | ||| - No duplicated effects (retries do not leak effects).
172 | ||| - No IO occurs during CAS evaluation.
173 | ||| - Effects are executed strictly after commit.
174 | |||
175 | ||| Design Notes:
176 | ||| - `stepfn` must be pure (no IO, no external mutation).
177 | ||| - All side effects must be encoded in `StripeEffect a`.
178 | ||| - This function is the only place where `Stripe a` transitions are committed.
179 | |||
180 | export
191 | --------------------------------------------------------------------------------
192 | -- Configuration
193 | --------------------------------------------------------------------------------
195 | ||| Set the number of stripes in the `PoolConfig a`.
196 | export
200 | setNumStripes (MkPoolConfig create free cachettl (maxres ** prfmaxres) _ pclabel) numstripes =
202 | free
203 | cachettl
205 | numstripes
206 | pclabel
208 | ||| Assign a label to the `PoolConfig a`.
209 | export
216 | --------------------------------------------------------------------------------
217 | -- Resource Management
218 | --------------------------------------------------------------------------------
220 | ||| Create a new striped resource pool.
221 | |||
222 | ||| Behavior:
223 | ||| - Allocates exactly `mstripes` independent `Stripe`s.
224 | ||| - Distributes the total capacity (`poolmaxresources`) across stripes as evenly as possible:
225 | ||| - Each stripe receives either `base` or `base + 1` capacity.
226 | ||| - The first `rest` stripes receive the extra unit.
227 | ||| - Initializes each stripe with:
228 | ||| - `available = assigned capacity`
229 | ||| - empty cache
230 | ||| - empty waiter queues
231 | ||| - fresh waiter id supply
232 | ||| - Constructs a `LocalPool1` for each stripe and stores them in a mutable array.
233 | |||
234 | ||| Resource Distribution:
235 | ||| - Let:
236 | ||| - `base = div maxres mstripes`
237 | ||| - `rest = mod maxres mstripes`
238 | ||| - Then:
239 | ||| - Total capacity is preserved: sum(stripes) = maxres
240 | ||| - Load is balanced with minimal skew (difference ≤ 1).
241 | |||
242 | ||| Concurrency Model:
243 | ||| - Each stripe is independent and owns its own:
244 | ||| - resource cache
245 | ||| - waiter queues
246 | ||| - capacity accounting
247 | ||| - Threads interact with exactly one stripe at a time (via `getLocalPool`).
248 | ||| - This minimizes contention and improves scalability.
249 | |||
250 | ||| Cleanup Model:
251 | ||| - No global collector thread is created.
252 | ||| - Resource cleanup is performed opportunistically via `cleanStripeIfNeeded`.
253 | ||| - This ensures:
254 | ||| - No background threads.
255 | ||| - Cleanup proportional to usage.
256 | ||| - Deterministic behavior (no GC reliance).
257 | |||
258 | ||| Guarantees:
259 | ||| - Total capacity never exceeds `poolmaxresources`.
260 | ||| - Each stripe starts empty but with full creation capacity.
261 | ||| - No IO occurs during stripe initialization except allocation of refs.
262 | ||| - Array is fully initialized before being returned.
263 | |||
264 | ||| Failure Conditions:
265 | ||| - Crashes if:
266 | ||| - An impossible index is encountered during initialization (should be unreachable).
267 | ||| - A `Nat -> Fin` conversion fails (indicates internal inconsistency).
268 | |||
269 | ||| Notes:
270 | ||| - `numstripes` is explicit, avoiding runtime dependency on capabilities.
271 | ||| - The caller is responsible for eventual cleanup via `destroyAllResources`.
272 | ||| - This function performs no resource creation; resources are created lazily on demand.
273 | |||
274 | ||| Invariants Established:
275 | ||| - Each `LocalPool1` corresponds to exactly one stripe.
276 | ||| - Stripe state is valid and consistent for CAS-based transitions.
277 | ||| - Waiter queues and cancellation queues start empty.
278 | |||
279 | export
291 | where
296 | []
304 | []
323 | []
324 | QEnd
325 | QEnd
326 | 0
327 | empty
333 | (assert_total $ idris_crash "Data.Pool.newPool.saturatePools: couldn't convert Nat to Fin") # t
338 | ||| Select a `LocalPool1 World a` for the current thread.
339 | |||
340 | ||| This function deterministically maps the calling thread to one of the
341 | ||| available stripes using a modulo-based hashing scheme.
342 | |||
343 | ||| Behavior:
344 | ||| - Computes a stripe index `sid`:
345 | ||| - If `n == 1`, always selects index `0` (fast path).
346 | ||| - Otherwise:
347 | ||| - Retrieves the current thread id (`getThreadId`).
348 | ||| - Maps it into `[0, n)` via modulo arithmetic.
349 | ||| - Converts the resulting index into a `Fin n`.
350 | ||| - Returns the corresponding `LocalPool1` from the array.
351 | |||
352 | ||| Thread-to-Stripe Mapping:
353 | ||| - Mapping is stable for a given thread id.
354 | ||| - Different threads are distributed across stripes.
355 | ||| - Collisions are possible but minimized under uniform thread ids.
356 | |||
357 | ||| Concurrency Implications:
358 | ||| - Each thread interacts primarily with a single stripe.
359 | ||| - Reduces contention compared to a single global pool.
360 | ||| - Enables scalable parallel access under CAS-based updates.
361 | |||
362 | ||| Arithmetic Details:
363 | ||| - Uses a custom `remInt` implementation to ensure:
364 | ||| - Correct behavior for negative thread ids (if any).
365 | ||| - Avoidance of undefined behavior from division/modulo edge cases.
366 | ||| - Conversion pipeline:
367 | ||| - Int (thread id)
368 | ||| - modulo n
369 | ||| - Nat
370 | ||| - Fin n
371 | |||
372 | ||| Guarantees:
373 | ||| - Always returns a valid `LocalPool1` when invariants hold.
374 | ||| - No mutation of pool state occurs.
375 | ||| - No blocking or waiting.
376 | |||
377 | ||| Failure Conditions:
378 | ||| - Crashes if:
379 | ||| - Conversion from `Nat` to `Fin n` fails (should be impossible if modulo is correct).
380 | ||| - Division by zero is attempted (guarded by invariant `n >= 1`).
381 | |||
382 | ||| Performance:
383 | ||| - O(1) selection.
384 | ||| - Minimal overhead in the `n == 1` case (no IO, no modulo).
385 | ||| - Single IO call (`getThreadId`) in the general case.
386 | |||
387 | ||| Design Notes:
388 | ||| - This function is intentionally simple and deterministic.
389 | ||| - It avoids randomness or hashing to keep behavior predictable.
390 | ||| - Stripe selection is orthogonal to resource availability:
391 | ||| - Load balancing is achieved probabilistically via thread distribution.
392 | |||
393 | ||| Invariants:
394 | ||| - `n >= 1` (guaranteed by `PoolConfig` construction).
395 | ||| - `pools` contains exactly `n` initialized entries.
396 | ||| - Each index in `[0, n)` maps to a valid `LocalPool1`.
397 | |||
398 | ||| Relationship to the system:
399 | ||| - This is the entry point for all pool operations:
400 | ||| - `takeResource`
401 | ||| - `tryTakeResource`
402 | ||| - `putResource`
403 | ||| - It determines which stripe's CAS state machine is used.
404 | |||
405 | private
427 | where
433 | 1
437 | -1
439 | 0
450 | q
461 | ||| Deliver a value to a `Stripe a` state.
462 | |||
463 | ||| This function:
464 | ||| - Updates Stripe state
465 | ||| - Emits wake effects
466 | |||
467 | ||| Invariants:
468 | ||| - Each wake corresponds to a committed state transition.
469 | ||| - Queue ordering is preserved.
470 | ||| - No side effects occur during evaluation.
471 | |||
472 | export
479 | -- no waiting thread
487 | [None]
490 | [None]
495 | ||| Block until a resource is delivered to this waiter.
496 | |||
497 | ||| Behavior:
498 | ||| - Waits on the provided `Channel (Maybe a)` for a wakeup signal.
499 | ||| - Returns:
500 | ||| - `Just a` if a resource is successfully delivered.
501 | ||| - `Nothing` if the waiter is cancelled or destroyed.
502 | |||
503 | ||| Cancellation:
504 | ||| - If the waiting thread is aborted, the `cleanup` handler is invoked.
505 | ||| - This atomically marks the waiter as cancelled by inserting its `wid` into the Stripe's `cancelled` queue.
506 | ||| - Cancellation is lazy, cancelled waiters are skipped during dequeue.
507 | |||
508 | ||| Guarantees:
509 | ||| - No busy waiting, the thread blocks on a channel.
510 | ||| - No lost wakeups, every successful `signal` results in exactly one `channelPut` to a live waiter.
511 | ||| - Safe under races:
512 | ||| - If cancellation happens before wake, waiter is skipped later.
513 | ||| - If wake happens before cancellation, value is delivered.
514 | ||| - Exactly-once semantics:
515 | ||| - Each waiter receives at most one wakeup.
516 | ||| - Each wakeup corresponds to a committed Stripe transition.
517 | |||
518 | ||| Design Notes:
519 | ||| - This function performs no direct Stripe mutation except in `cleanup`.
520 | ||| - All coordination with producers happens via `signal` + `StripeEffect`.
521 | ||| - The `Channel (Maybe a)` encodes both success (`Just`) and termination (`Nothing`).
522 | |||
523 | ||| Invariants:
524 | ||| - `wid` must be the same identifier used when enqueuing the waiter.
525 | ||| - The channel must be single-consumer and used exactly once.
526 | ||| - Stripe state remains the single source of truth for cancellation.
527 | |||
528 | export
540 | where
560 | ||| Destroy a resource instead of returning it to the `Pool1 World a`.
561 | |||
562 | ||| Behavior:
563 | ||| - If a waiter exists, they are woken with `Nothing`.
564 | ||| - Otherwise, no state change occurs (resource is discarded).
565 | |||
566 | ||| Guarantees:
567 | ||| - Waiters are not left blocked indefinitely.
568 | ||| - No resource is reinserted into the cache.
569 | |||
570 | export
580 | where
592 | ||| Free resource entries in the stripe that satisfy a predicate.
593 | |||
594 | ||| Behavior:
595 | ||| - Removes stale entries from cache atomically.
596 | ||| - Emits a batched free effect.
597 | ||| - Ensures no resource is freed twice or leaked.
598 | |||
599 | ||| Guarantees:
600 | ||| - Removal is atomic with respect to Stripe.
601 | ||| - Freeing happens after commit.
602 | ||| - Safe under contention and retries.
603 | |||
604 | private
616 | where
626 | [None]
629 | )
641 | ||| Opportunistically clean stale resources from a `Stripe1 World a`.
642 | |||
643 | ||| This function performs stripe-local garbage collection of cached resources
644 | ||| based on a time-to-live (TTL) policy. It replaces the need for a global
645 | ||| collector thread by tying cleanup to normal pool activity.
646 | |||
647 | ||| Behavior:
648 | ||| - Reads the current monotonic time.
649 | ||| - Constructs a staleness predicate using the provided TTL.
650 | ||| - Invokes `cleanStripe` to:
651 | ||| - Remove stale entries from the cache.
652 | ||| - Emit `FreeMany` effects for the removed resources.
653 | ||| - Effects are executed only after the CAS commit inside `cleanStripe`.
654 | |||
655 | ||| Staleness:
656 | ||| - A resource is considered stale if:
657 | ||| - now - lastUsed > ttl
658 | ||| - Time is measured using `CLOCK_MONOTONIC`, ensuring:
659 | ||| - No sensitivity to wall-clock changes.
660 | ||| - Stable elapsed-time semantics.
661 | |||
662 | ||| Concurrency Model:
663 | ||| - Cleanup is performed via `cleanStripe`, which uses CAS:
664 | ||| - Stripe state updates are atomic.
665 | ||| - Effects are executed exactly once after commit.
666 | ||| - Safe under contention:
667 | ||| - Multiple threads may attempt cleanup concurrently.
668 | ||| - Only one successful CAS applies each transition.
669 | ||| - No resource is freed more than once.
670 | |||
671 | ||| Execution Model:
672 | ||| - This function performs IO (time retrieval) outside CAS.
673 | ||| - The actual mutation and freeing are deferred via `StripeEffect`.
674 | ||| - No IO occurs during CAS evaluation.
675 | |||
676 | ||| Usage:
677 | ||| - Intended to be called opportunistically during:
678 | ||| - `takeResource`
679 | ||| - `putResource`
680 | ||| - `tryTakeResource`
681 | ||| - Provides amortized cleanup without background threads.
682 | |||
683 | ||| Guarantees:
684 | ||| - Stale resources are eventually freed under continued usage.
685 | ||| - No interference with active resources or waiters.
686 | ||| - No blocking or waiting is introduced.
687 | |||
688 | ||| Tradeoffs:
689 | ||| - Cleanup is activity-driven rather than time-driven.
690 | ||| - Idle stripes may retain stale resources longer.
691 | ||| - In exchange:
692 | ||| - No global thread.
693 | ||| - Lower runtime overhead.
694 | ||| - Fully local behavior.
695 | |||
696 | ||| Failure Handling:
697 | ||| - Crashes if time retrieval fails (consistent with module error policy).
698 | |||
699 | ||| Invariants:
700 | ||| - Only cached resources are considered for cleanup.
701 | ||| - Each freed resource is removed exactly once.
702 | ||| - Stripe structure remains consistent after cleanup.
703 | |||
704 | private
716 | where
720 | ||| Return a resource to the `Pool1 World n a`.
721 | |||
722 | ||| Behavior:
723 | ||| - If a waiter exists, the resource is delivered directly.
724 | ||| - Otherwise, it is inserted into the cache with a timestamp.
725 | |||
726 | ||| Guarantees:
727 | ||| - No resource is lost.
728 | ||| - Wakeups are ordered and deterministic.
729 | |||
730 | export
739 | ||| Destroy all resources in all stripes in the `Pool1 World n a`.
740 | |||
741 | ||| Behavior:
742 | ||| - Removes all cached resources from every stripe.
743 | ||| - Frees them via the provided `freeresource` function.
744 | ||| - Leaves wait queues untouched.
745 | |||
746 | ||| Guarantees:
747 | ||| - Each resource is freed exactly once.
748 | ||| - No IO occurs during Stripe state transitions.
749 | ||| - Safe under contention (uses CAS + effect model).
750 | |||
751 | ||| Notes:
752 | ||| - This only affects cached (idle) resources.
753 | ||| - Resources currently checked out are NOT affected.
754 | |||
755 | export
762 | where
775 | ||| Restore one unit of available capacity in the `Stripe a`.
776 | |||
777 | ||| Behavior:
778 | ||| - Increments `available` by 1.
779 | ||| - Does not modify cache or queue.
780 | ||| - Emits no effects.
781 | |||
782 | ||| Used when resource creation fails after capacity was reserved.
783 | |||
784 | ||| Guarantees:
785 | ||| - Atomic under CAS.
786 | ||| - No IO performed.
787 | ||| - Safe under contention.
788 | |||
789 | export
794 | where
798 | MkStripeStep
800 | [None]
802 | ||| Acquire a resource from the `Pool1 World n a`.
803 | |||
804 | ||| Behavior:
805 | ||| - Attempts to take a resource from the local stripe.
806 | ||| - Uses a single CAS step to atomically choose between:
807 | ||| - Reusing a cached resource.
808 | ||| - Reserving capacity for new resource creation.
809 | ||| - Enqueuing as a waiter when fully exhausted.
810 | |||
811 | ||| Fast Path (Cache Reuse):
812 | ||| - If a cached resource exists:
813 | ||| - Remove it from the cache.
814 | ||| - Return it immediately.
815 | ||| - `available` is not modified.
816 | ||| - The resource already exists and therefore does not consume creation capacity.
817 | |||
818 | ||| Creation Path:
819 | ||| - If the cache is empty but `available > 0`:
820 | ||| - Atomically decrement `available`.
821 | ||| - Reserve one unit of creation capacity.
822 | ||| - Create a fresh resource outside the CAS section.
823 | |||
824 | ||| Wait Path:
825 | ||| - If:
826 | ||| - cache is empty.
827 | ||| - and `available == 0`.
828 | ||| - Then:
829 | ||| - enqueue a `Waiter`.
830 | ||| - block on `waitForResource`.
831 | ||| - The waiter is eventually:
832 | ||| - woken with `Just a` when a resource is returned.
833 | ||| - or `Nothing` when capacity is restored.
834 | |||
835 | ||| Capacity Semantics:
836 | ||| - `available` represents remaining creation budget.
837 | ||| - It is decremented ONLY when creating a brand-new resource.
838 | ||| - It is restored when:
839 | ||| - resource creation aborts.
840 | ||| - resources are destroyed.
841 | |||
842 | ||| Concurrency Guarantees:
843 | ||| - Decision logic is atomic via CAS.
844 | ||| - No IO occurs during CAS evaluation.
845 | ||| - Effects execute exactly once after successful commit.
846 | ||| - Waiters are served FIFO (excluding cancelled waiters).
847 | ||| - No lost wakeups.
848 | |||
849 | ||| Invariants:
850 | ||| - Total live resources never exceeds stripe capacity.
851 | ||| - Cached resources are timestamped before insertion.
852 | ||| - Waiters exist only inside Stripe state.
853 | |||
854 | export
860 | -- clean stripe if needed
862 | -- pre-allocate channel for slow path
868 | -- fast path
873 | rest
874 | queue
875 | queuer
876 | nextid
877 | cancelled
882 | )
883 | -- slow path
887 | -- enqueue waiter
894 | cache
895 | queue
898 | cancelled
903 | )
905 | -- resource creation slot
909 | cache
910 | queue
911 | queuer
912 | nextid
913 | cancelled
918 | )
921 | -- Run effects after commit
924 | -- fast path
927 | -- create immediately
938 | -- woken with resource
941 | -- need to create
951 | where
958 | ||| Safely acquire and use a resource from the pool.
959 | |||
960 | ||| This is the primary high-level interface for working with `Pool1`.
961 | ||| It ensures that resources are correctly returned or destroyed,
962 | ||| even in the presence of exceptions or cancellation.
963 | |||
964 | ||| Behavior:
965 | ||| - Acquires a resource using `takeResource`.
966 | ||| - Executes the user action `f` with that resource.
967 | ||| - On normal completion:
968 | ||| - The resource is returned to the pool via `putResource`.
969 | ||| - On exception or cancellation:
970 | ||| - The resource is destroyed via `destroyResource`.
971 | |||
972 | ||| Concurrency & Masking:
973 | ||| - The entire operation runs inside `uncancelable`, ensuring:
974 | ||| - Resource acquisition and release cannot be interrupted.
975 | ||| - The user action `f` is executed via `poll`, meaning:
976 | ||| - It *can* be interrupted or cancelled.
977 | ||| - If cancellation occurs during `f`, the cleanup handler runs.
978 | |||
979 | ||| Cleanup Guarantees:
980 | ||| - Exactly one of the following happens:
981 | ||| - `putResource` (success path)
982 | ||| - `destroyResource` (failure or cancellation path)
983 | ||| - No resource is leaked or returned twice.
984 | ||| - Waiters are properly woken via Stripe effects.
985 | |||
986 | ||| Failure Handling:
987 | ||| - Exceptions from `f` are propagated.
988 | ||| - Exceptions during acquisition or cleanup cause a crash (consistent with the rest of the module’s error handling).
989 | |||
990 | ||| Returns:
991 | ||| - The result of applying `f` to the acquired resource.
992 | |||
993 | ||| Invariants:
994 | ||| - Resources are never duplicated or lost.
995 | ||| - Pool state remains consistent under concurrency.
996 | ||| - All Stripe effects are executed after CAS commit.
997 | |||
998 | export
1010 | where
1023 | ||| Attempt to take a resource without blocking.
1024 | |||
1025 | ||| Behavior:
1026 | ||| - Reads the local stripe and checks availability.
1027 | ||| - If no resources are available:
1028 | ||| - Returns `Nothing` immediately.
1029 | ||| - Does NOT enqueue a waiter.
1030 | ||| - Does NOT create a resource.
1031 | |||
1032 | ||| - If a resource is available:
1033 | ||| - Removes it atomically via CAS.
1034 | ||| - Returns `Just (resource, LocalPool1)`.
1035 | |||
1036 | ||| Guarantees:
1037 | ||| - Non-blocking: never waits on a channel.
1038 | ||| - No side effects inside CAS.
1039 | ||| - No waiter allocation.
1040 | ||| - Safe under contention via CAS retry.
1041 | |||
1042 | export
1053 | where
1060 | -- clean stripe if needed
1062 | -- attempt fast-path only
1066 | -- no capacity, do nothing
1070 | )
1071 | -- cache hit, consume
1074 | rest
1075 | queue
1076 | queuer
1077 | nextid
1078 | cancelled
1080 | )
1081 | -- available > 0, but cache empty
1085 | )
1100 | ||| Attempt to acquire and use a resource from the pool without blocking.
1101 | |||
1102 | ||| Behavior:
1103 | ||| - Tries to take a resource immediately using `tryTakeResource`.
1104 | ||| - If no resource is available:
1105 | ||| - Returns `Nothing` without blocking or creating a resource.
1106 | ||| - If a resource is available:
1107 | ||| - Executes the provided function `f` with the resource.
1108 | ||| - Returns `Just result` on success.
1109 | |||
1110 | ||| Resource Handling:
1111 | ||| - The acquired resource is always returned to the pool via `putResource` after successful execution of `f`.
1112 | ||| - If an exception or cancellation occurs during `f`:
1113 | ||| - The resource is destroyed using `destroyResource` instead of being returned.
1114 | |||
1115 | ||| Cancellation Semantics:
1116 | ||| - The outer operation is `uncancelable`, ensuring:
1117 | ||| - No resource is leaked between acquisition and release.
1118 | ||| - The user function `f` is executed under `poll`, meaning:
1119 | ||| - It remains cancelable.
1120 | ||| - If cancellation occurs during `f`:
1121 | ||| - The resource is safely discarded.
1122 | ||| - The pool remains in a consistent state.
1123 | |||
1124 | ||| Concurrency Guarantees:
1125 | ||| - Does not block waiting for a resource.
1126 | ||| - Does not enqueue a waiter.
1127 | ||| - All Stripe transitions (`putResource`, `destroyResource`) are performed via `casWithEffects`, ensuring:
1128 | ||| - Atomic state updates.
1129 | ||| - No duplicated side effects.
1130 | ||| - Deterministic wake behavior.
1131 | |||
1132 | ||| Failure Handling:
1133 | ||| - Any exception from `f` is propagated.
1134 | ||| - Internal pool errors result in a crash with diagnostic information.
1135 | |||
1136 | ||| Returns:
1137 | ||| - `Nothing` if no resource was immediately available.
1138 | ||| - `Just r` if a resource was acquired and `f` completed successfully.
1139 | |||
1140 | ||| Notes:
1141 | ||| - This function is the non-blocking counterpart to `withResource`.
1142 | ||| - It is useful when callers prefer to fallback rather than wait.
1143 | |||
1144 | export
1156 | where