Lore testing standards

This document defines the standard patterns for testing across the Lore codebase.

Overview

Type	Location	Framework	Purpose
Unit tests	Inline #[cfg(test)] modules	Rust/tokio	Module-level testing
Integration tests	lore-revision/tests/	Rust/tokio	Cross-module testing
Smoke tests	scripts/test/	Python/pytest	CLI and server testing
Load tests	Internal infrastructure	Internal harness	Performance testing

---

1. Target risk at the cheapest tier that can find the fault

Test effort follows risk, not line count. Business logic — session state, ordering guarantees, allocation and placement decisions — earns thorough tests. Plumbing and trivial code earn few or none. A test that restates the implementation costs maintenance and finds nothing.

Then split by what you are checking:

• Edge cases, variants, boundaries, and error paths → unit tests. They are cheap, so be exhaustive.
• Happy paths → tests that run against real things. Mocks cannot tell you whether this works wired to a real dependency or assembled into the binary we ship, and it is not worth asking twenty times over.

Assert on outcomes, not on wording. Check Ok or Err and the error variant, never a substring of the message. Message text is presentation: it changes for readability and the test fails for no reason, or it stops matching a real regression and passes for no reason.

The tiers

Cheapest first. Cost tracks how much must be standing before the test can run.

Tier	Answers	Location	Needs
Unit	Is this logic correct across all its cases?	Inline #[cfg(test)]	Nothing
Smoke	Does a real user flow work across the binaries?	scripts/test/, @pytest.mark.smoke	lore and loreserver binaries built

Edge cases belong in unit tests. The smoke test covers the happy path only.

---

2. Rust unit tests

Inline in source modules with #[cfg(test)]:

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_sync_example() {
        // synchronous test
    }
}

---

3. Rust async tests

Frameworks: tokio, mockall, async-trait

All async tests use the LORE_CONTEXT.scope() pattern:

#[tokio::test]
async fn test_example() {
    LORE_CONTEXT
        .scope(setup_test_execution(), async {
            // test code
        })
        .await;
}

Test independence

Tests MUST be independent and isolated. Avoid:

• #[serial] — Forces sequential execution, indicating shared mutable state.
• Test dependencies — Tests that rely on other tests running first.

If you find yourself needing #[serial], refactor the test to:

1. Create isolated test fixtures and state per test.
2. Use unique identifiers (for example, random repository names or unique temp directories).
3. Mock shared resources instead of using real shared state.

// Anti-pattern: shared state requiring serial execution
#[tokio::test]
#[serial]  // DON'T DO THIS
async fn test_with_shared_state() { ... }

// Preferred: isolated test with unique fixtures
#[tokio::test]
async fn test_with_isolated_state() {
    let test_repo = test_store_create();  // Creates unique test repository
    // Test uses only this isolated state
}

Key files:

• lore-revision/tests/helper.rs — test_store_create(), setup_test_execution().
• lore-revision/tests/ — Cross-module integration tests.
• lore-integration-tests/ — Tests against real infrastructure, without mocking

Scratch space on disk

lore_base::test_util::TempDir, and TempFile for a single file, are the only way a test asks for scratch space on disk. Both are behind lore-base's test-util feature, which the crates that need it enable from their dev-dependencies:

[dev-dependencies]
lore-base = { workspace = true, features = ["test-util"] }

let dir = TempDir::new("my-test-");       // <temp>/my-test-A1b2C3d4
std::fs::write(dir.child("data.bin"), b"...").expect("write");

let file = TempFile::with_contents("my-test-", b"payload");
some_api(file.path());

The name carries a random suffix, so two tests — or two runs on one machine — never share a directory, and removal happens in Drop, so it happens whether the test passes or fails an assertion.

Do not build a path under std::env::temp_dir() by hand, and do not remove a directory on the last line of a test body: a failing assertion panics before reaching that line, so the run that most needs its output is the one that leaks it.

Return the guard, never a path derived from it. A helper that ends temp.path().to_path_buf() drops the guard on return and deletes the directory its caller is about to read. Hand back the TempDir and let the caller hold it:

fn fixture() -> (TempDir, PathBuf) { ... }   // correct
fn fixture() -> PathBuf { ... }              // deletes the directory on return

Set LORE_KEEP_TEST_DATA=1 to keep the directories rather than remove them — the Rust equivalent of the smoke tests' --keep-test-data. The prefix is all there is to identify one afterwards, so give it the test's name wherever several tests in a module would otherwise share it.

tempfile is the implementation and a dependency of lore-base only. Do not add it to another crate: a second guard type is a second set of rules, and it does not honour LORE_KEEP_TEST_DATA.

Polling for async state

Poll for a condition instead of sleeping. Fixed sleeps are flaky on slow CI, wasteful on fast machines, and opaque — a failure does not say whether the timeout was wrong or the code is broken.

// BAD
sleep(Duration::from_millis(500)).await;
// GOOD
wait_for_condition(|| async { manager.session_count() == 2 }, Duration::from_secs(5))
    .await
    .expect("sessions should appear within 5 seconds");

Fixed sleeps are acceptable only for time-based behavior such as TTL expiry, or pauses under 20ms to let a spawned task start.

---

4. Smoke tests (scripts/test/)

Framework: pytest

Requirement: All Lore CLI commands must have smoke test coverage. When adding a new command, add corresponding tests to scripts/test/.

Key files

File	Purpose
conftest.py	Fixtures and server management
lore.py	Lore wrapper class for the CLI
error_types.py	Exception mapping from CLI output

Fixtures

• new_lore_repo — Creates a new test repository.
• scratch_dir — Hands out a path beside the repositories for the test to create something at: a shared store, a clone target, a moved instance.
• lore_executable_path — Path to the Lore client binary.
• auto_lore_local_server — Auto starts the server for the session.

Test data is removed as the run goes

A full run writes about ten gigabytes, so nothing is kept once it is no longer needed. Cleanup is in two layers:

1. Per test. new_lore_repo removes every repository it handed out and every repository those went on to clone; scratch_dir removes every path it handed out; global_dir_name removes the isolated global config and the shared stores under it. All of this runs whether the test passed or failed. A server built inside a test or a class fixture through generate_server_config is removed at that same scope, after the fixture that launched it has stopped it. An autouse fixture removes the per-test tmp_path, which pytest itself only removes under the failed policy and then only for tests that passed.
2. Per session. _SessionCleanup in lore_server.py empties basetemp once every worker has finished, taking the session server's root and anything a per-test cleanup could not.

A test that creates files outside its repository must take scratch_dir and create them at a path it hands out. Anything written straight to tmp_path_factory.getbasetemp() survives the test that made it and is only caught by the session sweep, which is a backstop, not the guard.

Removal is best effort and never fails a test: a locked file is logged and left to the sweep.

Pass --keep-test-data to keep everything on disk — repositories, stores, server roots and the server log — when investigating a failure.

Usage

@pytest.mark.smoke
def test_commit(new_lore_repo):
    repo: Lore = new_lore_repo()
    repo.stage(offline=True)
    repo.commit("Test commit", offline=True)
    repo.push()

Running with uv

Tests require Python 3.13+. Use uv to manage dependencies and run tests:

# Install dependencies
uv sync

# Run all smoke tests with local server
uv run pytest scripts/test/ --lore-client-binary=release --lore-server-binary=release

# Run only tests marked as smoke
uv run pytest scripts/test/ -m smoke

# Run tests in parallel (uses pytest-xdist)
uv run pytest scripts/test/ -n auto

# Against external server
uv run pytest scripts/test/ --disable-local-server --lore-remote-url=lore://host:port

Command-line options

Option	Default	Description
--lore-client-binary	release	Path or "release"/"debug"
--lore-server-binary	release	Path or "release"/"debug"
--lore-remote-url	lore://127.0.0.1:41338	Server address
--disable-local-server	false	Use external server
--disable-auto-server	false	Don't auto start the server
--keep-test-data	false	Leave repositories, stores and server roots on disk

Per-test timeout

Every test is bounded at ten minutes (timeout in pyproject.toml), and the stack of every thread is dumped a minute before that (faulthandler_timeout). Nothing else bounds a test: the helper runs the client with subprocess.run and no timeout, so a client that never returns would otherwise be a run that never returns.

A test that reaches the limit is reported as a failure and the run continues. On Linux the test fails in place; on Windows there is no SIGALRM, so the worker is killed and xdist reports worker 'gwN' crashed while running <test> and replaces it. Either way the log names the test and carries the stack it was stuck in.

Override with --timeout=0 to disable, or --timeout=<seconds> for a run that is legitimately slower.

pytest markers

Marker	Description
@pytest.mark.smoke	Smoke tests for basic functionality
@pytest.mark.slow	Slow running tests
@pytest.mark.disable_auto_server	Tests requiring the --disable-auto-server flag

---

5. Load tests

Lore has a load-testing suite that exercises concurrent clone, commit, sync, lock, and compaction workloads. It runs on internal infrastructure and isn't part of the open-source repository, so its harness and scenarios aren't documented here.

---

6. Best practices

1. Target risk, not line count — spend test effort on business logic; leave plumbing and trivial code with few or no tests.
2. All Lore commands must have smoke tests in scripts/test/.
3. Use LORE_CONTEXT.scope() for all async Rust tests.
4. Keep tests independent — Avoid #[serial] and test dependencies; use isolated fixtures.
5. Poll instead of sleeping for async state; reserve fixed sleeps for time-based behavior or sub-20ms task startup.
6. Use the new_lore_repo fixture for smoke tests, and scratch_dir for anything a test creates outside its repository — both remove what they hand out.
7. Mark tests with @pytest.mark.smoke for smoke test runs.
8. Use offline=True for operations that don't need the server.
9. Feature-gate integration tests that require external dependencies.