JsonStore

An embedded JSON document store. Inserts stream their input, searches return lazily read results (SPEC.md). This is the store's writer, which holds the directory's lock until it's closed, with the read operations of ReadableJsonStore; other processes can read the store beside it with openSecondary. After a failed write or sync, operations throw EngineFailedException until the store is reopened (SPEC.md §8).

Inserts of any size run concurrently: small documents are logged together in group commits, large ones stream to lanes of their own, so a huge or slow insert doesn't hold up small ones. Memory stays within the store's budgets whatever the document sizes; when indexes can't be flushed as fast as documents arrive, inserts wait (SPEC.md §8).

Objects can carry a type name (SPEC.md §6), given with the insert as JSON paths or assigned later. Types are metadata: they never change the stored JSON.

Documents are never changed in place: to change one, replace it, which inserts the new version and deletes the old one in one atomic change (SPEC.md §7.1).

Stored values and JSON text can be compared for equality or diffed (SPEC.md §9), however large they are.

Types

Link copied to clipboard
object Companion

Three ways to get a store's writer, one per intent, so that a mistyped path never quietly becomes a new, empty store: open for a store that exists, create for a new one, and openOrCreate where either will do. And openSecondary to read a store beside its writer.

Properties

Link copied to clipboard

What this store was created with; fixed for its life (see CreationOptions).

Functions

Link copied to clipboard
abstract suspend fun assignType(node: NodeRef, type: String?)

Sets the type of the object node to type, or removes it with null; returns once the change is acknowledged (SPEC.md §6.1). Assigning the type the object already has changes nothing.

Link copied to clipboard
abstract suspend fun assignTypes(assignments: List<Pair<NodeRef, String?>>)

Applies assignments in order as one atomic change; a later assignment to the same object sees the earlier ones. When one of them fails the checks of assignType, nothing changes.

Link copied to clipboard
abstract override fun blocking(): BlockingJsonStore

The same store without coroutines: a view that shares this store's state, valid until this store is closed. The view has no close of its own; closing this store is what releases it.

Link copied to clipboard
abstract suspend fun clearType(type: String): Long

Removes type from every object that has it, as one atomic change, and returns how many objects lost it; returns once the change is acknowledged. Every object that has the type when the change is applied loses it, so none of those typed by operations acknowledged before the call keeps it. The name stays in listTypes. The call holds 32 bytes per object of the type while it runs. Throws InvalidTypeNameException for an invalid name.

Link copied to clipboard
abstract fun close()
Link copied to clipboard
abstract suspend fun count(query: DocumentKeyQuery, atMost: Long = Long.MAX_VALUE): Long

How many documents the key-order search returns for query, counting no further than atMost; see the other count.

abstract suspend fun count(query: JsonPathQuery, atMost: Long = Long.MAX_VALUE): Long

How many hits the JSONPath search returns for query, counting no further than atMost; see the other count.

abstract suspend fun count(query: KeyValueQuery, atMost: Long = Long.MAX_VALUE): Long

How many hits search returns for query on the current state, counting no further than atMost (SPEC.md §4.7): the same candidates and matches, with the query's after, type, mode and nested-hit filter, but no hit's JSON is read, and limit and maxBytes don't apply. count(query, n) == n tells whether there are at least n hits, and stops at the n-th. Refuses what search refuses; a negative atMost throws IllegalArgumentException.

Link copied to clipboard
abstract suspend fun countOfType(type: String): Long

The exact number of objects of type; reads the entry of every one of them.

Link copied to clipboard
abstract suspend fun delete(docId: DocId): Boolean

Deletes the document docId with its type assignments (SPEC.md §7); returns once the delete is acknowledged. Returns true when this call deleted it, false when it was already deleted. Searches started before keep seeing the document until they're closed.

Link copied to clipboard
abstract suspend fun deleteAll(docIds: Collection<DocId>): Int

Deletes the documents of docIds as one atomic change; returns how many this call deleted, skipping documents already deleted and repeated docIds. When one of them fails the checks of delete, nothing changes.

Link copied to clipboard
abstract fun diff(a: CompareOperand, b: CompareOperand, options: DiffOptions = DiffOptions()): Flow<Difference>

The differences between a (left) and b (right), streamed in canonical order (SPEC.md §9.3): depth first, object members by key, array elements by index. Each collection runs the comparison on one new snapshot, on the store's dispatcher, and stops after DiffOptions.maxDifferences. Errors as in isEqual, thrown when collecting.

Link copied to clipboard
abstract fun explain(query: JsonPathQuery): QueryPlan

How search would find the candidate documents of query, without running it.

Link copied to clipboard
abstract suspend fun get(key: String): DocId?

The visible document whose key is key in a store with a CreationOptions.documentKey, or null when none is (SPEC.md §3.3); with Normalization.NFC, key is normalized first. A store without document keys throws IllegalStateException.

Link copied to clipboard
abstract fun info(): StoreInfo

How many documents the store holds, and has deleted; read from memory, so it costs nothing to ask.

Link copied to clipboard
abstract suspend fun insert(source: InputStream, types: Map<String, String> = emptyMap()): DocId

Inserts the JSON document read from source, which isn't closed; returns its docId once it's visible. Cancelling the call stops it at the next read of source, and nothing of the document becomes visible; once source is read to its end, the insert completes and returns even if the call is cancelled (SPEC.md §8).

abstract suspend fun insert(path: Path, types: Map<String, String> = emptyMap()): DocId

Inserts the document in the file path, read as it is split; the file is opened and closed by the call.

abstract suspend fun insert(json: String, types: Map<String, String> = emptyMap()): DocId

Inserts a document given as a string; meant for small documents.

Link copied to clipboard
abstract suspend fun insertAll(sources: Iterable<InsertSource>): List<InsertOutcome>

Inserts the documents of sources in order, reading each to its end (streams aren't closed). Small documents are logged in shared group commits, so bulk loads sync far less often than with one insert per document. Returns one outcome per source; a rejected document, or a replace that conflicts, doesn't stop the others. Any other error is thrown: documents whose outcome wasn't returned may or may not be visible.

Link copied to clipboard
abstract suspend fun isEqual(a: CompareOperand, b: CompareOperand, options: CompareOptions = CompareOptions()): Boolean

Whether the values a and b are equal (SPEC.md §9): typed, numbers by decimal value, strings by code points, object members in any order, array elements in order. Reads one snapshot of the store with memory independent of the values' sizes; values given as JSON are read and split first.

Link copied to clipboard
abstract fun listTypes(): List<TypeInfo>

Every type name ever assigned, sorted by name, with an approximate count of its objects.

Link copied to clipboard
abstract suspend fun objectsOfType(type: String, after: NodeRef? = null, limit: Int = 100): List<NodeRef>

Up to limit objects of type in (docId, nodePre) order from the current state, strictly after after when it's given; resume with the last one returned. Throws InvalidTypeNameException for an invalid name.

Link copied to clipboard
abstract fun objectsOfTypeFlow(type: String, after: NodeRef? = null): Flow<NodeRef>

Every object of type in (docId, nodePre) order, strictly after after when it's given, as a flow. Each collection reads one new snapshot on the store's dispatcher and holds it until the collection ends, so what one collection emits is consistent: the objects of the type as the operations acknowledged before it started left them. Throws InvalidTypeNameException for an invalid name, when collecting.

Link copied to clipboard
abstract suspend fun objectsOfTypePage(type: String, after: Cursor? = null, limit: Int = 100): TypePage

Up to limit objects of type in (docId, nodePre) order from the current state, strictly after after when it's given, and where the next page starts: TypePage.cursor, null when no objects were left. Only the cursor of a TypePage resumes it, anything else throws InvalidCursorException.

Link copied to clipboard
abstract suspend fun pathOf(node: NodeRef): String

Where node is in its document, as an RFC 9535 Normalized Path from the document root (SPEC.md §3.2): $['orders'][3], and $ for the root. Reads one fragment per level from the root down to the node.

Link copied to clipboard
abstract suspend fun replace(docIds: Collection<DocId>, source: InputStream, types: Map<String, String> = emptyMap()): DocId

Inserts the JSON document read from source as insert does, and deletes the documents of docIds in the same atomic change (SPEC.md §7.1): once it's acknowledged the new document is visible and they are deleted, and a crash leaves either both or neither. Returns the new document's docId.

abstract suspend fun replace(docIds: Collection<DocId>, path: Path, types: Map<String, String> = emptyMap()): DocId

replace with the document in the file path, opened and closed by the call.

abstract suspend fun replace(docIds: Collection<DocId>, json: String, types: Map<String, String> = emptyMap()): DocId

replace with the document given as a string; meant for small documents.

Link copied to clipboard

Starts a search of documents in key order (SPEC.md §3.3) on the current state; close the result when done. A store without document keys throws IllegalStateException.

abstract fun search(query: JsonPathQuery): SearchResult

Starts a JSONPath search on the current state (SPEC.md §10.4): the distinct nodes the query selects from every document, in the query's order of (docId, nodePre). Close the result when done.

abstract fun search(query: KeyValueQuery): SearchResult

Starts a search on the current state, reading on the caller's thread; close the result when done. A search the index controls of CreationOptions leave unanswerable throws QueryRejectedException with QueryRejectedException.Reason.NOT_INDEXED (SPEC.md §4.6).

Link copied to clipboard
abstract fun select(node: NodeRef, path: JsonPath): SelectResult

The nodes path selects from the value of node, as RFC 9535 defines its nodelist: in the RFC's order, with duplicates, each with its Normalized Path, where $ is node (SPEC.md §10.3). The result reads the current state lazily; close it when done.

Link copied to clipboard
abstract suspend fun typeOf(node: NodeRef): String?

The type of node, or null, which nodes that aren't objects always get. Errors as in JsonStore.assignType.

Link copied to clipboard
abstract suspend fun upsert(source: InputStream, types: Map<String, String> = emptyMap()): DocId

Inserts the JSON document read from source as insert does, in a store with a CreationOptions.documentKey, and deletes the visible document with the same key, if there is one, in the same atomic change (SPEC.md §3.3): once it's acknowledged the new document is visible and the old one deleted, and a crash leaves either both or neither. Of two upserts of one key, the one acknowledged last wins. Returns the new document's docId.

abstract suspend fun upsert(path: Path, types: Map<String, String> = emptyMap()): DocId

upsert with the document in the file path, opened and closed by the call.

abstract suspend fun upsert(json: String, types: Map<String, String> = emptyMap()): DocId

upsert with the document given as a string; meant for small documents.

Link copied to clipboard
abstract suspend fun verify(progress: VerifyProgress = VerifyProgress { _, _, _ -> }): VerifyReport

Reads everything the store can read back and reports what doesn't check out (SPEC.md §8): record and block checksums, the fragments each COMMIT still in the log names, and every fragment-index entry against the record it locates. Nothing is changed or repaired.