DocLite

A powerful PHP NoSQL document store built on top of SQLite.

Table of contents

• About DocLite
  • Why DocLite?
• Getting Started
• The Database
  • Creating a memory database
  • Creating a file database
  • Error Handling & Logging
  • Import and export data
    • Importing Data
    • Exporting data
  • Advanced options
    • Get DocLite version
    • Optimize database
    • Set synchronization mode
    • Set rollback journal mode
• Collections
  • About Collections
  • Obtain a collection
  • Create a document
  • Save a document
  • Retrieve a document
  • Map a document to a custom class
  • Delete a document
  • Query a collection
    • Find single document by values
    • Find all matching documents by values
    • Find all documents in collection
    • Advanced queries
    • Query operators
  • Join collections
  • Caching results
  • Index a collection
  • Unique indexes
  • Delete a collection
  • Collection transactions
  • Full text search
• Documents
  • About Documents
  • Getting and setting document data
  • Mapping document fields to objects
  • Document Unique Id
  • Saving a document
  • Deleting a document
  • Document validation
• Other info
  • Symfony integration
  • Licensing
  • Bugs, issues
  • Contact the author

About DocLite

DocLite is a powerful NoSQL document store for PHP built on top of SQLite. It uses the
PHP PDO SQLite library to access a SQLite database and automatically manage
documents organized in to named collections, which are stored as JSON.

DocLite takes advantage of the SQLite JSON1 extension (this is usually bundled in to
the libsqlite included with your PHP distribution, so you probably already have it)
to store, parse, index and query JSON documents - giving you the power and flexibility
of a fully transactional and ACID compliant NoSQL solution, yet contained within the
local file system. No need for more complex systems like Mongo, CouchDB or Elasticsearch
when your requirements are slim. No need for any external dependencies, just PHP with
PDO SQLite enabled.

DocLite provides a simple, intuitive, flexible and powerful PHP library that you can
learn, install and start using in minutes.

Why DocLite?

DocLite lends itself well to a variety of use cases, including but not limited to:

• Agile development and rapid prototyping while your requirements are evolving.

• Powerful, self-contained NoSQL database for small to medium websites or
  applications, such as blogs, business website, CMS, CRM or forums.

• A fast and reliable cache for data retrieved from remote databases, APIs or
  servers. Process your data in to documents, save in DocLite and easily query
  and filter your data as needed.

• Robust, performant, ACID compliant replacement for weaker, slower, flat-file
  data stores utilizing JSON, XML or YAML.

• Application database for web apps installed and run on a local environment.

• Database for microservices and middleware.

• Fast in-memory database for data processing or machine learning algorithms.

Broadly speaking, DocLite is suitable for the same uses cases
as the underlying SQLite engine it is built on, but where you desire a NoSQL solution.

Getting Started

use Gebler\Doclite\{FileDatabase, MemoryDatabase};

// To create or open an existing file database.
$db = new FileDatabase('/path/to/db');

// To open an existing file database in read-only mode.
$db = new FileDatabase('/path/to/existing/db', true);

// To create a new in-memory database.
$db = new MemoryDatabase();

$users = $db->collection("user"); 

// Create a new User in the collection
$user = $users->get();

// Get the automatically generated document ID
$id = $user->getId();

// Set properties by magic set* methods
$user->setUsername("dwgebler");
$user->setRole("admin");
$user->setPassword(password_hash("admin", \PASSWORD_DEFAULT));
$user->setCreated(new \DateTimeImmutable);

// Update the user in the collection
$user->save();

// Retrieve this user later on by ID
$user = $users->get($id);

// Or search for a user by any field
$user = $users->findOneBy(["username" => "dwgebler"]);

class CustomUser
{
    private $id;
    private $username;
    private $password;
    
    public function getId() {...}
    public function setId($id) {...}
    public function getUsername() {...}
    public function setUsername($username) {...}    
}

// Retrieve a previously created user and map the result on to a CustomUser object.
// You can also pass a null ID as the first parameter to create a new CustomUser.
$user = $users->get($id, CustomUser::class);

// $user is now an instance of CustomUser and can be saved through the Collection.
$users->save($user);

The Database

DocLite is built on top of SQLite 3 and supports two types of database; file and memory.
The corresponding classes are FileDatabase and MemoryDatabase.

Creating a memory database

MemoryDatabase is stored in volatile memory and is therefore ephemeral for the lifetime
of your application scripts. Its constructor takes optional parameters:

• a boolean flag indicating whether to enable full text search features (defaults to false) - this
  feature requires SQLite to have been compiled with the FTS5 extension.
• an integer representing the maximum connection timeout in seconds (defaults to 1) which
  is how long the connection should wait if the underlying SQLite database is locked.
• A PSR-3 compatible logger instance (defaults to null).

```php
use Gebler\Doclite\MemoryDatabase;

$db = new MemoryDatabase();

// With full text search enabled and a 2-second connection timeout
$logger = new \Monolog\Logger('my-logger');
$db = new MemoryDatabase(true, 2, $logger); 

Creating a file database

FileDatabase constructor takes one mandatory and then some optional parameters;
only the file or directory path to a new or existing database is required.

Optional parameters are:

• a boolean flag indicating whether the database should be opened in read-only mode, which defaults to false.
• a boolean flag indicating whether to enable full text search features (defaults to false) - this
  feature requires SQLite to have been compiled with the FTS5 extension.
• an integer representing the maximum connection timeout in seconds (defaults to 1) which
  is how long the connection should wait if the underlying SQLite database is locked.
• A PSR-3 Logger instance to use for logging database events.

The path supplied to FileDatabase can be a relative or absolute path which is any of:

• An existing directory with read and write access.
• A non-existent file in a directory with read-write access.
• An existing database in a directory with read-write or read-only access (read-only mode).
• A non-existing directory path which your script has permission to create.

If no file name is specified, a default file name data.db will be used for the underlying database.

use Gebler\Doclite\FileDatabase;

// Open a new database
$db = new FileDatabase('./data/mydb.db');

// Open an existing database in read-only mode
$db = new FileDatabase('./data/mydb.db', true);

// Open a new database called data.db in existing directory /home/data
$db = new FileDatabase('/home/data');

// All options - path, read-only mode, full text search, connection timeout and logger
$logger = new \Monolog\Logger('mylogger');
$db = new FileDatabase('./data/mydb.db', false, true, 1, $logger);

// Or, in PHP 8, named parameters:
$db = new FileDatabase(path: './data/mydb.db', readOnly: true, ftsEnabled: true);

If you open a database in read-only mode, you will be able to retrieve documents from a collection,
but you will not be able to save them or create new documents or collections.
Attempting to do so will trigger an error.

It is good practice wrapping FileDatabase creation in a try-catch block.
Initializing a FileDatabase may throw either an IOException (for errors relating to the file system)
or a DatabaseException (for errors establishing the DB connection).

use Gebler\Doclite\Exception\IOException;
use Gebler\Doclite\Exception\DatabaseException;
use Gebler\Doclite\FileDatabase;

try {
  $db = new FileDatabase('/path/to/db');
} catch (IOException $e) {
    var_dump($e->getMessage());
} catch (DatabaseException $e) {
    var_dump($e->getMessage());
}

Error Handling & Logging

To enable logging queries and parameters in the full and final SQL sent to the database, pass a PSR-3 logger instance in to you FileDatabase or MemoryDatabase
either via the constructor or the $database->setLogger(LoggerInterface $logger) method at any time.

Then call $database->enableQueryLogging() to enable logging of all queries. These will be logged at the debug level.

Or call $database->enableSlowQueryLogging() to enable logging of queries that take longer than 500ms.
These will be logged with a warning level.

As long as a LoggerInterface instance is set on a database, any exceptions will also be logged at the error level.

You can disable logging by calling $database->disableQueryLogging() or $database->disableSlowQueryLogging().

DocLite primarily throws a DatabaseException when any error occurs. This is true across
the Database, Collection and Document types. A Database exception will
include a message, an error code (see below), any underlying system exception if there was one (so normal
Exception behaviour up to this point), plus any SQL query which was being executed
(DocLite hides these from you during normal operation, of course, as it is a NoSQL solution,
but they are useful for filing bug reports!), and an array of any relevant parameters - these
may be things like a document ID, the document data, etc.

use Gebler\Doclite\Exception\DatabaseException;
...
try {
    $user->setUsername("dwgebler");
    $user->save();
} catch (DatabaseException $e) {
    var_dump($e->getMessage(), $e->getCode(), $e->getQuery(), $e->getParams());
}

A DatabaseException can occur on any Database, Collection or Document
method which interacts with the underlying database.

Error codes are represented by public constants in the DatabaseException class.
The full list of error codes are as follows:

Import and export data

DocLite can import data from and export data to JSON, YAML, XML and CSV files.
For this purpose, the Database object provides two methods, import() and
export().

⚠️ Import or export operations on very large collections may exhaust
memory. This feature will (probably) be improved and made more efficient for
working with large data sets in future.

It is recommended to use JSON for exports you intend to reload in
to a DocLite database. Support for other formats is experimental.

Importing Data

The data you want to import can be organized either in to files, where each file
represents a collection of multiple documents, or a directory where each
sub-directory represents a collection and contains a number of files each
representing a single document.

import(string $path, string $format, int $mode)

format can be any of json, yaml, xml, or csv. This should also
match the extension of the filename(s) containing your data.

When using the csv format, the first line of a CSV file is assumed to be a
header line containing field names.

mode can be either of the constants Database::MODE_IMPORT_COLLECTIONS or
Database::MODE_IMPORT_DOCUMENTS.

Collection names are inferred from the subdirectory or file names. For example,
/path/to/collections/users.json will import to the users collection, as will
a sub-directory /path/to/collections/users/ when importing a collection from
multiple files.

// Create a new, empty database
$db = new FileDatabase('/path/to/data.db');

// Import the contents of a directory where each file is a collection
$db->import('/path/to/collections', 'json', Database::IMPORT_COLLECTIONS);

// Import the contents of a directory where each sub directory is a collection 
// of files representing single documents.
$db->import('/path/to/collections', 'json', Database::IMPORT_DOCUMENTS);

When you import documents in to a collection, any documents which have a unique
ID matching an existing document in the database will overwrite that document.
Otherwise, new documents will be created for any documents with an unmatched or
missing ID.

💡 Each Collection import will be wrapped in a single transaction, so
imports are atomic per collection. You can also speed up bulk imports by
setting the advanced options (see below) to alter the database’s
synchronization and rollback journal modes to something a little more
permissive, if you understand the implications of doing so.

Exporting data

You can export the entire contents of one or more collections. Much like importing
data, you can choose whether DocLite should export this as one file per collection
containing multiple documents, or one directory per collection with one file per
document.

export(string $path, string $format, int $mode, array $collections = [])

format can be any of json, yaml, xml, or csv.

mode can be either of the constants Database::MODE_EXPORT_COLLECTIONS or
Database::MODE_EXPORT_DOCUMENTS.

collections can be a mix of strings of collection names and/or Collection
objects. If this is empty, all collections in the database will be exported.

// Export the entire database to one file per collection in the specified 
// output directory.
$db->export('/path/to/export', 'json', Database::EXPORT_COLLECTIONS);

// Export the entire database to a directory structure with one file per document.
$db->export('/path/to/export', 'json', Database::EXPORT_DOCUMENTS);

// Export only the "User" and "Person" collections.
// Assume Collection $persons = $db->get("Person");
$db->export(
    '/path/to/export', 
    'json',
    Database::EXPORT_COLLECTIONS,
    ['User', $persons]
);

⚠️ The XML standard imposes some restrictions on entity names. When
exporting to this format, DocLite will replace any invalid characters in
document fields with underscores. This means you may not be able to recreate
your document store exactly as it was should you subsequently import these
files in to a DocLite database.

Advanced options

DocLite Database objects have a few methods for more advanced options.

Get DocLite version

// Return the version of DocLite as a SemVer string, e.g. 1.0.0
$db->getVersion();

Optimize database

Call $db->optimize() to attempt database optimization. This function does
not return anything, though can throw a DatabaseException if something goes
wrong. Periodic optimization can reduce database file size and improve performance.

Set synchronization mode

The underlying SQLite sync mode can be set to one of the following constants in
the Database class.
See SQLite documentation
for details of the implications of changing this value; disabling sync can lead
to data loss in the event of a crash or power loss.

Call $db->setSyncMode(Database::MODE_CONSTANT) to set the mode.
For example to set Full Sync mode, call $db->setSyncMode(Database::MODE_SYNC_FULL).

This function returns true on success or false on failure.

Call $db->getSyncMode() to get the current mode which can be compared to one
of the constants. The return type is int.

Set rollback journal mode

The underlying SQLite management of the rollback journal can be set to one of
the following constants.
See SQLite documentation
for the implications of changing this value; disabling the rollback journal can
lead to unintended data state.

⚠️ Warning: If you disable the rollback journal, transactions, atomic commits
and rollbacks will no longer work. The behaviour of transaction methods on a
collection in this mode is undefined and may lead to unpredictable results or
data corruption. You should therefore not use the
transaction methods in MODE_JOURNAL_NONE.

Call $db->setJournalMode(Database::MODE_CONSTANT) to set the mode.
For example to set WAL mode, call $db->setJournalMode(Database::MODE_JOURNAL_WAL).

This function returns true on success or false on failure.

Call $db->getJournalMode() to get the current mode which can be compared to one
of the constants. The return type is string.

Collections

About Collections

Collections are at the heart of DocLite. A Collection represents a named group of documents
(for example, “Users”) and is analogous to a table in a structured database.

💡 Note: Collections are represented in the underlying SQLite database as tables.
They must therefore obey a few rules:

• Collection names cannot start with sqlite_
• Collection names cannot start with a number.
• Collection names may contain only alphanumeric characters and underscores.
• Collection names cannot be longer than 64 characters.

A Collection object is the means by which you create, find, update and delete documents.

Every document in a collection must have a unique ID. You can either supply this yourself,
or one will be created for you when you first instantiate a document. Auto generated IDs
take the form of a v1 UUID
which includes a timestamp of when the document was first created.

Obtain a collection

Collections are obtained from a FileDatabase or MemoryDatabase by calling the collection
method. If the collection does not exist, it will be automatically created.

$userCollection = $db->collection("Users");

Create a document

Once you have a collection, create a new document by calling the collection’s get method.

$newUser = $userCollection->get();

Save a document

By default, documents are returned as a DocLite Document object, which provides
a save() method. You can also save a document of any type by calling save()
on the collection with the document object as a parameter.

// works for DocLite Document objects
$newUser->save();

// works for both DocLite documents and documents mapped to custom types
$userCollection->save($newUser);

Retrieve a document

get can also be used to retrieve a document by its ID.

$existingUser = $userCollection->get($id);

Map a document to a custom class

By default, retrieving a document will return a DocLite Document object, which provides magic
methods and properties for you to access and manipulate the document data. It is however also
possible to create or retrieve a document as an object of any custom class, provided that class
has either public properties or getter/setter methods for the document fields you wish to hydrate.

// Get a user as an object of type CustomUser.
$user = $userCollection->get($id, CustomUser::class);

By default, DocLite will look for a property called id to populate with the document’s unique id.
If you want to use a different property on a custom class to store this id, for example because your class
does not have an id property, or you are using it for something else, you can specify a custom ID property
name as a third parameter to get.

$user = $userCollection->get($id, CustomUser::class, 'databaseId');

Alternatively, you can add a public property or getter/setter to your class called docliteId
and DocLite will automatically attempt to populate this instead in the absence of an id property.

While the Document class provides a built-in save() method as a convenience to update a Document
in storage, documents represented as your own custom classes must be saved through the collection object.

$userCollection->save($user);

If you are using a custom property on your class to hold the document’s unique ID, you
should supply the ID as an additional parameter.

$userCollection->save($user, $user->getDatabaseId());

Finally, when saving a document represented as a custom class, you can specify an optional
third parameter to list any properties on the object you do not want to be stored in the
document. It is only necessary to do this either for properties you wish to be excluded which
are public / have getter/setter methods, or public get methods which do not represent properties.

$userCollection->save($user, $user->getDatabaseId(), ['nonDatabaseField']);

Delete a document

Much like save(), there is both a convenience delete() method on DocLite
Document objects and a deleteDocument(object $document) method on the
collection itself.

// Works for DocLite Document objects.
$user->delete();

// works for both DocLite documents and documents mapped to custom types
$userCollection->deleteDocument($user);

Query a collection

The Collection object provides a range of methods to find documents by arbitrary criteria.

Find single document by values

Find a single document where all keys match the specified values by calling findOneBy.

$user = $userCollection->findOneBy([
    'role' => 'admin',
    'name' => 'Mr Administrator',
]);

findOneBy takes optional custom class name and custom class ID field parameters in the same
manner as get.

$user = $userCollection->findOneBy(['username' => 'admin'], CustomUser::class, 'databaseId');

If a document which matches the criteria cannot be found, null is returned.

Find all matching documents by values

The function findAllBy works the same way as findOneBy but will return a
generator which you can iterate over, or convert to array via PHP’s
iterator_to_array function.

foreach($userCollection->findAllBy(['active' => true]) as $user) {
   ...
}

Find all documents in collection

To retrieve all documents in a collection, use findAll(). Like the previous two functions,
findAll can take an optional custom classname and ID property as parameters.

foreach($userCollection->findAll() as $user) {
   ...
}

Advanced queries

DocLite includes a powerful query building mechanism to retrieve or delete all documents
in a collection matching arbitrary criteria.

To build a query, use any combination of the where(), and(), or(), limit(),
offset() and orderBy() functions on the collection object, followed by a
call to fetch(), delete() or count().

You can also run nested queries to group clauses together via union()
(for grouping clauses by OR) and intersect() (for grouping clauses by AND).

You can query a document to any depth by separating nested fields with a . dot character,
you can also add square brackets [] to the end of a field which is a list to query
all the values inside that list for any match.

The advanced queries APIs are better understood by example.

For the following code snippets, imagine each document of your user collection looks
like the following data example, expressed here as YAML:

username: adamjones
first_name: Adam
last_name: Jones
password: "$2y$10$LRS.0xUCJjWSmQuWMMRsuurZ0OGlU.NH7KYXsipzkfUa0YREEarj2"
address:
  street: 123 Fake Street
  area: Testville
  county: Testshire
  postcode: TE1 3ST
roles:
- USER
- EDITOR
telephone: "+441234567890"
registered: true
active: true
lastLogin: "2021-02-13T10:34:40+00:00"
email: [email protected]
api_access:
  "/v1/pages/":
  - POST
  - GET
  "/v1/contributors/":
  - GET

Here are some example queries you could run against a collection of these documents.

$users = $db->collection("Users");

$activeUsers = $users->where('active', '=', true)->fetch();

$gmailUsers = $users->where('email', 'ENDS', '@gmail.com')->fetch();

$registeredAndNotActiveUsers = $users->where('registered', '=', true)
                                     ->and('active', '=', false)
                                     ->fetch();

$usersInPostalArea = $users->where('address.postcode', 'STARTS', 'TE1')->fetch();

$usersWith123InPhone = $users->where('telephone', 'CONTAINS', '123')->fetch();

$usersWithNoNumbersInUsername = $users->where('username', 'MATCHES', '^[A-Za-z]*$')
                                       ->fetch();
                                       
$usersWithEditorRole = $users->where('roles[]', '=', 'EDITOR')->fetch();

$usersWithEditorOrAdminRole = $users->where('roles[]', '=', 'ADMIN')
                                    ->or('roles[]', '=', 'EDITOR')
                                    ->fetch();
                                    
$usersWithEditorAndAdminRole = $users->where('roles', '=', ['ADMIN', 'EDITOR']);                                    
                                    
$usersWhoHaveAtLeastOneRoleWhichIsNotAdmin = $users->where('roles[]', '!=', 'ADMIN')->fetch();

/* 
 * This next one is trickier. "roles" is a list of values in our document.
 * As we can see above, roles[] != ADMIN would return all users who
 * have at least one role in their list which is not ADMIN.
 * But this means if a user has roles ["USER","ADMIN"], they would
 * be matched.
 * So for users who do NOT have the ADMIN role at all, we can
 * quote the value "ADMIN" and ask for matches where the entire list of roles
 * (so no square brackets) does not contain this value.
*/
$usersDoNotHaveAdminRole = $users->where('roles', 'NOT CONTAINS', '"ADMIN"')->fetch();

$deleteAllUsersWithEditorRole = $users->where('roles[]', '=', 'EDITOR')->delete();

$first10UsersOrderedByFirstName = $users->orderBy('first_name', 'ASC')
                                        ->limit(10)
                                        ->fetch();                                                               

$next10UsersOrderedByFirstName = $users->orderBy('first_name', 'ASC')
                                       ->limit(10)
                                       ->offset(10)
                                       ->fetch();                                                               

// Use [] on any field which is a list to search within its sub-items
$usersWithPostAccessToPagesApi = $users->where(
    'api_access./v1/pages/[]', '=', 'POST')->fetch();                                     

$allUsersWithPostAccessToAnyApi = $users->where('api_access[]', '=', 'POST')
                                        ->fetch();

$start = new DateTimeImmutable('2021-03-01 00:00:00');
$end = new DateTime('2021-06-30');
$usersWhoSignedUpBetweenMarchAndJune = $users->where('date', 'BETWEEN', $start, $end)->fetchArray();                                     

/**
 * Nested queries are also possible.
 * To get all users where 
 * (active=true and address.postcode matches '^[A-Za-z0-9 ]*$')
 * OR
 * (roles[] list contains "EDITOR" and lastLogin > 2021-01-30)
 */
$nestedUsers = $users->where('active', '=', true)
                     ->and('address.postcode', 'MATCHES', '^[A-Za-z0-9 ]*$')
                     ->union()
                     ->where('roles[]', '=', 'EDITOR')
                     ->and('lastLogin', '>', '2021-01-30')
                     ->fetch();

💡 Like findAllBy(), the fetch() method returns a generator, not an
array. If you would like all results at once, replace fetch() with
fetchArray().

💡 The fetch() method on advanced queries can take a custom class name
and custom ID field as optional parameters, just like the findOneBy,
findAllBy and findAll() methods.

💡 Speed up complex queries by enabling DocLite’s caching feature.

Query operators

Advanced queries support the following operators:

Join Collections

It is possible to join a collection to one or more other collections when running a query,
to include matching results from these collections in the documents returned. This works much
like a foreign key in a relational database.

For example, if you have a users collection and a comments collection, where some documents in
comments contain a field user_id. You can query users and join on comments, such that any documents
matching in comments for the same user ID will be included in the users document, under a field called comments.

/**
 * Imagine a user document like:
 * {"__id":"1", "name":"John Smith"}
 * 
 * and a corresponding comments document like:
 * {"__id":"5", "user_id": "1", "comment":"Hello world!"} 
 * 
 * You can query the users collection with a join to retrieve an aggregated document like this:
 * {"__id":"1","name":"John Smith","comments":[{"__id":"5","comment":"Hello world!"}]}
 */
$users = $db->collection('Users');
$comments = $db->collection("Comments");
$users->where('__id', '=', '1')->join($comments, 'user_id', '__id')->fetchArray();

The Collection::join method takes the collection to join as the first parameter, the name of the
document field in that collection to use as a foreign key as the second parameter, and
the corresponding field in documents in the joining collection (e.g. Users) to match against.

The above example therefore is looking for documents in Comments where the field user_id matches the
field __id in Users.

As join is part of the standard query building interface on a Collection, you can combine with other
query operators such as where, and etc. or other joins.

Caching results

DocLite can cache the results of queries to speed up retrieval of complex result sets.
For very simple queries, however, this may provide no benefit or even incur a small
performance penalty, so you should only turn it on if you need to.

To turn on caching for a collection, call the collection object’s
enableCache() method.

Likewise, you can disable caching by calling disableCache().

Cache results are valid for the cache lifetime, which defaults to 60 seconds. You can
change the cache validity period by calling setCacheLifetime($seconds). A cache
lifetime of zero means cached results will never expire.

You can manually flush the cache by calling clearCache().

$userCollection->enableCache();

// Set the cache validity period to 1 hour.
$userCollection->setCacheLifetime(3600);

$userCollection->disableCache();

$userCollection->clearCache();

Finally, the Database object can be set to automatically prune expired
entries whenever the cache is queried. This behaviour is disabled by default;
to enable auto-pruning, call enableCacheAutoPrune() on the database object.

$db->enableCacheAutoPrune();

💡 For complex queries, the cache is very fast. If you are running a large
number of complex queries on a large data set and these queries are likely to
be repeated without the data changing in storage for the lifetime of the cache,
it is a good idea to make use of DocLite’s caching.

Index a collection

It is possible to build indexes on any document fields inside a collection to
speed up queries against that field.

When you create a collection, an index is automatically added for the internal
ID field. To add a custom index, call the addIndex method with the name of a
document field.

$userCollection->addIndex('email');

To add a single index on multiple fields (as per a multi-column index),
simply call addIndex with the additional field names as separate
parameters.

$userCollection->addIndex('first_name', 'last_name');

Unique Indexes

You can also add a unique index which acts as a constraint on the field, ensuring
that no two documents in the collection can have the same value for that field or field combination.

$userCollection->addUniqueIndex('email');

If you attempt to add a document to a collection with a unique index and a value
that already exists in the collection, a DatabaseException will be thrown with the
code DatabaseException::ERR_UNIQUE_CONSTRAINT.

💡 Note: indexes are an advanced feature which work the same way they do
in any other SQLite database, the only difference being they are created
on document fields rather than a table column. Poorly chosen indexes may
provide no benefit or even slow down queries.

Delete a collection

To delete all documents in a collection entirely, call the deleteAll() method.

$userCollection->deleteAll();

Collection transactions

It is possible to wrap a sequence of database operations inside a transaction.
To do this, use the Collection’s beginTransaction(), commit() and
rollback() methods.

$collection = $db->collection("Users");
$collection->beginTransaction();

// ...do some stuff, insert a bunch of records or whatever...

// commit the results and end the transaction
$collection->commit();

// or rollback the changes and end the transaction
$collection->rollback();

Full text search

DocLite is able to build powerful full text indexes against collections to allow you to
search and produce a list of documents, ordered by relevance, where specified fields match some text or phrase.

Full text search capability requires your PHP’s libsqlite to be built with the FTS5 extension.
Just like the JSON1 extension, this is usually bundled in to the standard distribution so you probably already have it.

To search a collection, ensure you have initialized your Database with the full text parameter set to true to enable
this feature, then simply call the search() method on any collection, with the search phrase followed by an array of
the names of any document fields you wish to search against.

$path = '/path/to/db';
$readOnly = false;
$ftsEnabled = true;
$timeout = 1;
$db = new FileDatabase($path, $readOnly, $ftsEnabled, $timeout);
$blogPosts = $db->collection("posts");
$results = $blogPosts->search('apache', ['title', 'summary', 'content']);

Results are automatically ordered by relevance.

💡 DocLite will intelligently manage your full text indexes to keep your database optimized.
When you call search(), if there is no index for the set of fields you are searching on, it will be created automatically on the first search.
If you later call search() on a superset of fields for an existing index, the original index
will be destroyed and a new, larger index encompassing all searched fields created. This is so DocLite can use the smallest possible index for all the fields you wish to search against.

On small collections, this process is so fast you may not see an impact. If, however, you have a very
large collection, the recommendation is to create your full text indexes by calling search() once from a
separate script, so that when your application first runs and calls search(), the relevant indexes already exist.

Because search() is part of the standard query fetching interface on a collection (same as fetch() and count()), it can be preceded by
normal query filters using where(), and() etc. Similar to fetch(), the search() method returns
a generator. You can convert the results to an array by using PHP’s iterator_to_array() function.

Documents

About Documents

Documents are a variadic structured store of data in the form of key-value pairs,
stored in the database as JSON; that is, each document inside a collection can have its own freeform
structure. It does not matter whether this matches the structure of any other documents
in the same collection, that is up to how your application decides to use DocLite.

A document will by default be represented by the DocLite Document class, however it is also
possible to create or retrieve documents from a collection and map them on to your own classes.
See the Collection documentation for more details on this.

Getting and setting document data

The Document class provides magic get and set methods and property accessors for arbitrary document
keys. That is, once you have a Document from a Collection, you can set or read any properties
you like by either method:

$users = $db->collection("Users");
// Create a new Document with an auto generated UUID.
$user = $db->get();
// Create a new property called username via a magic setter.
$user->setUsername('dwgebler');
// Create a new property called password via a magic property.
$user->password = password_hash("admin", \PASSWORD_DEFAULT);
// Read the username property via a magic property.
echo $user->username;
// Read the password property via a magic getter.
echo $user->getPassword();
// Properties can contain scalar values, arrays, or even other Documents and
// custom objects.
$user->setRoles(['user', 'admin']);

There is one small semantic difference between the magic method and property access
techniques; when using magic methods, the property names are converted from camelCase
to snake_case, whereas direct property access is literal e.g.

// setter uses camel case
$user->setFirstName('Dave');

// but the corresponding property created will be lower cased and snake_cased
echo $user->first_name;

// if you want a key in a document to be case sensitive, set it as a property only
$user->FirstName = 'Dave';

// you should now use the property access to retrieve its value later on
echo $user->FirstName;

// This will not work and will raise a ValueError on getFooBar(),
// because the method call will look for a property called foo_bar
$user->FooBar = 'baz';
$user->getFooBar();

The Document class also provides two further methods, getValue and setValue,
to query a document by nested keys using a path in dot . notation. These
methods can also be used to get or set fields with names which can’t be
expressed through magic set methods or properties.

getValue() raises a ValueError if the specified path cannot be found.

setValue() will automatically create any parent properties on a nested path.

// This is the same as:
// $address = $user->getAddress();
// $postcode = $address['postcode'];
$user->getValue('address.postcode');

// Assume "roles" is a list, this will return an array
$user->getValue('roles');

// Retrieve the first role
$user->getValue('roles.0');

// Assume api_access is a dictionary of keys mapped to lists.
// This will return the list of data under the /v1/users/ key
// as an array.
$access = $user->getValue('api_access./v1/users/');
if (!in_array('POST', $access)) { ... }

// If address does not exist, it will be created with postcode as a key.
$user->setValue('address.postcode', 'TE1 3ST');

// Or set a value with special characters in the name:
$user->setValue('api_access./v1/users/', ['GET', 'POST']);

💡 The values of document fields are arbitrary. Scalar values, arrays and
even objects of custom classes can all be stored in a document.

Mapping document fields to objects

If you’ve retrieved a document as a default Document object, it is still
possible to map document fields which represent custom objects to custom classes.
To do this, use the Document’s map() method and pass it a field name (which
can use the nested dot . notation as described above), along with either a
class name or existing object instance if you wish to populate an existing object.

Consider you have the following custom class in your application:

class Person
{
    private $id;

    private $firstName;

    private $lastName;

    private $address = [];

    private $postcode;

    private $dateOfBirth;

    private $identityVerified;

    public function getId(): ?int
    {
        return $this->id;
    }
    
    public function setId(string $id): self
    {
        $this->id = $id;
        return $this;
    }    

    public function getFirstName(): ?string
    {
        return $this->firstName;
    }

    public function setFirstName(string $firstName): self
    {
        $this->firstName = $firstName;
        return $this;
    }

    public function getLastName(): ?string
    {
        return $this->lastName;
    }

    public function setLastName(string $lastName): self
    {
        $this->lastName = $lastName;
        return $this;
    }

    public function getAddress(): ?array
    {
        return $this->address;
    }

    public function setAddress(array $address): self
    {
        $this->address = $address;
        return $this;
    }

    public function getPostcode(): ?string
    {
        return $this->postcode;
    }

    public function setPostcode(string $postcode): self
    {
        $this->postcode = $postcode;
        return $this;
    }

    public function getDateOfBirth(): ?\DateTimeImmutable
    {
        return $this->dateOfBirth;
    }

    public function setDateOfBirth(\DateTimeImmutable $dateOfBirth): self
    {
        $this->dateOfBirth = $dateOfBirth;
        return $this;
    }

    public function getIdentityVerified(): ?bool
    {
        return $this->identityVerified;
    }

    public function setIdentityVerified(bool $identityVerified): self
    {
        $this->identityVerified = $identityVerified;
        return $this;
    }
}

And a User document with the following structure:

__id: b83e319a-7887-11eb-8deb-b9e03d2e720d
username: daniel_johnson1
active: false
roles:
- CONTRIBUTOR
- AUTHOR
telephone: "+441254220959364"
password: "$2y$10$y8P2Cjph1F.iIc.s2j9aM.GW9qy8aOMeEfzDulQox465mgBJF.pPG"
person:
  firstName: Daniel
  lastName: Johnson
  address:
    house: '123'
    street: Test Road
    city: Testville
    Country: Testshire
  postcode: "TE1 3ST"
  dateOfBirth: '1980-03-23T00:00:00+00:00'
  identityVerified: true

When you initially retrieve the Document, the person key will contain an
array. But you can map this to your Person class as follows:

$user = $collection->get("b83e319a-7887-11eb-8deb-b9e03d2e720d");
$user->map('person', Person::class);

// $user->getPerson() now returns a Person object.

// Or you can map to an existing Person object.
$person = new Person();
$user->map('person', $person);

Document Unique Id

Every document in the same collection must have a unique ID.

By default, when you create a new document, an ID is generated for you as a
v1 UUID.

You can get or set a Document ID with the getId() and setId(string $id)
methods.

💡 Note: Changing a Document ID essentially treats it as a different document, i.e.
providing a new unique ID will result in a new document being inserted in to your database
when you save it. Likewise changing a document’s ID to the ID of another document in the
collection will cause that document to be overwritten.

If the ID was auto generated, you can obtain a DateTimeImmutable representing the
document’s time of creation by calling its getTime() method:

$users = $db->collection("Users");
// Create a new Document with an auto generated UUID.
$user = $users->get();
// $date is a \DateTimeImmutable
$date = $user->getTime();
echo $date->format('d m Y H:i');

If you don’t want to use an auto-generated ID for a new document, simply pass in your
own ID to the collection’s get() method. As long as the ID does not match any document
in the collection’s database storage, a new document will be created. Document IDs are
strings.

$users = $db->collection("Users");
// Create a new Document with a custom ID.
// If this ID already exists in the Users collection, that document will be returned.
$user = $users->get("user_3815");

Saving a document

Documents represented as a DocLite Document object provide a convenience method to
save the document to its collection. To save a Document in storage, call save().

$users = $db->collection("Users");
$user = $users->get();
$user->setUsername("admin");
$user->save();

If a document has been mapped on to a custom class, you will need to save it through
its collection instead.

$users = $db->collection("Users");
// Create a new document with an automatically generated UUID and
// retrieved as an object of type CustomUser.
$user = $users->get(null, CustomUser::class);
$user->setUsername("admin");
$users->save($user);

Deleting a document

Documents represented as a DocLite Document object provide a convenience method to
delete the document from its collection. To delete a Document in storage,
call delete().

$users = $db->collection("Users");
$user = $users->get("12345");
$user->delete();

If a document has been mapped on to a custom class, you will need to delete it
through its collection instead.

$users = $db->collection("Users");
// Create a new document with an automatically generated UUID and
// retrieved as an object of type CustomUser.
$user = $users->get("12345", CustomUser::class);
$users->deleteDocument($user);

Document validation

It is possible to add JSON Schema validation to a Document via the addJsonSchema()
method. This takes a single string parameter of a valid JSON schema. If the schema
cannot be validated, a DatabaseException will be thrown.

$user->addJsonSchema(file_get_contents('schema.json'));

Once you have loaded a schema, every time you set a document property or try to save
the document, the document data will be validated against your schema. If the data fails
to validate, a DatabaseException will be thrown.

You can also manually validate at any time by calling validateJsonSchema().

$user->addJsonSchema(file_get_contents('schema.json'));
try {
    $user->validateJsonSchema();
    // This will automatically call validateJsonSchema() anyway.
    $user->save();
    // As will this.
    $user->setUsername("foobar");
} catch (DatabaseException $e) {
    $params = $e->getParams();
    $error = $params['error'];
    echo "Document failed to validate against JSON Schema because:\n".$error;
}

Finally, you can unload a JSON Schema and remove the validaton by calling
removeJsonSchema().

$user->removeJsonSchema();

Other info

Symfony integration

Although there is not a specific integration with the Symfony framework, it’s
trivial to inject DocLite as a service in to any Symfony application. Simply
install DocLite via Composer as an app dependency, then modify your
services.yaml as per the following example.

    app.filedatabase:
        class: Gebler\Doclite\FileDatabase
        arguments:
            $path: "../var/data/app.db"
            $readOnly: false
    app.memorydatabase:
        class: Gebler\Doclite\MemoryDatabase

    Gebler\Doclite\DatabaseInterface: '@app.filedatabase'
    Gebler\Doclite\DatabaseInterface $memoryDb: '@app.memorydatabase'

You can now typehint a DatabaseInterface like any other service, using the
alias $memoryDb as the parameter name if you’d like a MemoryDatabase.

Licensing

DocLite is available under the MIT license as open source software.

If you use DocLite and find it useful, I am very grateful for any support
towards its future development.

Bugs, issues

Please raise an issue on the project GitHub if you encounter any problems. I am
always interested in improving the software.

Contact the author

You can email me on [email protected]