MongoKitten

Native MongoDB driver for Swift, written in Swift

orlandos-nl
721
103
Swift

A fast, pure swift MongoDB driver based on Swift NIO built for Server Side Swift. It features a great API and a battle-tested core. Supporting both MongoDB in server and embedded environments.

MongoKitten is a fully asynchronous driver, which means that it doesn’t block any threads. This also means that it can be used in any asynchronous environment, such as Vapor or Hummingbird.

Docs & Community

Join our Discord for any questions and friendly banter.

If you need hands-on support on your projects, our team is available at [email protected].

Look into Sample Code using MongoKitten & Vapor

Projects

A couple of MongoKitten based projects have arisen, check them out!

🕶 Installation

Set up MongoDB server

If you haven’t already, you should set up a MongoDB server to get started with MongoKitten. MongoKitten supports MongoDB 3.6 and above.

For development, this can be on your local machine.

Install MongoDB for Ubuntu, macOS or any other supported Linux Distro.

Alternatively, make use of a DAAS (Database-as-a-service) like MongoDB Atlas.

Add MongoKitten to your Swift project 🚀

MongoKitten uses the Swift Package Manager. Add MongoKitten to your dependencies in your Package.swift file:

.package(url: "https://github.com/orlandos-nl/MongoKitten.git", from: "7.2.0")

Also, don’t forget to add the product "MongoKitten" as a dependency for your target.

.product(name: "MongoKitten", package: "MongoKitten"),

Add Meow (Optional)

Meow is an ORM that resides in this same package.

.product(name: "Meow", package: "MongoKitten"),

FAQ

I can't connect to MongoDB, authentication fails!
  1. Make sure you’ve specified authSource=admin, unless you know what your authSource is. MongoDB’s default value is really confusing.
  2. If you’ve specified an authMechanism, try removing it. MongoKitten can detect the correct one automatically.

🚲 Basic usage

First, connect to a database:

import MongoKitten

let db = try await MongoDatabase.connect(to: "mongodb://localhost/my_database")

Vapor users should register the database as a service:

extension Request {
    public var mongo: MongoDatabase {
        return application.mongo.adoptingLogMetadata([
            "request-id": .string(id)
        ])
    }
}

private struct MongoDBStorageKey: StorageKey {
    typealias Value = MongoDatabase
}

extension Application {
    public var mongo: MongoDatabase {
        get {
            storage[MongoDBStorageKey.self]!
        }
        set {
            storage[MongoDBStorageKey.self] = newValue
        }
    }
    
    public func initializeMongoDB(connectionString: String) throws {
        self.mongo = try MongoDatabase.lazyConnect(to: connectionString)
    }
}

The same goes for Hummingbird users:

extension HBApplication {
    public var mongo: MongoDatabase {
        get { extensions.get(\.mongo) }
        set { extensions.set(\.mongo, value: newValue) }
    }
}

extension HBRequest {
    public var mongo: MongoDatabase {
        application.mongo.adoptingLogMetadata([
            "hb_id": .string(id)
        ])
    }
}

Make sure to instantiate the database driver before starting your application.

For Vapor:

try app.initializeMongoDB(connectionString: "mongodb://localhost/my-app")

For hummingbird:

app.mongo = try MongoDatabase.lazyConnect(to: "mongodb://localhost/my-app")

Connect vs. LazyConnect

In MongoKitten, you’ll find two main variations of connecting to MongoDB.

  • connect calls are async throws, and will immediately attempt to establish a connection. These functions throw an error if unsuccessful.
  • lazyConnect calls are throws, and will defer establishing a connection until it’s necessary. Errors are only thrown if the provided credentials are unusable.

Connect’s advantage is that a booted server is known to have a connection. Any issues with MongoBD will arise immediately, and the error is easily inspectable.

LazyConnect is helpful during development, because connecting to MongoDB can be a time-consuming process in certain setups. LazyConnect allows you to start working with your system almost immediately, without waiting for MongoKitten. Another advantage is that cluster outages or offly timed topology changes do not influence app boot. Therefore, MongoKitten can simply attempt to recover in the background. However, should something go wrong it can be hard to debug this.

CRUD (Create, Read, Update, Delete)

Before doing operations, you need access to a collection where you store your models. This is MongoDB’s equivalent to a table.

// The collection "users" in your database
let users = db["users"]

Create (insert)

// Create a document to insert
let myUser: Document = ["username": "kitty", "password": "meow"]

// Insert the user into the collection
// The _id is automatically generated if it's not present
try await users.insert(myUser)

Read (find) and the query builder

To perform the following query in MongoDB:

{
  "username": "kitty"
}

Use the following MongoKitten code:

if let kitty = try await users.findOne("username" == "kitty") {
  // We've found kitty!
}

To perform the following query in MongoDB:

{
  "$or": [
    { "age": { "$lte": 16 } },
    { "age": { "$exists": false } }
  ]
}

Use the following MongoKitten code:

for try await user in users.find("age" <= 16 || "age" == nil) {
  // Asynchronously iterates over each user in the cursor
}

You can also type out the queries yourself, without using the query builder, like this:

// This is the same as the previous example
users.findOne(["username": "kitty"])

Cursors

Find operations return a Cursor. A cursor is a pointer to the result set of a query. You can obtain the results from a cursor by iterating over the results, or by fetching one or all of the results.

Cursors will close automatically if the enclosing Task is cancelled.

Fetching results

You can fetch all results as an array:

// Fetch all results and collect them in an array
let users = try await users.find().drain()

Note that this is potentially dangerous with very large result sets. Only use drain() when you are sure that the entire result set of your query fits comfortably in memory.

Cursors are generic

Find operations return a FindQueryBuilder. You can lazily transform this (and other) cursors into a different result type by using map, which works similar to map on arrays or documents. A simple commonly used helper based on map is .decode(..) which decodes each result Document into a Decodable entity of your choosing.

let users: [User] = try await users.find().decode(User.self).drain()

Update & Delete

You can do updateOne/many and deleteOne/many the same way you’d see in the MongoDB docs.

try await users.updateMany(where: "username" == "kitty", setting: ["age": 3], unsetting: nil)

The result is implicitly discarded, but you can still get and use it.

try await users.deleteOne(where: "username" == "kitty")

let reply = try await users.deleteAll(where: "furType" == "fluffy")
print("Deleted \(reply.deletes) kitties 😿")

Indexes

You can create indexes on a collection using the buildIndexes method.

try await users.buildIndexes {
  // Unique indexes ensure that no two documents have the same value for a field
  // See https://docs.mongodb.com/manual/core/index-unique/s
  UniqueIndex(
    named: "unique-username", 
    field: "username"  
  )

  // Text indexes allow you to search for documents using text
  // See https://docs.mongodb.com/manual/text-search/
  TextScoreIndex(
    named: "search-description", 
    field: "description"
  )

  // TTL Indexes expire documents after a certain amount of time
  // See https://docs.mongodb.com/manual/core/index-ttl/
  TTLIndex(
    named: "expire-createdAt", 
    field: "createdAt", 
    expireAfterSeconds: 60 * 60 * 24 * 7 // 1 week
  )
}

Aggregation

MongoDB supports aggregation pipelines. You can use them like this:

let pipeline = try await users.buildAggregate {
  // Match all users that are 18 or older
  Match(where: "age" >= 18)

  // Sort by age, ascending
  Sort(by: "age", direction: .ascending)

  // Limit the results to 3
  Limit(3)
}

// Pipeline is a cursor, so you can iterate over it
// This will iterate over the first 3 users that are 18 or older in ascending age order
for try await user in pipeline {
  // Do something with the user
}

Transactions

try await db.transaction { transaction in
  // Do something with the transaction
}

GridFS

MongoKitten supports GridFS. You can use it like this:

let database: MongoDatabase = ...
let gridFS = GridFSBucket(in: database)

You can then use the GridFSBucket to upload and download files.

let blob: ByteBuffer = ...
let file = try await gridFS.upload(
  blob,
  filename: "invoice.pdf",
  metadata: [
    "invoiceNumber": 1234,
    "invoiceDate": Date(),
    "invoiceAmount": 123.45
  ]
)

Optionally, you can define a custom chunk size. The default is 255kb.

For chunked file uploads, you can use the GridFSFileWriter:

let writer = GridFSFileWriter(toBucket: gridFS)

do {
  // Stream the file from HTTP
  for try await chunk in request.body {
    // Assuming `chunk is ByteBuffer`
    // Write each HTTP chunk to GridFS
    try await writer.write(data: chunk)
  }

  // Finalize the file, making it available for reading
  let file = try await writer.finalize(filename: "invoice.pdf", metadata: ["invoiceNumber": 1234])
} catch {
  // Clean up written chunks, as the file upload failed
  try await writer.cancel()

  // rethrow original error
  throw error
}

You can read the file back using the GridFSReader or by iterating over the GridFSFile as an AsyncSequence:

// Find your file in GridFS
guard let file = try await gridFS.findFile("metadata.invoiceNumber" == 1234) else {
  // File does not exist
  throw Abort(.notFound)
}

// Get all bytes in one contiguous buffer
let bytes = try await file.reader.readByteBuffer()

// Stream the file
for try await chunk in file {
  // `chunk is ByteBuffer`, now do something with the chunk!
}

📦 About BSON & Documents

MongoDB is a document database that uses BSON under the hood to store JSON-like data. MongoKitten implements the BSON specification in its companion project, OpenKitten/BSON. You can find out more about our BSON implementation in the separate BSON repository, but here are the basics:

Literals

You normally create BSON Documents like this:

let documentA: Document = ["_id": ObjectId(), "username": "kitty", "password": "meow"]
let documentB: Document = ["kitty", 4]

From the example above, we can learn a few things:

  • A BSON document can represent an array or a dictionary
  • You can initialize a document like you initialize normal dictionaries and arrays, using literals
  • The values in a Document (either the array elements or the values of a dictionary pair) can be of any BSON primitive type
  • BSON primitives include core Swift types such as Int, String, Double and Bool, as well as Date from Foundation
  • BSON also features some unique types, such as ObjectId

Just Another Collection

Like normal arrays and dictionaries, Document conforms to the Collection protocol. Because of this, you can often directly work with your Document, using the APIs you already know from Array and Dictionary. For example, you can iterate over a document using a for loop:

for (key, value) in documentA {
	// ...
}

for value in documentB.values {
	// ...
}

Document also provides subscripts to access individual elements. The subscripts return values of the type Primitive?, so you probably need to cast them using as? before using them.

let username = documentA["username"] as? String

Think twice before converting between Document and Dictionary

Our Document type is implemented in an optimized, efficient way and provides many useful features to read and manipulate data, including features not present on the Swift Dictionary type. On top of that, Document also implements most APIs present on Dictionary, so there is very little learning curve.

💾 Codable

MongoKitten supports the Encodable and Decodable (Codable) protocols by providing the BSONEncoder and BSONDecoder types. Working with our encoders and decoders is very similar to working with the Foundation JSONEncoder and JSONDecoder classes, with the difference being that BSONEncoder produces instances of Document and BSONDecoder accepts instances of Document, instead of Data.

For example, say we want to code the following struct:

struct User: Codable {
	var profile: Profile?
	var username: String
	var password: String
	var age: Int?
	
	struct Profile: Codable {
		var profilePicture: Data?
		var firstName: String
		var lastName: String
	}
}

We can encode and decode instances like this:

let user: User = ...

let encoder = BSONEncoder()
let encoded: Document = try encoder.encode(user)

let decoder = BSONDecoder()
let decoded: User = try decoder.decode(User.self, from: encoded)

A few notes:

  • BSONEncoder and BSONDecoder work very similar to other encoders and decoders
  • Nested types can also be encoded and are encouraged
    • Nested structs and classes are most often encoded as embedded documents
  • You can customize the representations using encoding/decoding strategies

Meow

Meow works as a lightweight but powerful ORM layer around MongoKitten.

Setting up with Vapor

extension Application {
    public var meow: MeowDatabase {
        MeowDatabase(mongo)
    }
}

extension Request {
    public var meow: MeowDatabase {
        MeowDatabase(mongo)
    }
}

Setting up with Hummingbird

extension HBApplication {
    public var meow: MeowDatabase {
        MeowDatabase(mongo)
    }
}

extension HBRequest {
    public var meow: MeowDatabase {
        MeowDatabase(mongo)
    }
}

Models

There are two main types of models in Meow, these docs will focus on the most common one.

When creating a model, your type must implement the Model protocol.

import Meow

struct User: Model {
  ..
}

Each Model has an _id field, as required by MongoDB. The type must be Codable and Hashable, the rest is up to you. You can therefore also make _id a compound key such as a struct. It must still be unique and hashable, but the resulting Document is acceptable for MongoDB.

Each field must be marked with the @Field property wrapper:

import Meow

struct User: Model {
  @Field var _id: ObjectId
  @Field var email: String
}

You can also mark use nested types, as you’d expect of MongoDB. Each field in these nested types must also be marked with the @Field property wrapper to make it queryable.

import Meow

struct UserProfile: Model {
  @Field var firstName: String?
  @Field var lastName: String?
  @Field var age: Int
}

struct User: Model {
  @Field var _id: ObjectId
  @Field var email: String
  @Field var profile: UserProfile
}

Queries

Using the above model, we can query it from a MeowCollection. Get your instance from the MeowDatabase using a typed subscript!

let users = meow[User.self]

Next, run a find or count query, but using a type-checked syntax instead! Each portion of the path needs to be prefixed with $ to access the Field property wrapper.

let adultCount = try await users.count(matching: { user in
  user.$profile.$age >= 18
})

As meow just recycles common MongoKitten types, you can use a find query cursor as you’d do in MongoKitten.

let kids = try await users.find(matching: { user in
  user.$profile.$age < 18
})

for try await kid in kids {
  // TODO: Send verification email to parents
}

References

Meow has a helper type called Reference, you can use this in your model instead of copying the identifier over. This will give you some extra helpers when trying to resolve a models.

Reference is also Codable and inherit’s the identifier’s LosslessStringConvertible. So it can be used in Vapor’s JWT Tokens as a subject, or in a Vapor’s Route Parameters.

// GET /users/:id using Vapor
app.get("users", ":id") { req async throws -> User in
  let id: Reference<User> = req.parameters.require("id")
  return try await id.resolve(in: req.meow)
}

☠️ License

MongoKitten is licensed under the MIT license.

Backers