Introduction

Permakeep is a Rust implementation of Perkeep.

Why another Perkeep implementation?

  1. Having a second implementation is a good thing. It allows for validating the formats and protocols.
  2. Permakeep will be an attempt at simplifying some of the UX around Perkeep.

For now, everything that is implemented in Permakeep is compatible with Perkeep, especially the data models.

Now

  • Directory Blobserver
  • S3 Blobserver

Next

  • Initial CLI
  • Indexer
  • JSON Signing

Later / Ideas

  • Web App

Command Line

TODO

Terminology

blob

Immutable sequence of 0 or more bytes.

blobref

Reference to a blob in the <digalg>-<digest> format where:

  • <digalg> is a cryptographic hash function
  • <digest> is that function's digest, in hex

Examples of valid blobrefs:

sha1-f1d2d2f924e986ac86fdf7b36c94bcdf32beec15
md5-d3b07384d113edec49eaa6238ad5ff00
sha256-b5bb9d8014a0f9b1d61e21e796d78dccdf1352f23cd32812f4850b878ae4944c

JSON Signing

Permakeep claim objects are signed. This allows, for example, to share files publicly.

Design Notes

Permakeep implements Perkeep's JSON signing format, which is documented here. In this documentation page, we try to document the same format in a simpler manner.

The format was designed with the following goals in mind:

1. The final document must be valid JSON

  • We don't add any header or footer that wouldn't be compatible with standard JSON parsers.

2. Stay human-readable

  • Given that JSON is generally human-readable, this comes for free if we respect the first goal.
  • We should allow whitespaces and we should not require canonicalized JSON files.

3. Compatible with any standard JSON parsing/serializing library

  • It should be possible to implement our JSON signing format with any standard JSON parsing/serializing library.
  • This goal helps guarantee that the format will remain simple to implement in the future.

Signing

1. Initial JSON object

  • Must be an object (not an array).
  • Must contain the two following key/value pairs:
    • camliVersion: 1
    • camliSigner: <blobref> (Where <blobref> points to a blob that contains an ASCII-armored public key).

Example object "1-object":

{
    "camliVersion": 1,
    "camliSigner": "sha1-8616ebc5143efe038528c2ab8fa6582353805a7a",
    "foo": "bar"
}

2. Serialize the object

  • Remove the camliVersion key/value pair, we will add it back later.
  • Serialize the object with any standard JSON serialization library.
  • Whitespaces don't need to be exactly the same at this point.

Example JSON document "2-serialized":

{
    "camliSigner": "sha1-8616ebc5143efe038528c2ab8fa6582353805a7a",
    "foo": "bar"
}

3. Prepare the document for signing

  • Remove all leading and trailing whitespaces.
  • Remove the leading character {.
  • Remove the trailing character }.
  • Prepend {"camliVersion": 1 to the document.
    • This allows us to ensure that all signed claims begin with the string {"camliVersion":.

Example JSON document "3-bytes-payload":

{"camliVersion": 1,
  "camliSigner": "sha1-8616ebc5143efe038528c2ab8fa6582353805a7a",
  "foo": "bar"

4. Create a signature

Create an ASCII-armored detachehd signature of 3-bytes-payload.

For example, using gpg:

gpg \
    --detach-sign \
    --local-user=54F8A914 \
    --armor \
    -o 3-bytes-payload 4-signature.asc

Example signature:

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAABAgAGBQJO3/DNAAoJECkxpnwm9avaf6EH/3HVJC+6ybOJDTJIInQBum9Y
FzC1I8b6xNLN0yFdDtypZUotvW9pvU2pVpbfNSmcW/OL02eR2kgL55dHxbUjbN9C
vXlvSb2QAy8IQMdA3721pMR41rNNn08w5bbAWgW/suiyN5z0pIKn3vPEHbguGeNQ
BStgOSq1WkgCozNBxPA7V5mcUx2rUOsWHYSmEY8foPdeDYcrw2pvxPN8kXk6zBrZ
ilrtaY+Yx5zPLkq8trhHPgCdf4chL+Y2kmxXMKYjU+bkmJaNycUURdncZakTEv9Y
fbBp04kbHIaN6DttEoXuU96nTyuCFhIftmV+GPbvGpl3e2yhmae5hUUt1g0o8FE=
=aSCK
-----END PGP SIGNATURE-----

5. Attach the signature

  • Exract the base64 part of the ASCII detached signature 4-signature.asc, and call that S.
  • Append the following string to 3-before-signing: ,"camliSig:"<S>"}\n"

Example final document "5-signed":

{"camliVersion": 1,
  "camliSigner": "sha1-8616ebc5143efe038528c2ab8fa6582353805a7a",
  "foo": "bar"
,"camliSig":"iQEcBAABAgAGBQJO3/DNAAoJECkxpnwm9avaf6EH/3HVJC+6ybOJDTJIInQBum9YFzC1I8b6xNLN0yFdDtypZUotvW9pvU2pVpbfNSmcW/OL02eR2kgL55dHxbUjbN9CvXlvSb2QAy8IQMdA3721pMR41rNNn08w5bbAWgW/suiyN5z0pIKn3vPEHbguGeNQBStgOSq1WkgCozNBxPA7V5mcUx2rUOsWHYSmEY8foPdeDYcrw2pvxPN8kXk6zBrZilrtaY+Yx5zPLkq8trhHPgCdf4chL+Y2kmxXMKYjU+bkmJaNycUURdncZakTEv9YfbBp04kbHIaN6DttEoXuU96nTyuCFhIftmV+GPbvGpl3e2yhmae5hUUt1g0o8FE==aSCK"}

Verifying

1. Extract the camliSigner and fetch the public key

Parse the JSON document, and extract the value of the camliSigner key.

Use this blobref to fetch the public key of the signer from a store of trusted keys.

2. Extract the camliSig

Extract the value of camliSig and store it for verification.

3. Extract the bytes payload

Extract the bytes payload by finding the index of the last occurence of the following 13-bytes substring:

,"camliSig":"

Everything before that index is the bytes payload.

4. Verify the signature

Verify that the camliSig is a signature made by the camliSigner for the bytes payload.

Schema

While Perkeep can be used to store just bytes, it also adopts a standard schema that can be used to represent different data types.

TODO...

Bytes

TODO

Getting Started

Dependencies

You should install the following:

Commands

From the root directory, you may use the following commands:

  • Build using make build
  • Test using make test
  • Run CI equivalent using make ci

Project Structure

The project structure for permakeep was based on Astral's uv and ruff. It uses a flat crate structure.

Building the Documentation

Dependencies

You should install the following:

Both may be installed from source using the same versions as the CI with the following command:

PERMAKEEP_GIT_ROOT=$(git rev-parse --show-toplevel)
cargo install mdbook --version $(cat ${PERMAKEEP_GIT_ROOT}/docs/ci/mdbook_version.txt)
cargo install mdbook-linkcheck --version $(cat ${PERMAKEEP_GIT_ROOT}/docs/ci/mdbook_linkcheck_version.txt)

Commands

The source for the documentation can be found under docs/.

From that directory, you may run any of the following commands:

  • Build using make build
  • Test using make test
  • Run CI equivalent using make ci
  • View the docs locally with live reload using make serve

License

Permakeep is licensed under either of

at your option.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Permakeep by you, as defined in the Apache-2.0 license, shall be dually licensed as above, without any additional terms or conditions.

Permakeep was inspired by Perkeep, which is available under the Apache-2.0 license. The Perkeep source can be found at https://github.com/perkeep/perkeep.