Changelog History
Page 1
-
v0.10.0-alpha.0 Changes
September 27, 2022π This release ships the long-awaited Ory Permission Language (a.k.a. π userset-rewrites) π. You can now define global π rules for permissions, like "every user who is an owner also has read access", and many more. Best of all, you don't have to learn a new language to express these rules, but instead just π use a subset of TypeScript. Therefore syntax highlighting, formatting tools, π linters, unit test frameworks, ... work out of the box π¦! We will give a talk π£οΈ about how we ended up with this solution at the Ory Summit, so make sure to sign up or watch the recoding on YouTube later. Start exploring the Ory Permission Language π by following our guide π. π This is only the most shiny β¨ feature we packed into this release, see the full π changelog for all the other fixes and features we included.
π Bug Fixes
- Concurrency-safe graph utils (ea9dda9)
- Correct paths in TypeScript SDK (#1025) (8b30508)
- Do not setup /etc/nsswitch.conf on alpine (1f9fa96):
Go fixed the initial issue and does not rely on that file anymore, see https://github.com/golang/go/issues/35305
Tests now use the new httpclient to properly handle empty strings vs strings (where the value is omitted in the JSON request).
http*request** metrics contain data related only to /metrics/prometheus endpoint. This commit adds endpoints from non-monitoring routers.
fix: register read and write routers with PrometheusManager
fix: register read and write routers with PrometheusManager p2
fix: register read and write routers with PrometheusManager p3
feat: grpc request metrics
chore: add test
chore: revert gRPC metric test
chore: re-trigger ci
chore: re-trigger ci
chore: re-trigger ci
fix: lint
fix: cve
- Sdk generation (acc1546):
fix: better error handling
chore: remove old httpclient
Enable TLS and certificate checking in the gRPC client when communicating with remote hosts.
π· Build System
Code Generation
- π Pin v0.10.0-alpha.0 release commit (52259a3):
Bumps from v0.10.0-alpha.0.pre.0
π¨ Code Refactoring
- Generalize tree structure (6a0b2fe):
This will allow reusing the tree to provide debug info on how a check decision was reached.
π Documentation
π Features
The subject-set rewrites can now be configured through the Ory Permission Language (OPL), which is a subset of TypeScript. The OPL config is referenced in the central configuration under namespaces as such:
[...] namespaces: location: <location> [...]
The can be any valid file, directory or URI.
- π Fine-grained control over transport security (5f056b7):
This adds two new flags to the Keto CLI:
- --insecure-disable-transport-security: Use plaintext instead of TLS
- --insecure-skip-hostname-verification: Use TLS, but do not verify the certificate
By default, the Keto CLI now connects to the remote via TLS and verifies the hostname.
- OPL typescript library on npm (446fe7d)
- Simpler notation for subjects w/o relation (ec979df)
- Subject-set rewrites (6f61af8)
- π Support subject sets in check (1760459)
β Tests
-
v0.9.0-alpha.0 Changes
August 01, 2022π This release ships a few changes in the API paths. Requests and responses were π¨ not changed. However, we did A LOT of internal refactoring and improvements on the persistence layer. Some naming in the SDKs changed, it is a lot cleaner now. π One important change is that we removed the single table migrator. From now on to migrate from v0.6.0-alpha.1, please first migrate the legacy namespaces using v0.8.0-alpha.2 We also overhauled the whole persistence structure to ensure high scalability. This means that the migration might take a bit longer than usual, β so please test the process first on a backup or staging environment. For all the details, check out the full changelog.
π₯ Breaking Changes
π
keto namespace migrate ...
commands were removed. To migrate from v0.6.0-alpha.1, please first migrate the legacy namespaces using v0.8.0-alpha.2β¬οΈ The protobuf API was bumped to
v1alpha2
. Please upgrade your client dependency π to that version.v1alpha1
is still supported for now, but might be dropped soon.π° Some payload keys are now (not) required anymore. The generated SDKs will likely have breaking changes.
Co-authored-by: Patrik [email protected] Co-authored-by: hperl [email protected]
/check
is now/relation-tupes/check
/expand
is now/relation-tuples/expand
/relation-tuples
is now/admin/relation-tuples
for write APIsπ¦ gRPC package is now called
ory.keto.relation_tuples.v1alpha2
π gRPC relation-tuple-delta action enum names are prefixed with
ACTION_
π Bug Fixes
- cli: Make flag registration non-racy (8415ced)
- 0οΈβ£ Enable telemetry by default (9dc8c7c)
- Hide relation tuples with deleted namespace (cb1a2dd)
Code Generation
- π Pin v0.9.0-alpha.0 release commit (6a13898)
π¨ Code Refactoring
This change refactors the API paths to be consistent with the rest of the Ory ecosystem. This step is required for the unified Ory SDK. Additionally, as we plan to add high level APIs, e.g. for RBAC. The check and expand API paths changed to allow adding those.
- π Change pagination to use keyset pagination (7b861c9):
The page token now is the last ID of the previous page. This enables faster queries and more stable pagination. NOTE: in case an integration modified page tokens to control pagination, this change will break the integration. Page tokens are opaque strings and should never be messed with.
π Documentation
- π¦ Expose embedded OpenAPI spec (f9d20e3)
- π Fix docker compose demo setup (#872) (e89fbb0)
- π Improve wording in README (#881) (fd6af60)
- π· Shorten CI status badge (#928) (81d880d)
- π version schema: Require version or fall back to latest (#863) (5306c93)
π Features
- β Add check endpoints that do not mirror status code (#853) (07d0fbd)
- β Add reverse lookup indices (#875) (25af263)
- β Add spec for namespace configs (3d61b1c):
Co-authored-by: hackerman [email protected]
- π§ Make sensitive log value redaction text configurable (#860) (b8b1d81)
- Map strings to UUIDs (#809) (#840) (add6577):
With this change Keto now maps strings to UUIDv5 on the storage layer. This change allows unlimited strings to be used while maintaining good performance. Further, it reduces the likeliness of database hot-spots. The migration that applies this mapping might take some time, so please confirm that your migration strategy works for you.
- Metric names same as for Kratos (315ff41)
- tracing: Improved tracing for persisters and requests (#878) (eb62c50)
- tracing: Switch to opentelemetry (#861) (31f38ed)
β Tests
-
v0.8.0-alpha.2 Changes
March 04, 2022 -
v0.8.0-alpha.1 Changes
February 22, 2022π This is merly a cleanup release to fix automation issues.
π Bug Fixes
- β Add foreign key constraint for network ID (e815cb0)
- Phony (9f5d0aa)
- π Resolve post-release steps (#838) (d478567)
Code Generation
- π Pin v0.8.0-alpha.1 release commit (6daf88b)
β Tests
- π Fix inconsistencies (dab1b1d)
-
v0.8.0-alpha.0 Changes
February 10, 2022Ory Keto v0.8.0-alpha.0 mainly ships internal improvements next to one bigger, possibly breaking feature. With π PR #799 Keto now supports bulk deletion of relation-tuples. For gRPC clients we added a new rpc, while in the REST world we had to change the behavior of the existing delete handler.
IT NOW DELETES ALL MATCHING TUPLES.
Example:
curl -X DELETE "https://keto.local/relation-tuples?subject_id=foo"
will delete all relation tuples that have the subject ID
foo
, even across namespaces. Passing empty strings is equivalent to not setting a value. Please β‘οΈ test your integrations on a copy of your database before rolling out the update.If you don't use that REST endpoint, you are on the safe side.
π Bug Fixes
- β Add dummy sidebar (555ffca)
- β Add hiring notice to README (#798) (2a6ddae)
- CORS config values are ignored (#789) (ffeb5e3)
- π³ Docker compose migrate (#800) (f1599a4)
- π³ Docker-compose-postgres.yml SQL migration service (#779) (8f041bc)
- Namespace should not be required in List API (#796) (07be82e):
The
namespace
parameter is now not required anymore in the list REST API.- Openapi spec and internal SDK (#819) (a1b20c7)
- π Panic on macOS (059a6f9)
- Slow keto start up time (b7c620c):
Found a deeply nested dependency which was importing
https://github.com/markbates/pkger
, causing unreasonable CPU consumption and significant delay at start up time. With this patch, start up time was reduced from almost 1.7s to 0.02s.$ time keto keto 1.65s user 2.02s system 734% cpu 0.499 total $ time ./keto-patch ./keto-patch 0.02s user 0.01s system 6% cpu 0.425 total
Code Generation
- π Pin v0.8.0-alpha.0 release commit (85d59ec)
π¨ Code Refactoring
- π§ Configuration structure for limits (ffa99ec)
- π Move documentation to ory/docs and move to OAS3.0 generator (#833) (55d9d4e)
π Documentation
- β Add cloud (5c66087)
- β Add link to quickstart in config reference docs (#775) (25bc579)
- Clarify that CLI remotes should be addresses and not URIs (#808) (7ce7973)
- Heading caseing (#785) (563087d)
- β‘οΈ Update readme (d9397cc)
π Features
The parameter
max-depth
for the check command limits the depth of the search, a safeguard against particularly expensive queries. This allows users more fine-grain control.Furthermore, there is now a global max-depth configuration value that limits the overall max-depth of check and expand operations. It defaults to
5
, which is considered a very safe value. -
v0.7.0-alpha.1 Changes
October 19, 2021π This release provides small docs fixes especially for SDK clients.
Code Generation
- π Pin v0.7.0-alpha.1 release commit (0d1e33a)
π Documentation
-
v0.7.0-alpha.0 Changes
October 06, 2021π We are proud to present you a new release of Ory Keto! It has been a while, but we have been working hard not only on code, but also concepts and discussing many upcoming features. To join us on this exciting journey, watch :eyes: and start :star: the repository.
π At a first glance the release might not look too exciting from the outside, but we had 376 changed files with 47,578 additions and 25,418 deletions. In total, π 12 contributors worked on the 192 commits. The most changes were bug fixes, π¨ internal refactoring, and improving API consistency. Expect a more reliable Keto, that is also prepared to receive π many new exciting features.
Because the database schema changed significantly, and it is not possible to have SQL-only migrations, there is a special migration procedure needed to β¬οΈ upgrade from Ory Keto v0.6. Please follow the π migration guide and, as always, read the changelog before β¬οΈ upgrading.
π₯ Breaking Changes
This patch changes the payload of the REST API. The gRPC API is not affected. The parameter
subject
was previously an encoded string. With this π change clients have to explicitly use eithersubject_id
or (subject_set.namespace
andsubject_set.object
andsubject_set.relation
). The same is true for REST responses returned by Keto. An error with a hint will be returned ifsubject
is still used.π Bug Fixes
make sdk
dependency on the Ory CLI (#710) (0cb5706)- β Add missing tracers (#600) (aa263be), closes #593
- cli: Panic when printing empty expand trees (#686) (7956dec)
- π³ Dockerfiles (#737) (f10dec1)
- π Exclude
/health
endpoints from logs (#716) (7c27f92) - π Handle relation tuple cycles in expand and check engine (#623) (8e30119)
- π² Log all database connection errors (#588) (2b0fad8)
- π Move gRPC client module root up (#620) (3b881f6):
BREAKING: The npm package
@ory/keto-grpc-client
from now on includes all API versions. Because of that, the import paths changed. For migrating to the new client package, change the import path according to the following example:- import acl from '@ory/keto-grpc-client/acl_pb.js' + // from the latest version + import { acl } from '@ory/keto-grpc-client' + // or a specific one + import acl from '@ory/keto-grpc-client/ory/keto/acl/v1alpha1/acl_pb.js'
This change significantly improves and the config schema. Parts will now be taken from upstream to ensure a more up-to-date schema.
- Patch REST API input validation and SDK generation (#717) (d49e098)
- β Run a whole namespace migration as one transaction (#739) (142bd47)
- π Set version during release build and register version handler (#714) (8091475)
- β‘οΈ Update docker-compose.yml version (#595) (7fa4dca), closes #549
Chores
- β‘οΈ Update repository templates (f53d3eb)
Code Generation
- π Pin v0.7.0-alpha.0 release commit (7962e77)
π¨ Code Refactoring
- Ensure namespace manager reload is resource contained (#735) (5696fc6)
- π Make subject sets and subject IDs unambiguous (#729) (5a1b0ba)
- Persistence table structure (#638) (d02b818):
This big refactoring greatly reduces operation complexity and paves the way for upcoming performance improvements. From now on the relation tuples from all namespaces are stored in the same table, instead of having tables per namespace. A migration path will be provided separately.
π Documentation
- β Add migration guide to v0.7 (#758) (3335a4b)
- β Add new redoc docsc (#651) (435ae00)
- π Change npm org scope from @oryd to @ory (#730) (945005f)
- π Fix build and gen:config (#676) (f128732)
- π Fix erroneous sidebar commit (5cf2b4c)
- π Fix example not following best practice (#582) (a015818)
- π Fix link in migration guide (e995829)
- π Fix REST API reference (#718) (f989b39)
- β‘οΈ Update NPM links due to organisation move (#616) (6355bea)
- β‘οΈ Update reference to Zanzibar in README (#746) (a0d8caf)
π Features
Behold! The Keto gRPC client library now has useful helpers that allow you to replace:
- deltas := make([]*acl.RelationTupleDelta, len(tuples)) - for i := range rts { - deltas[i] = &acl.RelationTupleDelta{ - Action: acl.RelationTupleDelta_INSERT, - RelationTuple: rts[i], - } - } + deltas := acl.RelationTupleToDeltas(tuples, acl.RelationTupleDelta_INSERT)
and
- &acl.Subject{Ref: &acl.Subject_Set{Set: &acl.SubjectSet{ - Namespace: "directories", - Object: "/photos", - Relation: "access", - }}} + acl.NewSubjectSet("directories", "/photos", "access")
and
- &acl.Subject{Ref: &acl.Subject_Id{ - Id: "user1", - }} + acl.NewSubjectID("user1")
Enjoy these new treats :chocolate_bar: :lollipop: :icecream:
- Enable telemetry collection for gRPC (#738) (5ac8b0c)
- π Make generated gRPC client its own module (#583) (f0fbb64)
- Max_idle_conn_time (#605) (50a8623), closes #523
- Migration to single table SQL schema (#707) (00713bc):
This change adds a migration path from Keto version v0.6.x to the new persistence structure introduced by https://github.com/ory/keto/pull/638. Every namespace has to be migrated separately, or you can use the CLI to detect and migrate all namespaces at once. Have a look at
keto help namespace migrate legacy
for all details. Please make sure that you backup the database before running the migration command. Please note that this migration might be a bit slower than usual, as we have to pull the data from the database, transcode it in Keto, and then write it to the new table structure. Versions of Keto >v0.7 will not include this migration script, so you will first have to migrate to v0.7 and move on from there.The
keto namespace validate
command now additionally supports:- validation of namespaces in config files
- validation of a directory specified in config files
β Tests
Confirmed that the fix works because
$ go test -tags sqlite -run TestStatusCmd/server_type=read/case=block -count 1000 ./cmd/status
passed.
-
v0.6.0-alpha.3 Changes
April 29, 2021π Resolves CRDB and build issues.
Code Generation
- π Pin v0.6.0-alpha.3 release commit (d766968)
-
v0.6.0-alpha.2 Changes
April 29, 2021π This release improves stability and documentation.
π Bug Fixes
- β Add npm run format to make format (7d844a8)
- π Makefile target (0e6f612)
- π Move swagger to spec dir (7f6a061)
- Resolve clidoc issues (ef12b4e)
- β‘οΈ Update install.sh (#568) (86ab245)
- π Use correct id (5e02902)
- π Use correct id for api (32a6b04)
- π Use sqlite image versions (#544) (ec6cc5e)
Code Generation
- π Pin v0.6.0-alpha.2 release commit (470b2c6)
π Documentation
- β Add gRPC client README.md (#559) (9dc3596)
- π Change forum to discussions readme (#539) (ea2999d)
- π Fix cat videos example docker compose (#549) (b25a711)
- π Fix typo (#538) (99a9693)
- Include namespace in olymp library example (#540) (135e814)
- β‘οΈ Update install from source steps to actually work (#548) (e662256)
π Features
-
v0.6.0-alpha.1 Changes
April 07, 2021We are extremely happy to announce next-gen Ory Keto which implements Zanzibar: Googleβs Consistent, Global Authorization System:
π§ > Zanzibar provides a uniform data model and configuration language for
expressing a wide range of access control policies from hundreds of client services at Google, including Calendar, Cloud, Drive, Maps, Photos, and YouTube. Its authorization decisions respect causal ordering of user actions and thus provide external consistency amid changes to access control lists and object contents. Zanzibar scales to trillions of access control lists and π > millions of authorization requests per second to support services used by π€ > billions of people. It has maintained 95th-percentile latency of less than 10 milliseconds and availability of greater than 99.999% over 3 years of production use.
Ory Keto is the first open source planet-scale authorization system built with cloud native technologies (Go, gRPC, newSQL) and architecture. It is also the first open source implementation of Google Zanzibar :tada:!
Many concepts developer by Google Zanzibar are implemented in Ory Keto already. Let's take a look!
π As of this release, Ory Keto knows how to interpret and operate on the basic access control lists known as relation tuples. They encode relations between objects and subjects. One simple example of such a relation tuple could encode "
user1
has access to file/foo
", a more complex one could encode "everyone who has write access on/foo
has read access on/foo
".Ory Keto comes with all the basic APIs as described in the Zanzibar paper. All of them are available over gRPC and REST.
- List: query relation tuples
- Check: determine whether a subject has a relation on an object
- Expand: get a tree of all subjects who have a relation on an object β‘οΈ 4. Change: create, update, and delete relation tuples
For all details, head over to the π documentation.
π With this release we officially move the "old" Keto to the legacy-0.5 branch. We will only π provide security fixes from now on. A migration path to v0.6 is planned but not yet implemented, as the architectures are vastly different. Please refer to the issue.
π We are keen to bring more features and performance improvements. The next π features we will tackle are:
- Subject Set rewrites
- π Native ABAC & RBAC Support
- Integration with other policy servers
- π€ Latency reduction through aggressive caching
- Cluster mode that fans out requests over all Keto instances
π So stay tuned, :star: this repo, :eyes: releases, and π subscribe to our newsletter :email:.
π Bug Fixes
- β Add description attribute to access control policy role (#215) (831eba5)
- Add leak_sensitive_values to config schema (2b21d2b)
- β¬οΈ Bump CLI (80c82d0)
- β¬οΈ Bump deps and replace swagutil (#212) (904258d)
- Check engine overwrote result in some cases (#412) (3404492)
- Check health status in status command (21c64d4)
- Check REST API returns JSON object (#460) (501dcff), closes #406
- Empty relationtuple list should not error (#440) (fbcb3e1)
- Ensure nil subject is not allowed (#449) (7a0fcfc):
The nodejs gRPC client was a great fuzzer and pointed me to some nil pointer dereference panics. This adds some input validation to prevent panics.
- Ensure persister errors are handled by sqlcon (#473) (4343c4a)
- π Handle pagination and errors in the check/expand engines (#398) (5eb1a7d)
- Ignore dist (ba816ea)
- Ignore x/net false positives (d8b36cb)
- π Improve CLI remote sourcing (#474) (a85f4d7)
- π Improve handlers and add tests (#470) (ca5ccb9)
- π¨ Insert relation tuples without fmt.Sprintf (#443) (fe507bb)
- π Minor bugfixes (#371) (185ee1e)
- π³ Move dockerfile to where it belongs (f087843)
- Namespace migrator (#417) (ea79300), closes #404
- β Remove SQL logging (#455) (d8e2a86)
- π Rename /relationtuple endpoint to /relation-tuples (#519) (8eb55f6)
- π Resolve gitignore build (6f04bbb)
- π Resolve goreleaser issues (d32767f)
- π Resolve windows build issues (8bcdfbf)
- Rewrite check engine to search starting at the object (#310) (7d99694), closes #302
- π Secure query building (#442) (c7d2770)
- π³ Strict version enforcement in docker (e45b28f)
- β‘οΈ Update dd-trace to fix build issues (2ad489f)
- β‘οΈ Update docker to go 1.16 and alpine (c63096c)
- π Use errors.WithStack everywhere (#462) (5f25bce), closes #437:
Fixed all occurrences found using the search pattern
return .*, err\n
.- π¦ Use package name in pkger (6435939)
- schema: Add trace level to logger (a5a1402)
- π Use make() to initialize slices (#250) (84f028d), closes #217
π· Build System
Code Generation
- π Pin v0.6.0-alpha.1 release commit (875af25)
π¨ Code Refactoring
π Documentation
- β Add check- and expand-API guides (#493) (09a25b4)
- β Add current features overview (#505) (605afa0)
- β Add missing pages (#518) (43cbaa9)
- β Add namespace and relation naming conventions (#510) (dd31865)
- β Add performance page (#413) (6fe0639):
This also refactored the server startup. Functionality did not change.
- β Add production guide (a9163c7)
- β Add zanzibar overview to README.md (#265) (15a95b2)
- API overview (#501) (05fe03b)
- Concepts (#429) (2f2c885)
- β Delete old redirect homepage (c0a3784)
- Document gRPC SKDs (7583fe8)
- π Fix grammatical error (#222) (256a0d2)
- π Fix regression issues (9697bb4)
- Generate gRPC reference page (#488) (93ebe6d)
- π Improve CLI documentation (#503) (be9327f)
- π Minor fixes (#532) (638342e)
- π Move development section (9ff393f)
- π Move to json sidebar (257bf96)
- β Remove duplicate "is" (ca3277d)
- β Remove duplicate template (1d3b38e)
- β Remove old documentation (#426) (eb76913)
- Replace TODO links (#512) (ad8e20b)
- Resolve broken links (0d0a50b)
- Simple access check guide (#451) (e0485af):
This also enables gRPC go, gRPC nodejs, cURL, and Keto CLI code samples to be tested.
- β‘οΈ Update comment in write response (#329) (4ca0baf)
- β‘οΈ Update install instructions (d2e4123)
- β‘οΈ Update introduction (5f71d73)
- β‘οΈ Update README (#515) (18d3cd6):
Also format all markdown files in the root.
- β‘οΈ Update repository templates (db505f9)
- β‘οΈ Update repository templates (6c056bb)
- β‘οΈ Update SDK links (#514) (f920fbf)
- π Update swagger documentation for REST endpoints (c363de6)
- π Use mdx for api reference (340f3a3)
- β‘οΈ Various improvements and updates (#486) (a812ace)
π Features
- β Add .dockerignore (8b0ff06)
- β Add and automate version schema (b01eef8)
- β Add check engine (#277) (396c1ae)
- β Add gRPC health status (#427) (51c4223)
- Add is_last_page to list response (#425) (b73d91f)
- β Add POST REST handler for policy check (7d89860)
- β Add relation write API (#275) (f2ddb9d)
- β Add REST and gRPC logger middlewares (#436) (615eb0b)
- β Add SQA telemetry (#535) (9f6472b)
- β Add sql persister (#350) (d595d52)
- β Add tracing (#536) (b57a144)
- π Allow to apply namespace migrations together with regular migrations (#441) (57e2bbc)
- β Delete relation tuples (#457) (3ec8afa), closes #452
- π³ Dockerfile and docker compose example (#390) (10cd0b3)
- Expand API (#285) (a3ca0b8)
- Expand GPRC service and CLI (#383) (acf2154)
- First API draft and generation (#315) (bda5d8b)
- GRPC status codes and improved error messages (#467) (4a4f8c6)
- GRPC version API (#475) (89cc46f)
- π Implement goreleaser pipeline (888ac43), closes #410
- Incorporate new GRPC API structure (#331) (e0916ad)
- π§ Koanf and namespace configuration (#367) (3ad32bc)
- π§ Namespace configuration (#324) (b94f50d)
- Namespace migrate status CLI (#508) (e3f7ad9):
This also refactors the current
migrate
andnamespace migrate
commands.Includes Typescript definitions.
This is a first draft of the read API. It is reachable by REST and gRPC calls. The main purpose of this PR is to establish the basic repository structure and define the API.
This command parses the relation tuple format used in the docs. It greatly improves the experience when copying something from the documentation. It can especially be used to pipe relation tuples into other commands, e.g.:
echo "messages:02y_15_4w350m3#decypher@john" | \ keto relation-tuple parse - --format json | \ keto relation-tuple create -
The new PATCH handler allows transactional changes similar to the already existing gRPC service.
- Separate and multiplex ports based on read/write privilege (#397) (6918ac3)
- Swagger SDK (#476) (011888c)
β Tests
- β Add command tests (#487) (61c28e4)
- β Add dedicated persistence tests (#416) (4e98906)
- β Add handler tests (#478) (9315a77)
- β Add initial e2e test (#380) (dc5d3c9)
- β Add relationtuple definition tests (#415) (2e3dcb2)
- β Enable GRPC client in e2e test (#382) (4e5c6ae)
- π Improve docs sample tests (#461) (6e0e5e6)