« Changelog History


dgraph v21.12.0 Release Notes

Release Date: 2021-11-30 // over 2 years ago

  • 🔄 Changed

    • [BREAKING] feat(sroar): Bring sroar to Dgraph (#7840)
    • [BREAKING] Return error for illegal math operations. (#7631)
    • [BREAKING] feat: bring dgraph-lambda to dgraph, alpha launches lambda server (#7973)
    • [BREAKING] fix json marshal unmarshal for namespace > 127 (#7810)
    • [BREAKING] fix(usability): make force-namespace flag compulsory in live loader for galaxy user (#7731)
    • [BREAKING] perf(Transactions): Run transactions concurrently (#7694)
    • [BREAKING] feat(flags): expand badger to accept all valid options (#7677)
    • [BREAKING] fix(commit): make txn context more robust (#7659)
    • ⚡️ [BREAKING] Opt(Restore): Optimize Restore's new map-reduce based design (#7666)
    • [BREAKING] fix(metrics): Rename Badger metrics. (#7507)
    • ⏪ [BREAKING] Make backup-restore an open source feature (#8067)

    ➕ Added

    • GraphQL

      • Feat(GRAPHQL): adds @default directive for setting default field values at create and update (#8017)
      • Feat(GRAPHQL): Support auth with custom DQL (#7775)
      • Feat(GRAPHQL): This PR allows updatable and nullable @id fields. (#7736)
      • Feat(GRAPHQL): Disallow DQL schema changes for predicates used in GraphQL schema (DGRAPH-3245) (#7742)
      • Feat(GRAPHQL): This PR allows @id field in interface to be unique across all the implementing types. (#7710)
      • Feat(GRAPHQL): Add language tag support in GraphQL (#7663)
      • Feat(GRAPHQL): Zero HTTP endpoints are now available at GraphQL admin (GRAPHQL-1118) (#6649)
      • Feat(GRAPHQL): Webhooks on add/update/delete mutations (GRAPHQL-1045) (#7494)
      • Feat(GRAPHQL): Allow Multipe JWKUrls for auth. (#7528)
      • Feat(GRAPHQL): Add support for passing OAuth Bearer token as authorization JWT (#7490)

    • Core Dgraph

      • Feat(metrics): Add Badger metrics. (#8034)
      • Feat(magicNumber): Introduce magic number (#8032)
      • Feat(lambda): allow access to access jwt in lambda (#8023)
      • Feat(rdf-response): Support RDF response via http query request (#8004)
      • Feat(sroar): Use rank() API from sroar and some cleanup (#8002)
      • Feat(lambda): store lambda scripts within the dgraph (#7955)
      • Feat(query): handle extend keyword for Queries and Mutations (#7916)
      • Feat(Backup): Add native google cloud storage backup support (#7829)
      • Feat(Backup): Add native support for backup to Azure. (#7843)
      • Feat(cloud): add shared-instance flag in limit superflag in alpha (#7770)
      • Feat(Dgraph): Add task queue implementation (#7716)
      • Feat(DQL): @groupby on scalar fields and count duplicate (#7746)
      • Feat(Query): Add random keyword in DQL (#7693)
      • Feat(tool): Neo4j CSV to RDF Converter (#7545)
      • Feat(query): Add mechanism to have a limit on number of pending queries (#7603)
      • Feat(flag): remove unused badger.max-retries option from bulk command (#7591)
      • Feat(sentry): clusterID flag added for alpha sentry reports (gql-services) (#7580)
      • Feat(cmd/debuginfo) add new metrics to be collected (#7439)
      • Feat(flags): use Vault for ACL secrets (#7492)
      • Feat(Apollo): Add support for @provides and @requires directive. (#7503)
      • Feat(restore): Introduce incremental restore (#7942) (#7971)
      • Feat(schema): do schema versioning and make backup non-blocking for indexing (#7852)
      • Feat(zero bulk): adding bulk call for alpha to inform zero about the tablets (#8100)
      • Feat(cdc): Add superflag to enable TLS without CA or certs. (#8097)

    • Enterprise Features

      • Feat(Multi-tenancy): Add namespaces field to state (#7808)
      • Feat(multi-tenancy): make drop data namespace aware (#7789) (#7795)
      • Feat(cdc): Add support for SCRAM SASL mechanism (#7765)
      • Feat(acl): allow access to all the predicates using wildcard (#7991)
      • Feat(cdc): Add superflag to enable TLS without CA or certs. (#7946)

    🛠 Fixed

    • GraphQL

      • Fix(GRAPHQL): add validation of null values with correct order of graphql rule validation (#8007) (#8008)
      • Fix(GRAPHQL): fix type assertion failure in graphql if resolver is not defined (#8003)
      • Fix(GRAPHQL): fixing graphql schema update when the data is restored (#7970)
      • Fix(GRAPHQL): Nested Auth Rules not working properly. (#7915)
      • Fix(GRAPHQL): optimize eq filter queries (#7895)
      • Fix(GRAPHQL): Fix duplicate XID error in case of interface XIDs (#7776)
      • Fix(GRAPHQL): Pass on HTTP request headers for subscriptions (#7806)
      • Fix(GRAPHQL): Make mutation rewriting tests more robust (#7768)
      • Fix(GRAPHQL): Fix error message of lambdaOnMutate directive (#7751)
      • Fix(GRAPHQL): fix auth query rewriting with ID filter (#7740)
      • Fix(GRAPHQL): Add filter in DQL query in case of reverse predicate (#7728)
      • Fix(GRAPHQL): Fix GraphQL encoding in case of empty list (#7726)
      • Fix(GRAPHQL): fix @cascade with Pagination for @auth queries. (#7695)
      • Fix(GRAPHQL): Fix Execution Trace for Add and Update Mutations (#7656)
      • Fix(GRAPHQL): Log query along with the panic (#7638)
      • Fix(GRAPHQL): Add error handling for unrecognized args to generate directive. (#7612)
      • Fix(GRAPHQL): Fix panic when no schema exists for a new namespace (#7630)
      • Fix(GRAPHQL): fixed output coercing for admin fields. (#7617)
      • Fix(GRAPHQL): fix lambda querying a lambda field in case of no data. (#7610)
      • Fix(GRAPHQL): Add extra checks for deleting UpdateTypeInput (#7595)
      • Fix(GRAPHQL): remove support of @id directive on Float (#7583)
      • Fix(GRAPHQL): Fix mutation with Int Xid variables. (#7565)
      • Fix(GRAPHQL): Fix custom(dql: ...) with __typename (GRAPHQL-1098) (#7569)
      • Fix(GRAPHQL): Change variable name generation for interface auth rules (#7559)
      • Fix(GRAPHQL): Apollo federation now works with lambda (GRAPHQL-1084) (#7558)
      • Fix(GRAPHQL): fix empty remove in update mutation patch, that remove all the data for nodes in filter. (#7563)
      • Fix(GRAPHQL): fix order of entities query result (#7542)
      • Fix(GRAPHQL): Change variable name generation from Type to Type_ (#7556)
      • Fix(GRAPHQL): fix duplicate xid error for multiple xid fields. (#7546)
      • Fix(GRAPHQL): Added support for exact index on field having @id directive. (#7534) (#7551)
      • Fix(GRAPHQL): fix query rewriting for multiple order on nested field. (#7523)
      • Fix(GRAPHQL) fix empty type Query with single extended type definition in the schema. (#7517)

    • Core Dgraph

      • Fix(sort): Only filter out nodes with positive offsets. (#8077)
      • Fix(fragment): merge the nested fragments fields (#8075)
      • Fix(lambda): upgrade lambda dependencies to fix vulnerabilities (#8074)
      • Fix(magic): fix the magic version in bulk loader etc (#8070)
      • Fix(split): enable split of posting list with single plist (#8062)
      • Fix(restore): Do not retry restore proposal (#8058)
      • Fix(txn): Fix data races in transaction code (#8060)
      • Fix(shutdown): wait for pending queries to process on alpha shutdown (#8057)
      • Fix(restore-test): Make offline restore use separate map directory for each group (#8047)
      • Fix(lambda-logs): extract namespace from body.namespace (#8043)
      • Fix(lambda): make lambda active only after successful start (#8036)
      • Fix(probe): do not contend for lock in lazy load (#8037)
      • Fix(lambda): shutdown node processes when alpha gets killed (#8027)
      • Fix(snapshot): Fix snapshot calculation after restore (#8024)
      • Fix(badger): Upgrade badger to fix deadlock (#8025)
      • Fix(bulk): quote the schema correctly in bulk loader (#8019)
      • Fix(sbs): handle response error gracefully (#8018)
      • Fix(admin): make config changes to pass through gog middlewares (#8014)
      • Fix(lambda): fix race condition in lambda server spin up (#8013)
      • Fix(proposals): Incremental proposal key for zero proposals (#8005)
      • Fix(lambda): monitor lambda server, fix performance issue, remove lambda logs from extensions (#8006)
      • Fix(live): quote the xid when doing upsert (#7983)
      • Fix(sroar): Bring latest sroar to master (#7977)
      • Fix(query): Do not execute filters if there are no source uids (#7962) (#7969)
      • Fix(snapshot): update last snapshot time across members (#7968)
      • Fix(pool): use write lock when getting health info (#7963)
      • Fix(JoinCluster): Avoid retrying JoinCluster indefinitely (#7961)
      • Fix(rollups): Write rolled-up keys at ts+1 (#7957) (#7959)
      • Fix(conn): JoinCluster loop should use latest conn (#7950)
      • Fix(restore): Set kv version to restoreTs for rolled up keys and schema keys (#7930) (#7935)
      • Fix(backup): Fix full backup request (#7932)
      • Fix(cmd/debug): Print banned namespaces correctly. (#7929)
      • Reconnect via a redial in case of disconnection. (#7918)
      • Fix(metrics): Expose dgraph_num_backups_failed_total metric view. (#7900)
      • Fix(sroar): Fix TestAuthWithCustomDQL failure because of roaring bitmaps (#7902)
      • Fix(DQL): revert changes related to cascade pagination with sort (#7885)
      • Fix(restore): append galaxy namespace to type name (#7880)
      • Fix(Backup): use validReadTs from manifest for backward compatibility (#7601) (#7863)
      • fix the predicate move (#7862)
      • Fix(restore): consider the banned namespaces while bumping (#7839)
      • Fix(restore): update the schema and type from 2103 (#7838)
      • Fix(updatemanifest): update the version of manifest after update (#7828)
      • Fix(backup): handle manifest version logic, update manifest version to 2105 (#7825)
      • Fix(schema-update): Start opIndexing only when index creation is required. (#7845)
      • Fix(admin): remove exportedFiles field (#7835)
      • Fix(restore): reset the kv.StreamId before sending to stream writer (#7833)
      • Fix(auth): preserve the status code while returning error (#7832)
      • bug fix to permit audit streaming to stdout writer(#7803)
      • Fix(lease): don't do rate limiting when not limit is not specified (#7787) (#7801)
      • Fix(restore): Bump uid and namespace after restore (#7790)
      • Fix(txn): ensure that txn hash is set (#7782)
      • Fix(export-backup): Fix double free in export backup (#7780)
      • Fix(Dgraph): Forward task status requests to correct Alpha (#7774)
      • Fix(bulk): upsert guardian/groot for all existing namespaces (#7759)
      • Fix(export): Fix facet export of reference type postings to JSON format (#7744)
      • fix: Prevent proposal from being dropped accidentally (#7741)
      • Fix(live): make live loader progress on a cluster with very high maxUid (#7743)
      • Fix(Chunker): don't delete node with empty facet in mutation (#7737)
      • Fix(DQL): optimize query for has function with offset. (#7727)
      • fixing readme for dgraph after latest release (#7732)
      • Fix(lease): prevent ID lease overflow (#7724)
      • Fix(lsbackup): Fix profiler in lsBackup (#7729)
      • Fix(bulk): throw the error instead of crashing (#7722)
      • Fix(ee): GetKeys should return an error (#7713)
      • Fix(raftwal): take snapshot after restore (#7719)
      • Fix(pagination): Fix after for regexp, match functions (#7700)
      • Fix(query): Prevent multiple entries for same predicate/type in schema mutations. (#7715)
      • Fix(vault): Hide ACL flags when not required (#7701)
      • Fix(flag): fix bulk loader flag and remove flag parsing from critical path (#7679)
      • Fix(upgrade): make upgrade tool to work with non-acl cluster (#7674)
      • Fix(query): Fix pagination with match functions (#7668)
      • Fix(postingList): Acquire lock before reading the cached posting list (#7632)
      • Fix(zero): add a ratelimiter to limit the uid lease per namespace (#7568)
      • Fix(export): use UriHandler for exports (#7690)
      • Fix s3 backup copy (#7669)
      • return if no uids exist in queries for Geo (#7651)
      • Fix(DGRAPH): fix @normalize response when multiple fields at different levels with same alias are selected. (#7639)
      • Fix(/commit): protect the commit endpoint via acl (#7608)
      • Use GetString for vault path (#7605)
      • Fix query logging for mutations (#7646)
      • Fix(login): fix login based on refresh token logic (#7637)
      • Fix(Query): Fix cascade pagination with 0 offset. (#7636)
      • feat(flags): Add query timeout as a limit config (#7599)
      • Fix(flags): Add empty defaults to Vault superflag (#7598)
      • Fix(persistent): make persistent query namespace aware (#7570)
      • Fix(rollups): Fix splits in roll-up (#7609)
      • fix for xgo version to use (#7620)
      • Fix(flags): Expose global flags to dgraph subcommands. (#7530)
      • Fix(telemetry): fix zero crash due to telemetry (#7575)
      • Fix(telemetry): Track enterprise feature usage (#7495)
      • Fix(release): update support for xgo tool (#7576)
      • Fix(super-flags): Use GetPath for path arguments in superflags (#7541)
      • Fix(dql): Fix error message in case of wrong argument to val() (#7543)
      • Fix(Roaring): Remove pack from posting list (#7535)
      • Fix(Flags): immediately panic on SuperFlag user errors (#7529)
      • Fix(Rollups): Don't try splitting a posting list with cardinality less than 2. (#7525)
      • Fix(export): fix namespace parameter in export (#7524)
      • Fix(live): fix usage of force-namespace parameter in export (#7526)
      • Fix largeSchemaUpdate test (#7522)
      • Fix(Configs): Allow hierarchical notation in JSON/YAML configs (#7498)
      • Fix(Bulk): Remove stale allocator in reduce (#7510)
      • Fix upsert mutations (#7515)
      • Fix(standalone): Set whitelist flag using superflag. (#7512)
      • Fix(admin-endpoints): Error out if the request is rejected by the server (#7511)
      • Fix(Dgraph): Throttle number of files to open while schema update (#7480)
      • Fix(metrics): Expose Badger LSM and vlog size bytes. (#7488)
      • Fix(schema): log error instead of panic if schema not found for predicate (#7502)
      • Fix(tool): Don't ban namespace in export_backup (#8099)
      • Fix(state): fix hex to uint64 response of list of namespaces (#8101)
      • Fix(restore): return nil if there is error (#8098)

    • Enterprise Features

      • Fix(audit): fixing audit logs for websocket connections (#8048)
      • Fix(acl): subscribe for the correct predicates (#7992)
      • Fix(acl): filter out the results based on type (#7978) (#7980)
      • Fix(groot): do not upsert groot for all namespaces on restart (#7917)
      • Fix(cdc): Show namespace info in event meta (#7721)
      • Fix(learner): Don't start a learner node with no peers (#7582)
      • Fix(audit): logs not getting deleted after N days (#7567)
      • Fix(release/v21.03) - Use worker.GetEEFeatureList instead of ee.GetEEFeatureList (#7564)
      • Fix(multi-tenancy): Format namespace to human readable form (#7552)
      • Fix(learner nodes): Reconnect to learner nodes after restart (#7554)
      • Fix(multi-tenancy): fix live loader for case when namespace does not exist for data (#7505)

    🐎 Performance:

    • ⚡️ Opt(schema): Optimize populateSchema() by avoiding repeated lock acquisition (#8068)
    • 📜 Perf: Speed up parsing of a huge query with a lot of conditional mutations (#7871)
    • ⏪ Opt(Restore): Make restore map phase faster (#8038)
    • Opt(codec): return nil instead of a new bitmap (#7997)
    • Opt(cache): Use Ristretto to store posting lists (#7995)
    • Opt(rdf-output): Make RDF output generation concurrent (#7988)
    • Opt(recurse): Optimise recurse and bring range iterators from sroar (#7989)
    • ⏪ Opt(restore): Sort the buffer before spinning the writeToDisk goroutine (#7984)
    • ✅ Perf(sroar): Use latest sroar and add histogram in the sbs tool (#7982)
    • Opt(Alpha): Load schema and types using Stream framework (#7938) (#7940)
    • Opt(query): Use sroar in pb.List (#7864)
    • Opt(snapshot): use full table copy when streaming the entire data (#7870)
    • ⚡️ Opt(snapshot): Optimize snapshot by using sinceTs (#7826)
    • 🚚 Opt(predMove): iterate Phase I till there is major data to move (#7792)
    • Opt(dropPrefix): allow logical drop for deleting predicates and indexing (#7764)
    • ⚡️ Opt(txn commits): Optimize txns by passing Skiplists to Badger (#7777)
    • Opt(GraphQL): filter existence queries on GraphQL side instead of using @filter(type) (#7757)
    • 🚚 Opt(predMove): hot tablet move (#7703)
    • Opt(Backup): Make backups faster (#7680)
    • ⏪ Perf(restore): Implement map-reduce based restore (#7664)
    • 🏗 Opt(reindex): do not try building indices when inserting a new predicate (#7109)
    • Perf(txn): de-duplicate the context keys and predicates (#7478)
    • 🐎 perf(rollup): use NSplit API from sroar to improve rollup performance (#8092)

Previous changes from v21.03.2

  • 🛠 Fixed

    • GraphQL

      • Handle extend keyword for Queries and Mutations (#7923)

    • Core Dgraph

      • fix(Raft): Detect network partition when streaming (#7908)
      • fix(Raft): Reconnect via a redial in case of disconnection. (#7921)
      • fix(conn): JoinCluster loop should use latest conn (#7952)
      • fix(pool): use write lock when getting health info (#7967)
      • fix(acl): The Acl cache should be updated on restart and restore. (#7964)
      • fix(acl): filter out the results based on type (#7981)
      • fix(backup): Fix full backup request (#7934)
      • fix(live): quote the xid when doing upsert (#7999)
      • fix(export): Write temporary files for export to the t directory. (#7998)

    🔄 Changed

    • ⬆️ protobuf: upgrade golang/protobuf library v1.4.1 -> v1.5.2 (#7949)
    • 🌲 chore(raft): Log packets message less frequently. (#7913)

    ➕ Added

    • feat(acl): allow access to all the predicates using wildcard. (#7993)
    • feat(Multi-tenancy): Add namespaces field to state. (#7936)