All Versions
26
Latest Version
Avg Release Cycle
37 days
Latest Release
1248 days ago

Changelog History
Page 1

  • v8.0.0 Changes

    October 27, 2020

    πŸš€ This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

    Incompatible Changes

    πŸš€ The following PRs made changes to behavior that clients might be relying on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

    • πŸ”„ Change error code from 1203 to 1153 for three specific error conditions. #6630
      The three errors being changed:
      When a GRPC message coming back to the vtgate exceeds the configured size limit (configured via grpc_max_message_size)
      πŸ”§ If a write is attempting to make a change with a payload that is larger than the configured allowable size
      If a scatter query is attempting to collect and process too many rows in memory within the vtgate to construct a response
    • 0️⃣ Zero auto-increment mode: This enables inserting a 0 value (and not just NULL) into an auto-increment column to generate sequence values. This matches the default behavior of MySQL. #6749
    • 0️⃣ Turn off schema tracker by default #6712

    πŸ› Bugs Fixed

    VTGate / MySQL compatibility

    • πŸ›  Fix where clause in information schema with correct database name #6599
    • πŸ›  Fix DDL execution on reserved connection without in active transaction #6514
    • Fix allow passed table_schema in information_schema queries to vttablet if the keyspace is not found #6653
    • πŸ›  Fix information schema for non-existent schema #6667
    • πŸ›  Fix reserved connection in autocommit mode on DML #6748
    • πŸ›  Fix dbname that needs escaping that we broke in fce4cfd #6794
    • MySQL CLI, mysql --ssl does not work in MySQL 8.0 #5556
    • OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') #6208
    • Fix support for show tables where tables_in_db to use keyspace name, not underlying db name #6446
    • Common empty result set query on MySQL returns a row in Vitess #6663
    • Operator precedence issue #6734
    • Table names out-of-order between replica and primary #6738
    • βͺ Close Idle reserved connections and Rollback Idle transactions #6552
    • πŸ›  Minor Vindex fixes: #6526
    • VTgate does not report autocommit system variable and SERVER_STATUS_IN_TRANS flag correctly #5825
    • 0️⃣ MariaDB JDBC Connector fails to connect to vtgate with default options #5851
    • VTgate panics if there are no vttablets and vtctlds #6117
    • βœ‚ Remove shard session held in vtgate for reserved connection on connection failure #6522
    • Session variable does not work as expected in 7.0 #6559
    • Retry should not happen when in dedicated connection #6523
    • Correctly report AUTOCOMMIT status in network packets #6551
    • βͺ Close Idle reserved connections and Rollback Idle transactions #6552
    • Reset Session for Reserved Connection Query Failure #6673
    • Errors in multi-statement queries not fatal to statement sequence #6747 #6808
    • πŸ”’ Lock wait timeout on DELETE from table using lookup index #6698
    • Application couldn't connect to external db giving error tabletserver.go:1522] Code: INVALID_ARGUMENT syntax error at position 33 near ':vtg2' (CallerID: gnocchi) : Sql: "select convert(:vtg1, CHAR(:vtg2)) as anon_1 from dual", #6466
    • The client got the wrong database name #6753
    • πŸ‘» DBAPIError exception wrapped from (pymysql.err.InternalError) (1105, u'vtgate: http://gnocchi-gnocchizone-vtgate:15000/: reserved connections are not supported on old gen gateway') #6577
    • πŸ‘ Provide support for β€œSELECT INTO OUTFILE S3” #6811
    • πŸ›  Fix error around breaking of multi statements #6824
    • πŸ‘Œ Support SHOW TABLE STATUS FROM db type queries in vtgate #6354
    • πŸ‘‰ SHOW FULL FIELDS doesn't follow routing rules #6803
    • VDiff/Tablet Picker: fix picker retry logic to handle canceled context #6954
    • Healthcheck: Use isIncluded correctly to fix replica/rdonly routing bug #6922

    Other

    • VReplication tablet_picker should keep trying to find a tablet until context expires #6546
    • ⚑️ VReplication improve reverse workflow: update cells/tablet_types #6556
    • VDiff: fix panic for tables with a unicode_loose_md5 vindex #6640
    • πŸ’» Vtctld UI: Add mysqld port display back to vtctld web UI #6598 #6597
    • πŸ–¨ Vtctld: Fix workflow CopyState fetching (and thus printing). #6657
    • πŸ›  Fix unsaved event regression #6710
    • πŸ”„ Change tracker to use allprivs instead of dba #6720
    • πŸ›  Fixes long wait filter keyspace #6721
    • s3 ListBackups doesn’t work from root directory #6732
    • Vtctld: ExecuteFetchAsDba and ExecuteFetchAsAllPrivs do not escape db names. #6545
    • Can't switch on restore_from_backup on existing shard with no tables #4896
    • πŸ”¨ Refactor conn.go #6818

    Functionality Added or Changed

    VTGate / MySQL compatibility

    • βž• Add support for DELETE IGNORE #6722
    • Only take on simple dual queries in the vtgate #6666
    • πŸ‘Œ Support mycli access to vtgate #4365
    • Add support for unicode_loose_xxhash #6457
    • Provide virtual information_schema #5394
    • information_schema.tables_schema comparison with dynamic value no longer works #6827
    • Mysqldump: Allow setting multiple variables in one statement #5401
    • πŸ”’ Block lock function usage other than dual table #6470
    • πŸ”’ Lock Session Support #6517
    • πŸ‘‰ Make sure lookup vindexes are queryable inside transaction #6499
    • Add unicode_loose_xxhash Vindex type #6549
    • πŸ‘Œ Support vindex update or delete when destination is provided by client in dml query #6554
    • πŸ‘‰ Make sure to handle EXPLAIN on tablets #6579
    • Rewrite SHOW TABLES #6615
    • πŸ‘‰ Make offset work in OLAP mode #6655
    • Support sql_calc_found_rows with limit in sharded keyspace #6680
    • πŸ”’ Lock shard session heartbeat #6683
    • πŸ—„ Mysql -ssl tag is deprecated in the later versions of mysql #6682
    • πŸ‘‰ Use statement support in olap mode #6692
    • βœ‚ Delete row using consistent lookup vindex in where clause #6700
    • βž• Add vtexplain support for non-evenly sharded keyspaces #6705
    • πŸ‘‰ Make sure dual queries set found_rows correctly #6718
    • βž• Add if not exists support to database ddl in parsing #6724
    • πŸ‘ Allow Update of same vindex using same vindex lookup on consistent lookup vindex #6736
    • ⚑️ Update Vindex only on changes #6737
    • Add support for SHOW (DATABASES|VITESS_SHARDS|VITESS_TABLETS) LIKE #6750
    • πŸ‘ 'CASCADE' support #6743
    • Mask database name #6791
    • πŸ‘‰ Make sure to backtick schema names properly #6550
    • Transaction Limiter: fix quota leak if Begin() errors or if the tx killer runs #6731
    • πŸ‘ Allow Load Data From S3 #6823

    OLAP Functioanlity

    • πŸ‘ Allow switching between OLAP and OLTP #6691
    • πŸ‘‰ Use statement support in olap mode #6692
    • πŸ›  Fix error handling in olap mode #6940

    πŸ‘ Set Statement Support

    Set statement support is added in Vitess. There are some system variables which are disabled by default and can be enabled using flag -enable_system_settings on VTGate.These system variables are set on the backing MySQL instance, and will force the connection to be dedicated instead of part of the connection pool.

    • 0️⃣ Disabled passthrough system variables by default. #6859
    • πŸ‘ Allow switching workload between OLAP and OLTP #4086 #6691
    • πŸ‘Œ Support for multiple session set statement syntax #6508 #6495 #6494
    • πŸ‘ Allow enumeration in set system variables #6493
    • πŸ– Handle boolean settings better #6501
    • πŸ”¨ Set statements refactored to use plan building #6487
    • βž• Add more system settings #6486
    • Evaluate system variables #6708

    VReplication

    • VReplication: _vt.vreplication source column VARBINARY->BLOB, allowing more tables to participate in a workflow #6421
    • 🐎 VReplication and vstreamer performance and error metrics #6519, #6132
    • VReplication: Find collations for char primary keys and cast comparison during catchup #6568
    • Varchar primary keys with collations can cause vreplication to miss rows #6622
    • VReplication: Timestamp w/ Heartbeat #6635
    • 🌲 VReplication: Improve few error/log messages #6669
    • VReplication: handle SET statements in the binlog stream #6772
    • πŸ‘· VReplication: e2e Tests added to CI #6730
    • πŸ”Š VDiff logs: additional logging around vdiff/vstreamer for observability #6684
    • VStream: experimental POC of mysql protocol #6670
    • πŸ‘ Materialize: Support rollups using a literal value as a column #6733
    • Materialization: Ignore non-participating tables that are renamed/deleted #6778
    • Materialize: add ability for maintaining accurate table-level aggregates #6767
    • 0️⃣ Schema Tracker: Turn off schema tracker by default #6712 (breaking change)
    • Workflow: Add List-All Command #6533
    • πŸ“‡ Workflow: List -> Show, and Expand Metadata #6544
    • πŸ”¨ Refactor: Cleanup Noisy Workflow Logging #6744
    • Schema tracker: Change tracker to use the allprivs user instead of dba #6720
    • vtctl: Adding VExec and Workflow commands #6410
    • vtctl Workflow cmd: don't expect all shards to have workflow streams #6576
    • Reshard: Sort table definitions before comparing them #6765 #6738
    • πŸ”¦ Expose a shard range calculator as a vtctl command GenerateShardRanges #6751 #6752
    • VStream: Field event now has all column attributes #6525
    • VStream events do not populate field "Flags" on FIELD events #5914
    • πŸ‘― RFC: VReplication based SplitClone #4604
    • πŸ‘ Experimental: Mysql Protocol support for VStream #6675
    • VReplication: Add metrics around vstreamer and vreplication metrics #6787
    • πŸ›  Fix issues with missing tables in vstreams #6789
    • βž• Add validations and logs to fix common issues faced during MoveTables/Reshard #6814
    • πŸ‘ Allow multiple blacklists for master #6816

    VTtablet

    • VTtablet throttling #6668
      πŸ“„ https://vitess.io/docs/reference/features/tablet-throttler/
    • TabletManager: publish displayState #6648
    • TabletManager: call setReadOnly inside ChangeTabletType #6762
    • TabletManager: change how SetReadOnly is called to avoid errors from externally managed tablets #6786
    • TabletManager: call SetReadOnly inside ChangeTabletType #6804
    • vttablet: Open and Close healthStreamer #6515
    • vttablet: create database if not present #6490
    • βͺ Restore: do not change tablet type to RESTORE if not actually performing a restore #6679
    • βͺ Restore: checkNoDB should not require tables to be present #6695
    • Heartbeat: Additional metrics/stats (lag histogram) #6528 #6634
    • VTtablet throttler feature flag: -enable-lag-throttler #6815
    • VTtablet two-phase commit design doc #6498

    VTorc - Orchestrator

    Following PRs are experimental version of Vitess-native Orchestrator 'vtorc' is ready for users to try out.

    • Orchestrator: initial import of Orchestrator #6582
    • Orchestrator: vitess mode #6613
    • Orchestrator: more changes #6714
    • ⏱ Orchestrator: use contexts with timeout for remote operations #6780
    • πŸ›  Orchestrator: fixed orchestrator govet errors #6781
    • Orchestrator: Add clusters_to_watch flag to. Defaults to all clusters. #6821

    Point in Time Recovery

    • PITR: Fix deadlock in ff code #6774
    • βž• Add flags to allow PITR binlog client TLS support #6775

    Other

    • βž• Add realtime health stats to vtctld /api/keyspace/ks/tablets API #6569
    • πŸ’» vtctld UI: Fix logic for displaying vindexes #6603
    • πŸ“„ Online schema changes #6547
    • πŸ“„ Table lifecycle management #6719, docs
    • Online-DDL: migration uses low priority throttling #6837
    • πŸ”¨ Emergency Reparent Refactor #6449
      • Make emergency reparents more robust. #6206
    • βœ… Reparent test failures #6706
    • ⚑️ Update user-facing terminology in the vtctld2 UI #6481
    • 🚀 GetSchema: Batch/parallel access to underlying mysqld for lower latency #6491
    • Prometheus interface: vtgate_buffer_* metrics cardinality #6326
    • πŸ’» Show more vindex details in vtctld /app/schema UI #6450
    • ⚑️ GRPC: update enforcement policy on server to match the one from the client #6629

    Examples / Tutorials

    • region_sharding: working resharding example #6565
    • 🐳 vtcompose/docker-compose: fix InitShardMaster #6584
    • 🐳 [docker] fix tablet hostname #6509

    πŸ“š Documentation

    • Replace more uses of slave with replica/replication #6520
    • ⚑️ Update some option docstrings for accuracy. #6590
    • Minor addition to Materialize help text. #6627
    • βœ‚ Remove auto-generation comment #6728
    • βž• add skip flag that can skip comparing source & destination schema when run splitdiff #6477
    • πŸ›  Fix and clarify some help options #6817

    πŸ— Build Environment Changes

    • βž• Address #6631 by reducing optimization level #6632
    • βž• Add front-end build tooling + fix build error #6473
    • Replace gogo proto with golang #6571
    • πŸ‘Œ Improve make proto #6580
    • βœ‚ Remove chromium hard-coding #6636
    • πŸ— Docker - allow BUILD_NUMBER variable override #6628
    • 🐳 Docker/lite: Fix package URLs for ubi images. #6704
    • ⚑️ Helm & docker/k8s update #6723
    • πŸ›  Fix build errors in ./go/vt/vtgate for go1.15 #6800
    • πŸ›  Fix build error on vcursor_impl tests on newer versions of go #6799
    • πŸš€ [java] bump java version to 8.0.0-SNAPSHOT for next release #6478
    • πŸ— Helm/docker: fix Docker builds + tag 7.0.2 #6773
    • πŸš€ Docker: Upgrade to Debian Buster (release-8.0) #6888
    • 🐳 Docker: Pin mysql 5.7 and 8.0 versions for xtrabackup compatibility
    • πŸš€ Docker: Revert "Docker - upgrade to Debian Buster (release-8.0)" #6929
    • Zookeper: Download zookeeper 3.4.14 from archive site #6867

    Functionality Neutral Changes

    • 🚚 Routing cleanup: remove routeOptions #6531
    • βœ… PITR: testcase #6594
    • Replaced Error with a warning in case parsing of VT_SPAN_CONTEXT fails #6766
    • πŸ”¨ Reparent tests refactoring: setup/teardown for each test, reduce cut/paste, improve readability #6726
    • βœ… Decider: endtoend test infrastructure + tests #6770
    • βœ… Orchestrator: add more test cases #6801
    • βœ… Orchestrator: don't set DbName for tests that don't need it. #6812
    • βž• Add unit test case to improve test coverage for go/sqltypes/named_result.go #6672
    • βœ… End to end: deflake sharding tests #6715
    • 🚚 Workflow test: fix flaky test. remove obsolete code #6694
    • βž• Add unit test for func Proto3ValuesEqual. #6649
    • βœ… Ensure tests for discoverygateway #6536
    • 🌲 Convert syslog events to normal log events #6511
    • βž• Add diagnostic logging to healthcheck. #6512
    • Healthcheck should receive healthcheck updates from all tablets in cells_to_watch #6852
    • Named columns and convenience SQL query interface #6543
    • βœ… Check http response code in vtctld API tests #6570
    • πŸ”’ Reverting package-lock.json to the one pre-PR #6603 #6611
    • βœ‚ Removed sqlparser.preview to set logstats stmtType and use plan.type #6637
    • AST struct name rewording #6642
    • πŸ›  Fix: Remove SetMaster Query Expectation from ERS Test #6617
    • πŸ›  Fix path for vttablet query log in end to end test #6656
    • πŸ›  Fixes error log #6496
    • βž• Add diagnostic logging to healthcheck. #6535
    • Try to install apt keys from list of different keyservers #6674
    • Trivial copy-pasta comment fixup #6592
    • βœ‚ Remove long-unused memcache and cacheservice #6596
    • πŸ›  Fix MoveTables docstring; was not valid JSON. #6606
    • βœ… Tablet gateway: unit tests #6608
    • Tablet gateway doesn't see all tablets in cells from cells_to_watch #6846
    • βž• Add Sleep to ERS Unit Test #6610
    • Strings to enum #6729
    • βœ‚ Remove syslog dependency #3563
    • Operator precedence must take associativity into consideration #6758
    • βž• Add optional ignore to delete statement #6802
    • πŸ›° Invalid trace payload causes vitess to fail the request and return an error #6759
    • 🌲 Vttablet: minor logging fix #6564
    • 🌲 Vttablet: add more logging to checkMastership #6618
  • v8.0.0-test

    October 26, 2020
  • v8.0.0-rc1 Changes

    October 07, 2020

    πŸš€ This release complies with VEP-3 which removes the upgrade order requirement. Components can be upgraded in any order. It is recommended that the upgrade order should still be followed if possible, except to canary test the new version of VTGate before upgrading the rest of the components.

    Incompatible Changes

    πŸš€ The following PRs made changes to behavior that clients might be relying on. They should be reviewed carefully so that client code can be changed in concert with a Vitess release deployment.

    • πŸ”„ Change error code from 1203 to 1153 for three specific error conditions. #6630
      The three errors being changed:
      When a GRPC message coming back to the vtgate exceeds the configured size limit (configured via grpc_max_message_size)
      πŸ”§ If a write is attempting to make a change with a payload that is larger than the configured allowable size
      If a scatter query is attempting to collect and process too many rows in memory within the vtgate to construct a response
    • 0️⃣ Zero auto-increment mode: This enables inserting a 0 value (and not just NULL) into an auto-increment column to generate sequence values. This matches the default behavior of MySQL. #6749
    • 0️⃣ Turn off schema tracker by default #6712

    πŸ› Bugs Fixed

    VTGate / MySQL compatibility

    • πŸ›  Fix where clause in information schema with correct database name #6599
    • πŸ›  Fix DDL execution on reserved connection without in active transaction #6514
    • Fix allow passed table_schema in information_schema queries to vttablet if the keyspace is not found #6653
    • πŸ›  Fix information schema for non-existent schema #6667
    • πŸ›  Fix reserved connection in autocommit mode on DML #6748
    • πŸ›  Fix dbname that needs escaping that we broke in fce4cfd #6794
    • MySQL CLI, mysql --ssl does not work in MySQL 8.0 #5556
    • OperationalError: (_mysql_exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query') #6208
    • Fix support for show tables where tables_in_db to use keyspace name, not underlying db name #6446
    • Common empty result set query on MySQL returns a row in Vitess #6663
    • Operator precedence issue #6734
    • Table names out-of-order between replica and primary #6738
    • βͺ Close Idle reserved connections and Rollback Idle transactions #6552
    • πŸ›  Minor Vindex fixes: #6526
    • VTgate does not report autocommit system variable and SERVER_STATUS_IN_TRANS flag correctly #5825
    • 0️⃣ MariaDB JDBC Connector fails to connect to vtgate with default options #5851
    • VTgate panics if there are no vttablets and vtctlds #6117
    • βœ‚ Remove shard session held in vtgate for reserved connection on connection failure #6522
    • Session variable does not work as expected in 7.0 #6559
    • Retry should not happen when in dedicated connection #6523
    • Correctly report AUTOCOMMIT status in network packets #6551
    • βͺ Close Idle reserved connections and Rollback Idle transactions #6552
    • Reset Session for Reserved Connection Query Failure #6673
    • Errors in multi-statement queries not fatal to statement sequence #6747 #6808
    • πŸ”’ Lock wait timeout on DELETE from table using lookup index #6698
    • Application couldn't connect to external db giving error tabletserver.go:1522] Code: INVALID_ARGUMENT syntax error at position 33 near ':vtg2' (CallerID: gnocchi) : Sql: "select convert(:vtg1, CHAR(:vtg2)) as anon_1 from dual", #6466
    • The client got the wrong database name #6753
    • πŸ‘» DBAPIError exception wrapped from (pymysql.err.InternalError) (1105, u'vtgate: http://gnocchi-gnocchizone-vtgate:15000/: reserved connections are not supported on old gen gateway') #6577
    • πŸ‘ Provide support for β€œSELECT INTO OUTFILE S3” #6811

    Other

    • VReplication tablet_picker should keep trying to find a tablet until context expires #6546
    • ⚑️ VReplication improve reverse workflow: update cells/tablet_types #6556
    • VDiff: fix panic for tables with a unicode_loose_md5 vindex #6640
    • πŸ’» Vtctld UI: Add mysqld port display back to vtctld web UI #6598 #6597
    • πŸ–¨ Vtctld: Fix workflow CopyState fetching (and thus printing). #6657
    • πŸ›  Fix unsaved event regression #6710
    • πŸ”„ Change tracker to use allprivs instead of dba #6720
    • πŸ›  Fixes long wait filter keyspace #6721
    • s3 ListBackups doesn’t work from root directory #6732
    • Vtctld: ExecuteFetchAsDba and ExecuteFetchAsAllPrivs do not escape db names. #6545
    • Can't switch on restore_from_backup on existing shard with no tables #4896
    • πŸ”¨ Refactor conn.go #6818

    Functionality Added or Changed

    VTGate / MySQL compatibility

    πŸ†• New functionality to set workload = 'olap'; can not repeat, but set workload = 'oltp' can; #4086

    βž• Add support for DELETE IGNORE #6722

    Only take on simple dual queries in the vtgate #6666

    πŸ‘Œ Support mycli access to vtgate #4365

    Add support for unicode_loose_xxhash #6457

    Provide virtual information_schema #5394

    Mysqldump: Allow setting multiple variables in one statement #5401

    SET planning #6487

    πŸ‘ Allow enumeration in set system variables #6493

    πŸ‘ Allow set to have scope for each setting #6494

    πŸ– HandleSet in executor to handle multiple settings in single set statement #6495

    0️⃣ Passthrough system settings flag: enabled by default #6586

    πŸ– Handle boolean settings better #6501

    βž• Add more system settings #6486

    πŸ”’ Block lock function usage other than dual table #6470

    πŸ”’ Lock Session Support #6517

    πŸ‘‰ Make sure lookup vindexes are queryable inside transaction #6499

    Add unicode_loose_xxhash Vindex type #6549

    πŸ‘Œ Support vindex update or delete when destination is provided by client in dml query #6554

    πŸ‘‰ Make sure to handle EXPLAIN on tablets #6579

    Rewrite SHOW TABLES #6615

    πŸ‘‰ Make offset work in OLAP mode #6655

    Support sql_calc_found_rows with limit in sharded keyspace #6680

    πŸ”’ Lock shard session heartbeat #6683

    πŸ—„ Mysql -ssl tag is deprecated in the later versions of mysql #6682

    πŸ‘ Allow switching between OLAP and OLTP #6691

    πŸ‘‰ Use statement support in olap mode #6692

    βœ‚ Delete row using consistent lookup vindex in where clause #6700

    βž• Add vtexplain support for non-evenly sharded keyspaces #6705

    Evaluate system variables #6708

    πŸ‘‰ Make sure dual queries set found_rows correctly #6718

    βž• Add if not exists support to database ddl in parsing #6724

    πŸ‘ Allow Update of same vindex using same vindex lookup on consistent lookup vindex #6736

    ⚑️ Update Vindex only on changes #6737

    Add support for SHOW (DATABASES|VITESS_SHARDS|VITESS_TABLETS) LIKE #6750

    πŸ‘Œ Support multiple session set statement syntax #6508

    πŸ‘ 'CASCADE' support #6743

    Mask database name #6791

    πŸ‘‰ Make sure to backtick schema names properly #6550

    Transaction Limiter: fix quota leak if Begin() errors or if the tx killer runs #6731

    OLAP Functioanlity

    • πŸ‘ Allow switching between OLAP and OLTP #6691
    • πŸ‘‰ Use statement support in olap mode #6692

    System Variables

    • πŸ‘ Allow enumeration in set system variables #6493
    • πŸ‘ Allow set to have scope for each setting #6494
    • πŸ– HandleSet in executor to handle multiple settings in single set statement #6495

    VReplication

    • VReplication: _vt.vreplication source column VARBINARY->BLOB, allowing more tables to participate in a workflow #6421
    • 🐎 VReplication and vstreamer performance and error metrics #6519, #6132
    • VReplication: Find collations for char primary keys and cast comparison during catchup #6568
    • Varchar primary keys with collations can cause vreplication to miss rows #6622
    • VReplication: Timestamp w/ Heartbeat #6635
    • 🌲 VReplication: Improve few error/log messages #6669
    • VReplication: handle SET statements in the binlog stream #6772
    • πŸ‘· VReplication: e2e Tests added to CI #6730
    • πŸ”Š VDiff logs: additional logging around vdiff/vstreamer for observability #6684
    • VStream: experimental POC of mysql protocol #6670
    • πŸ‘ Materialize: Support rollups using a literal value as a column #6733
    • Materialization: Ignore non-participating tables that are renamed/deleted #6778
    • Materialize: add ability for maintaining accurate table-level aggregates #6767
    • 0️⃣ Schema Tracker: Turn off schema tracker by default #6712 (breaking change)
    • Workflow: Add List-All Command #6533
    • πŸ“‡ Workflow: List -> Show, and Expand Metadata #6544
    • πŸ”¨ Refactor: Cleanup Noisy Workflow Logging #6744
    • Schema tracker: Change tracker to use the allprivs user instead of dba #6720
    • vtctl: Adding VExec and Workflow commands #6410
    • vtctl Workflow cmd: don't expect all shards to have workflow streams #6576
    • Reshard: Sort table definitions before comparing them #6765 #6738
    • πŸ”¦ Expose a shard range calculator as a vtctl command GenerateShardRanges #6751 #6752
    • VStream: Field event now has all column attributes #6525
    • VStream events do not populate field "Flags" on FIELD events #5914
    • πŸ‘― RFC: VReplication based SplitClone #4604
    • πŸ‘ Experimental: Mysql Protocol support for VStream #6675
    • VReplication: Add metrics around vstreamer and vreplication metrics #6787
    • πŸ›  Fix issues with missing tables in vstreams #6789
    • βž• Add validations and logs to fix common issues faced during MoveTables/Reshard #6814

    VTtablet

    • VTtablet throttling #6668
    • TabletManager: publish displayState #6648
    • TabletManager: call setReadOnly inside ChangeTabletType #6762
    • TabletManager: change how SetReadOnly is called to avoid errors from externally managed tablets #6786
    • TabletManager: call SetReadOnly inside ChangeTabletType #6804
    • vttablet: Open and Close healthStreamer #6515
    • vttablet: create database if not present #6490
    • βͺ Restore: do not change tablet type to RESTORE if not actually performing a restore #6679
    • βͺ Restore: checkNoDB should not require tables to be present #6695
    • Heartbeat: Additional metrics/stats (lag histogram) #6528 #6634
    • VTtablet throttler feature flag: -enable-lag-throttler #6815

    VTorc - Orchestrator

    • Orchestrator: initial import of Orchestrator #6582
    • Orchestrator: vitess mode #6613
    • Orchestrator: more changes #6714
    • ⏱ Orchestrator: use contexts with timeout for remote operations #6780
    • πŸ›  Orchestrator: fixed orchestrator govet errors #6781
    • Orchestrator: Add clusters_to_watch flag to. Defaults to all clusters. #6821

    Point in Time Recovery

    • PITR: Fix deadlock in ff code #6774
    • βž• Add flags to allow PITR binlog client TLS support #6775

    Other

    • βž• Add realtime health stats to vtctld /api/keyspace/ks/tablets API #6569
    • πŸ’» vtctld UI: Fix logic for displaying vindexes #6603
    • Online schema changes #6547
    • πŸ”¨ Emergency Reparent Refactor #6449
      • Make emergency reparents more robust. #6206
    • ⚑️ Update user-facing terminology in the vtctld2 UI #6481
    • 🚀 GetSchema: Batch/parallel access to underlying mysqld for lower latency #6491
    • Prometheus interface: vtgate_buffer_* metrics cardinality #6326
    • πŸ’» Show more vindex details in vtctld /app/schema UI #6450
    • ⚑️ GRPC: update enforcement policy on server to match the one from the client #6629

    Examples / Tutorials

    • region_sharding: working resharding example #6565
    • 🐳 vtcompose/docker-compose: fix InitShardMaster #6584
    • 🐳 [docker] fix tablet hostname #6509

    πŸ“š Documentation

    • Replace more uses of slave with replica/replication #6520
    • ⚑️ Update some option docstrings for accuracy. #6590
    • Minor addition to Materialize help text. #6627
    • βœ‚ Remove auto-generation comment #6728
    • βž• Add skip flag that can skip comparing source & destination schema when run splitdiff #6477
    • πŸ›  Fix and clarify some help options #6817

    πŸ— Build Environment Changes

    • βž• Address #6631 by reducing optimization level #6632
    • βž• Add front-end build tooling + fix build error #6473
    • Replace gogo proto with golang #6571
    • πŸ‘Œ Improve make proto #6580
    • βœ‚ Remove chromium hard-coding #6636
    • πŸ— Docker - allow BUILD_NUMBER variable override #6628
    • 🐳 Docker/lite: Fix package URLs for ubi images. #6704
    • ⚑️ Helm & docker/k8s update #6723
    • πŸ›  Fix build errors in ./go/vt/vtgate for go1.15 #6800
    • πŸ›  Fix build error on vcursor_impl tests on newer versions of go #6799
    • πŸš€ [java] bump java version to 8.0.0-SNAPSHOT for next release #6478
    • πŸ— Helm/docker: fix Docker builds + tag 7.0.2 #6773

    Functionality Neutral Changes

    • 🚚 Routing cleanup: remove routeOptions #6531
    • βœ… PITR: testcase #6594
    • Replaced Error with a warning in case parsing of VT_SPAN_CONTEXT fails #6766
    • πŸ”¨ Reparent tests refactoring: setup/teardown for each test, reduce cut/paste, improve readability #6726
    • βœ… Decider: endtoend test infrastructure + tests #6770
    • βœ… Orchestrator: add more test cases #6801
    • βœ… Orchestrator: don't set DbName for tests that don't need it. #6812
    • βž• Add unit test case to improve test coverage for go/sqltypes/named_result.go #6672
    • βœ… End to end: deflake sharding tests #6715
    • 🚚 Workflow test: fix flaky test. remove obsolete code #6694
    • βž• Add unit test for func Proto3ValuesEqual. #6649
    • βœ… Ensure tests for discoverygateway #6536
    • 🌲 Convert syslog events to normal log events #6511
    • βž• Add diagnostic logging to healthcheck. #6512
    • Named columns and convenience SQL query interface #6543
    • βœ… Check http response code in vtctld API tests #6570
    • πŸ”’ Reverting package-lock.json to the one pre-PR #6603 #6611
    • βœ‚ Removed sqlparser.preview to set logstats stmtType and use plan.type #6637
    • AST struct name rewording #6642
    • πŸ›  Fix: Remove SetMaster Query Expectation from ERS Test #6617
    • πŸ›  Fix path for vttablet query log in end to end test #6656
    • πŸ›  Fixes error log #6496
    • βž• Add diagnostic logging to healthcheck. #6535
    • Try to install apt keys from list of different keyservers #6674
    • Trivial copy-pasta comment fixup #6592
    • βœ‚ Remove long-unused memcache and cacheservice #6596
    • πŸ›  Fix MoveTables docstring; was not valid JSON. #6606
    • βœ… Tablet gateway: unit tests #6608
    • βž• Add Sleep to ERS Unit Test #6610
    • Strings to enum #6729
    • βœ‚ Remove syslog dependency #3563
    • Operator precedence must take associativity into consideration #6758
    • βž• Add optional ignore to delete statement #6802
    • πŸ›° Invalid trace payload causes vitess to fail the request and return an error #6759
    • 🌲 Vttablet: minor logging fix #6564
    • 🌲 Vttablet: add more logging to checkMastership #6618
  • v7.0.3 Changes

    October 22, 2020

    πŸš€ This update fixes several regressions that were deemed significant enough to be backported to the release branch.

    πŸ› Bugs Fixed

    • Vtgate : Should receive healthcheck updates from all tablets in cells_to_watch #6857
    • Vtgate : Fix error around breaking of multistatements #6824
    • Vtgate : Fix bug in REPLICA/RDONLY routing #6904
    • Vttablet : checkNoDB should not require tables to be present #6788

    Known Issues

    πŸš€ This release includes MySQL 5.7.32 and Percona Xtrabackup 2.4.20 which have a known incompatibility. Percona is working on a resolution.

    The issue will manifest if a user using 5.7.32:
    * Creates a new compressed table.
    * Runs any ALTER TABLE that uses the algorithm copy (table rebuild) on a compressed table.
    

    πŸ— Build Changes

    • πŸ— Build : Download zookeeper 3.4.14 from archive site #6868
  • v7.0.2 Changes

    September 22, 2020

    πŸš€ This update fixes several regressions that were deemed significant enough to be backported to the release branch.

    πŸ› Bugs Fixed

    • vtgate : Operator precedence must take associativity into consideration #6764
    • vtgate: Fix reserved connection in autocommit mode on DML #6748
    • VDiff: fix panic for tables with a unicode_loose_md5 vindex #6745
    • 0️⃣ vttablet: Turn off schema tracker by default #6746
    • βͺ restore: do not change tablet type to RESTORE if not actually performing a restore #6687
    • vttablet : Tablet manager init should publish displayState #6686
    • 🌲 vttablet: Tablet manager add more logging to checkMastership #6685
    • vttablet : Fix replmanager deadlock #6634
  • v7.0.1 Changes

    August 25, 2020

    πŸš€ This update fixes several regressions that were deemed significant enough to be backported to the release branch.
    0️⃣ In addition, the experimental feature to set system variables is now behind a flag that disables it by default.

    πŸ”§ Configuration Changes

    • Vtgate: added flag enable_system_settings to enable passthrough system settings, defaults to false #6562

    πŸ› Bugs Fixed

    • Vreplication: Materialize should only get schema from source tablets if target is missing tables #6623
    • πŸ’» Vtctld UI: mysql port is not displayed in tablet list #6597 #6600
    • Vtgate: transaction killer was closing reserved connection after the specified time. Only idle connections should be closed. Idle transactions should be rolled back. #6583
    • 🐳 Vtcompose: docker-compose example was broken by tabletmanager changes #6587
    • Vreplication: tablet_picker should respect canceled context. Also, keep trying to find a tablet until context expires. #6567
    • ⚑️ Vtgate: vindex update/delete by destination #6541 #6572
    • Vtgate: some usages of EXPLAIN were broken #6574 #6581
    • Vtgate: handle multiple settings in single SET statement #6504
    • Vtgate: cleanup shard session for reserved connection on failure #6563
    • Vtgate: if executing DDL on reserved connection, start transaction if needed #6555
    • Vtgate: make sure lookup vindexes use same transaction if one is open #6505
    • 🌲 Healthcheck: add diagnostic logging, do not add tablet if it is shutting down #6512
  • v7.0.0 Changes

    July 28, 2020

    Incompatible Changes

    πŸš€ This release includes the following changes which may result in incompatibilities when upgrading from a previous release. It is important that Vitess components are upgraded in the recommended order. This will change in the next release as documented in VEP-3.

    πŸ‘€ VTGate: The default gateway (-gateway_implementation flag value) is now "tabletgateway". The old default was β€œdiscoverygateway”. To use the old gateway, specify the flag in the command line. Anyone already using the flag will see no change; to use the new gateway, remove the -gateway_implementation flag. The new gateway is simpler and uses a new healthcheck module with no loss of functionality. #6155

    ⚠ VTGate: VTGate incorrectly returned Warning Code field name as Type. This has now been fixed. Code or scripts that depend on the incorrect behavior might break. #6437

    🚚 VReplication: The "-cell" parameter to MoveTables has been renamed to β€œ-cells”, since it now accepts multiple comma-separated cells or cellsAliases. The parameter specifies which cells can provide tablets as the source for vreplication streams. #6456

    VTGate: The possibility of sending OpenTracing trace spans through query comments has been changed. The old way of doing it was severely broken, so our assumption is that no-one has successfully been using this feature. Still, this is an API change, and we are reporting it as a breaking change. #6463

    πŸ”¨ VTTablet has been refactored to be generally more responsive to state changes and its environment: #6131 #6139 #6241 #6263 #6311 #6345 #6348 #6384 #6396 #6432 #6461

    Some initialization parameters are now mandatory: init_keyspace, init_shard and init_tablet_type. The flow where you could issue a vtctlclient InitTablet and then invoke vttablet without those init parameters is no longer supported.

    ⚑️ Consequently vtctl's InitTablet and UpdateTabletAddrs are deprecated.

    demote_master_type is deprecated: Now that init_tablet_type is mandatory, that value can act as the demote_master_type. There is no need for a separate flag.

    ⚑️ VTTablet will publish the discovered MySQL port on startup, and will not update it during the life of the process. There is no known use case where a MySQL port is expected to change. So, there is no need to keep polling MySQL for its port.

    Decouple enable_replication_watcher from disable_active_reparents. Currently, vttablet attempts to fix replication only if enable_replication_watcher is true and disable_active_reparents is false. This behavior has changed to depend only on the disable_active_reparents flag.

    Transitioning from a serving type to SPARE used to go through a brief lameduck period specified by the serving_state_grace_period (default 0). This has been removed. The transition will now happen immediately even if a grace period is specified.

    πŸš€ It was possible to enable health reporting and heartbeat at the same time. Performing the two at the same time is not useful. As of this release, enabling heartbeat automatically disables the polling mode of checking for replica health.

    🚚 The ACL rules used to be reported on the vttablet status page. The design was using an incorrect framework. This has been removed.

    🌐 Build: make targets "build_web" and β€œembed_static” have been removed. Instead there are new targets β€œweb_bootstrap”, β€œweb_build” and β€œweb_start” #6473

    πŸ—„ Deprecations

    πŸš€ The following vtctl commands have been deprecated. Deprecated commands will continue to work until the next release (8.0). #6428 #6345 #6461

    StartSlave -> Replaced with StartReplication

    StopSlave -> Replaced with StopReplication

    πŸ”„ ChangeSlaveType -> Replaced with ChangeTabletType

    InitTablet

    ⚑️ UpdateTabletAddrs

    Various vtctl commands had a flag called "-wait_slave_timeout". It has been replaced with β€œ-wait_replicas_timeout”. Scripts using the old flag will continue to work until the next major release (8.0). #6428

    πŸ—„ Helm charts: The current helm charts have not been keeping up with the forward progress in vitess, and there has been no maintainer for it. Additionally, the operator has pulled ahead of what the helm charts could do. The operator can also be functionally more versatile than the helm charts because custom code can be written to manage complex workflows. To maintain focus and velocity in the project, we are deprecating the helm charts. #6439

    πŸš€ Consul: The usage of Consul has been problematic for vitess users. The main difficulty comes from the fact that it's non-trivial for a multi-cell deployment. Due to this, we want to discourage the use of consul as a topo server, and are deprecating support for it. #6440

    πŸ› Bugs Fixed

    VTGate / MySQL compatibility

    πŸ– Handle non-keyword tokens FORMAT, TREE and VITESS. (This was broken by #6110) #6445

    When using user-defined variables that have not been initialized first, a non-fatal panic occurs and the connection is dropped. #6294 #6308

    Setting system variable panics vtgate. #6162 #6163

    regression: USE does not work from MySQL client. #6153 #6157

    πŸ‘‰ USE does not return an error if a keyspace doesn't exist. #3060 #6157

    Dotted table names are not escaped properly in lookup vindex flow. #6142 #6145

    Produce correct error code (1105 instead of 1062) when inserting duplicates into consistent_lookup vindex. #6144

    Rewriting expressions fail when used as function parameters. #6281

    ⚑️ Rollback after error clobbers updates. #6285

    Queries should not be wrapped into transactions if autocommit=1 is set. #6264

    Prepared statements with bind variables in column specification produce β€˜missing bind var’ error. #6287 #6298

    πŸ“œ Prepared statement queries returning NULL results are not parsed. #6240 #6249

    0️⃣ DEFAULT function not working with status column. #6221 #6224

    VTGate panic kills connection when selecting database() when using OLAP workload. #6200 #6205

    Wrong column value can be inserted due to parameterization variable confusion. #6266 #6270

    πŸ›  Fix owned table name formatting and duplicate error code in consistent lookup. #6145

    πŸ‘Œ Support for case-insensitive user defined variables. #6225

    πŸ›  Fix performance regression on VTGate. #6203

    Return NULL for non-existent user-defined variables. #6308

    MySQL connection closing should not inflate query counts. #6226

    πŸ‘‰ show vitess_shards intermittently returns empty list. #5038 #5189

    Protect insert lookup queries from bind variable name clashes #6328

    CAST/CONVERT should work correctly #6304

    πŸ‘ Allow charset introducers for vindex columns #6301

    πŸ›  Fix panic from EXPLAIN #6204

    πŸ– Handle special comments that start with ! #6186

    πŸ›  Fix panic with uppercase types #6140

    πŸ‘ Allow β€˜$’ in identifiers #6136

    Other

    ⏱ VTTablet: Etcd query timeout because auto-committable transactions are not properly rolled back. #6258 #6299

    πŸ›  VTTablet: Fixed race in schema engine during notification of schema changes. #6268

    πŸ›  VTTablet: Reloading ACL config was throwing misleading error messages.his has now been fixed and the error messages improved. #6309

    πŸš€ VTTablet: Panic during backup caused an error in the pgzip library has been fixed by contributing a fix to pgzip and upgrading to a new release that includes the fix. #6189

    VTTablet: Improvement of error handling by retrying while uploading to S3. #6182 #6395 #6405

    πŸ›  VTTablet: builtinbackupengine was uploading a manifest despite errors from S3. Error collection during backup has been fixed to address this. #6349 #6350

    VTTablet: Fix dba connection leak in LockTables. #6424

    VTTablet: Fix connection pool leak in schema engine. #6426

    VTTablet: Fix regression where a heartbeat interval flag setting of 1s would have resulted in an internal value of 1000s #6356 #6357

    VTTablet: Fix panic from binlog_streamer when β€˜-watch_replication_stream’ is set., #6253

    VTTablet: Add context to MysqlDaemon schema functions. This lets us handle disconnected RPCs instead of the client hanging forever. #6243

    VTTablet: fix panic from txlogz #6238

    πŸ‘ VTTablet: Add back support for millisecond values for flags that was dropped when yaml config was introduced #6250

    πŸ’» Vtctld UI: Change all external links in vitess to use https #6170 #6169

    Ensure SrvVSchema gets rebuilt for all cells #6276

    πŸ›  Fix MariadbGTIDSet multi-domain support #6184

    mysqlctl: Fix connection leak during killConnection() #6245

    πŸ›  Fix release script to install examples in the correct directory #6159

    examples/operator: fix port-forward command #6418

    VStream client: should send current position events immediately #6391 #6397

    Functionality Added or Changed

    VTGate/ MySQL compatibility

    Transactions on read-only replicas are now supported. Start a transaction by first choosing the desired tablet_type ("use keyspace@replica"), then issuing a β€œBEGIN” statement. Internally, this will issue a β€œSTART TRANSACTION READ ONLY” on the underlying database. This is only available if using the new gateway_implementation of tabletgateway (which is now the default). #6166 #6244

    πŸ‘ Create better plans for NULL comparisons. Instead of using a SelectScatter, now vtgate will produce empty results when the predicate is a comparison against literal NULLs. #6133 #6425

    πŸ‘Œ Support IN, NOT IN and equality (=) with null values #6152 #6146

    βž• Add EXPLAIN support for vtgate plans. View the produced vtgate query plan by using explain format=vitess #6110

    πŸ‘ Allow users to perform savepoint related constructs. #6355 #6404 #6412 #6414

    πŸš€ Added support for user-level advisory lock functions like get_lock and release_lock. This is considered experimental and will be improved in future releases based on user feedback. #6367 #6370 #6470

    πŸ‘ Allow users to change system variable settings at session level using set statements. This is a major change that was merged later in the development cycle. It is considered experimental and there may be edge cases. #6107 #6459 #6472 #6488

    "ignore_nulls" option to lookup vindexes #6147 #6222: There are situations where the "from" columns of a lookup vindex can be null. Such columns cannot be inserted in the lookup due to the uniqueness constraints of a lookup. The new β€œignore_nulls” option of the lookup vindex allows you to accommodate null values in rows. Such rows can still be found through other means, like the primary vindex.

    βž• Add support for row_count() function. #6174 #6179

    πŸ‘ XOR Operator is now supported. #6369 #6371

    Allow 'binary' for character_set_results. It is equivalent to null. #6237

    πŸ‘Œ Improve logging by adding keyspace, table, and tablet_type. #6282

    πŸ›° VTGate now takes in optional flags for β€˜max_payload_size’ and β€˜warn_payload_size’. A payload size above the warning threshold will only produce a warning. A payload size that exceeds β€˜max_payload_size’ will result in an error. The default behavior is not to check payload size, which is backwards-compatible. #6143 #6375

    A new comment directive (β€˜/*vt+ IGNORE_MAX_PAYLOAD_SIZE=1 */) has been added to provide the ability to override the payload size checking per query. #6143

    A new comment directive (β€˜/*vt+ IGNORE_MAX_MEMORY_ROWS=1 */) has been added to provide the ability to ignore the "-max_memory_rows" setting per query. #6430

    Support β€˜select * from information_schema.processlist’. #6310

    πŸ‘Œ Support batch lookup queries for integer and binary types. Other types will continue to require one query per row. #6420

    πŸ‘Œ Support system settings at session level. #6149

    Prepare for future features by adding reserved connections. This is the ability for a connected client to have a dedicated connection instead of using the connection pool. #6303 #6313 #6402 #6441 #6451

    πŸ‘ Allow charset introducers with vindex column values. #6236 #6314

    πŸ‘Œ Support Union All between different shards. #6351

    Exclude KeyspaceId(s) from plan cache key.This improves the cache hit rate. #6233

    Reload TLS config on SIGHUP. #6215

    ⬇️ Drop vindex DDL. #6151

    show vitess_tablets now reports master_term_start_time. We ensure that this is reported correctly after reparents. #6135 #6292 #6293

    πŸ‘ Allow parenthesized selects #6342

    πŸ‘Œ Support DO statements #6239

    πŸ‘Œ Support calling database() inside other functions #6291

    πŸ‘Œ Support IN, NOT IN and equality (=) with null values #6152 #6146

    VReplication

    VStream Copy extends the VStream API to allow streaming of the contents of databases or tables. Earlier you had to specify a position to start streaming events from. #6277

    0️⃣ VReplication workflows now accept "-cells" as an option. This can be a comma-separated list of cells or cellsAliases. The source tablet for vreplication will be chosen randomly among the tablets that belong to the specified cells/cellsAliases. If the option is not provided, we default to the cell that is local to the vreplication target. #6442, #6456 #6465

    Schema Versioning. Tablets now have a schema tracker that can be enabled to store schema versions as DDLs are applied. The _vt.schema_version table contains the schema with the corresponding ddl and gtid position when the ddl was applied. The tracked schemas are used by the new Historian service to provide vstreams with the correct schema while streaming old events. #6300 #6164 #6278 #6399 #6411 #6435 #6448

    πŸ‘Œ Support for specifying multiple external sources to facilitate migration into vitess.

    πŸ‘ Foreign Keys are now supported on the target database during vreplication. In the copy phase we disable FK checks and re-enable them once the replication phase starts. #6284

    Materialize: added a drop_constraint option during table copy to strip constraints while creating target tables #6422

    🐎 Materialize: optimizations while creating target tables resulting in a significant improvement in performance when moving a large number of tables from an external MySQL instance #6207, #6251

    ⬇️ DropSources: Added β€˜-rename_tables’ option to rename the source tables rather than dropping them entirely. #6383

    πŸ’Ύ Savepoints: Vreplication streams are not able to handle savepoint events in the binlog. #6438

    βž• Add keyspace, shard and workflow labels to VReplicationSecondsBehindMaster metrics #6279

    πŸ’» Correct command line help for MigrateServedTypes, MigrateServedFrom and SwitchWrites #6187

    VExec and Workflow commands have been added to vtctld as a layer above VReplicationExec. They allow listing/stopping/starting/deleting workflows and ability to run custom queries on the _vt.vreplication table for all shards in a workflow. #6410

    Point-in-time Recovery

    πŸš€ In the 4.0 release we introduced functionality to recover a keyspace into a snapshot based on a specified time. That feature only allowed you to restore a backup, but did not have the ability to automatically replay binlogs up to the specified time. The feature has now been completed so that true point-in-time recovery is possible. This is currently only supported for MySQL compatible installations, not MariaDB. #4857 #6267 #6408

    Healthcheck

    πŸ—„ VTGate can now use a new healthcheck that is logically simpler while providing the same functionality. This is the default. The older healthcheck has been deprecated but is still available to use if desired by setting "-gateway_implementation=discoverygateway" #5750 #6155

    VTCombo has been migrated to use the new healthcheck #6302

    VTExplain has been migrated to use new healthcheck #6460

    MiniVitess

    🐳 A docker container that can be used to bring up a self-contained vitess cluster, which can automatically resolve and self-configure to meet an external MySQL cluster. While this might sound very similar to VTCombo and vttestserver, it is not quite the same.

    🐳 MiniVitess spins up in docker. Given a MySQL server hostname, schema name & credentials, it runs an automated topology analysis via orchestrator, identifying the cluster members. It proceeds to bring up distinct tablet for each server and opens up vtgate connections. #6377

    Examples / Tutorials

    Vitess operator example #6154

    Region sharding example #6209 #6275

    πŸ”„ Change local example to use new command DropSources #6178

    βœ… Are-you-alive example now makes it easier to test multiple endpoints #6202

    🐳 Local docker tutorial added to website

    πŸ—„ Simplified helm chart for examples (however, helm is now deprecated) #6168

    Other

    ⚑️ vtctlclient ListAllTablets now shows the MasterTermStartTime. This is shown as "" for read-only tablet types. In case of an old master that has not yet updated the topo record, TabletType will be reported as β€œunknown” #6135

    πŸ†• New debug handlers have been added to enable/disable golang block and mutex profiling. #6137

    0️⃣ Vitessdriver: Allow overriding driver name. Default stays as "vitess" #6138

    βœ… VTCombo: add options to allow starting MySQL within VTCombo. Note that vttestserver provides similar functionality. #6265

    βœ… Disable foreign key checks during preflight schema change tests #4696

    πŸ›  Fix labels for opentsdb #6289

    πŸ‘ Allow empty shards when running ValidateSchemaKeyspace behind flag β€˜-skip-no-master’ #6216

    Death by SIGPIPE on stdout/stderr can be avoided by setting the new flag β€˜catch-sigpipe’ #6297

    πŸ‘ Prepared Statement support added when using special vitess construct of use <keyspace>:<shard> by adding GetField support in engine plan execute #6132

    GTIDSet Union #6180

    βž• Add relay log position and file based positions to ReplicationStatus #6217

    Function that can find errant GTIDs #6296

    VTTablet: Add StopReplicationMode to StopReplicationAndGetStatus RPC #6335

    VTTablet: DemoteMaster RPC returns full status instead of string representation of current position #6365

    VTTablet: WaitForPosition RPC can now accept either file-based or GTID-based positions #6374

    πŸ“š Documentation

    πŸ›  Fix links to local example and sample client in vitessdriver godoc #6254

    πŸ“š Flag documentation has been improved #6261

    πŸ“„ All flags are now documented in the program reference on the website.

    πŸ— Build Environment Changes

    πŸ‘• golangci-lint now runs as a pre-commit hook and CI check #6385 #6406

    βœ… CI test for region_sharding example #6275

    βœ‚ Removed staticcheck as a pre-commit hook because it is included in golangci-lint #6416

    βœ… Faster unit tests #6415

    ⬆️ Consul-api version upgraded to 1.8.0 #6358

    ⬆️ Upgrade pgzip to v1.2.4 #6189

    ⬆️ Upgrade log4j2 version to 2.13.3 #6403

    ⬆️ Upgrade netty version to 4.1.48.Final #6317

    ⬆️ Upgrade tcnative version to 2.0.30.Final #6317

    βž• Add docker_local target to Makefile and a docker/local/run.sh script #6393

    🐧 Only check for k3s on Linux #6353

    πŸ— UI build tooling was broken, it has now been fixed. #6473

    Functionality Neutral Changes

    πŸ”¨ Refactor table filtering logic #6242 #6259

    πŸ–¨ VTGate: Replace Sprintf with string concatenation in normalizer, this reduces CPU usage on benchmarks #6127

    VTTablet: Reduce the number of packets sent from vttablet to MySQL #6130

    πŸ›  Fixed flakiness in External Connector Test: fixed race #6201

    πŸ›  Fixed flakiness in Message Test: test had not been updated after jitter functionality was added #6283

    πŸ›  Fix flakiness in TestConnectTimeout #6188

    πŸ›  Fix TLS test for Go 1.13+ #6185

    πŸ›  Fixed linter errors #6416 #6364 #6307

    βœ… More dollar-sign tests #6363

    VTExplain: use DiscoveryGateway (and old healthcheck) until migration can be completed #6248

    πŸ—„ Deprecated RPCs from 6.0 have been deleted #6380

    ⚑️ Update references to orchestrator repo to point to openark instead of github #6360

    πŸ—„ Terminology: deprecate or replace references to "slave" with β€œreplica” except where referring to actual sql commands or fields returned from MySQL. #6428 #6392 #6379

    πŸ’» Terminology: replace user-facing references to "slave" with β€œreplica” or β€œreplication” in UI #6481

    πŸ›  Fix log format - use Infof instead of Info #6413

    βœ… Cleanup temp files used in tests #6400

    βž• Add unit test for func stringMapToString #6280

    βœ… Check errors in etcd2topo unit test #6128

    βœ… Unit test for atomic.go #6120

    βœ… Unit test for ResolveIPv4Addrs #6230

    βœ… Enabled set statement tests #6167

    Replaced VTTablet ExecuteBatch api call with Execute for autocommit query. #6407 #6431

    πŸ›  Fixed flaky TestQueryPlanCache #6454

    πŸ›  Fix test compile errors #6434

    βœ… Scatter_conn unit test using new healthcheck #6458

    βœ… Test that vtexplain works with healthcheck #6252

    🚚 Tests: Removed error check that caused spurious test failures #6462

    More test cases for IGNORE_MAX_PAYLOAD_SIZE #6467

    βœ… VSCopy Test: Mods/bug fixes/test logging to make test repeatable/debuggable, fix flakiness #6341 #6235

    Cleanup scatter conn #6257

    Simplify vtgate executor #6256

    βž• Add tests for SHOW COLUMNS #6192

    βž• Added fuzzer #6175

    Simplified TxPool and ConnPool code #6150

  • v7.0.0-beta Changes

    July 21, 2020

    Incompatible Changes

    πŸ‘€ VTGate: The default gateway ("-gateway_implementation" flag value) is now "tabletgateway". The old default was "discoverygateway”. To use the old gateway, specify the flag in the command line. Anyone already using the flag will see no change; to use the new gateway, remove the -gateway_implementation flag. The new gateway is simpler and uses a new healthcheck module with no loss of functionality. #6155

    ⚠ VTGate: VTGate incorrectly returned Warning Code field name as Type. This has now been fixed. Code or scripts that depend on the incorrect behavior might break. #6437

    πŸ—„ Deprecations

    πŸš€ The following vtctl commands have been deprecated. Deprecated commands will continue to work until the next release (8.0). #6428

    StartSlave -> Replaced with StartReplication

    StopSlave -> Replaced with StopReplication

    πŸ”„ ChangeSlaveType -> Replaced with ChangeTabletType

    Various vtctl commands had a flag called "-wait_slave_timeout". It has been replaced with β€œ-wait_replicas_timeout”. Scripts using the old flag will continue to work until the next major release (8.0). #6428

    Helm chart (more details to be added)

    Consul (more details to be added)

    Replaced VTTablet ExecuteBatch api call with Execute for autocommit query. #6407 #6431

    πŸ› Bugs Fixed

    VTGate / MySQL compatibility

    VTGate: When using user-defined variables that have not been initialized first, a non-fatal panic occurs and the connection is dropped. #6294 #6308

    VTGate: Setting system variable panics vtgate. #6162 #6163

    regression: USE does not work from mysql client. #6153 #6157

    πŸ‘‰ USE does not return an error if a keyspace doesn't exist. #3060 #6157

    Dotted table names are not escaped properly in lookup vindex flow. #6142 #6145

    Error 1105 instead of 1062 when inserting duplicate into consistent_lookup vindex. #6144

    VTGate: Rewriting expressions fail when used as function parameters. #6281

    ⚑️ Rollback after error clobbers updates. #6285

    Queries should not be wrapped into transactions if autocommit=1 is set. #6264

    Prepared statements with bind variables in column specification produce β€˜missing bind var’ error. #6287 #6298

    πŸ“œ Prepared statement queries returning NULL results are not parsed. #6240 #6249

    0️⃣ DEFAULT function not working with status column. #6221 #6224

    VTGate panic kills connection when selecting database() when using OLAP workload. #6200 #6205

    Wrong column value can be inserted due to parameterization variable confusion. #6266 #6270

    πŸ›  Fix owned table name formatting and duplicate error code in consistent lookup. #6145

    πŸ‘Œ Support for case-insensitive user defined variable. #6225

    πŸ›  Fix performance regression on VTGate. #6203

    Return NULL for non-existent user-defined variables. #6308

    πŸ”’ Lock function support. #6370

    VTGate: MySQL connection closing should not inflate query counts. #6226

    πŸ›  VTGate: Show vitess_shards intermittently returns empty list. This has been fixed #5038 #5189

    Other

    ⏱ VTTablet: Etcd query timeout because auto-committable transactions are not properly rolled back. #6258 #6299

    πŸ›  VTTablet: Fixed race in schema engine during notification of schema changes. #6268

    πŸ›  VTTablet: Reloading ACL config was throwing misleading error messages.his has now been fixed and the error messages improved. #6309

    πŸš€ VTTablet: Panic during backup because of an error in the pgzip library has been fixed by contributing a fix to pgzip and upgrading to a new release that includes the fix. #6189

    VTTablet: Improvement of error handling by retrying while uploading to S3. #6182 #6395 #6405

    πŸ›  VTTablet: builtinbackupengine was uploading a manifest despite errors from S3. Error collection during backup has been fixed to address this. #6349 #6350

    VTTablet: Fix dba connection leak in LockTables. #6424

    VTTablet: Fix connection pool leak in schema engine. #6426

    VTTablet: Fix panic from binlog_streamer when β€˜-watch_replication_stream’ is set., #6253

    πŸ’» Vtctld UI: Change all external links in vitess to use https #6170 #6169

    Ensure SrvVSchema gets rebuilt for all cells #6276

    πŸ›  Fix MariadbGTIDSet multi-domain support #6184

    mysqlctl: Fix connection leak during killConnection() #6245

    πŸ›  Fix release script to install examples in the correct directory #6159

    Functionality Added or Changed

    VTGate/ MySQL compatibility

    πŸ‘ Transactions on read-only replicas are now supported. Start a transaction by first choosing the desired tablet_type ("use @replica"), then issuing a β€œBEGIN” statement. Internally, this will issue a β€œSTART TRANSACTION READ ONLY” on the underlying database. This is only available if using tabletgateway. #6166 #6244

    πŸ‘ Create better plans for NULL comparisons. Instead of using a SelectScatter, now vtgate will produce empty results when the predicate is a comparison against literal NULLs. #6133 #6425

    βž• Add EXPLAIN support for vtgate plans. View the produced vtgate query plan by using EXPLAIN FORMAT=VITESS.

    βž• Added savepoint support. #6355 #6404 #6412 #6414

    "ignore_nulls" option to lookup vindexes #6147 #6222

    βž• Add support for row_count() function. #6174 #6179

    πŸ‘ XOR Operator is now supported in sql WHERE clause. #6369 #6371

    Allow 'binary' for character_set_results. It is equivalent to null. #6237

    πŸ‘Œ Improve logging by adding keyspace, table, and tablet_type. #6282

    πŸ›° VTGate now takes in optional flags for β€˜max_payload_size’ and β€˜warn_payload_size’. A payload size above the warning threshold will only produce a warning. A payload size that exceeds β€˜max_payload_size’ will result in an error. The default behavior is not to check payload size, which is backwards-compatible. #6143 #6375

    A new comment directive (β€˜/*vt+ IGNORE_MAX_PAYLOAD_SIZE=1 */) has been added to provide the ability to override the payload size checking per query. #6143

    Support β€˜select * from information_schema.processlist’. #6310

    πŸ‘ VTGate: Support batch lookup queries for integer and binary types. Other types will continue to require one query per row. #6420

    πŸ‘Œ Support system settings at session level. #6149

    Prepare for future features by adding reserved connections. This is the ability for a connected client to have a dedicated connection instead of using the connection pool. #6303 #6313

    πŸ‘ Allow charset introducers with vindex column values. #6236 #6314

    πŸ‘Œ Support Union All between different shards. #6351

    VTGate: Exclude KeyspaceId(s) from plan cache key.This improves the cache hit rate. #6233

    VTGate: Reload TLS config on SIGHUP. #6215

    VTGate: Drop vindex DDL. #6151

    VTGate: Show vitess_tablets now reports master_term_start_time. We ensure that this is reported correctly after reparents. #6135 #6292 #6293

    VReplication

    VStream Copy extends the VStream API to stream entire databases or tables. Earlier you specified a position to start streaming events from. #6277

    0️⃣ VReplication workflows now accept "-cells" as an option. This can be a comma-separated list of cells or cellsAliases. The source tablet for vreplication will be chosen randomly among the tablets that belong to the specified cells/cellsAliases. If the option is not provided, we default to the cell that is local to the vreplication target. #6442, #6456

    Schema Versioning. Tablets now have a schema tracker that can be enabled to store schema versions as DDLs are applied. The _vt.schema_version table contains the schema with the corresponding ddl and gtid position when the ddl was applied. The tracked schemas are used by the new Historian service to provide vstreams with the correct schema while streaming old events. #6300

    πŸ‘ Foreign Keys are now supported on the target database during vreplication. In the copy phase we disable FK checks and re-enable them once the replication phase starts. #6284

    Materialize: added a drop_constraint option during table copy to strip constraints while creating target tables #6422

    🐎 Materialize: optimizations while creating target tables resulting in a significant improvement in performance when moving a large number of tables from an external MySQL instance #6207, #6251

    ⬇️ DropSources: Added β€˜-rename_tables’ option to rename the source tables rather than dropping them entirely. #6383

    βž• Add keyspace, shard and workflow labels to VReplicationSecondsBehindMaster metrics #6279

    πŸ’» Correct command line help #6187

    Point-in-time Recovery

    πŸš€ In the 4.0 release we introduced functionality to recover a keyspace into a snapshot based on a specified time. That feature only allowed you to restore a backup, but did not have the ability to replay binlogs up to the specified time. The feature has now been completed so that true point-in-time recovery is possible. This is currently only supported for MySQL compatible installations, not MariaDB. #4857 #6267 #6408

    Healthcheck

    πŸ—„ VTGate can now use a new healthcheck that is logically simpler while providing the same functionality. This is the default. The older healthcheck has been deprecated but is still available to use if desired by setting "-gateway_implementation=discoverygateway" #5750 #6155

    VTCombo has been migrated to use the new healthcheck #6302

    MiniVitess

    🐳 A docker container that can be used to bring up a self-contained vitess cluster, which can automatically resolve and self-configure to meet an external MySQL cluster. While this might sound very similar to VTCombo and vttestserver, it is not quite the same. MiniVitess runs in docker, brings up distinct processes for each vitess component and works with an external MySQL cluster. #6377

    Examples / Tutorials

    Vitess operator example #6154

    Region sharding example #6209 #6275

    πŸ”„ Change local example to use new command DropSources #6178

    βœ… Are-you-alive example now makes it easier to test multiple endpoints #6202

    🐳 Local docker tutorial added to website

    Other

    ⚑️ Vtctl ListAllTablets now shows the MasterTermStartTime. This is shown as "" for read-only tablet types. In case of an old master that has not yet updated the topo record, TabletType will be reported as β€œunknown” #6135

    πŸ†• New debug handlers have been added to set block and mutex profiling. #6137

    0️⃣ Vitessdriver: Allow overriding driver name. Default stays as "vitess" #6138

    βœ… VTCombo: add options to allow starting MySQL within VTCombo. Note that vttestserver provides similar functionality. #6265

    βœ… Disable foreign key checks during preflight schema change tests #4696

    πŸ›  Fix labels for opentsdb #6289

    πŸ‘ Allow empty shards when running ValidateSchemaKeyspace behind flag β€˜-skip-no-master’ #6216

    Death by SIGPIPE on stdout/stderr can be avoided by setting the new flag β€˜catch-sigpipe’ #6297

    πŸ‘ Field query support in shard targeting #6132

    πŸ“š Documentation

    πŸ›  Fix links to local example and sample client in vitessdriver godoc #6254

    πŸ“š Flag documentation has been improved #6261

    πŸ“„ All flags are now documented in the program reference on the website.

    πŸ— Build Environment Changes

    πŸ‘• golangci-lint now runs as a pre-commit hook and CI check #6385 #6406

    βœ… CI test for region_sharding example #6275

    βœ‚ Removed staticcheck as a pre-commit hook because it is included in golangci-lint #6416

    βœ… Faster unit tests #6415

    ⬆️ Consul-api version upgraded to 1.8.0 #6358

    ⬆️ Upgrade pgzip to v1.2.4 #6189

    ⬆️ Upgrade log4j2 version to 2.13.3 #6403

    βž• Add docker_local target to Makefile and a docker/local/run.sh script #6393

    🐧 Build: Only check for k3s on Linux #6353

    Functionality Neutral Changes

    βž• Add context to MysqlDaemon schema functions. This lets us handle disconnected RPCs instead of the client hanging forever. #6243

    πŸ”¨ Refactor table filtering logic #6242 #6259

    πŸ–¨ VTGate: Replace Sprintf with string concatenation in normalizer, this reduces CPU usage on benchmarks #6127

    VTTablet: Reduce the number of packets sent from vttablet to MySQL #6130

    πŸ›  Fixed flakiness in External Connector Test: fixed race #6201

    πŸ›  Fixed flakiness in Message Test: test had not been updated after jitter functionality was added #6283

    πŸ›  Fix flakiness in TestConnectTimeout #6188

    πŸ›  Fix TLS test for Go 1.13+ #6185

    πŸ›  Fixed linter errors #6416 #6364 #6307

    βœ… More dollar-sign tests #6363

    VTExplain: use DiscoveryGateway (and old healthcheck) until migration can be completed #6248

    πŸ—„ Deprecated RPCs from 6.0 have been deleted #6380

    ⚑️ Update references to orchestrator repo to point to openark instead of github #6360

    πŸ—„ Terminology: deprecate or replace references to "slave" with β€œreplica” except where referring to actual sql commands or fields returned from MySQL. #6428 #6392 #6379

    πŸ›  Fix log format - use Infof instead of Info #6413

    βœ… Cleanup temp files used in tests #6400

    βž• Add unit test for func stringMapToString #6280

    βœ… Check errors in etcd2topo unit test #6128

    βœ… Unit test for atomic.go #6120

    βœ… Unit test for ResolveIPv4Addrs #6230

    βœ… Enabled set statement tests #6167

  • v6.0.20-20200818 Changes

    August 18, 2020

    πŸš€ This update fixes two regressions that were deemed significant enough to be backported to the release branch.

    πŸ› Bugs Fixed

    • Vtgate: delete by destination broken for tables with owned vindexes #6541 #6554
    • Vtgate: set statements with multiple settings should not error out #6566
  • v6.0.20-20200624 Changes

    June 24, 2020

    πŸš€ This update fixes a regression which was significant enough that it was decided to backport to the release branch.

    πŸ› Bug fixed

    πŸ”§ #6250 added support for float64 durations in vttablet's yaml configuration. The change introduced a regression where the heartbeat interval unit was changed to seconds, but the conversion was not correspondingly changed. In other words, a heartbeat interval of 1s would have resulted in a value of 1000s instead. The net effect is that healthy replicas would be reported as heavily lagged.

    This issue affects only those who enable heartbeats.

    πŸš€ This regression as merged into https://github.com/vitessio/vitess/releases/tag/v6.0.20-20200617.
    πŸ›  #6359 fixes #6356