v0.10.0-prerelease

This release contains a large number of changes over the previous, and introduces the concept of experimental features to Spirit.

With experimental features:

• We will try to introduce them in ways where the code is well isolated, and you will always have to opt-in to using them.
• We will always document why they are still considered experimental.

Highlights

• Support for --force-kill to kill connections preventing lock acquisition by @kolbe, @morgo in #428 #458 #438 #506 #503
• Support for atomic multi-table changes by @morgo in #469 #431 #434 #437 #505 #482 #516
• The --force-inplace option has been removed, as we instead rely on parsing and explicitly allowing known safe inplace operations by @Kiran01bm, @kolbe in #470 #429
• The --checksum option has been removed and is now always enabled by @morgo in #489
• When flushing replication changes, Spirit now waits for the copier to advance past that point. This helps reduce potential contention scenarios at the cost of checkpoints sometimes being delayed. We call this optimization keyBelowLowWatermark by @morgo in #490

Additional Changes

• Improved TLS support by @Kiran01bm #443 #510
• Improved handling for cases where a table is externally modified, or can't write checkpoint handling by @morgo in #483
• The sentinel table option is now consistently named _spirit_sentinel #447 by @morgo
• Refactor alter visibility to the checks sub-system by @Kiran01bm in #484
• Fix RDS hostname detection to prevent potential subdomain spoofing by @Kiran01bm in #509
• move checksum retries into checksum package by @morgo in #507
• make tests parallel by @morgo in #514
• panic: could not convert datum to uint64 by @tyd in #518
• Have metadatalocks support multiple locks in one connection by @morgo in #525
• feat: Add maxRecreateAttempts logic by @tyd in #527
• Add additional info to ErrNotSupportedStatement error by @kolbe in #528
• Experimental support for buffered changes based on the DBLog algorithm by @morgo in #460 #453 #452 #459 #473
• Experimental support for native linting by @kolbe #486 #492 #501 #511 #529 #500
• Experimental support for a move command by @morgo #467 #496 #515 #523

New Contributors

• @tyd made their first contribution in #518

Full Changelog: v0.9.0...v0.10.0

v0.9.2

This addresses a critical issue where after several retries (handling errors such as deadlocks or lock waits), the replication applier will drop changes that could not be applied.

In many cases, this corruption would have been caught by the checksum, but regardless, all users are encouraged to upgrade immediately.

See issue #472

v0.9.1

This addresses a critical issue where updating rows such that the value of a PRIMARY KEY column changed were not tracked correctly, potentially leading to changes being lost in the new table.

In many cases, this corruption would have been caught by the checksum, but regardless, all users are encouraged to upgrade immediately.

See issue #417

v0.9.0

This release includes bug fixes and some refactoring so that the replication client can support multiple subscriptions. This is preparation work for a planned feature (atomic changes across multiple tables).

Multiple replication subscriptions took a few bug fixes to get right, but now that it is looking stable I want to issue a release ahead of new features.

What's Changed

• Refactor repl to support multiple subscriptions by @morgo in #389
• remove unnecessary test by @morgo in #398
• Change NewTableLock to take []tables as input by @morgo in #400
• fix: Add ServerName to TLSConfig by @kolbe in #402
• Support add partition. Fix syntax errors for partition statements. by @JashLal in #403
• Add poison pill protection by @morgo in #407
• remove query from log by @morgo in #408
• Fix the type conversion issue when resuming from checkpoint for PK ty… by @prudhvi in #409
• Bump tidb parser version by @morgo in #410
• Add logging and increase timeout of BlockWait calls by @morgo in #415
• use defer for MDL by @morgo in #413
• Fix repl client getting stuck when encountering errors by @morgo in #416
• [checks] Reintroduce hasTriggers check as a Preflight check by @brirams in #420
• update interface types to any - no functional change by @Kiran01bm in #423
• bump tidb parser version by @kolbe in #424

New Contributors

• @brirams made their first contribution in #420

Full Changelog: v0.8.0...v0.9.0

v0.8.0

New Features

• The minimum golang version required is now 1.24 by @kolbe
• We've added support for --statement instead of specifying --table and --alter. For now, both are supported, but we may prefer --statement moving forward by @morgo
• The inplace alter operations for DROP/TRUNCATE partition are explicitly marked as safe by @JashLal

Enhancements and Fixes

• minor - update/fix lint command in dev guide by @Kiran01bm in #368
• dev/test improvement - log test output in json and summarise using tparse by @Kiran01bm in #369
• certbundle reorg - change to load rds cert bundle from a file by @Kiran01bm in #370
• enhance the mdl refresh runner to handle connection closures and temporary network issues resiliently by @Kiran01bm in #371
• Fix converting latin1 to utf8 charset checksum failures by @morgo in #373
• embed certificate instead of file load by @morgo in #376
• compose lock name on spirit side - clients acquiring locks can get to know their exact lock name by @Kiran01bm in #374
• Fix non printable characters in checkpoint by @morgo in #382
• trim trailing semicolon from alter statement by @kolbe in #385
• Add test leak detection by @morgo in #390
• exclude generated columns from checksum by @morgo in #396

New Contributors

• @Kiran01bm made their first contribution in #368
• @JashLal made their first contribution in #387

Full Changelog: v0.7.0-prerelease...v0.8.0

v0.7.0-prerelease

What's Changed

• support 4hr delayed replicas by @morgo in #354
• binary type conversion by @morgo in #356
• use a hash for metadata lock name by @kolbe in #357
• bump linter to 1.62.2 by @kolbe in #359
• Bump go by @kolbe in #358
• Upgrade go-mysql version to v1.9.1 by @prudhvi in #363
• sort USAGE alphabetically, add lock-wait-timeout docs by @morgo in #366

Full Changelog: v0.6.0-prerelease...v0.7.0-prerelease

v0.6.0-prerelease

This release contains a number of improvements over previous releases.

What's Changed

• update throttler latency query by @morgo in #294
• Add asserty feature by @morgo in #293
• reduce deadlocks in recopying chunks by @morgo in #295
• Add --strict option by @samongyr-sq in #304
• Add support for running tests with docker-compose by @sahilm in #300
• [#306] Upgrade to go 1.22 by @sahilm in #309
• [#310] upgrade to latest golangci-lint and enable new linters by @sahilm in #311
• prevent concurrent table modifications by @jayjanssen in #308
• Jayj/old table timestamped by @jayjanssen in #307
• cleanup server lock code by @morgo in #302
• Reject read only connections to primary nodes by @sahilm in #313
• rename test tables to avoid conflicts by @jayjanssen in #318
• add CopierEstimateHistory by @jayjanssen in #316
• fix unlock in progress API by @morgo in #322
• Add virtual column support by @morgo in #324
• dbconn: improve MDL lock output by @morgo in #326
• Use dbconn.Exec in checkpoint by @morgo in #328
• Remove copier word from log to have the throttler be used outside of spirit by @prudhvi in #334
• Fix changes not synced error by @morgo in #332
• repl: fix div by zero error by @morgo in #336
• workaround timeout issues by @morgo in #338
• repl: revert to inject-noise based master wait by @morgo in #339
• require RELOAD privilege by @morgo in #340
• Use quoted table name as lock name by @prudhvi in #342
• Add resume from checksum by @morgo in #344
• Add support for interpolate params by @morgo in #346
• repl: Improve Flush() to handle BlockWait() timeouts by @morgo in #349
• Use tablename for metadatalock name as tablename respsects both the m… by @prudhvi in #350

New Contributors

• @samongyr-sq made their first contribution in #304
• @sahilm made their first contribution in #300
• @jayjanssen made their first contribution in #308

Full Changelog: v0.1.0-prerelease...v0.6.0-prerelease

v0.1.0-prerelease

This is the looking good release.

We have all the expected minimum functionality, and have been running spirit in production for quite some time.

However, everyone's scenario is slightly different. We are not calling this v1.0.0 yet :-)