Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
# Changelog

## Unreleased

Features:

* Add Tencent Cloud KMS support for encryption and decryption.
The integration supports environment variable authentication
(``TENCENTCLOUD_SECRET_ID``, ``TENCENTCLOUD_SECRET_KEY``),
optional STS token (``TENCENTCLOUD_TOKEN``),
configurable region and custom endpoint.
New CLI flags: ``--tencent-kms``, ``--add-tencent-kms``,
``--rm-tencent-kms``.
Environment variable: ``SOPS_TENCENT_KMS_IDS``.
Configuration file key: ``tencent_kms``.

## 3.13.1

Improvements:
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ SOPS: Secrets OPerationS
========================

**SOPS** is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY
formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, HuaweiCloud KMS, age, and PGP.
formats and encrypts with AWS KMS, GCP KMS, Tencent Cloud KMS, Azure Key Vault, HuaweiCloud KMS, age, and PGP.
(`demo <https://www.youtube.com/watch?v=YTEVyLXFiq0>`_)

.. image:: https://i.imgur.com/X0TM5NI.gif
Expand Down
72 changes: 63 additions & 9 deletions cmd/sops/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ import (
"github.com/getsops/sops/v3/keyservice"
"github.com/getsops/sops/v3/kms"
"github.com/getsops/sops/v3/logging"
"github.com/getsops/sops/v3/tencentkms"

"github.com/getsops/sops/v3/pgp"
"github.com/getsops/sops/v3/stores"
"github.com/getsops/sops/v3/stores/dotenv"
Expand Down Expand Up @@ -91,7 +93,8 @@ func main() {
},
}
app.Name = "sops"
app.Usage = "sops - encrypted file editor with AWS KMS, GCP KMS, HuaweiCloud KMS, Azure Key Vault, age, and GPG support"
app.Usage = "sops - encrypted file editor with AWS KMS, GCP KMS, Tencent Cloud KMS, HuaweiCloud KMS, Azure Key Vault, age, and GPG support"

app.ArgsUsage = "sops [options] file"
app.Version = version.Version
app.Authors = []cli.Author{
Expand All @@ -110,7 +113,14 @@ func main() {
(You need to setup Google application default credentials. See
https://developers.google.com/identity/protocols/application-default-credentials)

To encrypt or decrypt a document with Tencent Cloud KMS, specify the
Tencent Cloud KMS key ID in the --tencent-kms flag or in the
SOPS_TENCENT_KMS_IDS environment variable.
(You need to setup Tencent Cloud credentials via TENCENTCLOUD_SECRET_ID,
TENCENTCLOUD_SECRET_KEY, optional TENCENTCLOUD_TOKEN for STS, and TENCENTCLOUD_REGION)

To encrypt or decrypt a document with HuaweiCloud KMS, specify the

HuaweiCloud KMS key ID (format: region:key-uuid) in the --hckms flag or in the
SOPS_HUAWEICLOUD_KMS_IDS environment variable.
(You need to setup HuaweiCloud credentials via environment variables:
Expand Down Expand Up @@ -955,6 +965,7 @@ func main() {
Usage: "comma separated list of Azure Key Vault URLs",
EnvVar: "SOPS_AZURE_KEYVAULT_URLS",
},

cli.StringFlag{
Name: "hc-vault-transit",
Usage: "comma separated list of vault's key URI (e.g. 'https://vault.example.org:8200/v1/transit/keys/dev')",
Expand All @@ -970,6 +981,11 @@ func main() {
Usage: "comma separated list of age recipients",
EnvVar: "SOPS_AGE_RECIPIENTS",
},
cli.StringFlag{
Name: "tencent-kms",
Usage: "comma separated list of Tencent Cloud KMS key IDs",
EnvVar: tencentkms.TencentKmsEnvVar,
},
cli.StringFlag{
Name: "input-type",
Usage: "currently json, yaml, dotenv and binary are supported. If not set, sops will use the file's extension to determine the type",
Expand Down Expand Up @@ -1183,6 +1199,14 @@ func main() {
Name: "rm-pgp",
Usage: "remove the provided comma-separated list of PGP fingerprints from the list of master keys on the given file",
},
cli.StringFlag{
Name: "add-tencent-kms",
Usage: "add the provided comma-separated list of Tencent Cloud KMS key IDs to the list of master keys on the given file",
},
cli.StringFlag{
Name: "rm-tencent-kms",
Usage: "remove the provided comma-separated list of Tencent Cloud KMS key IDs from the list of master keys on the given file",
},
cli.StringFlag{
Name: "filename-override",
Usage: "Use this filename instead of the provided argument for loading configuration, and for determining input type and output type",
Expand All @@ -1209,8 +1233,8 @@ func main() {
return toExitError(err)
}
if _, err := os.Stat(fileName); os.IsNotExist(err) {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hckms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hckms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hckms") != "" || c.String("add-tencent-kms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hckms") != "" || c.String("rm-tencent-kms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
return common.NewExitError(fmt.Sprintf("Error: cannot add or remove keys on non-existent file %q, use the `edit` subcommand instead.", fileName), codes.CannotChangeKeysFromNonExistentFile)
}
}
Expand Down Expand Up @@ -1306,6 +1330,7 @@ func main() {
Usage: "comma separated list of Azure Key Vault URLs",
EnvVar: "SOPS_AZURE_KEYVAULT_URLS",
},

cli.StringFlag{
Name: "hc-vault-transit",
Usage: "comma separated list of vault's key URI (e.g. 'https://vault.example.org:8200/v1/transit/keys/dev')",
Expand All @@ -1321,6 +1346,11 @@ func main() {
Usage: "comma separated list of age recipients",
EnvVar: "SOPS_AGE_RECIPIENTS",
},
cli.StringFlag{
Name: "tencent-kms",
Usage: "comma separated list of Tencent Cloud KMS key IDs",
EnvVar: tencentkms.TencentKmsEnvVar,
},
cli.StringFlag{
Name: "input-type",
Usage: "currently json, yaml, dotenv and binary are supported. If not set, sops will use the file's extension to determine the type",
Expand Down Expand Up @@ -1719,6 +1749,7 @@ func main() {
Usage: "comma separated list of Azure Key Vault URLs",
EnvVar: "SOPS_AZURE_KEYVAULT_URLS",
},

cli.StringFlag{
Name: "hc-vault-transit",
Usage: "comma separated list of vault's key URI (e.g. 'https://vault.example.org:8200/v1/transit/keys/dev')",
Expand All @@ -1734,6 +1765,11 @@ func main() {
Usage: "comma separated list of age recipients",
EnvVar: "SOPS_AGE_RECIPIENTS",
},
cli.StringFlag{
Name: "tencent-kms",
Usage: "comma separated list of Tencent Cloud KMS key IDs",
EnvVar: "SOPS_TENCENT_KMS_IDS",
},
cli.BoolFlag{
Name: "in-place, i",
Usage: "write output back to the same file instead of stdout",
Expand Down Expand Up @@ -1810,6 +1846,14 @@ func main() {
Name: "rm-pgp",
Usage: "remove the provided comma-separated list of PGP fingerprints from the list of master keys on the given file",
},
cli.StringFlag{
Name: "add-tencent-kms",
Usage: "add the provided comma-separated list of Tencent Cloud KMS key IDs to the list of master keys on the given file",
},
cli.StringFlag{
Name: "rm-tencent-kms",
Usage: "remove the provided comma-separated list of Tencent Cloud KMS key IDs from the list of master keys on the given file",
},
cli.BoolFlag{
Name: "ignore-mac",
Usage: "ignore Message Authentication Code during decryption",
Expand Down Expand Up @@ -1904,8 +1948,8 @@ func main() {
return toExitError(err)
}
if _, err := os.Stat(fileName); os.IsNotExist(err) {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hckms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hckms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
if c.String("add-kms") != "" || c.String("add-pgp") != "" || c.String("add-gcp-kms") != "" || c.String("add-hckms") != "" || c.String("add-tencent-kms") != "" || c.String("add-hc-vault-transit") != "" || c.String("add-azure-kv") != "" || c.String("add-age") != "" ||
c.String("rm-kms") != "" || c.String("rm-pgp") != "" || c.String("rm-gcp-kms") != "" || c.String("rm-hckms") != "" || c.String("rm-tencent-kms") != "" || c.String("rm-hc-vault-transit") != "" || c.String("rm-azure-kv") != "" || c.String("rm-age") != "" {
return common.NewExitError(fmt.Sprintf("Error: cannot add or remove keys on non-existent file %q, use `--kms` and `--pgp` instead.", fileName), codes.CannotChangeKeysFromNonExistentFile)
}
if isEncryptMode || isDecryptMode || isRotateMode {
Expand Down Expand Up @@ -2235,7 +2279,7 @@ func getEncryptConfig(c *cli.Context, fileName string, inputStore common.Store,
}, nil
}

func getMasterKeys(c *cli.Context, kmsEncryptionContext map[string]*string, kmsOptionName string, pgpOptionName string, gcpKmsOptionName string, hckmsOptionName string, azureKvOptionName string, hcVaultTransitOptionName string, ageOptionName string) ([]keys.MasterKey, error) {
func getMasterKeys(c *cli.Context, kmsEncryptionContext map[string]*string, kmsOptionName string, pgpOptionName string, gcpKmsOptionName string, hckmsOptionName string, azureKvOptionName string, hcVaultTransitOptionName string, ageOptionName string, tencentKmsOptionName string) ([]keys.MasterKey, error) {
var masterKeys []keys.MasterKey
for _, k := range kms.MasterKeysFromArnString(c.String(kmsOptionName), kmsEncryptionContext, c.String("aws-profile")) {
masterKeys = append(masterKeys, k)
Expand Down Expand Up @@ -2274,16 +2318,19 @@ func getMasterKeys(c *cli.Context, kmsEncryptionContext map[string]*string, kmsO
for _, k := range ageKeys {
masterKeys = append(masterKeys, k)
}
for _, k := range tencentkms.MasterKeysFromKeyIDString(c.String(tencentKmsOptionName)) {
masterKeys = append(masterKeys, k)
}
return masterKeys, nil
}

func getRotateOpts(c *cli.Context, fileName string, inputStore common.Store, outputStore common.Store, svcs []keyservice.KeyServiceClient, decryptionOrder []string) (rotateOpts, error) {
kmsEncryptionContext := kms.ParseKMSContext(c.String("encryption-context"))
addMasterKeys, err := getMasterKeys(c, kmsEncryptionContext, "add-kms", "add-pgp", "add-gcp-kms", "add-hckms", "add-azure-kv", "add-hc-vault-transit", "add-age")
addMasterKeys, err := getMasterKeys(c, kmsEncryptionContext, "add-kms", "add-pgp", "add-gcp-kms", "add-hckms", "add-azure-kv", "add-hc-vault-transit", "add-age", "add-tencent-kms")
if err != nil {
return rotateOpts{}, err
}
rmMasterKeys, err := getMasterKeys(c, kmsEncryptionContext, "rm-kms", "rm-pgp", "rm-gcp-kms", "rm-hckms", "rm-azure-kv", "rm-hc-vault-transit", "rm-age")
rmMasterKeys, err := getMasterKeys(c, kmsEncryptionContext, "rm-kms", "rm-pgp", "rm-gcp-kms", "rm-hckms", "rm-azure-kv", "rm-hc-vault-transit", "rm-age", "rm-tencent-kms")
if err != nil {
return rotateOpts{}, err
}
Expand Down Expand Up @@ -2433,6 +2480,7 @@ func keyGroups(c *cli.Context, file string, optionalConfig *config.Config) ([]so
var hcVaultMkKeys []keys.MasterKey
var hckmsMkKeys []keys.MasterKey
var ageMasterKeys []keys.MasterKey
var tencentkmsMkKeys []keys.MasterKey
kmsEncryptionContext := kms.ParseKMSContext(c.String("encryption-context"))
if c.String("encryption-context") != "" && kmsEncryptionContext == nil {
return nil, common.NewExitError("Invalid KMS encryption context format", codes.ErrorInvalidKMSEncryptionContextFormat)
Expand Down Expand Up @@ -2488,7 +2536,12 @@ func keyGroups(c *cli.Context, file string, optionalConfig *config.Config) ([]so
ageMasterKeys = append(ageMasterKeys, k)
}
}
if c.String("kms") == "" && c.String("pgp") == "" && c.String("gcp-kms") == "" && c.String("hckms") == "" && c.String("azure-kv") == "" && c.String("hc-vault-transit") == "" && c.String("age") == "" {
if c.String("tencent-kms") != "" {
for _, k := range tencentkms.MasterKeysFromKeyIDString(c.String("tencent-kms")) {
tencentkmsMkKeys = append(tencentkmsMkKeys, k)
}
}
if c.String("kms") == "" && c.String("pgp") == "" && c.String("gcp-kms") == "" && c.String("hckms") == "" && c.String("azure-kv") == "" && c.String("hc-vault-transit") == "" && c.String("age") == "" && c.String("tencent-kms") == "" {
conf := optionalConfig
var err error
if conf == nil {
Expand All @@ -2512,6 +2565,7 @@ func keyGroups(c *cli.Context, file string, optionalConfig *config.Config) ([]so
group = append(group, pgpKeys...)
group = append(group, hcVaultMkKeys...)
group = append(group, ageMasterKeys...)
group = append(group, tencentkmsMkKeys...)
log.Debugf("Master keys available: %+v", group)
return []sops.KeyGroup{group}, nil
}
Expand Down
37 changes: 29 additions & 8 deletions config/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ package config //import "github.com/getsops/sops/v3/config"

import (
"fmt"
"github.com/getsops/sops/v3/tencentkms"
"os"
"path"
"path/filepath"
Expand Down Expand Up @@ -130,14 +131,19 @@ type configFile struct {
}

type keyGroup struct {
Merge []keyGroup `yaml:"merge"`
KMS []kmsKey `yaml:"kms"`
GCPKMS []gcpKmsKey `yaml:"gcp_kms"`
HCKms []hckmsKey `yaml:"hckms"`
AzureKV []azureKVKey `yaml:"azure_keyvault"`
Vault []string `yaml:"hc_vault"`
Age []string `yaml:"age"`
PGP []string `yaml:"pgp"`
Merge []keyGroup `yaml:"merge"`
KMS []kmsKey `yaml:"kms"`
GCPKMS []gcpKmsKey `yaml:"gcp_kms"`
HCKms []hckmsKey `yaml:"hckms"`
AzureKV []azureKVKey `yaml:"azure_keyvault"`
Vault []string `yaml:"hc_vault"`
Age []string `yaml:"age"`
PGP []string `yaml:"pgp"`
TencentKMS []tencentKMSKey `yaml:"tencent_kms"`
}

type tencentKMSKey struct {
KeyID string `yaml:"key_id"`
}

type gcpKmsKey struct {
Expand Down Expand Up @@ -185,6 +191,7 @@ type creationRule struct {
HCKms []string `yaml:"hckms"`
AzureKeyVault interface{} `yaml:"azure_keyvault"` // string or []string
VaultURI interface{} `yaml:"hc_vault_transit_uri"` // string or []string
TencentKMS interface{} `yaml:"tencent_kms"` // string or []string
KeyGroups []keyGroup `yaml:"key_groups"`
ShamirThreshold int `yaml:"shamir_threshold"`
UnencryptedSuffix string `yaml:"unencrypted_suffix"`
Expand Down Expand Up @@ -221,6 +228,10 @@ func (c *creationRule) GetVaultURIs() ([]string, error) {
return parseKeyField(c.VaultURI, "hc_vault_transit_uri")
}

func (c *creationRule) GetTencentKMSKeys() ([]string, error) {
return parseKeyField(c.TencentKMS, "tencent_kms")
}

// Utility function to handle both string and []string
func parseKeyField(field interface{}, fieldName string) ([]string, error) {
if field == nil {
Expand Down Expand Up @@ -357,6 +368,9 @@ func extractMasterKeys(group keyGroup) (sops.KeyGroup, error) {
return nil, err
}
}
for _, k := range group.TencentKMS {
keyGroup = append(keyGroup, tencentkms.NewMasterKeyFromKeyID(k.KeyID))
}
return deduplicateKeygroup(keyGroup), nil
}

Expand Down Expand Up @@ -445,6 +459,13 @@ func getKeyGroupsFromCreationRule(cRule *creationRule, kmsEncryptionContext map[
for _, k := range vaultKeys {
keyGroup = append(keyGroup, k)
}
tencentkmsKeys, err := getKeysWithValidation(cRule.GetTencentKMSKeys, "tencent_kms")
if err != nil {
return nil, err
}
for _, k := range tencentkms.MasterKeysFromKeyIDString(strings.Join(tencentkmsKeys, ",")) {
keyGroup = append(keyGroup, k)
}
groups = append(groups, keyGroup)
}
return groups, nil
Expand Down
42 changes: 42 additions & 0 deletions config/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,18 @@ creation_rules:
hc_vault_uris: http://4:8200/v1/4/keys/4
`)

var sampleConfigWithTencentKMS = []byte(`
creation_rules:
- path_regex: scalar\.yaml$
tencent_kms: key-scalar
- path_regex: list\.yaml$
tencent_kms:
- key-list-1
- key-list-2
- path_regex: csv\.yaml$
tencent_kms: key-csv-1,key-csv-2
`)

var sampleConfigWithGroups = []byte(`
creation_rules:
- path_regex: foobar*
Expand Down Expand Up @@ -577,6 +589,36 @@ func TestLoadConfigFileWithMerge(t *testing.T) {
}, ids(conf.KeyGroups[1]))
}

func TestLoadConfigFileWithTencentKMS(t *testing.T) {
for _, tc := range []struct {
filePath string
keys []string
}{
{
filePath: "scalar.yaml",
keys: []string{"tencent_kms: key-scalar"},
},
{
filePath: "list.yaml",
keys: []string{
"tencent_kms: key-list-1",
"tencent_kms: key-list-2",
},
},
{
filePath: "csv.yaml",
keys: []string{
"tencent_kms: key-csv-1",
"tencent_kms: key-csv-2",
},
},
} {
conf, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithTencentKMS, t), "/conf/path", tc.filePath, nil)
assert.Nil(t, err)
assert.Equal(t, tc.keys, ids(conf.KeyGroups[0]))
}
}

func TestLoadConfigFileWithNoMatchingRules(t *testing.T) {
_, err := parseCreationRuleForFile(parseConfigFile(sampleConfigWithNoMatchingRules, t), "/conf/path", "foobar2000", nil)
assert.NotNil(t, err)
Expand Down
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ require (
github.com/pkg/errors v0.9.1
github.com/sirupsen/logrus v1.9.4
github.com/stretchr/testify v1.11.1
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.3.24
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/kms v1.3.7
github.com/urfave/cli v1.22.17
go.yaml.in/yaml/v3 v3.0.4
golang.org/x/crypto v0.52.0
Expand Down
5 changes: 5 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -327,6 +327,11 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.3.7/go.mod h1:r5r4xbfxSaeR04b166HGsBa/R4U3SueirEUpXGuw+Q0=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.3.24 h1:A0FLutAc8Qvzb4Ulz7e0otGwksM7dR9no8/AiIZj9kM=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/common v1.3.24/go.mod h1:r5r4xbfxSaeR04b166HGsBa/R4U3SueirEUpXGuw+Q0=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/kms v1.3.7 h1:xcpb2zxYNLoARq+hk91TWDtthIeuTWkhmRkZmSdbOOw=
github.com/tencentcloud/tencentcloud-sdk-go/tencentcloud/kms v1.3.7/go.mod h1:88xdoOQOMaevSlWGF5Ig7g6iNOSjTvAaUzAcOS8xnJc=
github.com/tjfoc/gmsm v1.4.1 h1:aMe1GlZb+0bLjn+cKTPEvvn9oUEBlJitaZiiBwsbgho=
github.com/tjfoc/gmsm v1.4.1/go.mod h1:j4INPkHWMrhJb38G+J6W4Tw0AbuN8Thu3PbdVYhVcTE=
github.com/urfave/cli v1.22.17 h1:SYzXoiPfQjHBbkYxbew5prZHS1TOLT3ierW8SYLqtVQ=
Expand Down
Loading