How to configure dbt pipeline

[GollaSaiVenkatesh]

I have following doubts regarding dbt plugin. I am using giturl for dbt project and i have deployed devlake on k8s.

where i can securely provide git creds of projectGitUrl?

where should i put profiles.yml securely?

[dosubot[bot]]

Hi @GollaSaiVenkatesh! I'm Dosu and I’m helping the incubator-devlake team.

To securely provide Git credentials for your dbt project's Git URL in a DevLake deployment on Kubernetes, use standard Kubernetes practices: store your credentials (like SSH keys, personal access tokens, or a .netrc file) in a Kubernetes Secret, then mount that Secret as a file or inject it as environment variables into the DevLake pod. For SSH, mount your private key at the default location (/root/.ssh/id_rsa or similar) and ensure the pod has the right permissions. For HTTPS with username/password or token, you can mount a .netrc file or set the appropriate environment variables so the git clone command (used by DevLake's dbt plugin) can access the repo securely. DevLake itself does not provide a built-in mechanism for handling these secrets, so this is left to your Kubernetes setup (see code reference).

For securely providing profiles.yml, create a Kubernetes Secret or ConfigMap containing your profiles.yml content, then mount it as a file to the path specified by the profilesPath option in your dbt pipeline config (or to the default project path if not specified). This ensures the file is available to the dbt process at runtime (see code reference). If you don't provide a profiles.yml, DevLake can generate one using the DB_URL environment variable, but for custom configs, mounting your own file is the secure approach.

In summary: use Kubernetes Secrets for sensitive files, mount them into your DevLake pod, and point your dbt pipeline config to the correct paths. There are no DevLake-specific instructions for this, so follow Kubernetes best practices for secret management (deployment example).

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.