Skip to content

Commit

Permalink
Add support multiple config for token providers (#99)
Browse files Browse the repository at this point in the history 
* Add support multiple config for token providers

- This is a change to the existing functionality but is backwards compatible and hence feature parity.
- Code before this change was supporting just one configuration for an ID provider. So, if an org had multiple azure configurations which needed supporting, that was not possible. The usecase could be to have different azure configuration for say prod and non-prod.
- The configurations could now be provided as a list against the ID provider. The targets can then be added within that configuration rather than a global configuration for all the targets which was the case before. Readme has been updated to reflect the new configuration along with the deprecation information of the old config format.
  • Loading branch information
@supreethrao
supreethrao authored Mar 11, 2024
1 parent 1f72b12 commit 0dbe1e5
Show file tree
Hide file tree
Showing 21 changed files with 722 additions and 304 deletions.
210 changes: 125 additions & 85 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -100,16 +100,16 @@ With a [configuration](#client-configuration) file like:
```yaml
providers:
osprey:
targets:
local.cluster:
server: https://osprey.local.cluster
foo.cluster:
server: https://osprey.foo.cluster
alias: [foo]
groups: [foo, foobar]
bar.cluster:
server: https://osprey.bar.cluster
groups: [bar, foobar]
- targets:
local.cluster:
server: https://osprey.local.cluster
foo.cluster:
server: https://osprey.foo.cluster
alias: [foo]
groups: [foo, foobar]
bar.cluster:
server: https://osprey.bar.cluster
groups: [bar, foobar]
```
The `groups` are labels that allow the targets to be organised into categories.
Expand Down Expand Up @@ -239,7 +239,12 @@ installed version.
The client uses a YAML configuration file. Its recommended location is:
`$HOME/.osprey/config`. Its contents are as follows:
### V2 Config
The structure of the osprey configuration supports multiple configuration for a provider type.
This structure will support scenarios where different azure providers can be configured for prod and non-prod targets.
```yaml
apiVersion: v2
# Optional path to the kubeconfig file to load/update when loging in.
# Uses kubectl defaults if absent ($HOME/.kube/config).
# kubeconfig: /home/jdoe/.kube/config
Expand All @@ -248,75 +253,107 @@ The client uses a YAML configuration file. Its recommended location is:
# When this value is defined, all targets must define at least one group.
# default-group: my-group
# Named map of supported providers (currently `osprey` and `azure`)
## Named map of supported providers (currently `osprey` and `azure`)
providers:
osprey:
# CA cert to use for HTTPS connections to Osprey.
# Uses system's CA certs if absent.
# certificate-authority: /tmp/osprey-238319279/cluster_ca.crt
# Alternatively, a Base64-encoded PEM format certificate.
# This will override certificate-authority if specified.
# certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk5vdCB2YWxpZAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
# Named map of target Osprey servers to contact for access-tokens
targets:
# Target Osprey's environment name.
# Used for the name of the cluster, context, and users generated
foo.cluster:
# hostname:port of the target osprey server
server: https://osprey.foo.cluster
# list of names to generate additional contexts against the target.
aliases: [foo.alias]
# list of names that can be used to logically group different Osprey servers.
groups: [foo]
# CA cert to use for HTTPS connections to Osprey.
# Uses system's CA certs if absent.
# certificate-authority: /tmp/osprey-238319279/cluster_ca.crt
# Alternatively, a Base64-encoded PEM format certificate.
# This will override certificate-authority if specified.
# certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk5vdCB2YWxpZAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
- provider-name: (Optional)
# CA cert to use for HTTPS connections to Osprey.
# Uses system's CA certs if absent.
# certificate-authority: /tmp/osprey-238319279/cluster_ca.crt
# Alternatively, a Base64-encoded PEM format certificate.
# This will override certificate-authority if specified.
# certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk5vdCB2YWxpZAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
# Named map of target Osprey servers to contact for access-tokens
targets:
# Target Osprey's environment name.
# Used for the name of the cluster, context, and users generated
foo.cluster:
# hostname:port of the target osprey server
server: https://osprey.foo.cluster
# list of names to generate additional contexts against the target.
aliases: [foo.alias]
# list of names that can be used to logically group different Osprey servers.
groups: [foo]
# CA cert to use for HTTPS connections to Osprey.
# Uses system's CA certs if absent.
# certificate-authority: /tmp/osprey-238319279/cluster_ca.crt
# Alternatively, a Base64-encoded PEM format certificate.
# This will override certificate-authority if specified.
# certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk5vdCB2YWxpZAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
# Authenticating against Azure AD
azure:
# These settings are required when authenticating against Azure
tenant-id: your-azure-tenant-id
server-application-id: azure-ad-server-application-id
client-id: azure-ad-client-id
client-secret: azure-ad-client-secret
# List of scopes to request as part of the request. This should be an Azure link to the API exposed on the server application
scopes:
- "api://azure-tenant-id/Kubernetes.API.All"
# This is required for the browser-based authentication flow. The port is configurable, but it must conform to
# the format: http://localhost:<port>/auth/callback
redirect-uri: http://localhost:65525/auth/callback
targets:
foo.cluster:
server: http://osprey.foo.cluster
# If "use-gke-clientconfig" is specified (default false) Osprey will fetch the API server URL and its
# CA cert from the GKE-specific ClientConfig resource in kube-public. This resource is created automatically
# by GKE when you enable to OIDC Identity Service. The "api-server" config element is also required.
# Usually "api-server" would be set to the public API server endpoint; the fetched API server URL will be
# the internal load balancer that proxies requests through the OIDC service.
# use-gke-clientconfig: true
#
# If "skip-tls-verify" is specified (default false) Osprey will skip TLS verification when attempting
# to make the connection to the specified server. This can be used in conjunction with `server` or `api-server`.
# skip-tls-verify: true
#
# If api-server is specified (default ""), Osprey will fetch the CA cert from the API server itself.
# Overrides "server". A ConfigMap in kube-public called kube-root-ca.crt should be made accessible
# to the system:anonymous group. This ConfigMap is created automatically with the Kubernetes feature
# gate RootCAConfigMap which was alpha in Kubernetes v1.13 and became enabled by default in v1.20+
# api-server: http://apiserver.foo.cluster
aliases: [foo.alias]
groups: [foo]
- name: (Optional)
# These settings are required when authenticating against Azure
tenant-id: your-azure-tenant-id
server-application-id: azure-ad-server-application-id
client-id: azure-ad-client-id
client-secret: azure-ad-client-secret
# List of scopes to request as part of the request. This should be an Azure link to the API exposed on the server application
scopes:
- "api://azure-tenant-id/Kubernetes.API.All"
# This is required for the browser-based authentication flow. The port is configurable, but it must conform to
# the format: http://localhost:<port>/auth/callback
redirect-uri: http://localhost:65525/auth/callback
targets:
foo.cluster:
server: http://osprey.foo.cluster
# If "use-gke-clientconfig" is specified (default false) Osprey will fetch the API server URL and its
# CA cert from the GKE-specific ClientConfig resource in kube-public. This resource is created automatically
# by GKE when you enable to OIDC Identity Service. The "api-server" config element is also required.
# Usually "api-server" would be set to the public API server endpoint; the fetched API server URL will be
# the internal load balancer that proxies requests through the OIDC service.
# use-gke-clientconfig: true
#
# If "skip-tls-verify" is specified (default false) Osprey will skip TLS verification when attempting
# to make the connection to the specified server. This can be used in conjunction with `server` or `api-server`.
# skip-tls-verify: true
#
# If api-server is specified (default ""), Osprey will fetch the CA cert from the API server itself.
# Overrides "server". A ConfigMap in kube-public called kube-root-ca.crt should be made accessible
# to the system:anonymous group. This ConfigMap is created automatically with the Kubernetes feature
# gate RootCAConfigMap which was alpha in Kubernetes v1.13 and became enabled by default in v1.20+
# api-server: http://apiserver.foo.cluster
aliases: [foo.alias]
groups: [foo]
```

### V1 Config (Deprecated)
This is the previously supported format.
The fields are the same but, the provider configuration is mapped to a provider type as opposed to being a list.
The config parsing will use this format unless specified to v2 on the apiVersion field in the config.
```yaml
providers:
osprey:
targets:
local.cluster:
server: https://osprey.local.cluster
foo.cluster:
server: https://osprey.foo.cluster
alias: [foo]
groups: [foo, foobar]
bar.cluster:
server: https://osprey.bar.cluster
groups: [bar, foobar]

# Authenticating against Azure AD
azure:
tenant-id: your-tenant-id
server-application-id: api://SERVER-APPLICATION-ID # Application ID of the "Osprey - Kubernetes APIserver"
client-id: azure-application-client-id # Client ID for the "Osprey - Client" application
client-secret: azure-application-client-secret # Client Secret for the "Osprey - Client" application
scopes:
# This must be in the format "api://" due to non-interactive logins appending this to the audience in the JWT.
- "api://SERVER-APPLICATION-ID/Kubernetes.API.All"
redirect-uri: http://localhost:65525/auth/callback # Redirect URI configured for the "Osprey - Client" application
targets: ...
```
The name of the configured targets will be used to name the managed clusters,
Expand Down Expand Up @@ -625,8 +662,12 @@ $ make
We use [Cobra](https://github.com/spf13/cobra), to generate the client and server commands.

### E2E tests

The e2e tests are executed against local Dex and LDAP servers.

Note: The below docker image is only for linux/amd64. You might be able to get it working with other architectures, but it's not officially supported yet.
The e2e tests are executed against local Dex and LDAP servers. There is a Dockerfile located in the `e2e/` directory that will handle the dependencies for you. (This came around due to dependency issues with older versions of openldap in ubuntu).

The setup is as follows:

Osprey Client (1) -> (*) Osprey Server (1) -> (1) Dex (*) -> (1) LDAP
Expand Down Expand Up @@ -654,8 +695,7 @@ To run the test locally, run the following command
`docker run -it -v <osprey root folder>:/osprey local-osprey-e2etest:1`
3. Inside the container run make test
```
cd /osprey
export PATH=$PATH:/osprey/build/bin/linux_amd64
make build
make test
```

Expand Down Expand Up @@ -730,14 +770,14 @@ The client ID and secrets generated in this section are used to fill out the Osp
```yaml
providers:
azure:
tenant-id: your-tenant-id
server-application-id: api://SERVER-APPLICATION-ID # Application ID of the "Osprey - Kubernetes APIserver"
client-id: azure-application-client-id # Client ID for the "Osprey - Client" application
client-secret: azure-application-client-secret # Client Secret for the "Osprey - Client" application
scopes:
# This must be in the format "api://" due to non-interactive logins appending this to the audience in the JWT.
- "api://SERVER-APPLICATION-ID/Kubernetes.API.All"
redirect-uri: http://localhost:65525/auth/callback # Redirect URI configured for the "Osprey - Client" application
- tenant-id: your-tenant-id
server-application-id: api://SERVER-APPLICATION-ID # Application ID of the "Osprey - Kubernetes APIserver"
client-id: azure-application-client-id # Client ID for the "Osprey - Client" application
client-secret: azure-application-client-secret # Client Secret for the "Osprey - Client" application
scopes:
# This must be in the format "api://" due to non-interactive logins appending this to the audience in the JWT.
- "api://SERVER-APPLICATION-ID/Kubernetes.API.All"
redirect-uri: http://localhost:65525/auth/callback # Redirect URI configured for the "Osprey - Client" application
```
Kubernetes API server flags:
Expand Down
24 changes: 13 additions & 11 deletions client/azure.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ const (

// AzureConfig holds the configuration for Azure
type AzureConfig struct {
// Name provides a named reference to the provider. For e.g sky-azure, nbcu-azure etc. Optional field
Name string `yaml:"name,omitempty"`
// ServerApplicationID is the oidc-client-id used on the apiserver configuration
ServerApplicationID string `yaml:"server-application-id,omitempty"`
// ClientID is the oidc client id used for osprey
Expand Down Expand Up @@ -78,27 +80,27 @@ func (ac *AzureConfig) ValidateConfig() error {
}

// NewAzureRetriever creates new Azure oAuth client
func NewAzureRetriever(provider *AzureConfig, options RetrieverOptions) (Retriever, error) {
func NewAzureRetriever(provider *ProviderConfig, options RetrieverOptions) (Retriever, error) {
config := oauth2.Config{
ClientID: provider.ClientID,
ClientSecret: provider.ClientSecret,
RedirectURL: provider.RedirectURI,
Scopes: provider.Scopes,
ClientID: provider.clientID,
ClientSecret: provider.clientSecret,
RedirectURL: provider.redirectURI,
Scopes: provider.scopes,
}
if provider.IssuerURL == "" {
provider.IssuerURL = fmt.Sprintf("https://login.microsoftonline.com/%s/%s", provider.AzureTenantID, wellKnownConfigurationURI)
if provider.issuerURL == "" {
provider.issuerURL = fmt.Sprintf("https://login.microsoftonline.com/%s/%s", provider.azureTenantID, wellKnownConfigurationURI)
} else {
provider.IssuerURL = fmt.Sprintf("%s/%s", provider.IssuerURL, wellKnownConfigurationURI)
provider.issuerURL = fmt.Sprintf("%s/%s", provider.issuerURL, wellKnownConfigurationURI)
}

oidcEndpoint, err := oidc.GetWellKnownConfig(provider.IssuerURL)
oidcEndpoint, err := oidc.GetWellKnownConfig(provider.issuerURL)
if err != nil {
return nil, fmt.Errorf("unable to query well-known oidc config: %w", err)
}
config.Endpoint = *oidcEndpoint
retriever := &azureRetriever{
oidc: oidc.New(config, provider.ServerApplicationID),
tenantID: provider.AzureTenantID,
oidc: oidc.New(config, provider.serverApplicationID),
tenantID: provider.azureTenantID,
}
retriever.useDeviceCode = options.UseDeviceCode
retriever.loginTimeout = options.LoginTimeout
Expand Down
Loading

0 comments on commit 0dbe1e5

Please sign in to comment.