Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,29 @@ Similarly, the deployment(s) can be torn down using:
./bin/roxie teardown [ <component> ]
```

### Multi-cluster deployments

roxie supports hub + spoke architectures where Central and SecuredCluster run on separate clusters.

1. Deploy Central on the hub cluster:
```bash
MAIN_IMAGE_TAG=4.9.2 ./roxie deploy central
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the (new) docs, I think we should prever roxie deploy -t <version> over MAIN_IMAGE_TAG, which was primarily added for backwards compat reasons with deployment scripts.

```

2. Switch kubectl context to the spoke cluster and deploy SecuredCluster:
```bash
ROX_ADMIN_PASSWORD=<admin-password> \
ROX_CA_CERT_FILE=<path-to-ca-cert> \
./roxie deploy secured-cluster \
--set securedCluster.centralEndpoint=<central-loadbalancer-ip>:443
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The external endpoint is also contained in some environment variable written by roxie, right?

```

> **Note:** The Central endpoint is printed during deployment. If you are in the roxie subshell,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, there it comes.

Maybe move this a bit up, because the above instructions as written direct the user to this mode?

Maybe also mention that for some use cases (automation), --envrc might be the better fit?

> `ROX_ADMIN_PASSWORD` and `ROX_CA_CERT_FILE` are already set, so you can run:
> ```bash
> ./roxie deploy secured-cluster --set securedCluster.centralEndpoint=$API_ENDPOINT
> ```

## Development

Enter the dev shell:
Expand Down
12 changes: 12 additions & 0 deletions cmd/deploy.go
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,18 @@ func runDeploy(cmd *cobra.Command, args []string) error {
}
}

if deploySettings.SecuredCluster.CentralEndpoint != "" {
if !components.IncludesSensor() {
return errors.New("securedCluster.centralEndpoint can only be used when deploying secured-cluster")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about this kind of validation.

I have always thought that roxie just takes what it needs from the provided config. Even if it has a full configuration for component A and only deployment of component B is requested, the config should be usable unmodified.

This validation step would prevent the user from using a single, merged configuration, no?

}
if os.Getenv("ROX_ADMIN_PASSWORD") == "" {
return errors.New("securedCluster.centralEndpoint requires ROX_ADMIN_PASSWORD to be set")
}
if os.Getenv("ROX_CA_CERT_FILE") == "" {
return errors.New("securedCluster.centralEndpoint requires ROX_CA_CERT_FILE to be set")
}
}

d, err := deployer.New(log)
if err != nil {
return fmt.Errorf("failed to create deployer: %w", err)
Expand Down
67 changes: 67 additions & 0 deletions internal/deployer/central_endpoint_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
package deployer

import (
"testing"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)

func TestConfigureSpec_CentralEndpoint(t *testing.T) {
tests := []struct {
name string
centralEndpoint string
centralNamespace string
expected string
}{
{
name: "falls back to internal endpoint",
centralEndpoint: "",
centralNamespace: "acs-central",
expected: "central.acs-central.svc:443",
},
{
name: "falls back to internal endpoint with custom namespace",
centralEndpoint: "",
centralNamespace: "stackrox",
expected: "central.stackrox.svc:443",
},
{
name: "uses provided central endpoint",
centralEndpoint: "central.example.com:443",
centralNamespace: "acs-central",
expected: "central.example.com:443",
},
{
name: "provided endpoint takes precedence over namespace",
centralEndpoint: "10.0.0.1:443",
centralNamespace: "stackrox",
expected: "10.0.0.1:443",
},
}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
sc := &SecuredClusterConfig{
CentralEndpoint: tt.centralEndpoint,
Spec: make(map[string]interface{}),
}
roxie := &RoxieConfig{FeatureFlags: make(map[string]bool)}
central := &CentralConfig{Namespace: tt.centralNamespace}

if err := sc.ConfigureSpec(roxie, central); err != nil {
t.Fatalf("ConfigureSpec failed: %v", err)
}

got, found, err := unstructured.NestedString(sc.Spec, "centralEndpoint")
if err != nil {
t.Fatalf("failed to get centralEndpoint from spec: %v", err)
}
if !found {
t.Fatal("centralEndpoint not found in spec")
}
if got != tt.expected {
t.Errorf("got %q, want %q", got, tt.expected)
}
})
}
}
8 changes: 7 additions & 1 deletion internal/deployer/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ func (c *CentralConfig) CustomResource() (map[string]interface{}, error) {
// SecuredClusterConfig holds deployment settings for the SecuredCluster component.
type SecuredClusterConfig struct {
Namespace string `yaml:"namespace,omitempty"`
CentralEndpoint string `yaml:"centralEndpoint,omitempty"`
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thought about this since yesterday and I keep coming back to the question: why not simply use spec["centralEndpoint"]? This is already a field in the CR and if we want to keep roxie a relative light-weight layer on top of our operator-based installation, we could naturally just use the existing spec field.

ResourceProfile types.ResourceProfile `yaml:"resourceProfile,omitempty"`
PauseReconciliation bool `yaml:"pauseReconciliation,omitempty"`
DeployTimeout time.Duration `yaml:"deployTimeout,omitempty"`
Expand Down Expand Up @@ -234,8 +235,13 @@ func (s *SecuredClusterConfig) ConfigureSpec(roxieConfig *RoxieConfig, centralCo
return err
}

centralEndpoint := internalCentralEndpoint(centralConfig.Namespace)
if s.CentralEndpoint != "" {
centralEndpoint = s.CentralEndpoint
}

if err := helpers.DeepMerge(s.Spec, map[string]interface{}{
"centralEndpoint": internalCentralEndpoint(centralConfig.Namespace),
"centralEndpoint": centralEndpoint,
}); err != nil {
return err
}
Expand Down
4 changes: 4 additions & 0 deletions internal/deployer/deployer.go
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,10 @@ func (d *Deployer) PrintSecuredClusterDeploymentSummary() {
log.Info(cyan.Sprint("│") + createRow("OLM", "Yes"))
}

if d.config.SecuredCluster.CentralEndpoint != "" {
log.Info(cyan.Sprint("│") + createRow("Central Endpoint", d.config.SecuredCluster.CentralEndpoint))
}

log.Info(cyan.Sprint("└" + strings.Repeat("─", boxWidth) + "┘"))
log.Info("")
}
Expand Down
Loading