$ ./deployment/scripts/deploy-openshift.sh
=========================================
🚀 MaaS Platform OpenShift Deployment
=========================================
📋 Checking prerequisites...
Required tools:
- oc:
- jq: jq-1.7
- kustomize: {v5.8.0 2025-11-09T14:39:49Z }
- git: git version 2.43.0
- openssl: OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024)
ℹ️ Note: OpenShift Service Mesh should be automatically installed when GatewayClass is created.
If the Gateway gets stuck in 'Waiting for controller', you may need to manually
install the Red Hat OpenShift Service Mesh operator from OperatorHub.
1️⃣ Checking OpenShift version and Gateway API requirements...
OpenShift version: 4.19.20
✅ OpenShift 4.19.20 supports Gateway API via GatewayClass (no feature gates needed)
2️⃣ Creating namespaces...
ℹ️ Note: If ODH/RHOAI is already installed, some namespaces may already exist
MaaS API namespace: maas-api (set MAAS_API_NAMESPACE env var to override)
namespace/opendatahub created
namespace/kserve created
namespace/kuadrant-system created
namespace/llm created
namespace/maas-api created
3️⃣ Installing dependencies...
Checking for existing Kuadrant installation...
No existing installation found, checking for leftover CRDs...
Installing Kuadrant...
✅ Namespace kuadrant-system already exists
🚀 Creating Kuadrant OperatorGroup...
operatorgroup.operators.coreos.com/kuadrant-operator-group created
🚀 Creating Kuadrant CatalogSource...
catalogsource.operators.coreos.com/kuadrant-operator-catalog created
🚀 Installing kuadrant (via OLM Subscription)...
subscription.operators.coreos.com/kuadrant-operator created
⏳ Waiting for kuadrant-operator-controller-manager deployment to be created... (attempt 1/7)
⏳ Waiting for kuadrant-operator-controller-manager deployment to be created... (attempt 2/7)
⏳ Waiting for operators to be ready...
deployment.apps/kuadrant-operator-controller-manager condition met
deployment.apps/limitador-operator-controller-manager condition met
deployment.apps/authorino-operator condition met
Patching Kuadrant operator...
clusterserviceversion.operators.coreos.com/kuadrant-operator.v1.3.0 patched
✅ Kuadrant operator patched
✅ Successfully installed kuadrant
4️⃣ Deploying Gateway infrastructure...
Cluster domain: apps.rosa.p2ncv-5hnzo-wnj.pehe.p3.openshiftapps.com
Deploying Gateway and GatewayClass...
gatewayclass.gateway.networking.k8s.io/openshift-default serverside-applied
gateway.gateway.networking.k8s.io/openshift-ai-inference serverside-applied
gateway.gateway.networking.k8s.io/maas-default-gateway serverside-applied
5️⃣ Checking for OpenDataHub/RHOAI KServe...
⚠️ KServe not detected. Deploying ODH KServe components...
🚀 Installing odh...
=========================================
🚀 OpenDataHub (ODH) Installation
=========================================
1️⃣ Installing ODH Operator from repository manifests...
Using operator image: quay.io/opendatahub/opendatahub-operator:latest
namespace/opendatahub-operator-system created
/tmp/tmp.paNWyiEk5Y ~/maas-prs/maas-billing
/tmp/tmp.paNWyiEk5Y/opendatahub-operator /tmp/tmp.paNWyiEk5Y ~/maas-prs/maas-billing
go: downloading go1.24.4 (linux/amd64)
mkdir -p /tmp/tmp.paNWyiEk5Y/opendatahub-operator/bin
Downloading sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.3
go: downloading sigs.k8s.io/controller-tools v0.17.3
go: downloading github.com/spf13/cobra v1.9.1
go: downloading golang.org/x/tools v0.30.0
go: downloading github.com/gobuffalo/flect v1.0.3
go: downloading k8s.io/apiextensions-apiserver v0.32.2
go: downloading k8s.io/apimachinery v0.32.2
go: downloading k8s.io/utils v0.0.0-20241104100929-3ea5e8cea738
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading k8s.io/api v0.32.2
go: downloading sigs.k8s.io/yaml v1.4.0
go: downloading github.com/spf13/pflag v1.0.6
go: downloading github.com/google/gofuzz v1.2.0
go: downloading sigs.k8s.io/structured-merge-diff/v4 v4.4.2
go: downloading github.com/mattn/go-colorable v0.1.13
go: downloading golang.org/x/sync v0.11.0
go: downloading sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3
go: downloading github.com/go-logr/logr v1.4.2
go: downloading golang.org/x/mod v0.23.0
go: downloading golang.org/x/sys v0.30.0
go: downloading github.com/fxamacker/cbor/v2 v2.7.0
go: downloading golang.org/x/net v0.35.0
go: downloading github.com/modern-go/reflect2 v1.0.2
go: downloading golang.org/x/text v0.22.0
Downloading sigs.k8s.io/kustomize/kustomize/v5@v5.7.0
go: downloading sigs.k8s.io/kustomize/kustomize/v5 v5.7.0
go: downloading github.com/spf13/cobra v1.8.0
go: downloading sigs.k8s.io/kustomize/api v0.20.0
go: downloading sigs.k8s.io/kustomize/cmd/config v0.20.0
go: downloading sigs.k8s.io/kustomize/kyaml v0.20.0
go: downloading sigs.k8s.io/yaml v1.5.0
go: downloading golang.org/x/text v0.21.0
go: downloading github.com/blang/semver/v4 v4.0.0
go: downloading github.com/sergi/go-diff v1.2.0
go: downloading github.com/go-errors/errors v1.4.2
go: downloading go.yaml.in/yaml/v3 v3.0.3
go: downloading k8s.io/kube-openapi v0.0.0-20241212222426-2c72e554b1e7
go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00
go: downloading github.com/xlab/treeprint v1.2.0
go: downloading gopkg.in/evanphx/json-patch.v4 v4.12.0
go: downloading github.com/google/gnostic-models v0.6.9
go: downloading google.golang.org/protobuf v1.36.1
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/carapace-sh/carapace-shlex v1.0.1
go: downloading github.com/go-openapi/jsonreference v0.20.2
go: downloading github.com/go-openapi/swag v0.23.0
go: downloading github.com/go-openapi/jsonpointer v0.21.0
go: downloading github.com/mailru/easyjson v0.7.7
/tmp/tmp.paNWyiEk5Y/opendatahub-operator/bin/controller-gen --load-build-tags=odh rbac:roleName=controller-manager-role crd:ignoreUnexportedFields=true webhook paths="./..." output:crd:artifacts:config=config/crd/bases output:rbac:artifacts:config=config/rbac output:webhook:artifacts:config=config/webhook
/tmp/tmp.paNWyiEk5Y/opendatahub-operator
namespace/opendatahub-operator-system configured
customresourcedefinition.apiextensions.k8s.io/auths.services.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/dashboards.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/datascienceclusters.datasciencecluster.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/datasciencepipelines.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/dscinitializations.dscinitialization.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/feastoperators.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/featuretrackers.features.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/gatewayconfigs.services.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/hardwareprofiles.infrastructure.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/kserves.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/kueues.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/llamastackoperators.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/modelcontrollers.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/modelregistries.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/modelsasservices.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/monitorings.services.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/rays.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/trainers.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/trainingoperators.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/trustyais.components.platform.opendatahub.io created
customresourcedefinition.apiextensions.k8s.io/workbenches.components.platform.opendatahub.io created
serviceaccount/opendatahub-operator-controller-manager created
clusterrole.rbac.authorization.k8s.io/opendatahub-operator-controller-manager-role created
clusterrole.rbac.authorization.k8s.io/opendatahub-operator-metrics-reader created
clusterrolebinding.rbac.authorization.k8s.io/opendatahub-operator-controller-manager-rolebinding created
service/opendatahub-operator-controller-manager-metrics-service created
service/opendatahub-operator-webhook-service created
deployment.apps/opendatahub-operator-controller-manager created
mutatingwebhookconfiguration.admissionregistration.k8s.io/opendatahub-operator-mutating-webhook-configuration created
validatingwebhookconfiguration.admissionregistration.k8s.io/opendatahub-operator-validating-webhook-configuration created
/tmp/tmp.paNWyiEk5Y ~/maas-prs/maas-billing
~/maas-prs/maas-billing
Waiting for operator to be ready (this may take a few minutes)...
deployment.apps/opendatahub-operator-controller-manager condition met
2️⃣ Creating DSCInitialization resource...
dscinitialization.dscinitialization.opendatahub.io/default-dsci created
Waiting for DSCInitialization to be ready...
Waiting for DSCInitialization to be ready... (1/30)
✅ DSCInitialization is ready
3️⃣ Creating DataScienceCluster...
datasciencecluster.datasciencecluster.opendatahub.io/default-dsc created
Waiting for DataScienceCluster to be ready...
Status: Phase=, Ready= (1/60)
Status: Phase=Not Ready, Ready=False (2/60)
Status: Phase=Not Ready, Ready=False (3/60)
Status: Phase=Not Ready, Ready=False (4/60)
Status: Phase=Not Ready, Ready=False (5/60)
Status: Phase=Not Ready, Ready=False (6/60)
✅ DataScienceCluster is ready
=========================================
📊 Verification
=========================================
DSCInitialization Status:
NAME AGE PHASE CREATED AT
default-dsci 72s Ready 2025-12-11T06:02:34Z
DataScienceCluster Status:
NAME READY REASON
default-dsc True
=========================================
✅ ODH Installation Complete!
=========================================
Next steps:
1. Deploy your models using KServe InferenceService
If you encounter issues, check the logs:
- ODH Operator: kubectl logs -n openshift-operators deployment/opendatahub-operator-controller-manager
- DSCInitialization: kubectl describe dscinitializations default-dsci
- DataScienceCluster: kubectl describe datasciencecluster default-dsc
🎉 Selected components set up successfully for OpenShift!
Setting MAAS_NAMESPACE for odh-model-controller deployment...
Waiting for odh-model-controller deployment to be ready...
deployment.apps/odh-model-controller condition met
✅ MAAS_NAMESPACE already set to maas-api
Waiting for deployment to update...
deployment "odh-model-controller" successfully rolled out
✅ odh-model-controller deployment patched
6️⃣ Waiting for Kuadrant operators to be installed by OLM...
⏳ Waiting for CSV kuadrant-operator.v1.3.0 to succeed (timeout: 300s)...
✅ CSV kuadrant-operator.v1.3.0 succeeded
⏳ Waiting for CSV authorino-operator.v0.22.0 to succeed (timeout: 60s)...
✅ CSV authorino-operator.v0.22.0 succeeded
⏳ Waiting for CSV limitador-operator.v0.16.0 to succeed (timeout: 60s)...
✅ CSV limitador-operator.v0.16.0 succeeded
⏳ Waiting for CSV dns-operator.v0.15.0 to succeed (timeout: 60s)...
✅ CSV dns-operator.v0.15.0 succeeded
Verifying Kuadrant CRDs are available...
⏳ Waiting for CRD kuadrants.kuadrant.io to appear (timeout: 30s)…
✅ CRD kuadrants.kuadrant.io detected, waiting for it to become Established...
customresourcedefinition.apiextensions.k8s.io/kuadrants.kuadrant.io condition met
⏳ Waiting for CRD authpolicies.kuadrant.io to appear (timeout: 10s)…
✅ CRD authpolicies.kuadrant.io detected, waiting for it to become Established...
customresourcedefinition.apiextensions.k8s.io/authpolicies.kuadrant.io condition met
⏳ Waiting for CRD ratelimitpolicies.kuadrant.io to appear (timeout: 10s)…
✅ CRD ratelimitpolicies.kuadrant.io detected, waiting for it to become Established...
customresourcedefinition.apiextensions.k8s.io/ratelimitpolicies.kuadrant.io condition met
⏳ Waiting for CRD tokenratelimitpolicies.kuadrant.io to appear (timeout: 10s)…
✅ CRD tokenratelimitpolicies.kuadrant.io detected, waiting for it to become Established...
customresourcedefinition.apiextensions.k8s.io/tokenratelimitpolicies.kuadrant.io condition met
7️⃣ Deploying Kuadrant configuration (now that CRDs exist)...
kuadrant.kuadrant.io/kuadrant created
8️⃣ Deploying MaaS API with in-cluster TLS...
Ensuring self-signed TLS materials and CA ConfigMap for MaaS API...
🔐 Generating self-signed certificate for maas-api
.+...+...+...+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+......+.+.....+.............+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+.....................+..+.............+..+.....................+.+....................+......+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
..........+......+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+.....+.......+...+..+................+.....+...+.........+.+.....+......+...+..........+...+........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+....+...+...........+.......+..+...+....+...+..+.........+....+..+..................+...+.................................+................+......+.........+.....+..........+...+...+.....+............+................+....................+...............+......+.........+.........+..........+........+...............+.............+.........+.....+.+......+.....+...+.............+..+......+...............+.+.....+......+........................+...+...+...+.......+..+.+.....+....+...........+.+.....+...+.......+........+.......+.....+....+..+........................+...+......+.......+..+.......+.....+...+.............+...+..+...+.......+........+......+................+..+.+.....................+......+.....+....+...+...........+.......+..+...+.........+......+...+....+......+.....+....+...+...+...........+..........+...+.....+...+.........................+..+.+..+...+.........+...+....+.........+...............+...............+.....................+...+............+.....+.........+................+...+..+............................+.....+...+...+.......+.....+.........+.........+................+...+..+.............+........+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----
📦 Creating/Updating secret maas-api-backend-tls in namespace maas-api
secret/maas-api-backend-tls created
✅ Secret maas-api-backend-tls is ready
Applying MaaS API TLS overlay...
serviceaccount/maas-api created
clusterrole.rbac.authorization.k8s.io/maas-api created
clusterrolebinding.rbac.authorization.k8s.io/maas-api created
configmap/maas-api-ca-cert created
configmap/tier-to-group-mapping created
service/maas-api created
persistentvolumeclaim/maas-api-db created
Warning: resource deployments/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/authorino configured
deployment.apps/maas-api created
httproute.gateway.networking.k8s.io/maas-api-route created
authpolicy.kuadrant.io/maas-api-auth-policy created
destinationrule.networking.istio.io/maas-api-backend-tls created
Warning: resource authorinos/authorino is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
authorino.operator.authorino.kuadrant.io/authorino configured
Restarting MaaS API to pick up TLS configuration...
deployment.apps/maas-api restarted
Waiting for MaaS API rollout to complete...
Waiting for deployment "maas-api" rollout to finish: 1 old replicas are pending termination...
kubectl get pods --all-namespaces^[[A^[[AWaiting for deployment "maas-api" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "maas-api" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "maas-api" rollout to finish: 1 old replicas are pending termination...
deployment "maas-api" successfully rolled out
Restarting Kuadrant operator to apply Gateway API provider recognition...
deployment.apps/kuadrant-operator-controller-manager restarted
Waiting for Kuadrant operator to be ready...
Waiting for deployment "kuadrant-operator-controller-manager" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment spec update to be observed...
Waiting for deployment spec update to be observed...
Waiting for deployment spec update to be observed...
Waiting for deployment "kuadrant-operator-controller-manager" rollout to finish: 1 old replicas are pending termination...
deployment "kuadrant-operator-controller-manager" successfully rolled out
🔟 Waiting for Gateway to be ready...
Note: This may take a few minutes if Service Mesh is being automatically installed...
✅ Service Mesh operator already detected
Waiting for Gateway to become ready...
gateway.gateway.networking.k8s.io/maas-default-gateway condition met
1️⃣1️⃣ Applying Gateway Policies...
authpolicy.kuadrant.io/gateway-auth-policy serverside-applied
ratelimitpolicy.kuadrant.io/gateway-rate-limits serverside-applied
tokenratelimitpolicy.kuadrant.io/gateway-token-rate-limits serverside-applied
TLS backend enabled; patching AuthPolicy metadata lookup to HTTPS
authpolicy.kuadrant.io/gateway-auth-policy patched
1️⃣3️⃣ Patching AuthPolicy with correct audience...
Attempting to detect audience...
Token created successfully
JWT payload extracted
Payload decoded successfully
Detected audience: https://rh-oidc.s3.us-east-1.amazonaws.com/27bd6cg0vs7nn08mue83fbof94dj4m9a
authpolicy.kuadrant.io/maas-api-auth-policy patched
✅ AuthPolicy patched
1️⃣4️⃣ Updating Limitador image for metrics exposure...
limitador.limitador.kuadrant.io/limitador patched
✅ Limitador image updated
=========================================
⚠️ TEMPORARY WORKAROUNDS (TO BE REMOVED)
=========================================
Applying temporary workarounds for known issues...
🔧 Restarting Kuadrant, Authorino, and Limitador operators to refresh webhook configurations...
pod "authorino-86568c87d7-k6f7c" deleted from kuadrant-system namespace
pod "kuadrant-operator-controller-manager-6464cc7dd4-mkkkj" deleted from kuadrant-system namespace
pod "kuadrant-operator-controller-manager-7f88c6995-47f9q" deleted from kuadrant-system namespace
pod "limitador-operator-controller-manager-84d8fbb794-wpz2k" deleted from kuadrant-system namespace
✅ Kuadrant operator restarted
deployment.apps/authorino-operator restarted
✅ Authorino operator restarted
deployment.apps/limitador-operator-controller-manager restarted
✅ Limitador operator restarted
Waiting for operators to be ready...
Waiting for deployment "kuadrant-operator-controller-manager" rollout to finish: 0 of 1 updated replicas are available...
deployment "kuadrant-operator-controller-manager" successfully rolled out
deployment "authorino-operator" successfully rolled out
Waiting for deployment "limitador-operator-controller-manager" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "limitador-operator-controller-manager" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "limitador-operator-controller-manager" rollout to finish: 1 old replicas are pending termination...
deployment "limitador-operator-controller-manager" successfully rolled out
=========================================
Deploying observability components...
telemetrypolicy.extensions.kuadrant.io/user-group created
servicemonitor.monitoring.coreos.com/limitador-metrics created
✅ Observability components deployed
=========================================
✅ Deployment Complete!
=========================================
📊 Status Check:
Component Status:
MaaS API pods running: 1
Kuadrant pods running: 9
KServe pods running: 2
Gateway Status:
Accepted: True
Programmed: True
Policy Status:
AuthPolicy: False
TokenRateLimitPolicy: False
Policy Enforcement Status:
AuthPolicy Enforced:
RateLimitPolicy Enforced:
TokenRateLimitPolicy Enforced:
TelemetryPolicy Enforced: True
=========================================
🔧 Troubleshooting:
=========================================
If policies show 'Not enforced' status:
1. Check if Gateway API provider is recognized:
kubectl describe authpolicy gateway-auth-policy -n openshift-ingress | grep -A 5 'Status:'
2. If Gateway API provider is not installed, restart all Kuadrant operators:
kubectl rollout restart deployment/kuadrant-operator-controller-manager -n kuadrant-system
kubectl rollout restart deployment/authorino-operator -n kuadrant-system
kubectl rollout restart deployment/limitador-operator-controller-manager -n kuadrant-system
3. Check if OpenShift Gateway Controller is available:
kubectl get gatewayclass
4. If policies still show 'MissingDependency', ensure environment variable is set:
kubectl get deployment kuadrant-operator-controller-manager -n kuadrant-system -o jsonpath='{.spec.template.spec.containers[0].env[?(@.name=="ISTIO_GATEWAY_CONTROLLER_NAMES")]}'
5. If environment variable is missing, patch the deployment:
kubectl -n kuadrant-system patch deployment kuadrant-operator-controller-manager --type='json' \
-p='[{"op": "add", "path": "/spec/template/spec/containers/0/env/-", "value": {"name": "ISTIO_GATEWAY_CONTROLLER_NAMES", "value": "openshift.io/gateway-controller/v1"}}]'
6. Restart Kuadrant operator after patching:
kubectl rollout restart deployment/kuadrant-operator-controller-manager -n kuadrant-system
kubectl rollout status deployment/kuadrant-operator-controller-manager -n kuadrant-system --timeout=60s
7. Wait for policies to be enforced (may take 1-2 minutes):
kubectl describe authpolicy gateway-auth-policy -n openshift-ingress | grep -A 10 'Status:'
If metrics are not visible in Prometheus:
1. Check ServiceMonitor:
kubectl get servicemonitor limitador-metrics -n kuadrant-system
2. Check Prometheus targets:
kubectl port-forward -n openshift-monitoring svc/prometheus-k8s 9090:9091 &
# Visit http://localhost:9090/targets and look for limitador targets
If webhook timeout errors occur during model deployment:
1. Restart ODH model controller:
kubectl rollout restart deployment/odh-model-controller -n opendatahub
2. Temporarily bypass webhook:
kubectl patch validatingwebhookconfigurations validating.odh-model-controller.opendatahub.io --type='json' -p='[{"op": "replace", "path": "/webhooks/1/failurePolicy", "value": "Ignore"}]'
# Deploy your model, then restore:
kubectl patch validatingwebhookconfigurations validating.odh-model-controller.opendatahub.io --type='json' -p='[{"op": "replace", "path": "/webhooks/1/failurePolicy", "value": "Fail"}]'
If API calls return 404 errors (Gateway routing issues):
1. Check HTTPRoute status:
kubectl get httproute -A
kubectl describe httproute facebook-opt-125m-simulated-kserve-route -n llm
2. Check if model is accessible directly:
kubectl get pods -n llm
kubectl port-forward -n llm svc/facebook-opt-125m-simulated-kserve-workload-svc 8080:8000 &
curl -k https://localhost:8080/health
3. Test model with correct name and HTTPS:
curl -k -H "Content-Type: application/json" -d '{"model": "facebook/opt-125m", "prompt": "Hello", "max_tokens": 50}' https://localhost:8080/v1/chat/completions
4. Check Gateway status:
kubectl get gateway -A
kubectl describe gateway maas-default-gateway -n openshift-ingress
If metrics are not generated despite successful API calls:
1. Verify policies are enforced:
kubectl describe authpolicy gateway-auth-policy -n openshift-ingress | grep -A 5 'Enforced'
kubectl describe ratelimitpolicy gateway-rate-limits -n openshift-ingress | grep -A 5 'Enforced'
2. Check Limitador metrics directly:
kubectl port-forward -n kuadrant-system svc/limitador-limitador 8080:8080 &
curl http://localhost:8080/metrics | grep -E '(authorized_hits|authorized_calls|limited_calls)'
3. Make test API calls to trigger metrics:
# Use HTTPS and correct model name: facebook/opt-125m
for i in {1..5}; do curl -k -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"model": "facebook/opt-125m", "prompt": "Hello $i", "max_tokens": 50}' "https://${HOST}/llm/facebook-opt-125m-simulated/v1/chat/completions"; done
=========================================
📝 Next Steps:
=========================================
1. Deploy a sample model:
kustomize build docs/samples/models/simulator | kubectl apply -f -
2. Get Gateway endpoint:
CLUSTER_DOMAIN=$(kubectl get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')
HOST="maas.${CLUSTER_DOMAIN}"
3. Get authentication token:
TOKEN_RESPONSE=$(curl -sSk -H "Authorization: Bearer $(oc whoami -t)" -H "Content-Type: application/json" -X POST -d '{"expiration": "10m"}' "https://${HOST}/maas-api/v1/tokens")
TOKEN=$(echo $TOKEN_RESPONSE | jq -r .token)
4. Test model endpoint:
MODELS=$(curl -sSk ${HOST}/maas-api/v1/models -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" | jq -r .)
MODEL_NAME=$(echo $MODELS | jq -r '.data[0].id')
MODEL_URL="${HOST}/llm/facebook-opt-125m-simulated/v1/chat/completions" # Note: This may be different for your model
curl -sSk -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"prompt\": \"Hello\", \"max_tokens\": 50}" "${MODEL_URL}"
5. Test authorization limiting (no token 401 error):
curl -sSk -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"prompt\": \"Hello\", \"max_tokens\": 50}" "${MODEL_URL}" -v
6. Test rate limiting (200 OK followed by 429 Rate Limit Exceeded after about 4 requests):
for i in {1..16}; do curl -sSk -o /dev/null -w "%{http_code}\n" -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"prompt\": \"Hello\", \"max_tokens\": 50}" "${MODEL_URL}"; done
7. Run validation script (Runs all the checks again):
./deployment/scripts/validate-deployment.sh
8. Check metrics generation:
kubectl port-forward -n kuadrant-system svc/limitador-limitador 8080:8080 &
curl http://localhost:8080/metrics | grep -E '(authorized_hits|authorized_calls|limited_calls)'
9. Access Prometheus to view metrics:
kubectl port-forward -n openshift-monitoring svc/prometheus-k8s 9090:9091 &
# Open http://localhost:9090 in browser and search for: authorized_hits, authorized_calls, limited_calls
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ kustomize build docs/samples/models/simulator | kubectl apply -f -
llminferenceservice.serving.kserve.io/facebook-opt-125m-simulated created
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ CLUSTER_DOMAIN=$(kubectl get ingresses.config.openshift.io cluster -o jsonpath='{.spec.domain}')
HOST="maas.${CLUSTER_DOMAIN}"
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ TOKEN_RESPONSE=$(curl -sSk -H "Authorization: Bearer $(oc whoami -t)" -H "Content-Type: application/json" -X POST -d '{"expiration": "10m"}' "https://${HOST}/maas-api/v1/tokens")
TOKEN=$(echo $TOKEN_RESPONSE | jq -r .token)
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ echo $TOKEN_RESPONSE
{"token":"eyJhbGciOiJSUzI1NiIsImtpZCI6InA2NmxtWG5xbEtIaGMycW4xS2YteHlQY18zOG9CNUhPd1RyTjl3eGpCSjQifQ.eyJhdWQiOlsibWFhcy1kZWZhdWx0LWdhdGV3YXktc2EiXSwiZXhwIjoxNzY1NDMzODQxLCJpYXQiOjE3NjU0MzMyNDEsImlzcyI6Imh0dHBzOi8vcmgtb2lkYy5zMy51cy1lYXN0LTEuYW1hem9uYXdzLmNvbS8yN2JkNmNnMHZzN25uMDhtdWU4M2Zib2Y5NGRqNG05YSIsImp0aSI6IjBkMjVlNzZjLWM1MjYtNGIwOC1hOGRjLWJkMDgzNzQxMjQ0YiIsImt1YmVybmV0ZXMuaW8iOnsibmFtZXNwYWNlIjoibWFhcy1kZWZhdWx0LWdhdGV3YXktdGllci1mcmVlIiwic2VydmljZWFjY291bnQiOnsibmFtZSI6ImNsdXN0ZXItYWRtaW4tYjA5MDY3YTYiLCJ1aWQiOiIxN2RiYzNjOS03YTc0LTRiMGQtYWQ0OC04NTQ5YmQ2M2Y4MTMifX0sIm5iZiI6MTc2NTQzMzI0MSwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Om1hYXMtZGVmYXVsdC1nYXRld2F5LXRpZXItZnJlZTpjbHVzdGVyLWFkbWluLWIwOTA2N2E2In0.M23Gz6KV2QSO3nGgCsDD0MIpC1aul5y2KShri5g2nij7i7NEsDUsLgGebqkSFqLAThlgPsMA4fj1R0nFAHI6TFUUE70Zd_lA9_dMa2NSMTUt_YrMtHbHgTTVthKX-1IFoYy4xrmy5TV2DsRAzZvYjJ5PuH2RxzThmt8qDd-XYe2xCCEh_RSBz0FUQH_M5PyF3kMPlveQZkLCzJ_kFDRr82tnwVNwPLNgfB1C0tbuE_jsrD7tI5KIfPRAmBVpHdOO_hkyVTTfgHRBhO7QgD2HP4gRXYaE_Ju11dd7OtMGPqapO0ouFof5Qhb3E9fu-YX7xURaJUEgRYfo1UWrgEbqSUgCL0e8OYEvR4nJfUMpo9NXNaITCBIMOiiMePg8i-MPf7Nts0Qfvzo8vlAf_lXO8CR7EHffUF9MyfFPER5KtQq4mCODGyayi5IQPY0oEVtcLk2gj4RInpeggiUUy-CF7dvC7Xd91q9t7mJgZmPbBv39kf5O43cCVrDCL1gure6q-RAQqz0V2AtADWmWk1RLJJPNl5ea05j6jWRzXH6klue6_jqszvdMmwaKhcjLZtuhTSFadXnLwYUj8nFcuZcNovTpuIknB01SdbipEyZun35mVML1ThwBhpk3HEpI0pQQjF5VhoiM5Syv02FjOvhWlKzaxYNhbOS64rNZ_Q6Cl8k","expiration":"10m0s","expiresAt":1765433841,"issuedAt":1765433241,"jti":"0d25e76c-c526-4b08-a8dc-bd083741244b"}
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ k get llminferenceservices -A
NAMESPACE NAME URL READY REASON AGE
llm facebook-opt-125m-simulated http://maas.apps.rosa.p2ncv-5hnzo-wnj.pehe.p3.openshiftapps.com/llm/facebook-opt-125m-simulated True 53s
brent@ip-172-31-33-128:~/maas-prs/maas-billing$ MODELS=$(curl -sSk ${HOST}/maas-api/v1/models -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" | jq -r .)
MODEL_NAME=$(echo $MODELS | jq -r '.data[0].id')
MODEL_URL="${HOST}/llm/facebook-opt-125m-simulated/v1/chat/completions" # Note: This may be different for your model
curl -sSk -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d "{\"model\": \"${MODEL_NAME}\", \"prompt\": \"Hello\", \"max_tokens\": 50}" "${MODEL_URL}"
{"id":"chatcmpl-bed174f6-e3bf-40f5-a5fd-1deaf2a7b101","created":1765433291,"model":"facebook/opt-125m","usage":{"prompt_tokens":0,"completion_tokens":14,"total_tokens":14},"object":"chat.completion","do_remote_decode":false,"do_remote_prefill":false,"remote_block_ids":null,"remote_engine_id":"","remote_host":"","remote_port":0,"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"The rest is silence. To be or not to be that is the "}}]}
Created
December 11, 2025 06:40
-
-
Save nerdalert/de4941462dad202e9f389259a7e17ffd to your computer and use it in GitHub Desktop.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment