Skip to content

Feat/exclude values#1663

Closed
chaimleib wants to merge 19 commits into
securego:masterfrom
chaimleib:feat/exclude-values
Closed

Feat/exclude values#1663
chaimleib wants to merge 19 commits into
securego:masterfrom
chaimleib:feat/exclude-values

Conversation

@chaimleib
Copy link
Copy Markdown

@chaimleib chaimleib commented May 1, 2026

Adds .G101.keys and .G101.values fields to the PathExcludeRule type. If provided with a slice of regex patterns, G101 "Possible hardcoded credentials" issues will also be excluded if the hardcoded-value matches any of the values patterns, or if its key matches any of the keys patterns.

For example, the following code, common in test files, used to trigger an error:

const token = "fakeToken"

With this PR, a config can be set to exclude that issue when encountered in test files, based on the hardcoded value matching a pattern:

{
  "exclude-rules": [{
    "path": ".+_test\\.go$",
    "G101": {
      "values": [
        "(?i)^test",
        "(?i)^fake"
      ]
    }
  }]
}

Variable names can be excluded as well, even if the value looks like a credential:

testPassword := "|0oK5-l1k3_a=Pa5sw0Rd"
{
  "exclude-rules": [{
    "path": ".+_test\\.go$",
    "G101": {
      "keys": ["(?i)^test"]
    }
  }]
}

@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 00:13 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from 656e5c1 to 621cadb Compare May 1, 2026 00:29
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 00:29 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 22:45 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 22:46 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from efef057 to 39c5837 Compare May 1, 2026 22:59
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 22:59 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 23:17 — with GitHub Actions Inactive
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Barry Security Review

Comment thread exclusion_filter.go Outdated
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 23:25 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from e8ccb74 to bc9372b Compare May 1, 2026 23:26
@chaimleib chaimleib temporarily deployed to security-review May 1, 2026 23:26 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 2, 2026 00:11 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from 7927a9b to c8a4d2d Compare May 2, 2026 01:09
@chaimleib chaimleib temporarily deployed to security-review May 2, 2026 01:09 — with GitHub Actions Inactive
@ccojocar
Copy link
Copy Markdown
Member

ccojocar commented May 4, 2026

Thanks for this contribution. This seems AI generated.

Why was required to rename the methods/functions? Can you reduce the size of the changes only to the hardcode credentials?

@chaimleib
Copy link
Copy Markdown
Author

chaimleib commented May 4, 2026

Thanks for this contribution. This seems AI generated.

Why was required to rename the methods/functions? Can you reduce the size of the changes only to the hardcode credentials?

This was not AI generated. I did not even use AI for autocomplete.

I renamed some symbols because they mentioned "Path Exclusion", but I was adding functionality not related to paths. To avoid the misleading implication that PathExclusion symbols dealt only with paths, I removed "Path" from the names.

This is still WIP, and I'm looking into whether I should push the config deeper to be more tightly coupled to G101. I also want to write tests.

EDIT 2026-05-07: Since I really want to restrict the exclusions to particular paths, path_filters.go is the most natural place to trigger the key-value excluder. But I separated the logic out to its own files as much as I could.

@ccojocar
Copy link
Copy Markdown
Member

ccojocar commented May 4, 2026

Please do first the minimum change required to incorporate the functionality you mentioned without refactoring. You can follow up with refactoring if it is needed afterwards. This will make reviewing the change much easier. Thanks

@codecov
Copy link
Copy Markdown

codecov Bot commented May 6, 2026

Codecov Report

❌ Patch coverage is 69.95885% with 73 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.25%. Comparing base (5f4eec9) to head (c8a4d2d).
⚠️ Report is 3 commits behind head on master.

Files with missing lines Patch % Lines
exclusion_filter.go 63.44% 49 Missing and 4 partials ⚠️
rules/hardcoded_credentials.go 83.72% 13 Missing and 1 partial ⚠️
cmd/gosec/main.go 25.00% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1663      +/-   ##
==========================================
- Coverage   80.57%   80.25%   -0.32%     
==========================================
  Files         109      109              
  Lines       10181    10260      +79     
==========================================
+ Hits         8203     8234      +31     
- Misses       1495     1541      +46     
- Partials      483      485       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@ccojocar ccojocar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add some info in the README regarding the usage of this exclusion filter?

Is this introducing any breaking changes for the existing filter format?

Comment thread exclusion_filter.go Outdated
Comment thread exclusion_filter.go Outdated
Comment thread exclusion_filter.go
@chaimleib chaimleib force-pushed the feat/exclude-values branch from c8a4d2d to 8f6972a Compare May 7, 2026 04:44
@chaimleib chaimleib temporarily deployed to security-review May 7, 2026 04:45 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 7, 2026 05:03 — with GitHub Actions Inactive
Copy link
Copy Markdown
Member

@ccojocar ccojocar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also make sure to update the https://github.com/securego/gosec/blob/master/RULES.md#g101 with the added functionality when is implemented.

Comment thread rules/hardcoded_credentials.go
Comment thread rules/hardcoded_credentials.go
Comment thread path_filter.go Outdated
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 04:58 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:06 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:15 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from 99239e0 to 2cc4ef0 Compare May 8, 2026 05:19
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:19 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from 2cc4ef0 to 9d7c266 Compare May 8, 2026 05:30
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:30 — with GitHub Actions Inactive
@chaimleib chaimleib force-pushed the feat/exclude-values branch from 9d7c266 to a722960 Compare May 8, 2026 05:35
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:35 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 05:54 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 06:30 — with GitHub Actions Inactive
@chaimleib
Copy link
Copy Markdown
Author

Can you please add some info in the README regarding the usage of this exclusion filter?

Is this introducing any breaking changes for the existing filter format?

Done.

There should be no breaking changes for the existing filter format. The new feature is only configurable via JSON right now, via a new field in the path filter config. I weighed allowing configs via CLI arguments, but that seemed too complicated at this stage.

@chaimleib
Copy link
Copy Markdown
Author

Also make sure to update the https://github.com/securego/gosec/blob/master/RULES.md#g101 with the added functionality when is implemented.

Done!

@chaimleib chaimleib changed the title WIP: Feat/exclude values Feat/exclude values May 8, 2026
@chaimleib chaimleib marked this pull request as ready for review May 8, 2026 06:37
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 06:59 — with GitHub Actions Inactive
@chaimleib chaimleib temporarily deployed to security-review May 8, 2026 20:06 — with GitHub Actions Inactive
Comment thread issue/issue.go
NoSec bool `json:"nosec"` // true if the issue is nosec
Suppressions []SuppressionInfo `json:"suppressions"` // Suppression info of the issue
Autofix string `json:"autofix,omitempty"` // Proposed auto fix the issue
HardcodedCredentialAddenda *HardcodedCredentialAddenda `json:"hardcoded_credential_addenda,omitempty"` // Info for configurable Hardcoded Credential exclusion rules.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is generic. The name sounds very specific only for hardcoded credentials rule.

Why this information is required to be added into an issue since the goal of the issue is only to add dictional filters on the matching.

Please remove this from the Issue structure. Thanks

Copy link
Copy Markdown
Author

@chaimleib chaimleib May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In one of my previous commits, this field was Addenda any. The problem then was the use of any. I got a comment seeming to say that it was too generic then. Should I change this back to Addenda any so that it becomes generic again? EDIT: The way I was envisioning things at that commit, different rules might add different Addenda fulfilling different interfaces than Keyer and Valuer, which I have since removed. Should I go back to that plan with Addenda any? Or should I stay with what I have, and refrain from imposing an opinion on later features which are not fleshed out yet?

(*PathExclusionFilter).ShouldExclude(*issue.Issue) needs some sort of addenda field, because otherwise it would lack key-value information about the violation. It wouldn't be able to call my new rule.hardcodedCredentialsExcluder.ShouldExcludeKV.

If I removed the call to ShouldExcludeKV from ShouldExclude, and instead made the exclusions at the rule level in rules/hardcoded_credentials.go, then the excluded counter for path-based excludes would fail to increment. In concrete terms, this section in cmd/gosec/main.go would not count the new exclusions:

	// Apply path-based exclusions first
	var pathExcludedCount int
	issues, pathExcludedCount = pathFilter.FilterIssues(issues)
	if pathExcludedCount > 0 {
		logger.Printf("Excluded %d issues by path-based rules", pathExcludedCount)
	}

Do we want that?

}
if r.ignoreEntropy || r.isHighEntropyString(val) {
iss := ctx.NewIssue(n, r.ID(), r.What, r.Severity, r.Confidence)
iss.HardcodedCredentialAddenda = &issue.HardcodedCredentialAddenda{
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the value of this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ensures that all issues reported by this rule include key-value information in a consistent way.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In which context is the key/value information required in addition to the issue itself?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see what's the benefit of having this information into an issue.

Copy link
Copy Markdown
Author

@chaimleib chaimleib May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could avoid this, if path-based exclusions for this rule were processed in rules/hardcoded_credentials.go, rather than in path_filter.go.

However, without some data passed down in the Issue, that would mess up the pathExcludedCount statistics.

The existing logic in path_filter.go only has access to the configs and the issue, so that's why I injected the key and value there. I thought it was important for there to be a single place, path_filter.go, which computes path-based exclusions, and tallies them all up.

But maybe that is not so important. I could compute the path exclusion in the rule, and then have it signal in the issue that it was excluded by the exclude-rules.G101 config:

  • A new ExcludedByPath bool field in the Issue struct could work. That's generic.
  • Or, could I use the existing Suppressions field? I didn't use that, because I thought this was reserved for #nosec and nolint:gosec directives, which include a Justification string. If it's ok to use that field, maybe I should add a justification field to the new config file format?
{
  "exclude-rules": [{
    "path": ".+_test\\.go$",
    "G101": {
      "justification": "Mocked credentials for tests are not real.",
      "values": [
        "(?i)^test",
        "(?i)^fake"
      ]
    }
  }]
}

Comment thread path_filter.go
Rules []string `json:"rules"` // Rule IDs to exclude. Use "*" to exclude all rules
Path string `json:"path"` // Regex pattern for matching file paths
Rules []string `json:"rules"` // Rule IDs to exclude. Use "*" to exclude all rules
G101 HardcodedCredentialExcludeOptions `json:"G101"` // Per-path G101 options
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is generic code. Why G101 is here? This should be named in a generic way, not specifically to a rule.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the JSON configuration file, users are expected to know the rule codes, as in the Rules field just above. As a UI, I tried to stay consistent in how users will name rule-specific options, so I named the rule-specific path exclude options with json:"G101".

(I did not want to change the type of the Rules field to a map[string]any. That would break compatibility with previous config files.)

I was also trying not to be confusing by naming the field in Go too differently from the JSON name. That's why the field name is also G101. Would you prefer the following?

	CredentialExcludeOptions HardcodedCredentialExcludeOptions `json:"G101"`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that this is generic code, not rule specific. You should use generic names for variables, and it should be written in a generic way that it can be reused.

For example this should be:

ExcludeOptions ExcludeOptions 

Copy link
Copy Markdown
Author

@chaimleib chaimleib May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now, in the master branch, this is the best config I can manage for my purpose:

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "rules": [ "G101" ]
  }]
}

My idea was to allow being more particular about the exclusions like this:

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "G101": /* HardcodedCredentialExcludeOptions */ {
      "keys": [ "(?i)^test", "(?i)^fake" ],
      "values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
    }
  }]
}

Are you proposing this?

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "options": /* ExcludeOptions */ {
      "G101": /* HardcodedCredentialExcludeOptions */ {
        "keys": [ "(?i)^test", "(?i)^fake" ],
        "values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
      }
    }
  }]
}

Isn't that more awkward than my idea?

Copy link
Copy Markdown
Author

@chaimleib chaimleib May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, did you mean this?

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "options": /* ExcludeOptions */ {
      "keys": [ "(?i)^test", "(?i)^fake" ],
      "values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
    }
  }]
}

What other rules would benefit from this?

What if you don't want all the rules to use a shared config? There would be no way to provide different rules with separate configs, unless each config was named uniquely to that rule:

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "options": /* ExcludeOptions */ {
      "G101_keys": [ "(?i)^test", "(?i)^fake" ],
      "G101_values": [ "(?i)^test", "(?i)^fake", "\*{4}" ],
      "G999_keys": [ "(?i)^test", "(?i)^fake" ],
      "G999_values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
    }
  }]
}

But that would also get awkward and repetitive. Isn't it better to make a separate object for the configs of each rule?

/* Config */ {
  "exclude-rules": /* []PathExcludeRule */ [{
    "path": ".+_test\\.go$",
    "G101": /* HardcodedCredentialExcludeOptions */ {
      "keys": [ "(?i)^test", "(?i)^fake" ],
      "values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
    },
    "G999": /* TBDExcludeOptions */ {
      "keys": [ "(?i)^test", "(?i)^fake" ],
      "values": [ "(?i)^test", "(?i)^fake", "\*{4}" ]
    }
  }]
}

// HardcodedCredentialExcludeOptions configures whether
// a Hardcoded Credentials issue should be excluded
// from the results, based on options specific to this rule.
type HardcodedCredentialExcludeOptions struct {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be named in a generic way, it's not only meant to be used only by hardcoded credentials rule.

Copy link
Copy Markdown
Author

@chaimleib chaimleib May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of opening this up to allow other rules to use this. But when I looked at the other rules, they were different enough that either these options were irrelevant, or would increase the PR size by a lot.

Did you find another rule where you want to use this type?

To minimize the PR impact, and to keep type ownership and usage clear, I decided to not imply usefulness beyond "hardcoded credentials" (the name for the rule implied by the filename rules/hardcoded_credentials.go).

If a later PR does want to use this for a different rule, that PR can rename the types as needed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above, use a generic name for the variables.

Values []string `json:"values"`
}

type CompiledHardcodedCredentialsRule struct {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Comment thread path_filter.go
original PathExcludeRule // Keep original for error messages
pathRegex *regexp.Regexp
ruleSet map[string]bool // Set of rule IDs to exclude
hardcodedCredentialsExcluder *CompiledHardcodedCredentialsRule
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto, this should be named generically maybe CredentialsRule.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like shorter names. Should this be like so?

	credentailsRule *CompiledCredentialsRule

@ccojocar ccojocar closed this May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants