diff --git a/CHANGELOG.md b/CHANGELOG.md index 4eed82ff..e3fc605a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,40 @@ # Changelog +## 2.13.0 + +### Upgrade notes + +Exit codes change in this release. Check any CI step that matches on a specific +code; see [Exit codes](README.md#exit-codes). + +- Invalid flags exit `2` instead of `1`. `--disable-blocking` doesn't apply to + them. +- Ctrl+C exits `130` instead of `2`. +- Missing tokens and other setup failures now honor `--disable-blocking` and + `--exit-code-on-api-error`. +- `--strict-blocking` now fails on existing blocking alerts. + +### Added + +- Under `--strict-blocking`, PR/MR comments list existing alerts in their own + table. + +### Changed + +- An incomplete scan comparison fails the run with the infrastructure-error + code. +- GitLab security report IDs for CVE findings include the advisory ID, so + existing CVE findings get new IDs once. + +### Fixed + +- `--strict-blocking`, the GitLab security report and FOSSA output include + alerts on unchanged packages. +- `--strict-blocking` no longer lists findings on added packages twice. +- Every CVE on a package is reported, not one per severity. +- A token without `repo:list` is reported as a missing scope, not a missing + repository. + ## 2.12.1 ### Changed: bump pinned Coana CLI to 15.15.1 diff --git a/README.md b/README.md index 5221daf1..b4e8f842 100644 --- a/README.md +++ b/README.md @@ -217,12 +217,13 @@ Minimal pattern: ## Exit codes -| Code | Meaning | -|------|---------| -| `0` | Clean scan — no blocking issues (or `--disable-blocking` set) | -| `1` | Blocking security finding(s) detected | -| `2` | Scan interrupted (SIGINT / Ctrl+C) | -| `3` | Infrastructure or API error (timeout, network failure, unexpected error) | +| Code | Meaning | +|-------|---------| +| `0` | Clean scan — no blocking issues (or `--disable-blocking` set) | +| `1` | Blocking security finding(s) detected | +| `2` | Invalid flags or configuration (usage error), including argparse's own errors | +| `3` | Infrastructure or API error: missing API or SCM token, permission or plan problem, timeout, network failure, incomplete scan comparison, unexpected error | +| `130` | Scan interrupted (SIGINT / Ctrl+C) | `--exit-code-on-api-error ` remaps the infrastructure-error code (`3`) to any value — e.g. a Buildkite @@ -239,9 +240,11 @@ error, so the supervisor's exit status (commonly 124 or 137) takes precedence. The two flags that affect exit codes can cancel each other out, so the order of precedence matters: -- **`--disable-blocking` wins over everything.** It forces exit `0` for *all* - outcomes — security findings *and* infrastructure errors. If you set it, - `--exit-code-on-api-error` has no effect (you'll always get `0`). +- **`--disable-blocking` wins for every run that gets past flag parsing.** It + forces exit `0` for security findings, infrastructure errors and interrupts. + If you set it, `--exit-code-on-api-error` has no effect (you'll always get `0`). + It does not apply to usage errors (`2`): a run with invalid flags never scanned + anything, so it always fails. - **`--exit-code-on-api-error` only applies when `--disable-blocking` is *not* set.** It changes the infra-error code (and the generic-error code); it never touches the security-finding code (`1`). diff --git a/docs/cli-reference.md b/docs/cli-reference.md index 1dcec13e..76a8cdcc 100644 --- a/docs/cli-reference.md +++ b/docs/cli-reference.md @@ -437,7 +437,7 @@ The CLI runs a pinned version of the Coana CLI, the analysis engine; it does **n | Parameter | Required | Default | Description | |:-------------------------|:---------|:--------|:----------------------------------------------------------------------| | `--ignore-commit-files` | False | False | Compare regardless of which files changed, scanning every manifest | -| `--disable-blocking` | False | False | Non-blocking CI mode: the CLI always exits **0**, even when blocking alerts are present (including with `--strict-blocking`). Also exits 0 on uncaught runtime errors and Socket API failures, so the job is treated as successful while findings and errors are still logged. Takes precedence over `--strict-blocking`. | +| `--disable-blocking` | False | False | Non-blocking CI mode: the CLI exits **0** even when blocking alerts are present (including with `--strict-blocking`). Also exits 0 on infrastructure errors (missing API or SCM token, Socket API failures, uncaught runtime errors) and interrupts, so the job is treated as successful while findings and errors are still logged. Invalid flags still exit **2**. Takes precedence over `--strict-blocking`. | | `--disable-ignore` | False | False | Disable support for `@SocketSecurity ignore` commands in PR comments. When set, alerts cannot be suppressed via comments and ignore instructions are hidden from comment output. See [Who can ignore an alert](#who-can-ignore-an-alert). | | `--ignore-authorization` | False | enforce | Who may suppress alerts with `@SocketSecurity ignore`. `enforce` requires write access and honors the command with a warning when the provider cannot report it; `strict` rejects it in that case; `off` honors any commenter. See [Who can ignore an alert](#who-can-ignore-an-alert). | | `--strict-blocking` | False | False | Fail on ANY security policy violations (blocking severity), not just new ones. Only works in diff mode. See [Strict Blocking Mode](#strict-blocking-mode) for details. | @@ -643,6 +643,8 @@ Security issues detected by Socket Security: - **Diff Mode Only**: The flag only works in diff mode (with SCM integration). In API mode, a warning is logged. - **Error-Level Only**: Only fails on `error=True` alerts (blocking severity), not warnings. +- **Existing alerts come from the full scan**: The scan comparison doesn't carry alerts on unchanged packages, so the CLI reads them from the new full scan, at the cost of one more API request per run. If that request fails, the run fails with the infrastructure-error code rather than passing without them. +- **PR/MR comments**: Existing alerts are listed in their own table after the new ones, blocking first. `@SocketSecurity ignore` applies to them too. Very large sets are cut off before the comment reaches GitHub's size limit, with a link to the full report. - **Priority**: `--disable-blocking` takes precedence - if both flags are set, the build will always pass. - **First Scan**: On the very first scan of a repository, there are no "existing" violations, so behavior is identical to standard mode. diff --git a/pyproject.toml b/pyproject.toml index 3ee6508e..0924672a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -6,7 +6,7 @@ build-backend = "hatchling.build" [project] name = "socketsecurity" -version = "2.12.1" +version = "2.13.0" requires-python = ">= 3.11" license = {"file" = "LICENSE"} dependencies = [ diff --git a/socketsecurity/__init__.py b/socketsecurity/__init__.py index 01e27579..08835ec8 100644 --- a/socketsecurity/__init__.py +++ b/socketsecurity/__init__.py @@ -1,4 +1,4 @@ __author__ = 'socket.dev' -__version__ = '2.12.1' +__version__ = '2.13.0' USER_AGENT_NAME = 'socket-python-cli' USER_AGENT = f'{USER_AGENT_NAME}/{__version__}' diff --git a/socketsecurity/config.py b/socketsecurity/config.py index 9e8f1580..96d906bc 100644 --- a/socketsecurity/config.py +++ b/socketsecurity/config.py @@ -2,15 +2,22 @@ import json import logging import os +import sys import tomllib from dataclasses import asdict, dataclass, field -from typing import List, Optional +from typing import List, NoReturn, Optional from socketdev import INTEGRATION_TYPES, IntegrationType from socketsecurity import __version__ +from socketsecurity.exit_codes import EXIT_USAGE +def _usage_error(message: str) -> NoReturn: + """Log an invalid flag or config value and exit like an argparse usage error.""" + logging.error(message) + sys.exit(EXIT_USAGE) + def get_plugin_config_from_env(prefix: str) -> dict: config_str = os.getenv(f"{prefix}_CONFIG_JSON", "{}") try: @@ -65,18 +72,14 @@ def load_cli_config_file(config_path: str) -> dict: elif config_path.lower().endswith(".toml"): data = tomllib.load(f) else: - logging.error("--config must be a .json or .toml file") - exit(1) + _usage_error("--config must be a .json or .toml file") except FileNotFoundError: - logging.error(f"Config file not found: {config_path}") - exit(1) + _usage_error(f"Config file not found: {config_path}") except (json.JSONDecodeError, tomllib.TOMLDecodeError) as e: - logging.error(f"Invalid config file format: {e}") - exit(1) + _usage_error(f"Invalid config file format: {e}") if not isinstance(data, dict): - logging.error("Config file must contain a top-level object/table") - exit(1) + _usage_error("Config file must contain a top-level object/table") scoped = data.get("socketcli") if isinstance(scoped, dict): @@ -106,7 +109,7 @@ def validate_exclude_paths(patterns: List[str]) -> None: Patterns are scan-root-relative globs. Reject the cases coana's --exclude-dirs / fast-glob cannot honor: negation, absolute paths, ``..`` traversal, and degenerate match-everything. - Exits with code 1 on the first invalid pattern. + Exits with the usage-error code (2) on the first invalid pattern. """ # Degenerate match-everything forms, compared against the trailing-slash-stripped pattern # (so "**/" reduces to "**" and is rejected, matching Node's stripTrailingSlash + check). @@ -114,17 +117,13 @@ def validate_exclude_paths(patterns: List[str]) -> None: for p in patterns: norm = (p or "").strip().replace("\\", "/") if norm.startswith("!"): - logging.error(f"--exclude-paths: negation patterns are not supported: {p!r}") - exit(1) + _usage_error(f"--exclude-paths: negation patterns are not supported: {p!r}") if norm.startswith("/"): - logging.error(f"--exclude-paths: patterns must be scan-root relative (no leading '/'): {p!r}") - exit(1) + _usage_error(f"--exclude-paths: patterns must be scan-root relative (no leading '/'): {p!r}") if norm == ".." or norm.startswith("../") or "/../" in norm or norm.endswith("/.."): - logging.error(f"--exclude-paths: '..' path traversal is not allowed: {p!r}") - exit(1) + _usage_error(f"--exclude-paths: '..' path traversal is not allowed: {p!r}") if norm.rstrip("/") in degenerate: - logging.error(f"--exclude-paths: pattern would exclude everything: {p!r}") - exit(1) + _usage_error(f"--exclude-paths: pattern would exclude everything: {p!r}") @dataclass @@ -416,11 +415,9 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig': try: config_args["excluded_ecosystems"] = json.loads(excluded_ecosystems.replace("'", '"')) except json.JSONDecodeError: - logging.error(f"Unable to parse excluded_ecosystems: {excluded_ecosystems}") - exit(1) + _usage_error(f"Unable to parse excluded_ecosystems: {excluded_ecosystems}") else: - logging.error(f"Unable to parse excluded_ecosystems: {excluded_ecosystems}") - exit(1) + _usage_error(f"Unable to parse excluded_ecosystems: {excluded_ecosystems}") # Build Slack plugin config, merging CLI arg with env config slack_config = get_plugin_config_from_env("SOCKET_SLACK") if args.slack_webhook: @@ -444,21 +441,18 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig': # Validate that sub_paths and workspace_name are used together if args.sub_paths and not args.workspace_name: - logging.error("--sub-path requires --workspace-name to be specified") - exit(1) + _usage_error("--sub-path requires --workspace-name to be specified") if args.workspace_name and not args.sub_paths: - logging.error("--workspace-name requires --sub-path to be specified") - exit(1) + _usage_error("--workspace-name requires --sub-path to be specified") # The API validates the repo as a slug, which can't contain "/", so an # owner/repo value would otherwise fail later at repository creation. if args.repo and "/" in args.repo: suggested = args.repo.rstrip("/").rsplit("/", 1)[-1] - logging.error( + _usage_error( f"--repo takes the repository slug without its owner, but got '{args.repo}'. " f"Use --repo {suggested}, and --workspace for the owner if your repositories use one." ) - exit(1) if args.pending_head: logging.warning( @@ -469,21 +463,16 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig': # argparse only enforces the mutually exclusive group for real CLI args; # this also catches both values arriving via a --config file. if args.base_scan_id and args.base_commit_sha: - logging.error("--base-scan-id and --base-commit-sha are mutually exclusive") - exit(1) + _usage_error("--base-scan-id and --base-commit-sha are mutually exclusive") if args.sarif_scope == "full" and not args.reach: - logging.error("--sarif-scope full requires --reach to be specified") - exit(1) + _usage_error("--sarif-scope full requires --reach to be specified") if args.sarif_reachability != "all" and not args.reach: - logging.error("--sarif-reachability requires --reach to be specified") - exit(1) + _usage_error("--sarif-reachability requires --reach to be specified") if args.sarif_grouping == "alert" and args.sarif_scope != "full": - logging.error("--sarif-grouping alert currently requires --sarif-scope full") - exit(1) + _usage_error("--sarif-grouping alert currently requires --sarif-scope full") if args.sarif_reachability in ("potentially", "reachable-or-potentially") and args.sarif_scope != "full": - logging.error("--sarif-reachability potentially/reachable-or-potentially requires --sarif-scope full") - exit(1) + _usage_error("--sarif-reachability potentially/reachable-or-potentially requires --sarif-scope full") # Validate --exclude-paths patterns up front (mirrors Node's assertValidExcludePaths). if config_args.get("exclude_paths"): @@ -491,23 +480,19 @@ def from_args(cls, args_list: Optional[List[str]] = None) -> 'CliConfig': # Validate that only_facts_file requires reach if args.only_facts_file and not args.reach: - logging.error("--only-facts-file requires --reach to be specified") - exit(1) + _usage_error("--only-facts-file requires --reach to be specified") # Validate that reach_use_only_pregenerated_sboms requires reach if args.reach_use_only_pregenerated_sboms and not args.reach: - logging.error("--reach-use-only-pregenerated-sboms requires --reach to be specified") - exit(1) + _usage_error("--reach-use-only-pregenerated-sboms requires --reach to be specified") # Validate reach_concurrency is >= 1 if provided if args.reach_concurrency is not None and args.reach_concurrency < 1: - logging.error("--reach-concurrency must be >= 1") - exit(1) + _usage_error("--reach-concurrency must be >= 1") # Validate max_purl_batch_size is within allowed range if args.max_purl_batch_size < 1 or args.max_purl_batch_size > 9999: - logging.error("--max-purl-batch-size must be between 1 and 9999") - exit(1) + _usage_error("--max-purl-batch-size must be between 1 and 9999") return cls(**config_args) diff --git a/socketsecurity/core/__init__.py b/socketsecurity/core/__init__.py index f78f6c19..9debddd1 100644 --- a/socketsecurity/core/__init__.py +++ b/socketsecurity/core/__init__.py @@ -8,7 +8,6 @@ import random import re import shutil -import sys import tarfile import tempfile import time @@ -32,6 +31,7 @@ from socketsecurity import USER_AGENT, __version__ from socketsecurity.core.classes import Alert, Diff, FullScan, Issue, Package, Purl from socketsecurity.core.exceptions import APIResourceNotFound +from socketsecurity.exit_codes import IncompleteComparisonError, InfrastructureError from .resource_utils import check_file_count_against_ulimit from .socket_config import SocketConfig @@ -1357,13 +1357,13 @@ def create_full_scan_with_report_url( packages=packages ) - consolidated: Set[str] = set() + consolidated: Set[Tuple[str, str, str, str]] = set() for alert_key, alerts in all_alerts_collection.items(): for alert in alerts: - alert_str = f"{alert.purl},{alert.type}" - if (alert.error or alert.warn) and alert_str not in consolidated: + finding_key = alert.consolidation_key() + if (alert.error or alert.warn) and finding_key not in consolidated: diff.new_alerts.append(alert) - consolidated.add(alert_str) + consolidated.add(finding_key) diff.alerts_fetched = True sbom_end = time.time() @@ -1501,7 +1501,8 @@ def get_repo_info(self, repo_slug: str, default_branch: str = "socket-default-br RepositoryInfo object Raises: - Exception: If API request fails + InfrastructureError: If the token can't read the repository, or the + repository doesn't exist and can't be created. """ try: # Need to switch to either standard logger or not call our module logging so that there isn't a conflict @@ -1517,7 +1518,15 @@ def get_repo_info(self, repo_slug: str, default_branch: str = "socket-default-br if not response.success: log.error(f"Failed to get repository: {response.status}") # log.error(response.message) - except APIFailure: + except APIFailure as lookup_error: + # Only a missing repository should be created. A token without the + # repo:list scope gets 401/403 here, and treating that as "doesn't + # exist" would end in a confusing repo:create failure instead. + if lookup_error.status_code in (401, 403): + raise InfrastructureError( + f"Failed to get repository {repo_slug} (HTTP {lookup_error.status_code}). " + "The API token may be missing the repo:list scope." + ) from lookup_error log.warning(f"Failed to get repository {repo_slug}, attempting to create it") try: @@ -1541,10 +1550,10 @@ def get_repo_info(self, repo_slug: str, default_branch: str = "socket-default-br # The SDK appends the request path and response headers after the first line, # and some subclasses carry no message at all. reason = str(e).split("\n", 1)[0] or type(e).__name__ - log.error(f"API failure while creating repository {repo_slug}{status}: {reason}") + message = f"API failure while creating repository {repo_slug}{status}: {reason}" if e.status_code == 403 and not isinstance(e, (APIInsufficientQuota, APIOrganizationNotAllowed)): - log.error("A 403 here usually means the API token is missing the repo:create scope.") - sys.exit(2) # Exit here with code 2. Code 1 indicates a successfully-detected security issue. + message += "\nA 403 here usually means the API token is missing the repo:create scope." + raise InfrastructureError(message) from e return response.data @@ -1864,16 +1873,13 @@ def resolve_base_full_scan_id(self, params: FullScanParams) -> Optional[str]: "the merge base." ) else: - log.error( + raise InfrastructureError( f"No full scan found for commit {commit_sha} in repo {params.repo} " "(--base-commit-sha), and no scanned ancestor within " f"{ANCESTOR_WALK_MAX_DEPTH} commits of it. Ensure a scan was " "created for that commit (e.g. the CLI runs on default-branch " "pushes), or pass --base-scan-id instead." ) - if self.cli_config.disable_blocking: - sys.exit(0) - sys.exit(self.cli_config.exit_code_on_api_error) log.info( f"Baseline selected: source={baseline_source} " f"scan_id={json.dumps(scan_id)} commit={json.dumps(baseline_commit)}" @@ -1892,14 +1898,11 @@ def resolve_base_full_scan_id(self, params: FullScanParams) -> Optional[str]: # Returning None would make the caller create an empty baseline scan, # reporting every dependency as newly added, so fail loudly like the # --base-commit-sha path above. - log.error( - f"Failed to resolve the matching head scan for repo {params.repo}: {error}" - ) if self.cli_config is None: raise - if self.cli_config.disable_blocking: - sys.exit(0) - sys.exit(self.cli_config.exit_code_on_api_error) + raise InfrastructureError( + f"Failed to resolve the matching head scan for repo {params.repo}: {error}" + ) from error if scan_id: log.info( "Baseline selected: source=repository-head " @@ -2106,6 +2109,12 @@ def get_diff_scan_artifacts( f"Error fetching diff scan {diff_scan_id}: unexpected response: {str(response)[:500]}" ) artifacts_dict = scan["artifacts"] + if scan.get("incomplete") or response.get("incomplete"): + raise IncompleteComparisonError( + f"The scan comparison is incomplete (diff scan {diff_scan_id}): the API " + "could not compare every package, so alerts may be missing. Failing " + "instead of reporting a result that may not cover the whole change." + ) log.info( "Diff scan comparison ready in " f"{time.monotonic() - poll_start:.2f}s: id={diff_scan_id}, " @@ -2215,6 +2224,10 @@ def get_added_and_removed_packages( new_full_scan_id, external_href=external_href, ) + except IncompleteComparisonError: + # The streaming comparison reads the same scans, so falling back to it + # would only hide that the result is partial. + raise except Exception as error: # SDK error messages can span many lines (path + response headers); the # first line carries the status, which is all the warning needs. @@ -2480,11 +2493,9 @@ def create_new_diff( external_href=external_href, ) - # Separate unchanged packages from added/removed for --strict-blocking support - unchanged_packages = { - pkg_id: pkg for pkg_id, pkg in packages.items() - if pkg_id not in added_packages and pkg_id not in removed_packages - } + unchanged_packages = Core.get_unchanged_packages(packages, added_packages, removed_packages) + if unchanged_packages and self._requires_unchanged_alerts(): + self.add_existing_alerts_from_full_scan(unchanged_packages, new_full_scan.id) diff = self.create_diff_report(added_packages, removed_packages, unchanged_packages) diff.packages = packages @@ -2509,6 +2520,104 @@ def create_new_diff( return diff + def _requires_unchanged_alerts(self) -> bool: + """Whether an enabled output reads alerts on unchanged packages. + + ``--strict-blocking`` fails the run on them, and the GitLab security report + and FOSSA-format output list every current finding, not only new ones. + A subset of _requires_unchanged_artifacts. + """ + config = self.cli_config + if config is None: + return False + return bool( + getattr(config, "strict_blocking", False) + or getattr(config, "enable_gitlab_security", False) + or getattr(config, "legal_format", "socket") == "fossa" + ) + + def add_existing_alerts_from_full_scan( + self, unchanged_packages: Dict[str, Package], full_scan_id: str + ) -> None: + """Fill in the alerts on unchanged packages from the new full scan. + + The cached diff-scans endpoint returns unchanged artifacts with an empty + ``alerts`` list, because it keeps only the alerts it counts as new. The full + scan of the same commit carries every package's alerts with their policy + actions, so they are read from there. Packages are matched by artifact ID, + then by ecosystem, namespace, name and version. + + Skipped when any unchanged package already has alerts: the comparison then + came from an endpoint that includes them, such as the streaming fallback. + A failed fetch raises, because reporting existing alerts as absent would + let ``--strict-blocking`` pass a run it should fail. + """ + if any(package.alerts for package in unchanged_packages.values()): + log.debug("The comparison already includes alerts on unchanged packages") + return + + start = time.time() + by_id: Dict[str, List[dict]] = {} + by_identity: Dict[Tuple[str, str, str, str], List[dict]] = {} + for artifact in self.get_sbom_data(full_scan_id).values(): + data = asdict(artifact) + alerts = data.get("alerts") or [] + by_id[data["id"]] = alerts + by_identity[Core._package_identity( + data["type"], data.get("namespace"), data["name"], data["version"] + )] = alerts + + matched = 0 + alert_count = 0 + for package in unchanged_packages.values(): + alerts = by_id.get(package.id) + if alerts is None: + alerts = by_identity.get(Core._package_identity( + package.type, package.namespace, package.name, package.version + )) + if alerts is None: + continue + package.alerts = alerts + matched += 1 + alert_count += len(alerts) + + log.info( + f"Loaded {alert_count} alerts for {matched} of {len(unchanged_packages)} " + f"unchanged packages from full scan {full_scan_id} in {time.time() - start:.2f}s" + ) + if matched < len(unchanged_packages): + log.warning( + f"{len(unchanged_packages) - matched} unchanged packages were not found in " + f"full scan {full_scan_id}, so their existing alerts are not reported" + ) + + @staticmethod + def _package_identity(package_type, namespace, name, version) -> Tuple[str, str, str, str]: + return ( + str(Package.normalize_type(package_type) or ""), + (namespace or "").strip("/"), + name or "", + version or "", + ) + + @staticmethod + def get_unchanged_packages( + packages: Dict[str, Package], + added_packages: Dict[str, Package], + removed_packages: Dict[str, Package], + ) -> Dict[str, Package]: + """Return the packages in ``packages`` that were neither added nor removed. + + ``packages`` is keyed by ``type/purl`` while the added and removed dicts are + keyed by artifact ID, so membership is checked on ``Package.id``. The result + is keyed by artifact ID to match the other two, which is what + ``get_source_data`` expects when it resolves ``topLevelAncestors``. + """ + return { + pkg.id: pkg for pkg in packages.values() + if pkg.id not in added_packages and pkg.id not in removed_packages + } + def create_diff_report( self, added_packages: Dict[str, Package], @@ -2853,29 +2962,27 @@ def get_new_alerts( if alert_key not in removed_package_alerts: new_alerts = added_package_alerts[alert_key] for alert in new_alerts: - # Consolidate by package and alert type, not by manifest details - alert_str = f"{alert.purl},{alert.type}" + finding_key = alert.consolidation_key() if alert.error or alert.warn: - if alert_str not in consolidated_alerts: + if finding_key not in consolidated_alerts: alerts.append(alert) - consolidated_alerts.add(alert_str) + consolidated_alerts.add(finding_key) else: new_alerts = added_package_alerts[alert_key] removed_alerts = removed_package_alerts[alert_key] for alert in new_alerts: - # Consolidate by package and alert type, not by manifest details - alert_str = f"{alert.purl},{alert.type}" + finding_key = alert.consolidation_key() # Only add if: # 1. Alert isn't in removed packages (or we're not ignoring readded alerts) # 2. We haven't already recorded this alert # 3. It's an error or warning - if (not ignore_readded or alert not in removed_alerts) and alert_str not in consolidated_alerts: + if (not ignore_readded or alert not in removed_alerts) and finding_key not in consolidated_alerts: if alert.error or alert.warn: alerts.append(alert) - consolidated_alerts.add(alert_str) + consolidated_alerts.add(finding_key) return alerts @@ -2900,13 +3007,12 @@ def get_unchanged_alerts( for alert_key in unchanged_package_alerts: for alert in unchanged_package_alerts[alert_key]: - # Consolidate by package and alert type - alert_str = f"{alert.purl},{alert.type}" + finding_key = alert.consolidation_key() # Only include error or warning alerts - if (alert.error or alert.warn) and alert_str not in consolidated_alerts: + if (alert.error or alert.warn) and finding_key not in consolidated_alerts: alerts.append(alert) - consolidated_alerts.add(alert_str) + consolidated_alerts.add(finding_key) return alerts @@ -2930,10 +3036,10 @@ def get_removed_alerts( for alert_key in removed_package_alerts: for alert in removed_package_alerts[alert_key]: - alert_str = f"{alert.purl},{alert.type}" + finding_key = alert.consolidation_key() - if alert_str not in consolidated_alerts: + if finding_key not in consolidated_alerts: alerts.append(alert) - consolidated_alerts.add(alert_str) + consolidated_alerts.add(finding_key) return alerts diff --git a/socketsecurity/core/alert_selection.py b/socketsecurity/core/alert_selection.py index 132be294..d775b17b 100644 --- a/socketsecurity/core/alert_selection.py +++ b/socketsecurity/core/alert_selection.py @@ -15,14 +15,26 @@ def select_diff_alerts(diff: Diff, strict_blocking: bool = False) -> List[Issue] """Select diff alerts for output rendering. In strict blocking mode, include unchanged alerts so rendered output aligns - with pass/fail policy evaluation. + with pass/fail policy evaluation. An unchanged alert already listed as new + (same Issue.consolidation_key) is skipped so each finding is rendered once. """ selected = list(getattr(diff, "new_alerts", []) or []) if strict_blocking: - selected.extend(getattr(diff, "unchanged_alerts", []) or []) + seen = {_alert_identity(alert) for alert in selected} - {None} + for alert in getattr(diff, "unchanged_alerts", []) or []: + identity = _alert_identity(alert) + if identity is None or identity not in seen: + selected.append(alert) + seen.add(identity) return selected +def _alert_identity(alert: Issue) -> Optional[Tuple[str, str, str, str]]: + if not (getattr(alert, "purl", "") or ""): + return None + return alert.consolidation_key() + + def clone_diff_with_selected_alerts(diff: Diff, selected_alerts: List[Issue]) -> Diff: """Clone a diff object while replacing new_alerts with selected alerts.""" selected_diff = Diff( diff --git a/socketsecurity/core/classes.py b/socketsecurity/core/classes.py index fd8fabcd..98e8a213 100644 --- a/socketsecurity/core/classes.py +++ b/socketsecurity/core/classes.py @@ -1,7 +1,7 @@ import json import logging from dataclasses import dataclass, field -from typing import Dict, List, Optional, TypedDict +from typing import Dict, List, Optional, Tuple, TypedDict from socketdev.fullscans import ( FullScanMetadata, @@ -19,6 +19,11 @@ # one path segment that cannot be split back into two, and the page does not resolve. NAMESPACE_REQUIRED_TYPES = frozenset({"maven"}) +# Alert props that name a vulnerability advisory, in the order Issue.advisory_id +# reads them. Alerts reach Issue.props from several sources, so both spellings +# appear; "cve" is the older field name. +ADVISORY_ID_PROPS = ("ghsaId", "ghsa_id", "cveId", "cve_id", "cve") + __all__ = [ "Report", "Score", @@ -354,6 +359,34 @@ def __init__(self, **kwargs): if not hasattr(self, "ignore"): self.ignore = False + def advisory_id(self) -> str: + """The GHSA or CVE identifier in the alert's props, upper-cased, or "".""" + props = getattr(self, "props", None) or {} + for prop_name in ADVISORY_ID_PROPS: + value = props.get(prop_name) + if isinstance(value, (list, tuple)): + value = next((v for v in value if isinstance(v, str) and v.strip()), "") + if isinstance(value, str) and value.strip(): + return value.strip().upper() + return "" + + def consolidation_key(self) -> Tuple[str, str, str, str]: + """Identity used to collapse repeat sightings of one finding into one alert. + + The same alert arrives once per manifest that pulls the package in, so the + key leaves manifest details out. It keeps the advisory ID because every CVE + of a given severity on a package shares one alert type (``criticalCVE``, + ``cve`` and so on): without it, a package's second advisory would be dropped, + and with it any blocking action that advisory carried. It keeps the + ecosystem because ``purl`` holds only name and version. + """ + return ( + getattr(self, "pkg_type", "") or "", + getattr(self, "purl", "") or "", + getattr(self, "type", "") or "", + self.advisory_id(), + ) + def __str__(self): return json.dumps(self.__dict__) diff --git a/socketsecurity/core/messages.py b/socketsecurity/core/messages.py index b2229f65..48345384 100644 --- a/socketsecurity/core/messages.py +++ b/socketsecurity/core/messages.py @@ -622,6 +622,12 @@ def generate_uuid_from_alert_gitlab(alert: Issue) -> str: """ # Create unique string from alert key properties unique_str = f"{alert.pkg_name}:{alert.pkg_version}:{alert.type}:{alert.severity}" + # Every CVE of one severity on a package shares a type, so the advisory ID + # keeps them apart. It's appended only when present, so other findings + # keep the IDs GitLab already tracks them by. + advisory = alert.advisory_id() if hasattr(alert, "advisory_id") else "" + if advisory: + unique_str += f":{advisory}" # Generate UUID5 (deterministic) from namespace and unique string namespace = uuid.UUID('6ba7b810-9dad-11d1-80b4-00c04fd430c8') # DNS namespace @@ -772,9 +778,15 @@ def create_security_comment_gitlab(diff: Diff) -> dict: dep_files_map: dict = {} all_alerts = list(diff.new_alerts) + list(getattr(diff, 'unchanged_alerts', [])) + seen_ids = set() for alert in all_alerts: + vulnerability_id = Messages.generate_uuid_from_alert_gitlab(alert) + # GitLab expects each finding ID once per report. + if vulnerability_id in seen_ids: + continue + seen_ids.add(vulnerability_id) vulnerability = { - "id": Messages.generate_uuid_from_alert_gitlab(alert), + "id": vulnerability_id, "category": "dependency_scanning", "name": alert.title if hasattr(alert, 'title') else f"{alert.type} in {alert.pkg_name}", "message": f"{alert.pkg_name}@{alert.pkg_version}: {alert.title if hasattr(alert, 'title') else alert.type}", @@ -925,6 +937,130 @@ def get_view_report_url(diff: Diff) -> str: return diff.report_url return "" + @staticmethod + def license_group_row(alerts: list, diff: Diff, config, show_ignore: bool) -> str: + """One row of the security comment table for a package's license policy violations.""" + row = "" + action = "Block" if any(alert.error for alert in alerts) else "Warn" + first_alert = alerts[0] + + # Use orange diamond for license policy violations + license_icon = "🔶" + + license_label = Messages.html_text( + f"{first_alert.pkg_name}@{first_alert.pkg_version}" + ) + license_marker = Messages.comment_marker_text( + f"{first_alert.pkg_name}@{first_alert.pkg_version}" + ) + + # Build license findings list + license_findings = [] + for alert in alerts: + license_findings.append(alert.title) + + row += f""" + + + {action} + {license_icon} + +
+ {license_label} has a License Policy Violation. +

License findings:

+ +

From: Manifest File

+

ℹ️ Read more on: This package | What is a license policy violation?

+
+

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

+

Suggestion: Find a package that does not violate your license policy or adjust your policy to allow this package's license.

+ {license_ignore_html} +
+
+ + + + """ + return row + + @staticmethod + def security_alert_row(alert: Issue, diff: Diff, config, show_ignore: bool) -> str: + """One row of the security comment table for a non-license alert.""" + severity_icon = Messages.get_severity_icon(alert.severity) + action = "Block" if alert.error else "Warn" + details_open = "" + patched_version = Messages.get_patched_version(alert) + patched_version_html = ( + "

Patched version: " + f"{Messages.html_text(patched_version)}

" + if patched_version else "" + ) + # Generate proper manifest URL + manifest_url = Messages.get_manifest_file_url(diff, alert.manifests, config) + pkg_label = Messages.html_text(f"{alert.pkg_name}@{alert.pkg_version}") + pkg_marker = Messages.comment_marker_text(f"{alert.pkg_name}@{alert.pkg_version}") + alert_title = Messages.alert_title_with_advisory(alert) + ignore_html = ( + f"

Mark as acceptable risk: To ignore this alert only in this pull request, reply with:
" + f"@SocketSecurity ignore {Messages.html_text(alert.pkg_type)}/{pkg_label}
" + f"Or ignore all future alerts with:
" + f"@SocketSecurity ignore-all

" + ) if show_ignore else "" + return f""" + + + {action} + + {Messages.html_attr(alert.severity)} + + +
+ {pkg_label} - {Messages.html_text(alert_title)} +

Note: {Messages.html_text(alert.description)}

+ {patched_version_html} +

Source: Manifest File

+

ℹ️ Read more on: + This package | + This alert | + What is known malware?

+
+

Suggestion: {Messages.html_text(alert.suggestion)}

+ {ignore_html} +
+
+ + + + """ + + @staticmethod + def alert_title_with_advisory(alert: Issue) -> str: + """The alert title, followed by its GHSA or CVE ID when it has one. + + Every CVE of one severity shares a title ("Critical CVE"), so without the + ID two advisories on the same package render as identical rows. + """ + title = getattr(alert, "title", "") or "" + advisory = alert.advisory_id() if hasattr(alert, "advisory_id") else "" + if advisory and advisory not in title.upper(): + return f"{title} ({advisory})" + return title + @staticmethod def security_comment_template(diff: Diff, config=None) -> str: """ @@ -949,10 +1085,11 @@ def security_comment_template(diff: Diff, config=None) -> str: security_alerts.append(alert) view_report_url = Messages.get_view_report_url(diff) + existing_alerts = Messages.existing_comment_alerts(diff, config) # Without this the caution banner would sit above a table with no rows, # which is how a comment looks once every alert it raised is resolved. - if not security_alerts and not license_groups: + if not security_alerts and not license_groups and not existing_alerts: return Messages.security_comment_no_alerts_template(view_report_url) # Start of the comment @@ -980,122 +1117,115 @@ def security_comment_template(diff: Diff, config=None) -> str: # Loop through security alerts (non-license), dynamically generating rows for alert in security_alerts: - severity_icon = Messages.get_severity_icon(alert.severity) - action = "Block" if alert.error else "Warn" - details_open = "" - patched_version = Messages.get_patched_version(alert) - patched_version_html = ( - "

Patched version: " - f"{Messages.html_text(patched_version)}

" - if patched_version else "" - ) - # Generate proper manifest URL - manifest_url = Messages.get_manifest_file_url(diff, alert.manifests, config) - pkg_label = Messages.html_text(f"{alert.pkg_name}@{alert.pkg_version}") - pkg_marker = Messages.comment_marker_text(f"{alert.pkg_name}@{alert.pkg_version}") - # Generate a table row for each alert - ignore_html = ( - f"

Mark as acceptable risk: To ignore this alert only in this pull request, reply with:
" - f"@SocketSecurity ignore {Messages.html_text(alert.pkg_type)}/{pkg_label}
" - f"Or ignore all future alerts with:
" - f"@SocketSecurity ignore-all

" - ) if show_ignore else "" - comment += f""" - - - {action} - - {Messages.html_attr(alert.severity)} - - -
- {pkg_label} - {Messages.html_text(alert.title)} -

Note: {Messages.html_text(alert.description)}

- {patched_version_html} -

Source: Manifest File

-

ℹ️ Read more on: - This package | - This alert | - What is known malware?

-
-

Suggestion: {Messages.html_text(alert.suggestion)}

- {ignore_html} -
-
- - - - """ + comment += Messages.security_alert_row(alert, diff, config, show_ignore) # Add license policy violation entries grouped by PURL for purl_key, alerts in license_groups.items(): - action = "Block" if any(alert.error for alert in alerts) else "Warn" - first_alert = alerts[0] - - # Use orange diamond for license policy violations - license_icon = "🔶" - - license_label = Messages.html_text( - f"{first_alert.pkg_name}@{first_alert.pkg_version}" - ) - license_marker = Messages.comment_marker_text( - f"{first_alert.pkg_name}@{first_alert.pkg_version}" - ) + comment += Messages.license_group_row(alerts, diff, config, show_ignore) - # Build license findings list - license_findings = [] - for alert in alerts: - license_findings.append(alert.title) - - comment += f""" - + if not security_alerts and not license_groups: + comment += """ - {action} - {license_icon} - -
- {license_label} has a License Policy Violation. -

License findings:

- -

From: Manifest File

-

ℹ️ Read more on: This package | What is a license policy violation?

-
-

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

-

Suggestion: Find a package that does not violate your license policy or adjust your policy to allow this package's license.

- {license_ignore_html} -
-
- + No new alerts in this change. - - """ +""" # Close table - comment += f""" + comment += """ +""" + if existing_alerts: + comment += Messages.existing_alerts_section( + existing_alerts, diff, config, show_ignore, view_report_url, len(comment) + ) + comment += f""" [View full report]({view_report_url}?action=error%2Cwarn) """ return Messages.normalize_comment_html(comment) + # GitHub rejects comment bodies over 65,536 characters. The existing-alerts + # section is the open-ended part of the security comment, so it stops adding + # rows at this size and links to the full report for the rest. + SECURITY_COMMENT_SIZE_BUDGET = 60000 + + @staticmethod + def existing_comment_alerts(diff: Diff, config) -> list: + """Alerts on unchanged packages to show in the PR comment. + + Only under --strict-blocking, where they fail the run: without it they + don't affect the result, and listing them would bury the new ones. + """ + if not (config and getattr(config, "strict_blocking", False)): + return [] + new_keys = {alert.consolidation_key() for alert in getattr(diff, "new_alerts", []) or []} + return [ + alert for alert in getattr(diff, "unchanged_alerts", []) or [] + if (alert.error or alert.warn) and alert.consolidation_key() not in new_keys + ] + + @staticmethod + def existing_alerts_section( + alerts: list, diff: Diff, config, show_ignore: bool, view_report_url: str, size_so_far: int + ) -> str: + """Table of alerts on packages this change didn't touch, blocking ones first. + + License policy violations are grouped per package, as in the table of new + alerts. + """ + entries = [] + license_groups: dict = {} + for alert in alerts: + if alert.type == "licenseSpdxDisj": + purl_key = f"{alert.pkg_type}/{alert.pkg_name}@{alert.pkg_version}" + if purl_key not in license_groups: + license_groups[purl_key] = [] + entries.append(license_groups[purl_key]) + license_groups[purl_key].append(alert) + else: + entries.append([alert]) + entries.sort(key=lambda group: not any(alert.error for alert in group)) + + blocking = sum(1 for alert in alerts if alert.error) + section = f""" + +**Existing alerts ({len(alerts)}).** These are on dependencies this change didn't add or update. `--strict-blocking` fails the run on the {blocking} with a **Block** action. + + + + + + + + + + +""" + closing = """ + +
ActionSeverityAlert (click for details)
+""" + shown = 0 + for group in entries: + if group[0].type == "licenseSpdxDisj": + row = Messages.license_group_row(group, diff, config, show_ignore) + else: + row = Messages.security_alert_row(group[0], diff, config, show_ignore) + if size_so_far + len(section) + len(row) + len(closing) > Messages.SECURITY_COMMENT_SIZE_BUDGET: + break + section += row + shown += len(group) + section += closing + if shown < len(alerts): + section += ( + f"\n{len(alerts) - shown} more existing alert(s) are not shown here to keep " + f"this comment within GitHub's size limit. See the [full report]({view_report_url}?action=error%2Cwarn).\n" + ) + return section + "\n" + @staticmethod def get_severity_icon(severity: str) -> str: """ diff --git a/socketsecurity/core/scm/github.py b/socketsecurity/core/scm/github.py index dcfa5607..5248fc87 100644 --- a/socketsecurity/core/scm/github.py +++ b/socketsecurity/core/scm/github.py @@ -11,6 +11,7 @@ from socketsecurity.core.exceptions import APIFailure from socketsecurity.core.git_remote import parse_git_remote from socketsecurity.core.scm_comments import Comments +from socketsecurity.exit_codes import InfrastructureError from socketsecurity.socketcli import CliClient @@ -54,8 +55,7 @@ def from_env(cls, pr_number: Optional[str] = None) -> 'GithubConfig': """Create config from environment variables with optional overrides""" token = os.getenv('GH_API_TOKEN') if not token: - log.error("Unable to get Github API Token from GH_API_TOKEN") - sys.exit(2) + raise InfrastructureError("Unable to get Github API Token from GH_API_TOKEN") is_buildkite = os.getenv("BUILDKITE") == "true" buildkite_pr = os.getenv("BUILDKITE_PULL_REQUEST") @@ -172,8 +172,7 @@ def __init__( self._ignore_permission_cache: dict[str, Optional[bool]] = {} if not self.config.token: - log.error("Unable to get Github API Token") - sys.exit(2) + raise InfrastructureError("Unable to get Github API Token") def check_event_type(self) -> str: if self.config.event_name.lower() in ["push", "workflow_dispatch"]: diff --git a/socketsecurity/core/scm/gitlab.py b/socketsecurity/core/scm/gitlab.py index d1fd1119..a7b2bd3a 100644 --- a/socketsecurity/core/scm/gitlab.py +++ b/socketsecurity/core/scm/gitlab.py @@ -11,6 +11,7 @@ from socketsecurity.core import log from socketsecurity.core.classes import Comment from socketsecurity.core.scm_comments import Comments +from socketsecurity.exit_codes import InfrastructureError from socketsecurity.socketcli import CliClient @@ -37,8 +38,7 @@ class GitlabConfig: def from_env(cls) -> 'GitlabConfig': token = os.getenv('GITLAB_TOKEN') if not token: - log.error("Unable to get GitLab API Token from GITLAB_TOKEN") - sys.exit(2) + raise InfrastructureError("Unable to get GitLab API Token from GITLAB_TOKEN") project_name = os.getenv('CI_PROJECT_NAME', '') if "/" in project_name: diff --git a/socketsecurity/exit_codes.py b/socketsecurity/exit_codes.py new file mode 100644 index 00000000..2be3d2a7 --- /dev/null +++ b/socketsecurity/exit_codes.py @@ -0,0 +1,42 @@ +"""Process exit codes, and the exceptions that map onto them. + +``socketcli.cli()`` is the only place that turns an outcome into an exit code, +so ``--disable-blocking`` and ``--exit-code-on-api-error`` apply to every run +that gets past argument parsing: + +* ``EXIT_OK`` (0): no blocking findings, or ``--disable-blocking`` is set. +* ``EXIT_BLOCKING`` (1): blocking security findings. +* ``EXIT_USAGE`` (2): invalid flags or configuration. The same code argparse uses + for its own usage errors. ``--disable-blocking`` does not apply, because a + misconfigured run never got far enough to scan anything. +* ``EXIT_INFRASTRUCTURE`` (3): missing credentials, API or environment failures. + ``--exit-code-on-api-error`` replaces it; ``--disable-blocking`` turns it into 0. +* ``EXIT_INTERRUPTED`` (130): Ctrl+C / SIGINT, the POSIX 128 + signal number + convention. ``--disable-blocking`` turns it into 0. + +This module imports nothing from the package so config parsing can use it +without pulling in ``socketsecurity.core``. +""" + +EXIT_OK = 0 +EXIT_BLOCKING = 1 +EXIT_USAGE = 2 +EXIT_INFRASTRUCTURE = 3 +EXIT_INTERRUPTED = 130 + + +class InfrastructureError(Exception): + """A run that cannot continue for a reason other than its own findings. + + Raise it instead of calling ``sys.exit`` so ``cli()`` can apply + ``--disable-blocking`` and ``--exit-code-on-api-error``. The message is + logged as is, without a traceback. + """ + + +class IncompleteComparisonError(InfrastructureError): + """The API marked a scan comparison incomplete. + + An incomplete comparison can leave out alerts, so a passing result would not + mean the change is clean. + """ diff --git a/socketsecurity/socketcli.py b/socketsecurity/socketcli.py index 066fe2df..96b9d630 100644 --- a/socketsecurity/socketcli.py +++ b/socketsecurity/socketcli.py @@ -4,7 +4,7 @@ import sys import traceback from datetime import datetime, timezone -from typing import List, Optional, Tuple +from typing import List, NoReturn, Optional, Tuple from uuid import uuid4 from dotenv import load_dotenv @@ -29,6 +29,12 @@ from socketsecurity.core.scm_comments import Comments from socketsecurity.core.socket_config import SocketConfig, module_folder_dirs from socketsecurity.core.streaming import StreamingLogs +from socketsecurity.exit_codes import ( + EXIT_INTERRUPTED, + EXIT_OK, + EXIT_USAGE, + InfrastructureError, +) from socketsecurity.fossa_compat import build_fossa_attribution_payload from socketsecurity.output import OutputHandler @@ -257,25 +263,27 @@ def cli(): main_code() except KeyboardInterrupt: log.info("Keyboard Interrupt detected, exiting") - config = CliConfig.from_args() # Get current config - if not config.disable_blocking: - sys.exit(2) - else: - sys.exit(0) + config = CliConfig.from_args() + sys.exit(EXIT_OK if config.disable_blocking else EXIT_INTERRUPTED) + except InfrastructureError as error: + _exit_for_infrastructure_error(str(error), include_traceback=False) except Exception as error: - config = CliConfig.from_args() # Get current config - _emit_infrastructure_error( + _exit_for_infrastructure_error( f"Unexpected error when running the CLI: {error}", include_traceback=True, ) - # --disable-blocking forces a clean exit for ALL outcomes (it takes - # precedence over --exit-code-on-api-error); otherwise infra/API errors - # exit with the configurable code (default 3), keeping them distinct - # from blocking security findings (exit 1). - if not config.disable_blocking: - sys.exit(config.exit_code_on_api_error) - else: - sys.exit(0) + + +def _exit_for_infrastructure_error(message: str, include_traceback: bool) -> NoReturn: + """Exit for a failure that is not a security finding. + + --disable-blocking takes precedence and exits 0. Otherwise the code is + --exit-code-on-api-error (default 3), which keeps these failures distinct + from blocking findings (1) and usage errors (2). See socketsecurity.exit_codes. + """ + config = CliConfig.from_args() + _emit_infrastructure_error(message, include_traceback=include_traceback) + sys.exit(EXIT_OK if config.disable_blocking else config.exit_code_on_api_error) def main_code(): @@ -290,10 +298,11 @@ def main_code(): # Validate API token if not config.api_token: - log.info("Socket API Token not found. Please set it using either:\n" - "1. Command line: --api-token YOUR_TOKEN\n" - "2. Environment variable: SOCKET_SECURITY_API_TOKEN") - sys.exit(3) + raise InfrastructureError( + "Socket API Token not found. Please set it using either:\n" + "1. Command line: --api-token YOUR_TOKEN\n" + "2. Environment variable: SOCKET_SECURITY_API_TOKEN" + ) sdk = build_socket_sdk(config) # Suppress urllib3 InsecureRequestWarning when using --allow-unverified @@ -360,10 +369,11 @@ def main_code(): missing_deps.append(dep) if missing_deps: - log.error(f"Reachability analysis requires the following dependencies: {', '.join(required_deps)}") - log.error(f"Missing dependencies: {', '.join(missing_deps)}") - log.error("Please install the missing dependencies and try again.") - sys.exit(3) + raise InfrastructureError( + f"Reachability analysis requires the following dependencies: {', '.join(required_deps)}\n" + f"Missing dependencies: {', '.join(missing_deps)}\n" + "Please install the missing dependencies and try again." + ) log.info(f"All required dependencies found: {', '.join(found_deps)}") @@ -378,15 +388,17 @@ def main_code(): # Check if plan matches enterprise* pattern (enterprise, enterprise_trial, etc.) if not org_plan.startswith('enterprise'): - log.error("Reachability analysis is only available for enterprise plans.") - log.error(f"Your organization plan is: {org_plan}") - log.error("Please upgrade to an enterprise plan to use reachability analysis.") - sys.exit(3) + raise InfrastructureError( + "Reachability analysis is only available for enterprise plans.\n" + f"Your organization plan is: {org_plan}\n" + "Please upgrade to an enterprise plan to use reachability analysis." + ) log.info(f"Organization plan verified: {org_plan}") else: - log.error("Unable to retrieve organization information for plan verification.") - sys.exit(3) + raise InfrastructureError( + "Unable to retrieve organization information for plan verification." + ) # Parse files argument try: @@ -420,7 +432,7 @@ def main_code(): log.error(f"Error details: {error}") log.debug(f"Files type: {type(config.files)}") log.debug(f"Files repr: {repr(config.files)}") - sys.exit(3) + sys.exit(EXIT_USAGE) # Determine if files were explicitly specified files_explicitly_specified = config.files != "[]" and len(specified_files) > 0 @@ -579,9 +591,11 @@ def main_code(): log.info(f"Pre-generated SBOMs mode: will submit {len(sbom_files_to_submit)} files (CDX, SPDX, and facts file)") except Exception as e: - log.error(f"Reachability analysis failed: {str(e)}") + # With --disable-blocking the scan carries on without reachability + # data instead of exiting 0 before anything is reported. if not config.disable_blocking: - sys.exit(3) + raise InfrastructureError(f"Reachability analysis failed: {e}") from e + log.error(f"Reachability analysis failed: {str(e)}") log.info("Continuing with normal scan flow...") @@ -861,6 +875,13 @@ def _is_unprocessed(c): diff.new_alerts = Comments.remove_alerts(comments, diff.new_alerts) ignored_alerts = [a for a in alerts_before if a not in diff.new_alerts] + # Under --strict-blocking, existing alerts fail the run and appear in + # the comment with the same ignore instructions, so an ignore has to + # apply to them as well. + if config.strict_blocking and diff.unchanged_alerts: + existing_before = list(diff.unchanged_alerts) + diff.unchanged_alerts = Comments.remove_alerts(comments, diff.unchanged_alerts) + ignored_alerts += [a for a in existing_before if a not in diff.unchanged_alerts] # Emit telemetry per-comment so each event carries the comment author. unprocessed_ignore = [ c for c in comments.get("ignore", []) @@ -943,7 +964,7 @@ def _is_unprocessed(c): new_security_comment = should_write_comment( config.disable_security_issue, - len(diff.new_alerts) > 0, + len(diff.new_alerts) > 0 or bool(Messages.existing_comment_alerts(diff, config)), update_old_security_comment, ) if not new_security_comment: diff --git a/tests/core/test_alert_consolidation.py b/tests/core/test_alert_consolidation.py new file mode 100644 index 00000000..64c0c4d3 --- /dev/null +++ b/tests/core/test_alert_consolidation.py @@ -0,0 +1,136 @@ +"""Repeat sightings of one finding collapse into one alert; distinct advisories don't. + +The same alert arrives once per manifest that pulls a package in, so the diff +collapses alerts on Issue.consolidation_key. Every CVE of one severity on a +package shares an alert type, so the key has to include the advisory ID, or a +package's second advisory is dropped along with any blocking action it carries. +""" +from unittest.mock import Mock + +from socketsecurity.config import CliConfig +from socketsecurity.core import Core +from socketsecurity.core.classes import Diff, Issue +from socketsecurity.core.messages import Messages +from socketsecurity.output import OutputHandler + + +def _cve(advisory, error=False, warn=False, pkg_type="npm", key=None, props_field="ghsaId"): + return Issue( + pkg_type=pkg_type, + pkg_name="lodash", + pkg_version="4.17.20", + purl="lodash@4.17.20", + type="criticalCVE", + severity="critical", + title="Critical CVE", + description="A vulnerability.", + suggestion="Upgrade.", + url="https://socket.dev/npm/package/lodash", + key=key or f"key-{advisory}", + props={props_field: advisory}, + error=error, + warn=warn, + ) + + +def test_two_advisories_on_one_package_are_both_new_alerts(): + alerts = { + "k1": [_cve("GHSA-aaaa-aaaa-aaaa", error=True)], + "k2": [_cve("GHSA-bbbb-bbbb-bbbb", error=True)], + } + + new_alerts = Core.get_new_alerts(alerts, {}) + + assert sorted(a.advisory_id() for a in new_alerts) == [ + "GHSA-AAAA-AAAA-AAAA", + "GHSA-BBBB-BBBB-BBBB", + ] + + +def test_blocking_advisory_is_not_hidden_behind_a_warning(): + """The kept advisory is a warning; dropping the second would pass the run.""" + alerts = { + "k1": [_cve("GHSA-aaaa-aaaa-aaaa", warn=True)], + "k2": [_cve("GHSA-bbbb-bbbb-bbbb", error=True)], + } + diff = Diff() + diff.new_alerts = Core.get_new_alerts(alerts, {}) + + config = Mock(spec=CliConfig) + config.disable_blocking = False + config.strict_blocking = False + assert not OutputHandler(config, Mock()).report_pass(diff) + + +def test_same_advisory_from_two_manifests_is_one_alert(): + alerts = { + "k1": [_cve("GHSA-aaaa-aaaa-aaaa", error=True), _cve("GHSA-aaaa-aaaa-aaaa", error=True)], + } + + assert len(Core.get_new_alerts(alerts, {})) == 1 + + +def test_unchanged_and_removed_alerts_keep_each_advisory(): + alerts = { + "k1": [_cve("CVE-2026-0001", error=True, props_field="cveId")], + "k2": [_cve("CVE-2026-0002", error=True, props_field="cveId")], + } + + assert len(Core.get_unchanged_alerts(alerts)) == 2 + assert len(Core.get_removed_alerts(alerts)) == 2 + + +def test_consolidation_key_includes_ecosystem_and_ignores_advisory_case(): + npm = _cve("ghsa-aaaa-aaaa-aaaa") + pypi = _cve("GHSA-AAAA-AAAA-AAAA", pkg_type="pypi") + + assert npm.consolidation_key() != pypi.consolidation_key() + assert npm.consolidation_key()[3] == pypi.consolidation_key()[3] + + +def test_non_advisory_alerts_still_collapse_per_package_and_type(): + """Alerts with no advisory ID keep the per-package, per-type collapse.""" + first = Issue(pkg_type="npm", purl="a@1.0.0", type="envVars", props={}, warn=True, key="k1") + second = Issue(pkg_type="npm", purl="a@1.0.0", type="envVars", props={}, warn=True, key="k2") + + assert len(Core.get_new_alerts({"k1": [first], "k2": [second]}, {})) == 1 + + +def test_pr_comment_rows_name_the_advisory(): + diff = Diff() + diff.new_alerts = [_cve("GHSA-aaaa-aaaa-aaaa", error=True), _cve("GHSA-bbbb-bbbb-bbbb", error=True)] + diff.diff_url = "https://socket.dev/diff" + diff.report_url = "https://socket.dev/report" + + comment = Messages.security_comment_template(diff) + + assert "Critical CVE (GHSA-AAAA-AAAA-AAAA)" in comment + assert "Critical CVE (GHSA-BBBB-BBBB-BBBB)" in comment + + +def test_title_already_naming_the_advisory_is_left_alone(): + alert = _cve("GHSA-aaaa-aaaa-aaaa") + alert.title = "GHSA-aaaa-aaaa-aaaa in lodash" + + assert Messages.alert_title_with_advisory(alert) == "GHSA-aaaa-aaaa-aaaa in lodash" + + +def test_gitlab_report_keeps_each_advisory_as_its_own_finding(): + diff = Diff() + diff.new_alerts = [_cve("GHSA-aaaa-aaaa-aaaa", error=True), _cve("GHSA-bbbb-bbbb-bbbb", error=True)] + + report = Messages.create_security_comment_gitlab(diff) + + ids = [v["id"] for v in report["vulnerabilities"]] + assert len(ids) == 2 + assert len(set(ids)) == 2 + + +def test_gitlab_ids_for_findings_without_an_advisory_are_unchanged(): + """GitLab tracks findings by ID across pipelines, so these must not move.""" + import uuid + + alert = Issue(pkg_name="a", pkg_version="1.0.0", type="malware", severity="critical", props={}) + expected = uuid.uuid5(uuid.UUID("6ba7b810-9dad-11d1-80b4-00c04fd430c8"), "a:1.0.0:malware:critical") + + assert Messages.generate_uuid_from_alert_gitlab(alert) == str(expected) diff --git a/tests/core/test_existing_alerts.py b/tests/core/test_existing_alerts.py new file mode 100644 index 00000000..5f32f4c5 --- /dev/null +++ b/tests/core/test_existing_alerts.py @@ -0,0 +1,306 @@ +"""Alerts on unchanged packages come from the full scan when the comparison omits them. + +The cached diff-scans endpoint returns unchanged artifacts with an empty +``alerts`` list, so ``--strict-blocking`` had nothing to block on and passed. +The new full scan carries every package's alerts, so the CLI reads them from +there whenever an output needs them: ``--strict-blocking``, the GitLab security +report and FOSSA-format output. +""" +import json +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import Mock + +import pytest +from socketdev.exceptions import APIFailure +from socketdev.fullscans import ( + FullScanParams, + FullScanStreamResponse, + StreamDiffResponse, +) + +from socketsecurity.config import CliConfig +from socketsecurity.core import Core +from socketsecurity.core.classes import Diff, Issue +from socketsecurity.core.messages import Messages +from socketsecurity.core.socket_config import SocketConfig +from socketsecurity.output import OutputHandler + +BUCKETS = ("added", "removed", "unchanged", "replaced", "updated") + + +def _alert(key, action="error", alert_type="malware"): + return { + "key": key, + "type": alert_type, + "severity": "critical", + "category": "supplyChainRisk", + "props": {}, + "action": action, + } + + +def _diff_artifact(diff_type, artifact_id, name, alerts=None): + ref = { + "direct": True, + "manifestFiles": [{"file": "package.json", "start": 0, "end": 0}], + "topLevelAncestors": [], + } + return { + "diffType": diff_type, + "type": "npm", + "name": name, + "version": "1.0.0", + "id": artifact_id, + "head": [ref], + "base": [], + "license": "MIT", + "licenseDetails": [], + "author": [], + "size": 1, + "score": {}, + "alerts": alerts or [], + } + + +def _scan_artifact(artifact_id, name, alerts): + return { + "type": "npm", + "name": name, + "version": "1.0.0", + "id": artifact_id, + "direct": True, + "manifestFiles": [{"file": "package.json", "start": 0, "end": 0}], + "topLevelAncestors": [], + "license": "MIT", + "licenseDetails": [], + "author": [], + "size": 1, + "score": {}, + "alerts": alerts, + } + + +def _diff_response(**buckets): + raw = json.loads((Path(__file__).parent.parent / "data/fullscans/diff/stream_diff.json").read_text()) + artifacts = {name: buckets.get(name, []) for name in BUCKETS} + return StreamDiffResponse.from_dict({ + "success": raw["success"], + "status": raw["status"], + "data": {**raw["data"], "artifacts": artifacts}, + }) + + +def _scan_response(*artifacts): + return FullScanStreamResponse.from_dict({ + "success": True, + "status": 200, + "artifacts": {artifact["id"]: artifact for artifact in artifacts}, + }) + + +@pytest.fixture +def core(mock_sdk_with_responses): + core = Core(config=SocketConfig(api_key="test_key"), sdk=mock_sdk_with_responses) + core.sdk.diffscans.create_from_ids.side_effect = Exception("diff-scans unavailable") + core.sdk.fullscans.stream_diff.side_effect = None + core.sdk.fullscans.stream.side_effect = None + core.resolve_base_full_scan_id = Mock(return_value="head") + core.create_full_scan = Mock(return_value=SimpleNamespace(id="new")) + return core + + +def _run(core, *flags, unchanged_alerts=None, scan_alerts=None, scan_id="u1"): + core.cli_config = CliConfig.from_args(["--api-token", "test", *flags]) + core.sdk.fullscans.stream_diff.return_value = _diff_response( + added=[_diff_artifact("added", "a1", "alpha", [_alert("k1")])], + unchanged=[_diff_artifact("unchanged", "u1", "gamma", unchanged_alerts)], + ) + core.sdk.fullscans.stream.return_value = _scan_response( + _scan_artifact("a1", "alpha", [_alert("k1")]), + _scan_artifact(scan_id, "gamma", scan_alerts if scan_alerts is not None else [_alert("k3")]), + ) + params = FullScanParams(repo="repo", branch="feature") + params.include_license_details = True + return core.create_new_diff(["/repo"], params, explicit_files=["/repo/package.json"]) + + +def _handler(strict_blocking=True): + config = Mock(spec=CliConfig) + config.disable_blocking = False + config.strict_blocking = strict_blocking + return OutputHandler(config, Mock()) + + +def test_strict_blocking_reads_existing_alerts_from_the_full_scan(core): + diff = _run(core, "--strict-blocking") + + assert [a.pkg_name for a in diff.unchanged_alerts] == ["gamma"] + assert diff.unchanged_alerts[0].error + core.sdk.fullscans.stream.assert_called_once() + + +def test_strict_blocking_fails_on_an_existing_blocking_alert(core): + """The exit-0 case: the only blocking alert is on a package the change didn't touch.""" + core.cli_config = CliConfig.from_args(["--api-token", "test", "--strict-blocking"]) + core.sdk.fullscans.stream_diff.return_value = _diff_response( + unchanged=[_diff_artifact("unchanged", "u1", "gamma")], + ) + core.sdk.fullscans.stream.return_value = _scan_response(_scan_artifact("u1", "gamma", [_alert("k3")])) + params = FullScanParams(repo="repo", branch="feature") + params.include_license_details = True + + diff = core.create_new_diff(["/repo"], params, explicit_files=["/repo/package.json"]) + + assert diff.new_alerts == [] + assert not _handler(strict_blocking=True).report_pass(diff) + + +def test_existing_alerts_match_by_package_when_artifact_ids_differ(core): + diff = _run(core, "--strict-blocking", scan_id="other-id") + + assert [a.pkg_name for a in diff.unchanged_alerts] == ["gamma"] + + +def test_existing_warnings_are_kept_and_ignored_alerts_are_not(core): + diff = _run( + core, "--strict-blocking", + scan_alerts=[_alert("k3", action="warn"), _alert("k4", action="ignore", alert_type="envVars")], + ) + + assert [(a.pkg_name, a.warn, a.error) for a in diff.unchanged_alerts] == [("gamma", True, False)] + + +@pytest.mark.parametrize("flags", [("--enable-gitlab-security",), ("--legal-format", "fossa")]) +def test_full_scope_reports_also_read_existing_alerts(core, flags): + diff = _run(core, *flags) + + assert [a.pkg_name for a in diff.unchanged_alerts] == ["gamma"] + + +def test_no_extra_fetch_when_no_output_reads_existing_alerts(core): + diff = _run(core) + + core.sdk.fullscans.stream.assert_not_called() + assert diff.unchanged_alerts == [] + + +def test_no_extra_fetch_when_the_comparison_already_has_existing_alerts(core): + """The streaming comparison includes them, and so will diff-scans once the API does.""" + diff = _run(core, "--strict-blocking", unchanged_alerts=[_alert("k3")]) + + core.sdk.fullscans.stream.assert_not_called() + assert [a.pkg_name for a in diff.unchanged_alerts] == ["gamma"] + + +def test_failed_full_scan_fetch_fails_the_run(core): + """Reporting no existing alerts after a failed fetch would let the run pass.""" + core.cli_config = CliConfig.from_args(["--api-token", "test", "--strict-blocking"]) + core.sdk.fullscans.stream_diff.return_value = _diff_response( + unchanged=[_diff_artifact("unchanged", "u1", "gamma")], + ) + core.sdk.fullscans.stream.return_value = FullScanStreamResponse.from_dict( + {"success": False, "status": 500, "message": "boom"} + ) + params = FullScanParams(repo="repo", branch="feature") + params.include_license_details = True + + with pytest.raises(APIFailure): + core.create_new_diff(["/repo"], params, explicit_files=["/repo/package.json"]) + + +# --- PR comment ------------------------------------------------------------------------ + + +def _issue(name, error=True, warn=False): + return Issue( + pkg_type="npm", + pkg_name=name, + pkg_version="1.0.0", + purl=f"{name}@1.0.0", + type="malware", + severity="critical", + title="Known malware", + description="Bad.", + suggestion="Remove it.", + url=f"https://socket.dev/npm/package/{name}", + props={}, + key=f"key-{name}", + error=error, + warn=warn, + ) + + +def _comment_diff(new=(), existing=()): + diff = Diff() + diff.new_alerts = list(new) + diff.unchanged_alerts = list(existing) + diff.diff_url = "https://socket.dev/diff" + diff.report_url = "https://socket.dev/report" + return diff + + +def _config(strict_blocking=True): + return SimpleNamespace(strict_blocking=strict_blocking, disable_ignore=False, scm="github") + + +def test_pr_comment_lists_existing_alerts_under_strict_blocking(): + diff = _comment_diff(new=[_issue("alpha")], existing=[_issue("gamma", error=False, warn=True), _issue("delta")]) + + comment = Messages.security_comment_template(diff, _config()) + + assert "Existing alerts (2)" in comment + assert "fails the run on the 1 with a **Block** action" in comment + # Blocking existing alerts come first. + assert comment.index("delta@1.0.0 - Known malware") < comment.index("gamma@1.0.0 - Known malware") + assert comment.index("alpha@1.0.0") < comment.index("Existing alerts") + + +def test_pr_comment_leaves_existing_alerts_out_without_strict_blocking(): + diff = _comment_diff(new=[_issue("alpha")], existing=[_issue("gamma")]) + + comment = Messages.security_comment_template(diff, _config(strict_blocking=False)) + + assert "Existing alerts" not in comment + assert "gamma" not in comment + + +def test_pr_comment_with_only_existing_alerts_says_nothing_new(): + diff = _comment_diff(existing=[_issue("gamma")]) + + comment = Messages.security_comment_template(diff, _config()) + + assert "No new alerts in this change." in comment + assert "Existing alerts (1)" in comment + + +def test_pr_comment_does_not_repeat_an_existing_alert_listed_as_new(): + diff = _comment_diff(new=[_issue("alpha")], existing=[_issue("alpha"), _issue("gamma")]) + + comment = Messages.security_comment_template(diff, _config()) + + assert "Existing alerts (1)" in comment + + +def test_pr_comment_stays_within_the_size_budget(monkeypatch): + monkeypatch.setattr(Messages, "SECURITY_COMMENT_SIZE_BUDGET", 6000) + diff = _comment_diff(existing=[_issue(f"pkg{i}") for i in range(20)]) + + comment = Messages.security_comment_template(diff, _config()) + + assert len(comment) < 6500 + assert "more existing alert(s) are not shown" in comment + assert "Existing alerts (20)" in comment + + +def test_existing_license_violations_use_the_license_row(): + license_alert = _issue("gamma") + license_alert.type = "licenseSpdxDisj" + license_alert.title = "License policy violation: GPL-3.0" + diff = _comment_diff(new=[_issue("alpha")], existing=[license_alert]) + + comment = Messages.security_comment_template(diff, _config()) + existing = comment[comment.index("Existing alerts"):] + + assert "gamma@1.0.0 has a License Policy Violation." in existing + assert "What is known malware?" not in existing diff --git a/tests/core/test_exit_codes.py b/tests/core/test_exit_codes.py new file mode 100644 index 00000000..f9713b3e --- /dev/null +++ b/tests/core/test_exit_codes.py @@ -0,0 +1,158 @@ +"""Exit codes are decided in one place, so the exit flags apply to every run. + +Failures that are not security findings raise InfrastructureError and reach +``socketcli.cli()``, which applies ``--disable-blocking`` and +``--exit-code-on-api-error``. Invalid flags exit 2 like argparse's own usage +errors, and Ctrl+C exits 130. See socketsecurity.exit_codes. +""" +import sys +from unittest.mock import MagicMock + +import pytest +from socketdev.exceptions import ( + APIFailure, + APIInsufficientPermissions, + APIResourceNotFound, +) + +from socketsecurity import socketcli +from socketsecurity.config import CliConfig +from socketsecurity.core import Core +from socketsecurity.core.socket_config import SocketConfig +from socketsecurity.exit_codes import IncompleteComparisonError, InfrastructureError + + +def _run_cli(monkeypatch, argv, error): + def fail_main_code(): + raise error + + monkeypatch.setattr(socketcli, "main_code", fail_main_code) + monkeypatch.setattr(sys, "argv", ["socketcli", "--api-token", "test", *argv]) + with pytest.raises(SystemExit) as exc_info: + socketcli.cli() + return exc_info.value.code + + +@pytest.mark.parametrize( + "argv, expected", + [ + ([], 3), + (["--exit-code-on-api-error", "100"], 100), + (["--disable-blocking"], 0), + (["--disable-blocking", "--exit-code-on-api-error", "100"], 0), + ], +) +def test_infrastructure_error_honors_exit_flags(monkeypatch, capsys, argv, expected): + code = _run_cli(monkeypatch, argv, InfrastructureError("token missing a scope")) + + assert code == expected + # Expected failures are reported without a traceback. + assert "Traceback" not in capsys.readouterr().err + + +def test_infrastructure_error_message_is_logged(monkeypatch, caplog): + _run_cli(monkeypatch, [], InfrastructureError("Unable to get GitLab API Token")) + + assert "Unable to get GitLab API Token" in caplog.text + + +def test_keyboard_interrupt_with_disable_blocking_exits_0(monkeypatch): + assert _run_cli(monkeypatch, ["--disable-blocking"], KeyboardInterrupt()) == 0 + + +def test_invalid_flags_exit_2_even_with_disable_blocking(): + """A misconfigured run never scanned anything, so --disable-blocking doesn't apply.""" + with pytest.raises(SystemExit) as exc_info: + CliConfig.from_args([ + "--api-token", "test", "--disable-blocking", + "--base-scan-id", "a", "--base-commit-sha", "b", + ]) + assert exc_info.value.code == 2 + + +@pytest.mark.parametrize("argv, expected", [([], 3), (["--disable-blocking"], 0)]) +def test_missing_api_token_honors_exit_flags(monkeypatch, caplog, argv, expected): + """A fork pull request has no secrets; --disable-blocking should keep it green.""" + monkeypatch.delenv("SOCKET_SECURITY_API_TOKEN", raising=False) + monkeypatch.delenv("SOCKET_SECURITY_API_KEY", raising=False) + monkeypatch.setattr(sys, "argv", ["socketcli", *argv]) + + with pytest.raises(SystemExit) as exc_info: + socketcli.cli() + + assert exc_info.value.code == expected + assert "Socket API Token not found" in caplog.text + + +# --- Repository lookup --------------------------------------------------------------- + + +@pytest.fixture +def core(): + sdk = MagicMock() + return Core(config=SocketConfig(api_key="test_key"), sdk=sdk) + + +def test_repo_lookup_permission_error_does_not_try_to_create(core): + """A token without repo:list is reported as such, not as a missing repository.""" + core.sdk.repos.repo.side_effect = APIInsufficientPermissions(status_code=403) + + with pytest.raises(InfrastructureError, match="repo:list"): + core.get_repo_info("my-repo") + + core.sdk.repos.post.assert_not_called() + + +def test_repo_lookup_not_found_creates_the_repository(core): + created = MagicMock(data="repo-info") + core.sdk.repos.repo.side_effect = [APIResourceNotFound(status_code=404), created] + core.sdk.repos.post.return_value = {"id": "repo"} + + assert core.get_repo_info("my-repo") == "repo-info" + core.sdk.repos.post.assert_called_once() + + +def test_repo_creation_failure_raises_infrastructure_error(core): + core.sdk.repos.repo.side_effect = APIResourceNotFound(status_code=404) + core.sdk.repos.post.side_effect = APIFailure("forbidden", status_code=403) + + with pytest.raises(InfrastructureError, match="repo:create"): + core.get_repo_info("my-repo") + + +# --- Incomplete comparisons ---------------------------------------------------------- + + +@pytest.fixture +def incomplete_response(): + return {"diff_scan": {"id": "diff-scan-1", "incomplete": True, "artifacts": {}}} + + +def test_incomplete_comparison_raises(core, incomplete_response): + core.sdk.diffscans.create_from_ids.return_value = {"diff_scan": {"id": "diff-scan-1"}} + core.sdk.diffscans.get.return_value = incomplete_response + + with pytest.raises(IncompleteComparisonError, match="incomplete"): + core.get_diff_scan_artifacts("head", "new") + + +def test_incomplete_comparison_does_not_fall_back_to_streaming(core, incomplete_response): + """The streaming comparison reads the same scans, so it can't fill the gap.""" + core.sdk.diffscans.create_from_ids.return_value = {"diff_scan": {"id": "diff-scan-1"}} + core.sdk.diffscans.get.return_value = incomplete_response + + with pytest.raises(IncompleteComparisonError): + core.get_added_and_removed_packages("head", "new") + + core.sdk.fullscans.stream_diff.assert_not_called() + + +def test_complete_comparison_is_returned(core): + core.sdk.diffscans.create_from_ids.return_value = {"diff_scan": {"id": "diff-scan-1"}} + core.sdk.diffscans.get.return_value = { + "diff_scan": {"id": "diff-scan-1", "incomplete": False, "artifacts": {}}, + } + + artifacts = core.get_diff_scan_artifacts("head", "new") + + assert artifacts.added == [] diff --git a/tests/core/test_sdk_methods.py b/tests/core/test_sdk_methods.py index 9235bc6b..b69d57a1 100644 --- a/tests/core/test_sdk_methods.py +++ b/tests/core/test_sdk_methods.py @@ -13,6 +13,7 @@ from socketsecurity.config import CliConfig from socketsecurity.core import ANCESTOR_SCAN_LOOKUP_LIMIT, SCAN_LOOKUP_PAGE_SIZE, Core from socketsecurity.core.socket_config import SocketConfig +from socketsecurity.exit_codes import InfrastructureError @pytest.fixture @@ -54,7 +55,7 @@ def test_get_repo_info(core, mock_sdk_with_responses): def test_get_repo_info_logs_why_repo_creation_failed(core, mock_sdk_with_responses, caplog): - """A repo that isn't in Socket yet and can't be created exits 2 with the reason logged""" + """A repo that isn't in Socket yet and can't be created fails with the reason""" mock_sdk_with_responses.repos.repo.side_effect = APIResourceNotFound(status_code=404) # Mirrors the SDK, which appends the request path and response headers to the message mock_sdk_with_responses.repos.post.side_effect = APIAccessDenied( @@ -62,14 +63,13 @@ def test_get_repo_info_logs_why_repo_creation_failed(core, mock_sdk_with_respons status_code=403, ) - with caplog.at_level(logging.INFO, logger="socketdev"), pytest.raises(SystemExit) as exit_info: + with caplog.at_level(logging.INFO, logger="socketdev"), pytest.raises(InfrastructureError) as error_info: core.get_repo_info("new-repo") - assert exit_info.value.code == 2 messages = [record.getMessage() for record in caplog.records] assert "Failed to get repository new-repo, attempting to create it" in messages - assert "API failure while creating repository new-repo (HTTP 403): Insufficient permissions" in messages - assert any("repo:create" in message for message in messages) + assert "API failure while creating repository new-repo (HTTP 403): Insufficient permissions" in str(error_info.value) + assert "repo:create" in str(error_info.value) @pytest.mark.parametrize( @@ -86,12 +86,12 @@ def test_get_repo_info_repo_creation_failure_reason( mock_sdk_with_responses.repos.repo.side_effect = APIResourceNotFound(status_code=404) mock_sdk_with_responses.repos.post.side_effect = error - with caplog.at_level(logging.INFO, logger="socketdev"), pytest.raises(SystemExit): + with pytest.raises(InfrastructureError) as error_info: core.get_repo_info("new-repo") - messages = [record.getMessage() for record in caplog.records] - assert f"API failure while creating repository new-repo (HTTP 403): {expected_reason}" in messages - assert any("repo:create" in message for message in messages) is expects_scope_hint + message = str(error_info.value) + assert f"API failure while creating repository new-repo (HTTP 403): {expected_reason}" in message + assert ("repo:create" in message) is expects_scope_hint def test_get_repo_info_restores_log_level_after_creating_repo( @@ -339,28 +339,19 @@ def test_resolve_base_full_scan_id_scopes_head_to_workspace(core): }, ) -def test_resolve_base_full_scan_id_workspace_lookup_failure_exits(core): - """A failed workspace lookup fails the run instead of diffing against an empty scan""" - core.cli_config = make_cli_config() - core.sdk.fullscans.get.return_value = {} - - params = make_full_scan_params(workspace="customer-a") - - with pytest.raises(SystemExit) as exc_info: - core.resolve_base_full_scan_id(params) - assert exc_info.value.code == core.cli_config.exit_code_on_api_error - +@pytest.mark.parametrize("flags", [(), ("--disable-blocking",)]) +def test_resolve_base_full_scan_id_workspace_lookup_failure_fails(core, flags): + """A failed workspace lookup fails the run instead of diffing against an empty scan. -def test_resolve_base_full_scan_id_workspace_lookup_failure_disable_blocking(core): - """--disable-blocking keeps the failed lookup from failing the build""" - core.cli_config = make_cli_config("--disable-blocking") + The exit flags are applied by cli(), so the failure is raised either way. + """ + core.cli_config = make_cli_config(*flags) core.sdk.fullscans.get.return_value = {} params = make_full_scan_params(workspace="customer-a") - with pytest.raises(SystemExit) as exc_info: + with pytest.raises(InfrastructureError, match="Failed to resolve the matching head scan"): core.resolve_base_full_scan_id(params) - assert exc_info.value.code == 0 def test_resolve_base_full_scan_id_workspace_lookup_failure_without_cli_config(core): @@ -486,7 +477,7 @@ def test_resolve_base_full_scan_id_ancestor_fallback_skips_temporary_scans(core, lambda self, sha, depth: ["unscanned-sha", "ancestor-1"], ) - with pytest.raises(SystemExit): + with pytest.raises(InfrastructureError): core.resolve_base_full_scan_id(make_full_scan_params()) @@ -499,27 +490,18 @@ def test_resolve_base_full_scan_id_ancestor_fallback_needs_local_history(core, m ] monkeypatch.setattr(Core, "first_parent_commits", lambda self, sha, depth: []) - with pytest.raises(SystemExit): + with pytest.raises(InfrastructureError): core.resolve_base_full_scan_id(make_full_scan_params()) -def test_resolve_base_full_scan_id_commit_sha_not_found_exits(core): - """A --base-commit-sha with no scan is a hard error (exit_code_on_api_error)""" - core.cli_config = make_cli_config("--base-commit-sha", "abc123") - core.sdk.fullscans.get.return_value = {"results": [], "nextPage": None} - - with pytest.raises(SystemExit) as exc_info: - core.resolve_base_full_scan_id(make_full_scan_params()) - assert exc_info.value.code == core.cli_config.exit_code_on_api_error - -def test_resolve_base_full_scan_id_commit_sha_not_found_disable_blocking(core): - """--disable-blocking keeps the missing-base error from failing the build""" - core.cli_config = make_cli_config("--base-commit-sha", "abc123", "--disable-blocking") +@pytest.mark.parametrize("flags", [(), ("--disable-blocking",)]) +def test_resolve_base_full_scan_id_commit_sha_not_found_fails(core, flags): + """A --base-commit-sha with no scan is an infrastructure error; cli() applies the exit flags""" + core.cli_config = make_cli_config("--base-commit-sha", "abc123", *flags) core.sdk.fullscans.get.return_value = {"results": [], "nextPage": None} - with pytest.raises(SystemExit) as exc_info: + with pytest.raises(InfrastructureError, match="No full scan found for commit abc123"): core.resolve_base_full_scan_id(make_full_scan_params()) - assert exc_info.value.code == 0 def test_get_full_scan(core, mock_sdk_with_responses, head_scan_metadata, head_scan_stream): """Test getting an existing full scan""" diff --git a/tests/core/test_strict_blocking_duplicates.py b/tests/core/test_strict_blocking_duplicates.py new file mode 100644 index 00000000..a37f67cc --- /dev/null +++ b/tests/core/test_strict_blocking_duplicates.py @@ -0,0 +1,213 @@ +"""`--strict-blocking` lists each finding once, whatever the baseline. + +Added packages must not also count as unchanged. An added package that leaks into +the unchanged set has its alerts reported twice under `--strict-blocking`, once as +new and once as existing, doubling the JSON, SARIF, Slack and GitLab findings and +inflating the "EXISTING blocking issues" count in the summary. + +These tests run `Core.create_new_diff` against a mocked diff response, with the +scan upload mocked out, then check what the output layer renders. +""" +import json +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import Mock + +import pytest +from socketdev.fullscans import FullScanParams, StreamDiffResponse + +from socketsecurity.config import CliConfig +from socketsecurity.core import Core +from socketsecurity.core.alert_selection import select_diff_alerts +from socketsecurity.core.classes import Diff, Issue +from socketsecurity.core.socket_config import SocketConfig +from socketsecurity.output import OutputHandler + +BUCKETS = ("added", "removed", "unchanged", "replaced", "updated") + + +@pytest.fixture +def core(mock_sdk_with_responses): + core = Core(config=SocketConfig(api_key="test_key"), sdk=mock_sdk_with_responses) + # Force the streaming diff so the mocked response below is the artifact source. + core.sdk.diffscans.create_from_ids.side_effect = Exception("diff-scans unavailable") + core.sdk.fullscans.stream_diff.side_effect = None + return core + + +def _artifact(diff_type, artifact_id, name, alerts=None, direct=True, ancestors=None): + ref = { + "direct": direct, + "manifestFiles": [{"file": "package.json", "start": 0, "end": 0}] if direct else [], + "topLevelAncestors": ancestors or [], + } + return { + "diffType": diff_type, + "type": "npm", + "name": name, + "version": "1.0.0", + "id": artifact_id, + "head": [] if diff_type in ("removed", "replaced") else [ref], + "base": [ref] if diff_type in ("removed", "replaced") else [], + "license": "MIT", + "licenseDetails": [], + "author": [], + "size": 1, + "score": {}, + "alerts": alerts or [], + } + + +def _blocking_alert(key, alert_type="malware"): + return { + "key": key, + "type": alert_type, + "severity": "critical", + "category": "supplyChainRisk", + "props": {}, + "action": "error", + } + + +def _diff_response(**buckets): + raw = json.loads((Path(__file__).parent.parent / "data/fullscans/diff/stream_diff.json").read_text()) + artifacts = {name: buckets.get(name, []) for name in BUCKETS} + return StreamDiffResponse.from_dict({ + "success": raw["success"], + "status": raw["status"], + "data": {**raw["data"], "artifacts": artifacts}, + }) + + +def _build_diff(core, response): + """Run Core.create_new_diff with the upload mocked out and `response` as the diff.""" + core.sdk.fullscans.stream_diff.return_value = response + core.resolve_base_full_scan_id = Mock(return_value="head") + core.create_full_scan = Mock(return_value=SimpleNamespace(id="new")) + params = FullScanParams(repo="repo", branch="feature") + params.include_license_details = True + return core.create_new_diff( + ["/repo"], params, explicit_files=["/repo/package.json"], + ) + + +def _handler(strict_blocking=True): + config = Mock(spec=CliConfig) + config.disable_blocking = False + config.strict_blocking = strict_blocking + config.legal_format = "socket" + config.legal = False + config.repo = None + config.branch = None + config.commit_sha = None + return OutputHandler(config, Mock()) + + +def test_empty_baseline_lists_added_alerts_once(core): + """Against an empty baseline every package is added and none is unchanged.""" + diff = _build_diff(core, _diff_response(added=[ + _artifact("added", "a1", "alpha", [_blocking_alert("k1")]), + _artifact("added", "a2", "beta", [_blocking_alert("k2")]), + ])) + + assert len(diff.new_alerts) == 2 + assert diff.unchanged_alerts == [] + + handler = _handler() + report = handler.build_json_report(diff) + assert len(report["new_alerts"]) == 2 + assert not handler.report_pass(diff) + + summary = handler.build_summary_text(diff) + assert "NEW blocking issues: 2" in summary + assert "EXISTING blocking issues" not in summary + + +def test_real_baseline_lists_new_and_existing_alerts_once_each(core): + """With a baseline, added alerts are new and unchanged alerts are existing.""" + diff = _build_diff(core, _diff_response( + added=[_artifact("added", "a1", "alpha", [_blocking_alert("k1")])], + unchanged=[_artifact("unchanged", "u1", "gamma", [_blocking_alert("k3")])], + )) + + assert [a.pkg_name for a in diff.new_alerts] == ["alpha"] + assert [a.pkg_name for a in diff.unchanged_alerts] == ["gamma"] + + handler = _handler() + report = handler.build_json_report(diff) + assert sorted(a["pkg_name"] for a in report["new_alerts"]) == ["alpha", "gamma"] + assert not handler.report_pass(diff) + + summary = handler.build_summary_text(diff) + assert "NEW blocking issues: 1" in summary + assert "EXISTING blocking issues: 1" in summary + + +def test_real_baseline_fails_on_existing_alerts_alone(core): + """An unchanged blocking alert fails the run when the API returns it.""" + diff = _build_diff(core, _diff_response( + unchanged=[_artifact("unchanged", "u1", "gamma", [_blocking_alert("k3")])], + )) + + assert diff.new_alerts == [] + assert not _handler(strict_blocking=True).report_pass(diff) + assert _handler(strict_blocking=False).report_pass(diff) + + +def test_without_strict_blocking_output_is_unchanged(core): + diff = _build_diff(core, _diff_response( + added=[_artifact("added", "a1", "alpha", [_blocking_alert("k1")])], + unchanged=[_artifact("unchanged", "u1", "gamma", [_blocking_alert("k3")])], + )) + + report = _handler(strict_blocking=False).build_json_report(diff) + assert [a["pkg_name"] for a in report["new_alerts"]] == ["alpha"] + + +def test_unchanged_packages_are_keyed_by_artifact_id(core): + """Transitive unchanged alerts resolve their introducer through the ancestor ID.""" + diff = _build_diff(core, _diff_response(unchanged=[ + _artifact("unchanged", "top", "parent"), + _artifact( + "unchanged", "dep", "child", [_blocking_alert("k5")], + direct=False, ancestors=["top"], + ), + ])) + + assert diff.unchanged_alerts[0].introduced_by == [("npm/parent@1.0.0", "package.json")] + + +def _issue(name, alert_type="malware", pkg_type="npm"): + return Issue( + pkg_type=pkg_type, + pkg_name=name, + pkg_version="1.0.0", + type=alert_type, + purl=f"{name}@1.0.0", + error=True, + ) + + +def test_select_diff_alerts_skips_unchanged_alert_already_listed_as_new(): + diff = Diff() + diff.new_alerts = [_issue("alpha")] + diff.unchanged_alerts = [_issue("alpha"), _issue("alpha", "typosquat"), _issue("gamma")] + + selected = select_diff_alerts(diff, strict_blocking=True) + + assert [(a.pkg_name, a.type) for a in selected] == [ + ("alpha", "malware"), + ("alpha", "typosquat"), + ("gamma", "malware"), + ] + + +def test_select_diff_alerts_keeps_same_purl_from_another_ecosystem(): + """`Issue.purl` has no ecosystem, so npm and PyPI packages can share one.""" + diff = Diff() + diff.new_alerts = [_issue("alpha", pkg_type="npm")] + diff.unchanged_alerts = [_issue("alpha", pkg_type="pypi")] + + selected = select_diff_alerts(diff, strict_blocking=True) + + assert [a.pkg_type for a in selected] == ["npm", "pypi"] diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py index 50047caa..d22b81ff 100644 --- a/tests/unit/test_config.py +++ b/tests/unit/test_config.py @@ -94,7 +94,7 @@ def test_sarif_scope_full_without_reach_exits(self): """--sarif-scope full without --reach should exit with code 1""" with pytest.raises(SystemExit) as exc_info: CliConfig.from_args(self.BASE_ARGS + ["--sarif-scope", "full"]) - assert exc_info.value.code == 1 + assert exc_info.value.code == 2 def test_sarif_scope_full_with_reach_succeeds(self): """--sarif-scope full with --reach should parse successfully""" @@ -105,7 +105,7 @@ def test_sarif_scope_full_with_reach_succeeds(self): def test_sarif_reachability_without_reach_exits(self): with pytest.raises(SystemExit) as exc_info: CliConfig.from_args(self.BASE_ARGS + ["--sarif-reachability", "reachable"]) - assert exc_info.value.code == 1 + assert exc_info.value.code == 2 def test_sarif_reachability_with_reach_succeeds(self): config = CliConfig.from_args( @@ -117,7 +117,7 @@ def test_sarif_reachability_with_reach_succeeds(self): def test_sarif_grouping_alert_requires_full_scope(self): with pytest.raises(SystemExit) as exc_info: CliConfig.from_args(self.BASE_ARGS + ["--reach", "--sarif-grouping", "alert"]) - assert exc_info.value.code == 1 + assert exc_info.value.code == 2 def test_sarif_reachability_reachable_with_reach_succeeds(self): config = CliConfig.from_args(self.BASE_ARGS + ["--reach", "--sarif-reachability", "reachable"]) @@ -175,7 +175,7 @@ class TestRepoAndPendingHeadFlags: def test_repo_with_owner_exits_with_suggestion(self, repo, caplog): with pytest.raises(SystemExit) as exc_info: CliConfig.from_args(["--api-token", "test-token", "--repo", repo]) - assert exc_info.value.code == 1 + assert exc_info.value.code == 2 assert "Use --repo my-repo" in caplog.text def test_repo_slug_is_accepted(self): @@ -235,7 +235,7 @@ def test_config_file_values_are_mutually_exclusive(self, tmp_path): ) with pytest.raises(SystemExit) as exc_info: CliConfig.from_args(self.BASE_ARGS + ["--config", str(config_path)]) - assert exc_info.value.code == 1 + assert exc_info.value.code == 2 def test_config_file_sets_base_commit_sha(self, tmp_path): config_path = tmp_path / "socketcli.toml" diff --git a/tests/unit/test_exclude_paths.py b/tests/unit/test_exclude_paths.py index 32271a1a..0abce732 100644 --- a/tests/unit/test_exclude_paths.py +++ b/tests/unit/test_exclude_paths.py @@ -92,13 +92,13 @@ def test_reach_exclude_paths_still_works_and_warns(caplog): def test_exclude_paths_validation_rejects(bad): with pytest.raises(SystemExit) as exc: CliConfig.from_args(BASE_ARGS + ["--exclude-paths", bad]) - assert exc.value.code == 1 + assert exc.value.code == 2 def test_exclude_paths_validation_rejects_within_csv(): with pytest.raises(SystemExit) as exc: CliConfig.from_args(BASE_ARGS + ["--exclude-paths", "src,..,tests"]) - assert exc.value.code == 1 + assert exc.value.code == 2 def _write_config(tmp_path, value): @@ -126,7 +126,7 @@ def test_exclude_paths_from_config_file_is_validated(tmp_path): cfg = _write_config(tmp_path, ["../escape"]) with pytest.raises(SystemExit) as exc: CliConfig.from_args(BASE_ARGS + ["--config", cfg]) - assert exc.value.code == 1 + assert exc.value.code == 2 def test_exclude_paths_valid_globs_accepted(): diff --git a/tests/unit/test_gitlab_auth.py b/tests/unit/test_gitlab_auth.py index 42c5e6bc..65f3e180 100644 --- a/tests/unit/test_gitlab_auth.py +++ b/tests/unit/test_gitlab_auth.py @@ -6,6 +6,7 @@ from socketsecurity import USER_AGENT from socketsecurity.core.scm.gitlab import GitlabConfig +from socketsecurity.exit_codes import InfrastructureError class TestGitlabAuthHeaders: @@ -107,10 +108,10 @@ def test_from_env_with_private_token(self): assert config.headers['PRIVATE-TOKEN'] == 'custom-token' assert 'Authorization' not in config.headers - def test_from_env_missing_token_exits(self): - """from_env should exit when GITLAB_TOKEN is missing""" + def test_from_env_missing_token_raises_infrastructure_error(self): + """A missing GITLAB_TOKEN goes through cli()'s exit handling, so the exit flags apply.""" with patch.dict(os.environ, {}, clear=True): - with pytest.raises(SystemExit): + with pytest.raises(InfrastructureError, match="GITLAB_TOKEN"): GitlabConfig.from_env() diff --git a/tests/unit/test_socketcli.py b/tests/unit/test_socketcli.py index 765150e9..1f298e0c 100644 --- a/tests/unit/test_socketcli.py +++ b/tests/unit/test_socketcli.py @@ -59,11 +59,11 @@ def test_disable_blocking_overrides_exit_code_on_api_error(monkeypatch): assert code == 0 -def test_keyboard_interrupt_still_exits_2(monkeypatch): +def test_keyboard_interrupt_exits_130(monkeypatch): code = _run_cli_expecting_exit( monkeypatch, ["socketcli", "--api-token", "test"], boom=KeyboardInterrupt() ) - assert code == 2 + assert code == 130 @pytest.mark.parametrize("scm", ["github", "gitlab"]) diff --git a/uv.lock b/uv.lock index 6077c132..a6271a3d 100644 --- a/uv.lock +++ b/uv.lock @@ -1334,7 +1334,7 @@ wheels = [ [[package]] name = "socketsecurity" -version = "2.12.1" +version = "2.13.0" source = { editable = "." } dependencies = [ { name = "beautifulsoup4" },