Skip to content

Conversation

@AdamKorcz
Copy link
Contributor

What kind of change does this PR introduce?

New check

This PR adds a new check with 3 underlying probes to scorecard. The check is an implementation of #2458. In essence, the check works like this:

  1. First, it collects a list of all the project's direct dependencies at the time of checking.
  2. Then, it calls deps.dev for a list of releases of each dependency.
  3. Then, the check assesses whether each dependency is the latest available version according to deps.dev.
  4. With the dependencies that are not the latest available version, the check assesses how much time has passed between the release date of the version that the project uses and the release date of the version right after the project uses. For example, if a project uses 1.2.3, and 1.2.4 was available 100 days ago, the check will record 100 days, even though 1.2.5 is the newest available. This is a design part on my end.
  5. Finally, the check creates an average of the dependencies that are not the latest available version and scores accordingly.

As such, the check considers the negative findings in a project - the dependencies that have not been updated to its latest version - and the check can be considered one that reports how old a project's dependencies are rather than providing a full picture of how quickly a project updates its dependencies.

A few notes on implementation details:

  1. The check relies heavily on osv-scalibr and the deps.dev API which means that any shortcomings in these two libraries will have direct impact on the check. The most significant shortcoming I have found is that the check doesn't always process all dependencies that it finds in a dependency manifest. I haven't debugged this extensively. At the same time, osv-scalibr and deps.dev seem like great sources for this check, and a few edge cases shouldn't in my opinion prevent us from using them. Even if the check doesn't process all dependencies, it is still useful by assessing the majority of a projects dependencies. Initially, I was working with distinct parsers for different lock file types, but this is in my opinion not the way to go. osv-scalibr provides a lot of value out of the box, and given that it is a fairly new library, I think it is fair to expect that it will continue its development.
  2. The check only considers direct dependencies. This is a design decision as I don't see a point in evaluating transitive dependencies; The goal of the check is to determine a project's maintenance practices - not to find old dependencies. As such, I have made the decision to give maintainers as much control I could over their score; by only considering direct dependencies, maintainers can simply update their dependencies. If we were to consider transitive dependencies too, maintainers are at the mercy of their direct dependencies to keep their dependencies up to date.
  • PR title follows the guidelines defined in our pull request documentation

  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

Fixes #2458

Special notes for your reviewer

I would recommend reading the documentation in the .yml and .yaml files in this PR and testing the check with go run main.go --repo=github.com/owner/repo --checks=MTTUDependencies before reviewing the code.

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)


@AdamKorcz AdamKorcz requested a review from a team as a code owner December 3, 2025 20:37
@AdamKorcz AdamKorcz requested review from raghavkaul and spencerschrock and removed request for a team December 3, 2025 20:37
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Dec 3, 2025
@codecov
Copy link

codecov bot commented Dec 3, 2025

Codecov Report

❌ Patch coverage is 80.65173% with 95 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.42%. Comparing base (353ed60) to head (7a015f1).
⚠️ Report is 285 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4868      +/-   ##
==========================================
+ Coverage   66.80%   68.42%   +1.62%     
==========================================
  Files         230      259      +29     
  Lines       16602    16147     -455     
==========================================
- Hits        11091    11049      -42     
+ Misses       4808     4214     -594     
- Partials      703      884     +181     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Feature: New check for average time to update dependencies

1 participant