Skip to main content

Project Level Metrics

For a given open source project, here are some ways of measuring the health of the project, thereby determining Dependency Risk.

What To Measure​

Committer Strength

Committer Strength

Number of individual committers (within a given recency).

GitHub provides some indication of this for each project. First on the front page of the project it gives a committer count:

GitHub Committers

Secondly on the Insights/Pulse page (for example here) you can see the activity for each committer, like so:

Pulse for a committer

See:

  • Backstage: This builds a graph of the tech landscape by consuming other golden-sources of data, such as GitHub, Artifactory, Jenkins, JIRA, etc. This gives you a really good high-level view of the whole landscape. More and more sources are getting backstage plugins to allow them to connect their data.
  • The GitHub GraphQL API.
  • Cauldron.io: Open source ecosystem analytics
Committer Diversity

Committer Diversity

Diversity of committer organisations

If all the committers to a project hail from the same organisation, this might represent a single-point-of-failure if the organisation defunds the initiative.

Stars

Stars

GitHub and GitLab both allow you to "Star" projects you like. The number of stars helps as an indicator of all-time popularity but doesn't show whether a project is still_popular.

GitHub Stars

GitLab Stars

Issues

Issues

It can be useful to review the open issues for a project, as well as comparing this with the number of closed issues. A healthy project like the one below will have vastly more closed issues than open ones.

GitLab Stars

Badges

Badges

Projects will often advertise "badges" on their README Pages. This can be used as another signal for determining Dependency Risk

GitLab Stars

Of particular interest are badges such as:

  • OpenSSF: This is a checklist of security behaviours.
  • FINOS Active: indicates the project is now mature in terms of quality and community.

Foundations like Apache don't rely on badges, but instead rename their projects when active. Therefore Incubator-Pegasus will become simply "Pegasus" once it reaches Active state.

Release Cadence

Release Cadence

For an open source / inner source project, how quickly are changes incorporated and released? If this can be done automatically, this is often (but not always) a sign of a mature CI / CD Pipeline.

For GitHub projects, details of the releases and tags are readily available, both summarised on the front page:

GitLab Stars

Broken down more thoroughly in the /tags and /releases sub-pages:

GitLab Stars

... and available programmatically via GraphQL.

Total Cost of Ownership (TCO)

Total Cost of Ownership (TCO)

How does using open source software compare with buying off-the-shelf? The costs are very different.

See: