Project Overview
SpiderFoot is an open-source OSINT automation tool written in Python. You give it a single seed — a domain, IP address, netblock, ASN, email address, phone number, username, or Bitcoin address — and it fans out across more than 200 collection modules, querying public data sources and chaining results together until the picture stops growing. What comes back is a linked dataset of everything publicly discoverable about the target: subdomains, open ports, leaked credentials, related identities, hosting relationships, and more.
You can realistically use it for two jobs. Defensively, point it at your own organization to map your attack surface and find the things you forgot you exposed. Offensively, use it for reconnaissance during authorized penetration tests — authorized being the operative word, because several scan profiles actively probe the target rather than just reading public records.
The license is MIT, so embedding it in internal tooling won't trigger a legal review.
Now the part you need to hear before you commit: development has effectively stalled. Intel 471 acquired SpiderFoot in 2022 to build its commercial product, and the last major open-source release, version 4.0, shipped that same year. Commit activity on the repository has been minimal since. The core still works and the architecture is genuinely good, but OSINT tools live and die by their integrations — when a third-party API changes, the module that depends on it breaks, and in this repo broken modules tend to stay broken. Treat SpiderFoot as stable but frozen, and weigh that maintenance risk before you standardize on it.
Key Challenges Addressed
OSINT work has a few problems that bite everyone, and SpiderFoot has specific machinery for each.
- Manual collection doesn't scale. Checking a target against DNS records, certificate transparency logs, breach databases, Shodan, and social platforms by hand means a dozen browser tabs and a dozen output formats. SpiderFoot wraps each source in a module with a common event interface, so one scan replaces the tab farm and everything lands in one database.
- Pivots are where investigations win or die. A domain leads to an email, the email leads to a username, the username leads to a breach record. SpiderFoot's scanner is event-driven: every data element a module emits is fed back to every module that consumes that element type, so pivots happen automatically instead of depending on you noticing the thread.
- Attack surfaces rot silently. Forgotten subdomains, expired-but-still-resolving DNS, services listening on odd ports. Modules for passive DNS, certificate transparency via crt.sh, and internet-wide scan data surface those without you enumerating anything by hand.
- Raw data isn't a finding. Thousands of events per scan is normal, and nobody reads thousands of rows. Version 4 added a correlation engine with rules defined in YAML that post-processes scan data into flagged findings, so noteworthy combinations float to the top instead of drowning in noise.
Getting Started
You need Python 3.9 or newer. Install and launch take about a minute:
git clone https://github.com/smicallef/spiderfoot.git
cd spiderfoot
pip3 install -r requirements.txt
python3 sf.py -l 127.0.0.1:5001
Open http://127.0.0.1:5001 in your browser for the web UI. Create a new scan, pick the target type, and choose one of four use cases: All, Footprint, Investigate, or Passive. For headless operation, run scans straight from the shell with python3 sf.py -s example.com -u all, or drive a remote server with the bundled sfcli.py client. A Dockerfile ships in the repo if you'd rather not manage the Python environment yourself.
Sharp edges you'll hit in the first hour:
- The web UI has no authentication. If you bind to
0.0.0.0instead of127.0.0.1, anyone who can reach the port can launch scans and read every result you've ever collected. Keep it on localhost, or put a reverse proxy with auth in front of it. - Keyless modules fail silently. A scan with no API keys configured "completes" fine but comes back thin, and you'll wrongly conclude the tool is weak. Go to Settings and add keys for Shodan, VirusTotal, SecurityTrails, and HaveIBeenPwned before judging your first scan.
- Footprint and Investigate profiles actively touch the target — port checks, web crawling, the works. If your authorization only covers passive collection, pick the Passive use case, or you've just probed infrastructure you had no permission to probe.
- All scan data lives in a single SQLite file,
spiderfoot.db, in the install directory. Delete the directory or rebuild the container without a volume mount and your scan history is gone. Mount or back up that file before anything else.
Features and Use Cases
The module library is the headline feature: 200+ modules covering DNS enumeration, certificate transparency, breach lookups through HaveIBeenPwned, internet scan data through Shodan and Censys, threat intel feeds like AlienVault OTX, file metadata extraction, username enumeration across platforms, and cryptocurrency address checks. You select modules individually per scan, or let a use-case profile pick a sensible set for you.
The web UI organizes results by data element type, draws a node graph showing how each finding was derived from the seed, and exports to CSV, JSON, and GEXF for downstream graph tooling.
Concrete ways to run it:
- Attack surface review. Seed a scan with your primary domains, run Footprint, then review discovered subdomains and exposed services against your asset inventory. Schedule headless runs from cron with
sf.pyand diff the exports between runs to catch new exposure. - Pre-engagement recon. Run Passive against a pentest target before kickoff so you arrive with the public exposure already mapped, without having sent a single packet to their infrastructure.
- Identity investigation. Seed with an email address or username and let the breach, paste, and account-enumeration modules build out the linked identity graph — useful for fraud and insider-threat work.
- Threat intel enrichment. Seed with attacker infrastructure from your IDS alerts and let the reputation and passive DNS modules pull in blocklist history and related infrastructure.
Ecosystem and Dependencies
SpiderFoot is self-contained by design: Python, an embedded CherryPy web server, and SQLite for storage. There's no external database or queue to operate, which makes it one of the easiest security tools to stand up.
Its real ecosystem is API-side. Modules integrate with dozens of third-party services — Shodan, Censys, VirusTotal, SecurityTrails, Hunter.io, HaveIBeenPwned, AlienVault OTX, and many more. Free tiers exist for several, but the gap between a keyed scan and a keyless one is dramatic, so budget for at least a couple of paid keys if you're doing serious work. SpiderFoot can also route relevant modules through TOR when an investigation needs dark web content.
It also plays well next to narrower tools. A common pattern is pairing it with Amass or subfinder for deeper dedicated subdomain enumeration, then using SpiderFoot for the broader identity and exposure picture around what those tools find.
Architectural Overview
The core design is an event bus with typed data elements. Every module declares which element types it consumes (watchedEvents) and which it produces (producedEvents), and implements a handleEvent method. The scanner seeds the bus with your target, and modules fire as matching elements appear — module A's output becomes module B's input with no orchestration code anywhere. The scan ends when no module produces anything new.
Three consequences of that design are worth understanding:
- Pivoting is free. Nobody wrote "if you find an email, check it against breaches" — the breach module simply subscribes to email elements. Adding one module enriches every scan path that produces its input type.
- Failure is contained. A module that breaks because its upstream API changed degrades one data source; the scan keeps running. Given the repo's maintenance state, this isolation is the reason the tool stays usable at all.
- Writing modules is the extension path. Subclass
SpiderFootPlugin, declare your event types, implementhandleEvent. If you have an internal data source you want in every investigation, this is a genuinely small lift.
Storage is SQLite, with every element recorded alongside the element that produced it — that lineage is what powers the provenance graph in the UI. The correlation engine runs YAML-defined rules over stored elements and writes findings back. The CherryPy layer serves the UI and the endpoints that sfcli.py drives. The single-file SQLite design is also the scaling ceiling: one instance, modest concurrency, no horizontal story. Fine for a team; wrong for continuous monitoring across a large estate.
Pros and Cons
Pros:
- Broadest module coverage of any open-source OSINT tool — one install replaces a shelf of single-purpose scripts.
- Event-driven pivoting surfaces connections you wouldn't have thought to query for.
- Trivial to deploy: Python plus SQLite, no external services, Docker if you want it.
- MIT licensed, with both a web UI and a scriptable CLI.
- Clean module API makes private extensions practical.
Cons:
- Maintenance has been minimal since the 2022 acquisition; integration modules break as third-party APIs drift, and fixes are slow or absent.
- The web UI ships with no authentication — securing access is entirely your problem.
- Serious results require paid API keys; keyless scans undersell the tool.
- SQLite and the single-instance design cap it at team scale; continuous enterprise monitoring is what the commercial product exists for.
- Big scans are noisy — shared hosting and CDN relationships generate associations you'll spend time discarding.
Comparison and Alternatives
GitHub stars
Weekly star gains
Full metrics details
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
Spiderfoot
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
150 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Issues opened
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Spiderfoot
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Issues closed
Weekly total
Full metrics details
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
Spiderfoot
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests opened
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Spiderfoot
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests closed
Weekly total
Full metrics details
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
Spiderfoot
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests merged
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Spiderfoot
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub API observation. Historical values render only when the source provides a real observation for that day.
142 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Open/closed pull request ratio
Weekly average
Full metrics details
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
Spiderfoot
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
150 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Open/closed issues ratio
Weekly average
Full metrics details
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
Spiderfoot
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
150 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Releases
Weekly total
Full metrics details
GitHub release published_at events bucketed by UTC day; drafts are excluded.
Spiderfoot
GitHub release published_at events bucketed by UTC day; drafts are excluded.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Amass
GitHub release published_at events bucketed by UTC day; drafts are excluded.
150 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
theHarvester
GitHub release published_at events bucketed by UTC day; drafts are excluded.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
subfinder
GitHub release published_at events bucketed by UTC day; drafts are excluded.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Hotness score
Weekly average
Full metrics details
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
Spiderfoot
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Amass
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
150 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
theHarvester
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
178 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
subfinder
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Reliability score
Weekly average
Full metrics details
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
Spiderfoot
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Amass
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
150 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
theHarvester
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
178 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
subfinder
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
The honest comparison is by scope, because no single open-source tool matches SpiderFoot's breadth.
Amass is the strongest single competitor for attack surface mapping. It's narrower — DNS enumeration and network mapping rather than identity OSINT — but it's OWASP-governed, actively maintained, and goes deeper on its specialty than SpiderFoot does. recon-ng is the closest architectural cousin: a modular OSINT framework with a Metasploit-style console, better if you live in a terminal and want scriptable workflows. theHarvester is the quick-strike option — emails, subdomains, and hosts from public sources in one command, with none of SpiderFoot's depth or persistence. subfinder does exactly one thing, passive subdomain enumeration, extremely fast, and slots into the wider ProjectDiscovery toolchain. Photon is a fast OSINT crawler that extracts URLs, keys, and files from a target's web presence — a focused supplement rather than a replacement.
| Project | Repo | Hotness (~6mo) | PRs merged (~6mo) | Star growth (~6mo) |
|---|---|---|---|---|
| SpiderFoot | https://github.com/smicallef/spiderfoot | 4.37 | 0 | 3707 |
| Amass | https://github.com/owasp-amass/amass | metrics pending | 0 | metrics pending |
| recon-ng | https://github.com/lanmaster53/recon-ng | metrics pending | 0 | metrics pending |
| theHarvester | https://github.com/laramies/theHarvester | metrics pending | 0 | 249 |
| subfinder | https://github.com/projectdiscovery/subfinder | 17.18 | 0 | 2123 |
| Photon | https://github.com/s0md3v/Photon | metrics pending | 0 | metrics pending |
The best overall alternative is Amass: active maintenance under OWASP governance, strong documentation, and healthy adoption signals make it the safer long-term bet, even though you give up SpiderFoot's identity and breach coverage.
Conclusion
If you're a security engineer who needs broad OSINT on your own attack surface without buying a platform, install SpiderFoot — its module breadth is unmatched in open source, and you'll have a useful exposure map within an hour. Read the official documentation and configure your API keys before your first real scan.
Do not launch sf.py bound to 0.0.0.0 on any shared or internet-facing host: the web UI has no authentication, so anyone who can reach port 5001 can run scans against arbitrary targets and read your entire scan history. Do this first: run python3 sf.py -l 127.0.0.1:5001, confirm the UI answers only on localhost, and add an authenticating reverse proxy before you ever widen that bind address.