A professional-grade SIGINT forensics and analysis platform for wireless network data with advanced spatial correlation, real-time monitoring, and comprehensive surveillance detection.
🚀 Quick Start • 📖 Documentation • 🤝 Contributing • ⭐ Star this repo
ShadowCheck transforms raw wireless capture data into actionable intelligence through powerful spatial analysis and intuitive visualizations. Built for security researchers, forensic analysts, and SIGINT professionals who need enterprise-grade tools for wireless network investigation.
|
|
Real-time network visualization with clustering and signal range indicators
Comprehensive metrics, security analysis, and temporal patterns
Automated threat detection and behavioral analysis
Note: Add screenshots to
docs/images/directory for maximum visual impact!
Get ShadowCheck running in under 5 minutes:
# 1. Clone the repository
git clone https://github.com/cyclonite69/shadowcheck.git
cd shadowcheck
# 2. Configure environment
cp .env.example .env
# Edit .env with your secure passwords
# 3. Start all services
docker compose up --build
# 4. Access the platform
# Frontend: http://localhost:3001
# Grafana: http://localhost:3000
# API: http://localhost:5000⚡ That's it! Your SIGINT platform is now running with:
- PostgreSQL 18 + PostGIS spatial database
- Real-time monitoring (Prometheus + Grafana)
- Centralized logging (Loki + Promtail)
- Full-featured React frontend
- Why ShadowCheck?
- Architecture
- Installation
- Configuration
- Usage & API
- Data Pipelines
- Security
- Monitoring
- Contributing
- Roadmap
- License
┌─────────────────────────────────────────────────────────────┐
│ ShadowCheck Platform │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌───────────┐ │
│ │ React UI │─────▶│ Express API │─────▶│ PostgreSQL│ │
│ │ (Vite) │ │ TypeScript │ │ + PostGIS│ │
│ └──────────────┘ └──────────────┘ └───────────┘ │
│ │ │ │ │
│ │ ┌───────▼─────────┐ │ │
│ │ │ Data Pipelines │ │ │
│ │ ├─────────────────┤ │ │
│ │ │ • KML Import │ │ │
│ │ │ • Kismet CSV │ │ │
│ │ │ • WiGLE API │ │ │
│ │ └─────────────────┘ │ │
│ │ │ │
│ ┌──────▼─────────────────────────────────────────▼──────┐ │
│ │ Monitoring & Observability Stack │ │
│ ├────────────────────────────────────────────────────────┤ │
│ │ Prometheus │ Grafana │ Loki │ AlertManager │ │
│ └────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
| Component | Technology | Purpose |
|---|---|---|
| Frontend | React + TypeScript + Vite | Modern, responsive UI |
| Backend | Node.js + Express | RESTful API server |
| Database | PostgreSQL 18 + PostGIS | Spatial data storage |
| Mapping | Mapbox GL JS | Interactive visualizations |
| Monitoring | Prometheus + Grafana | Metrics and dashboards |
| Logging | Loki + Promtail | Centralized log aggregation |
| Container | Docker + Docker Compose | Deployment orchestration |
Requirements:
- Docker 20.10+
- Docker Compose 2.0+
- 4GB+ RAM
- 10GB+ disk space
Step-by-step:
- Clone and configure:
git clone https://github.com/cyclonite69/shadowcheck.git
cd shadowcheck
cp .env.example .env- Set secure passwords in
.env:
POSTGRES_PASSWORD=your_secure_postgres_password_here
GRAFANA_PASSWORD=your_secure_grafana_password_here
PGADMIN_PASSWORD=your_secure_pgadmin_password_here # Optional- Launch services:
# Core services
docker compose up -d
# With pgAdmin (database management UI)
docker compose --profile admin up -d- Verify deployment:
docker compose ps
docker compose logs -f backend- Access interfaces:
- Frontend: http://localhost:3001
- API Docs: http://localhost:5000/api/v1/health
- Grafana: http://localhost:3000 (admin / your_grafana_password)
- Prometheus: http://localhost:9091
- pgAdmin: http://localhost:8080 ([email protected] / your_pgadmin_password)
Click to expand manual installation guide
Prerequisites:
- Node.js 18+
- PostgreSQL 14+ with PostGIS
- npm/pnpm/yarn
Backend Setup:
cd server
npm install
cp .env.example .env
# Configure DATABASE_URL in .env
npm run devFrontend Setup:
cd client
npm install
cp .env.example .env
# Configure VITE_API_URL in .env
npm run devDatabase Setup:
-- Create user and database
CREATE USER shadowcheck_user WITH PASSWORD 'secure_password';
CREATE DATABASE shadowcheck OWNER shadowcheck_user;
-- Connect and enable PostGIS
\c shadowcheck
CREATE EXTENSION IF NOT EXISTS postgis;
CREATE EXTENSION IF NOT EXISTS postgis_topology;
CREATE EXTENSION IF NOT EXISTS pg_trgm;
-- Apply schema
\i schema/migration.sqlAll configuration is managed through environment files. Never commit .env to version control.
Required Variables:
# Database
POSTGRES_PASSWORD=strong_unique_password_32_chars_min
DATABASE_URL=postgresql://shadowcheck_user:${POSTGRES_PASSWORD}@localhost:5432/shadowcheck
# Monitoring
GRAFANA_PASSWORD=strong_unique_password_for_grafana
# Optional: Database Admin UI
PGADMIN_PASSWORD=strong_unique_password_for_pgadmin
# Optional: Mapbox (for enhanced mapping)
MAPBOX_TOKEN=your_mapbox_public_token_here- ✅ Use 32+ character passwords with mixed case, numbers, and symbols
- ✅ Different passwords for each service
- ✅ Use secrets manager in production (AWS Secrets, Vault, etc.)
- ✅ Enable TLS/HTTPS for all external connections
- ✅ Restrict database access to localhost or VPN
- ✅ Regular security updates and dependency scanning
ShadowCheck exposes a comprehensive RESTful API:
GET /api/v1/health # Health check
GET /api/v1/status # Detailed system status
GET /api/v1/version # API version info
GET /api/v1/metrics # System metricsGET /api/v1/networks # List all networks (paginated)
GET /api/v1/within # Spatial radius search
GET /api/v1/visualize # GeoJSON for mappingGET /api/v1/analytics # Comprehensive overview
GET /api/v1/security-analysis # Security breakdown
GET /api/v1/signal-strength # Signal distribution
GET /api/v1/radio-stats # Multi-radio statistics
GET /api/v1/timeline # Temporal patternsGET /api/v1/surveillance/stats # Detection statistics
GET /api/v1/surveillance/wifi/threats # WiFi threat analysis
GET /api/v1/surveillance/location-visits # Location patterns
GET /api/v1/surveillance/home-following # Following detectionGET /api/v1/access-points # Detailed AP list
GET /api/v1/access-points/:mac # Single AP details
GET /api/v1/access-points/:mac/observations # AP timelineGET /api/v1/wigle/stats # Enrichment statistics
GET /api/v1/wigle/queue # Pending enrichments
POST /api/v1/wigle/tag # Tag networks for enrichment
POST /api/v1/wigle/enrich # Trigger enrichment# Spatial query: networks within 500m radius
curl "http://localhost:5000/api/v1/within?lat=43.0234&lon=-83.6968&radius=500"
# Security analysis with filters
curl "http://localhost:5000/api/v1/networks?radio_types=W&min_signal=-70&security_types=Open"
# Surveillance threat detection
curl "http://localhost:5000/api/v1/surveillance/wifi/threats"ShadowCheck supports multiple ingestion formats:
# Place KML files in pipelines/kml/
# Import via admin UI or API
curl -X POST http://localhost:5000/api/v1/pipelines/kml/import \
-H "Content-Type: application/json" \
-d '{"filename": "my_networks.kml"}'# Place Kismet CSV exports in pipelines/kismet/
# Automatic detection and import# Place WiGLE CSV files in pipelines/wigle/
# Import with geolocation enrichment- Kismet remote capture
- Real-time MQTT ingestion
- Kafka stream processing
ShadowCheck handles sensitive SIGINT data. Security is paramount:
- Role-based access control (RBAC) on all API endpoints
- JWT authentication with refresh tokens
- Rate limiting and request throttling
- IP whitelisting for admin functions
- Encrypted connections (TLS 1.3+)
- Encrypted database backups
- PII scrubbing and anonymization options
- Audit logging for all data access
- GDPR-compliant data retention policies
- Configurable data retention periods
- Export and deletion capabilities
- Comprehensive audit trails
- Always use HTTPS in production
- Enable database encryption at rest
- Regular security audits and penetration testing
- Keep dependencies updated
- Monitor for suspicious activity
ShadowCheck includes enterprise-grade monitoring:
- System Overview: Resource usage, uptime, API performance
- Database Metrics: Connection pool, query performance, storage
- Application Metrics: Request rates, error rates, latency
- Security Dashboard: Failed auth, suspicious patterns
- Custom application metrics
- Database connection pool monitoring
- API endpoint performance tracking
- Real-time alerting
- Centralized logging from all containers
- Log correlation and search
- Alert triggers on log patterns
- Long-term log retention
Access monitoring:
- Grafana: http://localhost:3000 (username: admin)
- Prometheus: http://localhost:9091
- View logs:
docker compose logs -f [service]
We welcome contributions from the community! Here's how to get started:
- 🐛 Report bugs and issues
- 💡 Suggest new features
- 📝 Improve documentation
- 🧪 Add tests and improve coverage
- 🔧 Submit bug fixes
- ✨ Implement new features
- Fork the repository
- Clone your fork:
git clone https://github.com/YOUR_USERNAME/shadowcheck.git - Create a feature branch:
git checkout -b feat/amazing-feature - Make your changes with tests
- Commit with clear messages:
git commit -m "feat: Add amazing feature" - Push to your fork:
git push origin feat/amazing-feature - Open a Pull Request with detailed description
- Follow existing code formatting
- Use TypeScript for type safety
- Write meaningful commit messages (conventional commits)
- Add tests for new features
- Update documentation as needed
See CONTRIBUTING.md for detailed guidelines.
- ✅ PostgreSQL + PostGIS spatial database
- ✅ Interactive Mapbox visualizations
- ✅ Multi-source data ingestion (KML, CSV)
- ✅ WiGLE API enrichment
- ✅ Surveillance pattern detection
- ✅ Comprehensive monitoring stack
- 🔄 Real-time streaming ingestion (Kafka)
- 🔄 Advanced ML-based threat detection
- 🔄 Multi-tenancy support
- 🔄 Mobile app (React Native)
- 📋 Vector tile support for massive datasets
- 📋 3D visualization with Cesium
- 📋 MISP integration for threat intelligence
- 📋 Automated PDF report generation
- 📋 OAuth2/SAML authentication
- 📋 Elasticsearch integration
- 📋 Graph database correlation (Neo4j)
- 📋 AI-powered anomaly detection
- Signal triangulation algorithms
- RF spectrum analysis integration
- Hardware device integration (SDR)
- Blockchain-based evidence chain
Have ideas? Open an issue or start a Discussion!
This project is licensed under the MIT License - see the LICENSE file for details.
- ✅ Use for personal and commercial projects
- ✅ Modify and distribute
- ✅ Private use
- ❌ No liability or warranty
ShadowCheck is built on the shoulders of giants:
- PostgreSQL - World's most advanced open source database
- PostGIS - Spatial database extender for PostgreSQL
- React - JavaScript library for building user interfaces
- TypeScript - JavaScript with syntax for types
- Vite - Next generation frontend tooling
- Mapbox GL JS - Interactive maps library
- Express - Fast, unopinionated web framework
- Docker - Containerization platform
- Prometheus - Systems monitoring and alerting
- Grafana - Analytics and monitoring platform
- Loki - Log aggregation system
- WiGLE.net for wireless network database API
- Kismet project for wireless IDS inspiration
- The open-source security research community
- 📖 Documentation
- 💬 Discussions
- 🐛 Issue Tracker
- 📧 Contact: @cyclonite69
- ⭐ Star this repo to show support
- 👁️ Watch for updates
- 🍴 Fork to contribute
Built for security researchers, by security researchers.