RepoSummarizer

# RepoSummarizer > An AI-powered tool that automatically generates comprehensive documentation, summaries, and visual overviews for any public GitHub repository. ## Table of Contents - [Overview](#overview) - [Key Features](#key-features) - [Installation](#installation) - [Quick Start](#quick-start) - [Usage](#usage) - [Architecture](#architecture) - [Examples](#examples) - [Contributing](#contributing) - [License](#license) ## Overview Understanding an unfamiliar codebase is often time-consuming and frustrating. Documentation may be outdated, incomplete, or missing entirely. Traditional tools typically summarize individual files without providing the broader architectural context developers need. **RepoSummarizer** solves this problem by combining static analysis with AI-powered insights to deliver a holistic understanding of any public GitHub repository, as if a knowledgeable teammate were walking you through the code. ### The Problem We Solve - **Onboarding Delays**: New team members spend days or weeks understanding legacy codebases - **Incomplete Documentation**: Existing docs often lag behind actual code changes - **Context Gaps**: File-level tools miss architectural patterns and component relationships - **Evaluation Friction**: Assessing third-party libraries requires deep code diving ### Our Solution RepoSummarizer analyzes repository structure, source code, commit history, and metadata to generate: - High-level project summaries - Module and file-level documentation - Visual architecture diagrams - Interactive HTML reports - Contribution guidelines based on detected patterns ## Key Features ### Intelligent Repository Analysis - Automatically detects project structure, languages, and frameworks - Identifies entry points, configuration files, and key components - Analyzes commit history and contributor activity patterns - Triages files intelligently to focus on important code ### AI-Generated Documentation - High-level project purpose and architecture overview - Module-by-module descriptions with dependency mapping - File-level summaries with key functions and classes - Auto-generated "Getting Started" guides based on conventions - Identifies extension points and contribution opportunities ### Visual Representations - Interactive directory structure diagrams (Mermaid) - Module dependency graphs - Commit activity timelines - Language distribution charts - Component relationship visualizations ### Multiple Output Formats - **Markdown**: Clean documentation for READMEs and wikis - **HTML**: Interactive reports with embedded visualizations - **JSON**: Structured data for programmatic integration - **Diagrams**: Exportable Mermaid or SVG charts ### Performance Optimization - Intelligent file triage to reduce processing time - GPU-accelerated local LLM inference - Caching layer for repeated analysis - Batched MAP-then-REDUCE processing for large repositories ## Installation ### Prerequisites - **Python 3.10+** ([Download](https://www.python.org/downloads/)) - **Git** installed and available in PATH - **8GB+ RAM** recommended for local LLM inference - **GPU** (optional) for faster AI processing ### Setup 1. **Clone the repository** ```bash git clone https://github.com/sstraccialini/repo-summarizer.git cd repo-summarizer ``` 2. **Create a virtual environment** **Windows (PowerShell):** ```powershell python -m venv .venv .\.venv\Scripts\Activate.ps1 ``` **macOS/Linux:** ```bash python -m venv .venv source .venv/bin/activate ``` 3. **Install dependencies** ```bash pip install --upgrade pip pip install -r requirements.txt ``` 4. **Download an LLM model** (optional, will auto-download if not present) ```bash # Model will be automatically downloaded from Hugging Face on first run # Or manually place a GGUF model in ./models/ ``` For detailed installation instructions and troubleshooting, see [INSTALL.md](INSTALL.md). ## Quick Start Analyze any public GitHub repository with a single command: ```bash python main.py https://github.com/username/repo.git ``` The tool will: 1. Clone the repository 2. Scan and analyze the codebase 3. Triage important files 4. Generate AI-powered summaries 5. Create visualizations 6. Output reports to `SUMMARY.html` and `SUMMARY.md` ### Example Output After processing, you'll find: - `SUMMARY.html` - Interactive web report with all visualizations - `SUMMARY.md` - Markdown documentation Open `SUMMARY.html` in your browser for the full interactive experience! ## Usage ### Basic Usage ```bash python main.py ``` ### Advanced Options ```bash python main.py \ --output custom_output.md \ --model-path ./models/custom-model.gguf \ --max-tokens 4096 ``` ### Some Available Options | Option | Description | Default | |--------|-------------|---------| | `--output` | Output Markdown file path | `SUMMARY.md` | | `--html-output` | Output HTML file path | `SUMMARY.html` | | `--model-path` | Path to GGUF model file | Auto-detect | | `--max-tokens` | Maximum tokens per summary | Auto-detect | | `--no-cache` | Disable summary caching | `false` | | `--max-depth` | Limit directory scan depth | Unlimited | | `--ignore-folders` | Folders to ignore | None | Run `python main.py --help` for the full list of options. ### Interactive Model Selection If multiple models are detected, you'll be prompted to choose one. You may download your preferred model from [Hugging Face](https://huggingface.co/models?search=gguf) and place it in the `./models/` directory, or run the tool to auto-download a default model. ## Architecture ``` repo-summarizer/ ├── src/ │ ├── core/ # Core configuration and data models │ │ ├── config.py # SummarizerConfig, ModelConfig │ │ ├── models.py # Data structures (FileInfo, RepoInfo) │ │ └── paths.py # Path constants │ ├── llm/ # LLM model handling │ │ ├── model_client.py # LlamaClient wrapper │ │ ├── model_discovery.py # Model detection │ │ ├── model_selector.py # Interactive selection │ │ └── param_optimizer.py # GPU optimization │ ├── triage/ # File selection and analysis │ │ ├── scanner.py # Repository scanning │ │ ├── triage.py # File prioritization │ │ └── schema.py # Metadata schemas │ ├── summarization/ # AI summarization pipeline │ │ ├── prompts.py # LLM prompt templates │ │ ├── batching.py # MAP-then-REDUCE processing │ │ └── doc_builder.py # Documentation generation │ ├── rendering/ # Output generation │ │ ├── visualizer.py # Mermaid diagram generation │ │ ├── html_report.py # HTML report rendering │ │ └── artifacts.py # Artifact management │ └── utils/ # Utility functions │ ├── git_utils.py # Git operations │ ├── cache.py # SQLite caching │ └── formatting.py # CLI output ├── tests/ # Test suite ├── main.py # Entry point └── requirements.txt # Dependencies ``` ### Key Components - **Triage Engine**: Intelligently selects files for analysis using heuristics and optional LLM filtering - **LLM Client**: Abstraction layer supporting local GGUF models with GPU acceleration - **Batch Processor**: MAP-then-REDUCE pattern for efficient large repository processing - **Cache Layer**: SQLite-based caching to avoid redundant processing - **Visualizer**: Generates Mermaid diagrams and interactive HTML reports ## Examples Some examples can be found in the [examples/](examples/) folder. ## Contributing ### Quick Contribution Guide 1. Fork the repository 2. Create a feature branch: `git checkout -b feature/new-feature` 3. Commit your changes: `git commit -m 'Add new feature'` 4. Push to the branch: `git push origin feature/new-feature` 5. Open a Pull Request ### Development Setup ```bash # Install development dependencies pip install -r requirements.txt # Run tests pytest tests/ ``` ## Acknowledgments Built with: - [llama-cpp-python](https://github.com/abetlen/llama-cpp-python) - Local LLM inference - [GitPython](https://github.com/gitpython-developers/GitPython) - Git integration - [Mermaid](https://mermaid.js.org/) - Diagram generation ## License This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.