Skip to content

Commit 7716dae

Browse files
svngokusvngoku
authored
Merge pull request #2 from svngoku/add-docker-support
Add Docker support and Streamlit web interface
2 parents 80b2b11 + b99b80c commit 7716dae

File tree

5 files changed

+264
-11
lines changed

5 files changed

+264
-11
lines changed

‎Dockerfile‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
FROM python:3.10-slim
2+
3+
WORKDIR /app
4+
5+
# Copy requirements first for better caching
6+
COPY requirements.txt .
7+
8+
# Install dependencies
9+
RUN pip install --no-cache-dir -r requirements.txt
10+
11+
# Copy the rest of the application
12+
COPY . .
13+
14+
# Create necessary directories
15+
RUN mkdir -p logs output
16+
17+
# Expose the Streamlit port
18+
EXPOSE 8501
19+
20+
# Set environment variables
21+
ENV PYTHONUNBUFFERED=1
22+
23+
# Copy .env file into the container
24+
COPY .env .env
25+
26+
# Command to run the Streamlit app
27+
CMD ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

‎README.md‎

Lines changed: 39 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,29 @@ This is a tutorial project of [Pocket Flow](https://github.com/The-Pocket/Pocket
6363

6464
## 🚀 Getting Started
6565

66+
### Option 1: Using Docker (Recommended)
67+
68+
1. Clone this repository
69+
70+
2. Configure your environment variables in the `.env` file:
71+
```bash
72+
# Copy the sample .env file
73+
cp .env.sample .env
74+
75+
# Edit the .env file with your credentials
76+
# GEMINI_PROJECT_ID=your-project-id
77+
# GITHUB_TOKEN=your-github-token
78+
```
79+
80+
3. Run the application using Docker Compose:
81+
```bash
82+
docker-compose up -d
83+
```
84+
85+
4. Access the Streamlit web interface at http://localhost:8501
86+
87+
### Option 2: Manual Installation
88+
6689
1. Clone this repository
6790

6891
2. Install dependencies:
@@ -75,17 +98,22 @@ This is a tutorial project of [Pocket Flow](https://github.com/The-Pocket/Pocket
7598
python utils/call_llm.py
7699
```
77100

78-
4. Generate a complete codebase tutorial by running the main script:
79-
```bash
80-
python main.py https://github.com/username/repo --include "*.py" "*.js" --exclude "tests/*" --max-size 50000
81-
```
82-
- `repo_url` - URL of the GitHub repository (required)
83-
- `-n, --name` - Project name (optional, derived from URL if omitted)
84-
- `-t, --token` - GitHub token (or set GITHUB_TOKEN environment variable)
85-
- `-o, --output` - Output directory (default: ./output)
86-
- `-i, --include` - Files to include (e.g., "*.py" "*.js")
87-
- `-e, --exclude` - Files to exclude (e.g., "tests/*" "docs/*")
88-
- `-s, --max-size` - Maximum file size in bytes (default: 100KB)
101+
4. Run the Streamlit web interface:
102+
```bash
103+
streamlit run app.py
104+
```
105+
106+
Or generate a complete codebase tutorial directly using the command line:
107+
```bash
108+
python main.py https://github.com/username/repo --include "*.py" "*.js" --exclude "tests/*" --max-size 50000
109+
```
110+
- `repo_url` - URL of the GitHub repository (required)
111+
- `-n, --name` - Project name (optional, derived from URL if omitted)
112+
- `-t, --token` - GitHub token (or set GITHUB_TOKEN environment variable)
113+
- `-o, --output` - Output directory (default: ./output)
114+
- `-i, --include` - Files to include (e.g., "*.py" "*.js")
115+
- `-e, --exclude` - Files to exclude (e.g., "tests/*" "docs/*")
116+
- `-s, --max-size` - Maximum file size in bytes (default: 100KB)
89117

90118
The application will crawl the repository, analyze the codebase structure, generate tutorial content, and save the output in the specified directory (default: ./output).
91119

‎app.py‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
import streamlit as st
2+
import os
3+
import dotenv
4+
from flow import create_tutorial_flow
5+
6+
# Load environment variables
7+
dotenv.load_dotenv()
8+
9+
# Default file patterns
10+
DEFAULT_INCLUDE_PATTERNS = {
11+
"*.py", "*.js", "*.ts", "*.go", "*.java", "*.pyi", "*.pyx",
12+
"*.c", "*.cc", "*.cpp", "*.h", "*.md", "*.rst", "Dockerfile",
13+
"Makefile", "*.yaml", "*.yml"
14+
}
15+
16+
DEFAULT_EXCLUDE_PATTERNS = {
17+
"*test*", "tests/*", "docs/*", "examples/*", "v1/*",
18+
"dist/*", "build/*", "experimental/*", "deprecated/*",
19+
"legacy/*", ".git/*", ".github/*"
20+
}
21+
22+
# Set page config
23+
st.set_page_config(
24+
page_title="Codebase Tutorial Generator",
25+
page_icon="📚",
26+
layout="wide"
27+
)
28+
29+
# Title and description
30+
st.title("📚 Codebase Tutorial Generator")
31+
st.markdown("""
32+
This app generates comprehensive tutorials for GitHub codebases using AI.
33+
Simply provide a GitHub repository URL and customize the generation settings.
34+
""")
35+
36+
# Sidebar for configuration
37+
with st.sidebar:
38+
st.header("Configuration")
39+
40+
# GitHub token input
41+
github_token = st.text_input(
42+
"GitHub Token (optional)",
43+
value=os.environ.get("GITHUB_TOKEN", ""),
44+
type="password",
45+
help="Personal access token for GitHub API. Helps avoid rate limits."
46+
)
47+
48+
# Output directory
49+
output_dir = st.text_input(
50+
"Output Directory",
51+
value="output",
52+
help="Directory where the tutorial will be saved"
53+
)
54+
55+
# Advanced options
56+
with st.expander("Advanced Options"):
57+
# File size limit
58+
max_file_size = st.number_input(
59+
"Max File Size (bytes)",
60+
value=100000,
61+
min_value=1000,
62+
help="Maximum file size to process (in bytes)"
63+
)
64+
65+
# Include patterns
66+
include_patterns_str = st.text_area(
67+
"Include Patterns",
68+
value="\n".join(DEFAULT_INCLUDE_PATTERNS),
69+
help="File patterns to include (one per line)"
70+
)
71+
72+
# Exclude patterns
73+
exclude_patterns_str = st.text_area(
74+
"Exclude Patterns",
75+
value="\n".join(DEFAULT_EXCLUDE_PATTERNS),
76+
help="File patterns to exclude (one per line)"
77+
)
78+
79+
# Main form
80+
with st.form("tutorial_form"):
81+
# Repository URL
82+
repo_url = st.text_input(
83+
"GitHub Repository URL",
84+
placeholder="https://github.com/username/repository",
85+
help="URL of the public GitHub repository"
86+
)
87+
88+
# Project name (optional)
89+
project_name = st.text_input(
90+
"Project Name (optional)",
91+
help="Custom name for the project (derived from URL if omitted)"
92+
)
93+
94+
# Submit button
95+
submit_button = st.form_submit_button("Generate Tutorial")
96+
97+
# Process form submission
98+
if submit_button:
99+
if not repo_url:
100+
st.error("Please enter a GitHub repository URL")
101+
else:
102+
# Show progress
103+
progress_bar = st.progress(0)
104+
status_text = st.empty()
105+
106+
# Parse include/exclude patterns
107+
include_patterns = set(filter(None, include_patterns_str.split("\n")))
108+
exclude_patterns = set(filter(None, exclude_patterns_str.split("\n")))
109+
110+
# Initialize shared dictionary
111+
shared = {
112+
"repo_url": repo_url,
113+
"project_name": project_name if project_name else None,
114+
"github_token": github_token if github_token else os.environ.get("GITHUB_TOKEN"),
115+
"output_dir": output_dir,
116+
"include_patterns": include_patterns,
117+
"exclude_patterns": exclude_patterns,
118+
"max_file_size": max_file_size,
119+
"files": [],
120+
"abstractions": [],
121+
"relationships": {},
122+
"chapter_order": [],
123+
"chapters": [],
124+
"final_output_dir": None
125+
}
126+
127+
try:
128+
# Create and run the flow
129+
status_text.text("Starting tutorial generation...")
130+
progress_bar.progress(10)
131+
132+
tutorial_flow = create_tutorial_flow()
133+
134+
# Update status for each node
135+
status_text.text("Fetching repository...")
136+
progress_bar.progress(20)
137+
138+
# Run the flow with progress updates
139+
# Note: In a real implementation, you would need to modify the flow
140+
# to provide progress updates or use callbacks
141+
result = tutorial_flow.run(shared)
142+
143+
progress_bar.progress(100)
144+
status_text.text("Tutorial generation complete!")
145+
146+
# Display result
147+
if result.get("final_output_dir"):
148+
st.success(f"Tutorial generated successfully in: {result['final_output_dir']}")
149+
150+
# Provide download links if files are accessible
151+
st.markdown("### Download Tutorial Files")
152+
for file in os.listdir(result["final_output_dir"]):
153+
file_path = os.path.join(result["final_output_dir"], file)
154+
if os.path.isfile(file_path):
155+
with open(file_path, "rb") as f:
156+
st.download_button(
157+
label=f"Download {file}",
158+
data=f,
159+
file_name=file,
160+
mime="text/markdown"
161+
)
162+
else:
163+
st.warning("Tutorial generation completed but output directory not found.")
164+
165+
except Exception as e:
166+
st.error(f"Error generating tutorial: {str(e)}")
167+
st.exception(e)
168+
169+
# Display information about the app
170+
st.markdown("---")
171+
st.markdown("""
172+
### How it works
173+
1. The app clones the GitHub repository
174+
2. It analyzes the codebase structure and identifies key abstractions
175+
3. It determines relationships between components
176+
4. It generates tutorial chapters in a logical order
177+
5. Finally, it combines everything into a comprehensive tutorial
178+
179+
### Requirements
180+
- A public GitHub repository
181+
- Google Gemini API access (configured via environment variables)
182+
""")

‎docker-compose.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
version: '3'
2+
3+
services:
4+
tutorial-generator:
5+
build:
6+
context: .
7+
dockerfile: Dockerfile
8+
ports:
9+
- "8501:8501"
10+
volumes:
11+
- ./output:/app/output
12+
- ./logs:/app/logs
13+
env_file:
14+
- .env
15+
restart: unless-stopped

‎requirements.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,3 +5,4 @@ gitpython>=3.1.0
55
google-cloud-aiplatform>=1.25.0
66
google-genai>=1.9.0
77
python-dotenv>=1.0.0
8+
streamlit>=1.32.0

0 commit comments

Comments
 (0)