Armin Tavassoli - SE333 Final Project
Testing Agent with Decision Table Test Generation and Security Scanning
For my SE333 final project, I built an MCP (Model Context Protocol) server that helps automate testing workflows for Java Maven projects. The main features are decision table-based test generation and security vulnerability scanning, along with coverage analysis and Git automation tools.
What This Does
The agent provides a few key capabilities:
- Decision Table-Based Test Generation: Generates JUnit test cases by analyzing method logic and creating test matrices that cover different decision paths
- Security Vulnerability Scanning: Scans Java code for common security issues like SQL injection, XSS, command injection, etc.
- Coverage Analysis: Finds code that's missing test coverage and suggests what to test
- Git Automation: Handles common Git workflows like staging, committing, and creating PRs
Features
Coverage Tools
find_jacoco_path: Finds where the JaCoCo coverage reports are located (HTML, XML, or exec files)missing_coverage: Looks through JaCoCo XML reports to find uncovered code and shows coverage stats
Test Generation
generate_decision_table_tests: Creates JUnit tests using decision tables. It:- Looks at method signatures and logic
- Finds decision points (if/else, switch statements, ternary operators)
- Builds a test matrix that covers:
- Null inputs
- Empty inputs
- Valid inputs
- Boundary conditions
- Edge cases
- Exception scenarios
Security Scanning
scan_security_vulnerabilities: Scans Java source code for:- SQL Injection risks
- Command Injection vulnerabilities
- Path Traversal issues
- Hardcoded secrets/passwords
- Insecure random number generation
- Deserialization risks
- XSS vulnerabilities
- Each finding is classified by severity (high, medium, low)
Git Tools
git_status: Shows what's staged, unstaged, and untrackedgit_add_all: Stages everything (skips build artifacts)git_commit: Creates commits with messages that include coverage statsgit_push: Pushes to the remote repogit_pull_request: Creates PRs (needs GitHub CLI or you can do it manually)
Setup
What You Need
- Python 3.12 or newer
- Node.js 18+ (LTS version works best)
- VS Code with Chat view
- Java 11+ and Maven 3.6+
- Git and a GitHub account
- uv package manager (install here)
Installation Steps
-
Install uv
# macOS/Linux curl -LsSf https://astral.sh/uv/install.sh | sh # Windows powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" -
Set up Python environment
cd Armin_Tavassoli_SE333_Final_project uv init uv venv source .venv/bin/activate # On Windows: .venv\Scripts\activate -
Install dependencies
uv sync # Or if that doesn't work: uv add mcp fastmcp httpx pypdf python-dotenv -
Connect VS Code to the MCP server
- Start the server in HTTP mode:
The server will start onpython server.pyhttp://localhost:8001/sse(or port 8000 if MCP_PORT is not set) - Note: VS Code's HTTP MCP client has known compatibility issues with FastMCP's SSE transport. For VS Code, you can either:
- Option A (Recommended for VS Code): Use stdio mode by setting
MCP_USE_STDIO=trueand configuring VS Code to use local process (see.vscode/settings.json) - Option B: Use HTTP mode - the server runs on HTTP as required, but VS Code may have connection issues. HTTP mode works well with other MCP clients or for testing with curl.
- Option A (Recommended for VS Code): Use stdio mode by setting
- In VS Code, press
CTRL+SHIFT+P(orCMD+SHIFT+Pon Mac) and search for "MCP: Add Server" - Enter the server URL:
http://localhost:8001(orhttp://localhost:8000if using default port) - Name it something like "SE333 Testing Agent"
- Make sure the tools show up in the Chat view
- Start the server in HTTP mode:
-
Enable Auto-Approve
- Press
CTRL+SHIFT+Pand search for "Chat: Settings" - Turn on Auto-Approve
- Check that all tools are highlighted
- Press
-
Set up the Maven project
cd codebase mvn clean install mvn test jacoco:report
How to Use
Basic Workflow
-
Check coverage
- Use
find_jacoco_pathto find the reports - Use
missing_coverageto see what's not covered
- Use
-
Generate tests
- Use
generate_decision_table_testswith a class and method name - Example:
generate_decision_table_tests(class_name="org.apache.commons.lang3.StringUtils", method_name="isEmpty")
- Use
-
Scan for security issues
- Use
scan_security_vulnerabilitiesto check the codebase - Start with high-severity issues
- Use
-
Git workflow
git_status→git_add_all→git_commit→git_push→git_pull_request
Example: Generating Tests
# In VS Code Chat:
generate_decision_table_tests(
class_name="org.apache.commons.lang3.StringUtils",
method_name="isEmpty",
input_parameters='{"str": "String"}'
)
This will:
- Look at the method signature and code
- Find decision points in the logic
- Generate test cases
- Give you JUnit test code you can save
Example: Security Scan
scan_security_vulnerabilities(
class_name="org.apache.commons.lang3.StringUtils",
severity="high"
)
You'll get JSON back with:
- What vulnerabilities were found
- Where they are (file and line)
- How severe they are
- Code snippets
- How to fix them
Project Structure
Armin_Tavassoli_SE333_Final_project/
├── codebase/ # Apache Commons Lang3 Maven project
│ ├── pom.xml # Maven config with JaCoCo
│ ├── src/
│ │ ├── main/java/ # Source code
│ │ └── test/java/ # Test code
│ └── target/
│ └── site/jacoco/ # Coverage reports
├── .github/
│ └── prompts/
│ └── tester.prompt.md # Agent prompt config
├── server.py # MCP server code
├── pyproject.toml # Python dependencies
├── README.md # This file
├── demo/ # Demo materials
│ └── final_presentation.mp4 # Video (if applicable)
├── report/ # Written report
│ └── reflection.pdf # LaTeX reflection report
└── docs/ # Extra docs
Tool Reference
find_jacoco_path
Finds where the JaCoCo coverage reports are in the Maven project.
Inputs: None
Returns: Path to the report file, or an error message if reports haven't been generated yet
Example:
find_jacoco_path()
missing_coverage
Looks at JaCoCo XML reports to find code that isn't covered by tests.
Inputs:
class_name(optional): Specific class to check
Returns: JSON with:
- Coverage summary (lines, branches, methods, classes)
- List of classes with low coverage
- Suggestions for what to test
Example:
missing_coverage(class_name="org.apache.commons.lang3.StringUtils")
generate_decision_table_tests
Generates JUnit test cases using decision tables.
Inputs:
class_name(required): Full class name likeorg.apache.commons.lang3.StringUtilsmethod_name(required): Method to testinput_parameters(optional): JSON describing the parameters
Returns: JUnit test code as a string
Example:
generate_decision_table_tests(
class_name="org.apache.commons.lang3.StringUtils",
method_name="isEmpty"
)
scan_security_vulnerabilities
Scans Java code for security issues.
Inputs:
class_name(optional): Specific class to scanseverity(optional): Filter by "all", "high", "medium", or "low"
Returns: JSON with:
- Total count of vulnerabilities
- Breakdown by severity
- Details for each one:
- File and line number
- Type of vulnerability
- Severity
- Description
- Code snippet
- How to fix it
Example:
scan_security_vulnerabilities(severity="high")
Git Tools
git_status
Shows what files are staged, unstaged, or untracked.
git_add_all
Stages all changes (build artifacts are excluded).
git_commit(message)
Creates a commit. The message will include coverage stats if available.
git_push(remote, branch)
Pushes commits to the remote repo.
git_pull_request(base, title, body)
Creates a pull request. You'll need GitHub CLI installed, or you can create it manually.
Troubleshooting
Server Won't Start
If python server.py fails:
- Check Python version:
python --version(needs 3.12+) - Make sure dependencies are installed:
uv sync - Check that the virtual environment is activated
- See if port 8000 is already in use
Can't Find JaCoCo Reports
If find_jacoco_path says reports aren't found:
- Run the tests first:
cd codebase && mvn clean test - Generate the reports:
mvn jacoco:report - Check that the directory exists:
ls codebase/target/site/jacoco/
VS Code Won't Connect
If tools don't show up in Chat:
- Make sure the server is actually running (check the terminal)
- Double-check the server URL in VS Code settings
- Try restarting VS Code
- Make sure Auto-Approve is enabled
- Check the server logs for any errors
Test Generation Fails
If generate_decision_table_tests gives an error:
- Make sure the class name is fully qualified (e.g.,
org.apache.commons.lang3.StringUtils) - Check that the method name is spelled correctly
- Verify the source file exists in
codebase/src/main/java/ - Make sure the file is readable
Security Scan Finds Nothing
If scan_security_vulnerabilities returns empty results:
- This might be fine if the code is actually secure
- Try scanning a specific class:
scan_security_vulnerabilities(class_name="...") - Try
severity="all"to see everything - Make sure Java files are in
codebase/src/main/java/
Git Commands Fail
If Git tools return errors:
- Make sure Git is initialized:
git status - Check that your Git credentials are set up
- For
git_pull_request: Install GitHub CLI (gh) or create the PR manually - Make sure you have write access to the repository
Development Notes
Adding New Tools
If you want to add a new MCP tool:
- Create a function with the
@mcp.tool()decorator - Write a good docstring
- Add the tool name to the list in
tester.prompt.md - Update this README
Testing
To test the agent:
- Start the server:
python server.py - In VS Code Chat, try each tool one at a time
- Check that the outputs look right
- Try some invalid inputs to test error handling
Tracking Results
Coverage Metrics
To track coverage improvements, use JaCoCo reports. After running mvn test jacoco:report, you can:
- View HTML report: Open
codebase/target/site/jacoco/index.htmlin a browser - Use the MCP tools:
find_jacoco_pathto locate reportsmissing_coverageto get detailed statistics
The reports show:
- Line coverage percentage
- Branch coverage percentage
- Method coverage percentage
- Class coverage percentage
For Presentation: Compare before/after coverage by:
- Running initial coverage:
cd codebase && mvn clean test jacoco:report - Generating tests using the agent
- Running coverage again:
mvn test jacoco:report - Comparing the metrics
Security Metrics
Track security improvements:
- Total vulnerabilities found (use
scan_security_vulnerabilities) - Breakdown by severity (high/medium/low)
- Remediation progress
- Files affected
Git Commit History
If you use the Git automation tools, you can track improvements through commit history:
- Each commit includes coverage statistics
- Review commit messages to see progress over time
- Use
git logto see the improvement timeline
Future Ideas
Some things I'd like to add later:
- Integration with mutation testing (PIT)
- Support for other languages besides Java
- Better test case prioritization
- Automated test refactoring
- CI/CD integration
- Real-time coverage monitoring
- More advanced security pattern detection
License
This is my SE333 coursework project at DePaul University.
The codebase (Apache Commons Lang3) uses the Apache License 2.0.
Contact
Student: Armin Tavassoli
Course: SE333 - Software Agents
Institution: DePaul University