startup-starter/skybridge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cleanup

Browse Source
This commit is contained in:
Ryan Copley
2025-08-22 14:14:22 -04:00
parent 46264cb556
commit 98a299e7b2
4 changed files with 194 additions and 511 deletions
Download Patch File Download Diff File Expand all files Collapse all files

116
FAILING_TESTS_PROMPT.md
View File

@ -1,116 +0,0 @@
# KMS API - Failing Tests Analysis & Resolution Prompt
## Context
I have a Go-based Key Management Service (KMS) API with comprehensive end-to-end tests. The server is running successfully on port 8081, but several tests are failing due to database connectivity issues and some implementation gaps. I need help resolving these failing tests.
## Current Test Results
**Total Tests**: 22
**Passing**: 17
**Failing**: 6
## Failing Tests Analysis
### 1. Database Connectivity Issues (Primary Problem)
```
[FAIL] Readiness Check (Expected: 200, Got: 503)
Response: {"status":"not ready","timestamp":"2025-08-22T17:17:03Z","checks":{"database":"unhealthy: sql: database is closed"}}
[FAIL] List applications with auth (Expected: 200, Got: 500)
Response: {"error":"Internal Server Error","message":"Failed to list applications"}
[FAIL] List applications with pagination (Expected: 200, Got: 500)
Response: {"error":"Internal Server Error","message":"Failed to list applications"}
[FAIL] Create application (Expected: 201, Got: 500)
Response: {"error":"Internal Server Error","message":"Failed to create application"}
[FAIL] Missing content-type (Expected: 400, Got: 500)
Response: {"error":"Internal Server Error","message":"Failed to create application"}
```
**Root Cause**: The server logs show `"sql: database is closed"` errors. The PostgreSQL database is not running or not properly connected.
### 2. Edge Case Handling
```
[FAIL] Delete non-existent token (Expected: 500, Got: 204)
```
**Issue**: The delete token endpoint returns 204 (success) even for non-existent tokens, but the test expects 500 (error).
## What Needs to be Completed
### Priority 1: Database Setup & Connection
1. **Start PostgreSQL Database**
- Either start a local PostgreSQL instance
- Or set up Docker container with PostgreSQL
- Or configure the application to use an in-memory/mock database for testing
2. **Database Configuration**
- Verify database connection string in configuration
- Ensure database migrations run successfully
- Check that database tables are created properly
3. **Connection Pool Management**
- Fix the "database is closed" error
- Ensure proper database connection lifecycle management
- Verify connection pool settings
### Priority 2: Application Service Implementation
The application CRUD operations are failing, which suggests:
1. **Repository Layer**: Verify PostgreSQL repository implementations work correctly
2. **Service Layer**: Ensure application service properly handles database operations
3. **Error Handling**: Improve error responses for database connectivity issues
### Priority 3: Token Service Edge Cases
1. **Delete Token Validation**: The delete token endpoint should return appropriate error codes for non-existent tokens
2. **Error Response Consistency**: Ensure consistent error handling across all endpoints
## Technical Details
### Server Configuration
- **Server Port**: 8081 (8080 was in use)
- **Database**: PostgreSQL (currently not connected)
- **Framework**: Gin (Go web framework)
- **Architecture**: Clean architecture with repositories, services, and handlers
### Key Files to Examine
- `cmd/server/main.go` - Server startup and database initialization
- `internal/database/postgres.go` - Database connection management
- `internal/services/application_service.go` - Application CRUD operations
- `internal/services/token_service.go` - Token management
- `docker-compose.yml` - Database container configuration
- `migrations/001_initial_schema.up.sql` - Database schema
### Current Working State
✅ **Working Endpoints**:
- Health check (basic)
- Authentication (login, verify, renew)
- Token operations (create, list, delete - when not requiring complex validation)
- Error handling for malformed JSON
- API documentation
❌ **Failing Endpoints**:
- Readiness check (database health)
- Application CRUD operations
- Some edge case validations
## Success Criteria
After resolution, all 22 tests should pass:
1. Database connectivity restored (readiness check returns 200)
2. Application CRUD operations working (create, read, update, delete, list)
3. Proper error handling for edge cases
4. All endpoints returning expected HTTP status codes
## How to Test
Run the E2E test script to verify fixes:
```bash
BASE_URL=http://localhost:8081 ./test/e2e_test.sh
```
The test script provides detailed output showing exactly which endpoints are failing and what responses they're returning versus what's expected.
## Additional Context
- The server starts successfully and handles HTTP requests
- Authentication middleware is working correctly
- The issue appears to be primarily database-related
- Some service implementations may have placeholder code that needs completion
- The test suite is comprehensive and will clearly show when issues are resolved

247
test/E2E_TESTING.md
View File

@ -1,247 +0,0 @@
# End-to-End Testing Guide
This document describes how to run end-to-end tests for the KMS (Key Management Service) API using the provided bash script.
## Overview
The `e2e_test.sh` script provides comprehensive end-to-end testing of the KMS API using curl commands. It tests all major functionality including health checks, authentication, application management, and token operations.
## Prerequisites
- `curl` command-line tool installed
- KMS server running (either locally or remotely)
- Bash shell environment
## Quick Start
### 1. Start the KMS Server
First, make sure your KMS server is running. You can start it using Docker Compose:
```bash
# From the project root directory
docker-compose up -d
```
Or run it directly:
```bash
go run cmd/server/main.go
```
### 2. Run the E2E Tests
```bash
# Run with default settings (server at localhost:8080)
./test/e2e_test.sh
# Or run with custom configuration
BASE_URL=http://localhost:8080 USER_EMAIL=admin@example.com ./test/e2e_test.sh
```
## Configuration
The script supports several environment variables for configuration:
| Variable | Default | Description |
|----------|---------|-------------|
| `BASE_URL` | `http://localhost:8080` | Base URL of the KMS server |
| `USER_EMAIL` | `test@example.com` | User email for authentication headers |
| `USER_ID` | `test-user-123` | User ID for authentication headers |
### Examples
```bash
# Test against a remote server
BASE_URL=https://kms-api.example.com ./test/e2e_test.sh
# Use different user credentials
USER_EMAIL=admin@company.com USER_ID=admin-456 ./test/e2e_test.sh
# Test against local server on different port
BASE_URL=http://localhost:3000 ./test/e2e_test.sh
```
## Test Coverage
The script tests the following functionality:
### Health Endpoints
- `GET /health` - Basic health check
- `GET /ready` - Readiness check with database connectivity
### Authentication Endpoints
- `POST /api/login` - User login (with and without auth headers)
- `POST /api/verify` - Token verification
- `POST /api/renew` - Token renewal
### Application Management
- `GET /api/applications` - List applications (with pagination)
- `POST /api/applications` - Create new application
- `GET /api/applications/:id` - Get application by ID
- `PUT /api/applications/:id` - Update application
- `DELETE /api/applications/:id` - Delete application
### Token Management
- `GET /api/applications/:id/tokens` - List tokens for application
- `POST /api/applications/:id/tokens` - Create static token
- `DELETE /api/tokens/:id` - Delete token
### Error Handling
- Invalid endpoints (404 errors)
- Malformed JSON requests
- Missing authentication headers
- Invalid request formats
### Documentation
- `GET /api/docs` - API documentation endpoint
## Output Format
The script provides colored output with clear test results:
- 🔵 **[INFO]** - General information and test progress
- 🟢 **[PASS]** - Successful test cases
- 🔴 **[FAIL]** - Failed test cases
- 🟡 **[WARN]** - Warnings and non-critical issues
### Sample Output
```
[INFO] Starting End-to-End Tests for KMS API
[INFO] Base URL: http://localhost:8080
[INFO] User Email: test@example.com
[INFO] User ID: test-user-123
[INFO] Waiting for server to be ready...
[PASS] Server is ready!
[INFO] === Testing Health Endpoints ===
[INFO] Running test: Health Check
[PASS] Health Check (Status: 200)
Response: {"status":"healthy","timestamp":"2025-08-22T17:13:26Z"}
[INFO] Running test: Readiness Check
[PASS] Readiness Check (Status: 200)
Response: {"status":"ready","timestamp":"2025-08-22T17:13:26Z","checks":{"database":"healthy"}}
...
[INFO] === Test Summary ===
Tests Run: 25
Tests Passed: 23
Tests Failed: 2
Some tests failed!
```
## Features
### Automatic Server Detection
The script waits for the server to be ready before running tests, with a configurable timeout.
### Dynamic Test Data
- Creates test applications and extracts their IDs for subsequent tests
- Creates test tokens and uses them for deletion tests
- Automatically cleans up test data after completion
### Comprehensive Error Testing
Tests various error conditions including:
- Missing authentication
- Invalid JSON payloads
- Non-existent resources
- Malformed requests
### Robust Error Handling
- Graceful handling of network errors
- Automatic cleanup on script interruption
- Clear error messages and status codes
## Troubleshooting
### Common Issues
1. **Server Not Ready**
```
[FAIL] Server failed to start within timeout
```
- Ensure the KMS server is running
- Check if the server is accessible at the configured URL
- Verify database connectivity
2. **Authentication Failures**
```
[FAIL] List applications with auth (Expected: 200, Got: 401)
```
- Check if authentication middleware is properly configured
- Verify the authentication headers are being processed correctly
3. **Database Connection Issues**
```
[FAIL] Readiness Check (Expected: 200, Got: 503)
```
- Ensure PostgreSQL database is running
- Check database connection configuration
- Verify database migrations have been applied
### Debug Mode
For more detailed output, you can modify the script to include verbose curl output:
```bash
# Add -v flag to curl commands for verbose output
# Edit the run_test function in e2e_test.sh
```
### Manual Testing
You can also run individual curl commands manually for debugging:
```bash
# Test health endpoint
curl -v http://localhost:8080/health
# Test authentication
curl -v -X POST http://localhost:8080/api/login \
-H "Content-Type: application/json" \
-H "X-User-Email: test@example.com" \
-H "X-User-ID: test-user-123" \
-d '{"app_id": "test-app", "permissions": ["read"]}'
```
## Integration with CI/CD
The script is designed to work well in CI/CD pipelines:
```yaml
# Example GitHub Actions workflow
- name: Run E2E Tests
run: |
docker-compose up -d
sleep 10 # Wait for services to start
./test/e2e_test.sh
docker-compose down
env:
BASE_URL: http://localhost:8080
USER_EMAIL: ci@example.com
```
## Extending the Tests
To add new test cases:
1. Create a new test function following the pattern `test_*`
2. Use the `run_test` helper function for consistent output
3. Add the function call to the `main()` function
4. Update this documentation
Example:
```bash
test_new_feature() {
log_info "=== Testing New Feature ==="
run_test "New feature test" "200" \
-X GET "$API_BASE/new-endpoint" \
-H "X-User-Email: $USER_EMAIL" \
-H "X-User-ID: $USER_ID"
}

274
test/README.md
View File

@ -1,48 +1,60 @@
# Integration Tests for API Key Management Service
# Testing Guide for KMS API
This directory contains comprehensive end-to-end integration tests for the API Key Management Service.
This directory contains comprehensive testing for the Key Management Service (KMS) API, including both Go integration tests and end-to-end bash script tests.
## Test Coverage
## Test Types
The integration tests cover:
### 1. Go Integration Tests (`integration_test.go`)
Comprehensive Go-based integration tests using the testify framework.
### 1. **Health Check Endpoints**
### 2. End-to-End Bash Tests (`e2e_test.sh`)
Curl-based end-to-end tests that can be run against any running KMS server instance.
---
## Go Integration Tests
### Test Coverage
The Go integration tests cover:
#### **Health Check Endpoints**
- Basic health check (`/health`)
- Readiness check with database connectivity (`/ready`)
### 2. **Application CRUD Operations**
#### **Application CRUD Operations**
- Create new applications
- List applications with pagination
- Get specific applications by ID
- Update application details
- Delete applications
### 3. **Static Token Workflow**
#### **Static Token Workflow**
- Create static tokens for applications
- Verify static token permissions
- Token validation and permission checking
### 4. **User Token Authentication Flow**
#### **User Token Authentication Flow**
- User login process
- Token generation for users
- Permission-based access control
### 5. **Authentication Middleware**
#### **Authentication Middleware**
- Header-based authentication validation
- Unauthorized access handling
- User context management
### 6. **Error Handling**
#### **Error Handling**
- Invalid JSON request handling
- Non-existent resource handling
- Proper HTTP status codes
### 7. **Concurrent Load Testing**
#### **Concurrent Load Testing**
- Multiple simultaneous health checks
- Concurrent application listing requests
- Database connection pooling under load
## Prerequisites
### Prerequisites
Before running the integration tests, ensure you have:
@ -50,7 +62,7 @@ Before running the integration tests, ensure you have:
2. **Test Database**: Create a test database named `kms_test`
3. **Go Dependencies**: All required Go modules installed
### Database Setup
#### Database Setup
```bash
# Connect to PostgreSQL
@ -63,16 +75,16 @@ CREATE DATABASE kms_test;
GRANT ALL PRIVILEGES ON DATABASE kms_test TO postgres;
```
## Running the Tests
### Running Go Integration Tests
### Option 1: Run All Integration Tests
#### Run All Integration Tests
```bash
# From the project root directory
go test -v ./test/...
```
### Option 2: Run with Coverage
#### Run with Coverage
```bash
# Generate coverage report
@ -80,7 +92,7 @@ go test -v -coverprofile=coverage.out ./test/...
go tool cover -html=coverage.out -o coverage.html
```
### Option 3: Run Specific Test Suites
#### Run Specific Test Suites
```bash
# Run only health endpoint tests
@ -96,7 +108,7 @@ go test -v ./test/ -run TestStaticTokenWorkflow
go test -v ./test/ -run TestConcurrentRequests
```
### Option 4: Run with Docker/Podman
#### Run with Docker/Podman
```bash
# Start the services first
@ -112,9 +124,110 @@ go test -v ./test/...
podman-compose down
```
---
## End-to-End Bash Tests
### Overview
The `e2e_test.sh` script provides comprehensive end-to-end testing of the KMS API using curl commands. It tests all major functionality including health checks, authentication, application management, and token operations.
### Prerequisites
- `curl` command-line tool installed
- KMS server running (either locally or remotely)
- Bash shell environment
### Quick Start
#### 1. Start the KMS Server
First, make sure your KMS server is running. You can start it using Docker Compose:
```bash
# From the project root directory
docker-compose up -d
```
Or run it directly:
```bash
go run cmd/server/main.go
```
#### 2. Run the E2E Tests
```bash
# Run with default settings (server at localhost:8080)
./test/e2e_test.sh
# Or run with custom configuration
BASE_URL=http://localhost:8080 USER_EMAIL=admin@example.com ./test/e2e_test.sh
```
### Configuration
The script supports several environment variables for configuration:
| Variable | Default | Description |
|----------|---------|-------------|
| `BASE_URL` | `http://localhost:8080` | Base URL of the KMS server |
| `USER_EMAIL` | `test@example.com` | User email for authentication headers |
| `USER_ID` | `test-user-123` | User ID for authentication headers |
#### Examples
```bash
# Test against a remote server
BASE_URL=https://kms-api.example.com ./test/e2e_test.sh
# Use different user credentials
USER_EMAIL=admin@company.com USER_ID=admin-456 ./test/e2e_test.sh
# Test against local server on different port
BASE_URL=http://localhost:3000 ./test/e2e_test.sh
```
### E2E Test Coverage
The bash script tests the following functionality:
#### Health Endpoints
- `GET /health` - Basic health check
- `GET /ready` - Readiness check with database connectivity
#### Authentication Endpoints
- `POST /api/login` - User login (with and without auth headers)
- `POST /api/verify` - Token verification
- `POST /api/renew` - Token renewal
#### Application Management
- `GET /api/applications` - List applications (with pagination)
- `POST /api/applications` - Create new application
- `GET /api/applications/:id` - Get application by ID
- `PUT /api/applications/:id` - Update application
- `DELETE /api/applications/:id` - Delete application
#### Token Management
- `GET /api/applications/:id/tokens` - List tokens for application
- `POST /api/applications/:id/tokens` - Create static token
- `DELETE /api/tokens/:id` - Delete token
#### Error Handling
- Invalid endpoints (404 errors)
- Malformed JSON requests
- Missing authentication headers
- Invalid request formats
#### Documentation
- `GET /api/docs` - API documentation endpoint
---
## Test Configuration
The tests use a separate test configuration that:
### Go Integration Tests Configuration
The Go tests use a separate test configuration that:
- Uses a dedicated test database (`kms_test`)
- Disables rate limiting for testing
@ -122,44 +235,15 @@ The tests use a separate test configuration that:
- Uses debug logging level
- Configures shorter timeouts
## Test Data Management
### Test Data Management
The integration tests:
Both test suites:
- **Clean up after themselves**: Each test cleans up its test data
- **Use isolated data**: Test data is prefixed with `test-` to avoid conflicts
- **Reset state**: Database state is reset between test runs
- **Use transactions**: Where possible, tests use database transactions
## Expected Test Results
When all tests pass, you should see output similar to:
```
=== RUN TestIntegrationSuite
=== RUN TestIntegrationSuite/TestHealthEndpoints
=== RUN TestIntegrationSuite/TestApplicationCRUD
=== RUN TestIntegrationSuite/TestApplicationCRUD/CreateApplication
=== RUN TestIntegrationSuite/TestApplicationCRUD/ListApplications
=== RUN TestIntegrationSuite/TestApplicationCRUD/GetApplication
=== RUN TestIntegrationSuite/TestStaticTokenWorkflow
=== RUN TestIntegrationSuite/TestStaticTokenWorkflow/CreateStaticToken
=== RUN TestIntegrationSuite/TestStaticTokenWorkflow/VerifyStaticToken
=== RUN TestIntegrationSuite/TestUserTokenWorkflow
=== RUN TestIntegrationSuite/TestUserTokenWorkflow/UserLogin
=== RUN TestIntegrationSuite/TestAuthenticationMiddleware
=== RUN TestIntegrationSuite/TestAuthenticationMiddleware/MissingAuthHeader
=== RUN TestIntegrationSuite/TestAuthenticationMiddleware/ValidAuthHeader
=== RUN TestIntegrationSuite/TestErrorHandling
=== RUN TestIntegrationSuite/TestErrorHandling/InvalidJSON
=== RUN TestIntegrationSuite/TestErrorHandling/NonExistentApplication
=== RUN TestIntegrationSuite/TestConcurrentRequests
=== RUN TestIntegrationSuite/TestConcurrentRequests/ConcurrentHealthChecks
=== RUN TestIntegrationSuite/TestConcurrentRequests/ConcurrentApplicationListing
--- PASS: TestIntegrationSuite (2.34s)
PASS
```
## Troubleshooting
### Common Issues
@ -187,18 +271,17 @@ PASS
- The test server uses random ports, but check if other services are running
- Stop any running instances of the API service
4. **Test Timeouts**
4. **Server Not Ready (E2E Tests)**
```
Error: test timed out
[FAIL] Server failed to start within timeout
```
- Increase test timeout values
- Check database performance
- Verify network connectivity
- Ensure the KMS server is running
- Check if the server is accessible at the configured URL
- Verify database connectivity
### Debug Mode
To run tests with additional debugging:
#### For Go Tests
```bash
# Enable debug logging
LOG_LEVEL=debug go test -v ./test/...
@ -210,33 +293,26 @@ go test -race -v ./test/...
go test -memprofile=mem.prof -v ./test/...
```
## Test Architecture
#### For E2E Tests
For more detailed output, you can modify the script to include verbose curl output by adding `-v` flag to curl commands.
The integration tests use:
## Integration with CI/CD
- **testify/suite**: For organized test suites with setup/teardown
- **httptest**: For HTTP server testing
- **testify/assert**: For test assertions
- **testify/require**: For test requirements
The test structure follows the same clean architecture as the main application:
Both test suites work well in CI/CD pipelines:
```yaml
# Example GitHub Actions workflow
- name: Run Integration Tests
run: |
docker-compose up -d
sleep 10 # Wait for services to start
go test -v ./test/...
./test/e2e_test.sh
docker-compose down
env:
BASE_URL: http://localhost:8080
USER_EMAIL: ci@example.com
```
test/
├── integration_test.go # Main integration test suite
├── test_helpers.go # Test utilities and mocks
└── README.md # This documentation
```
## Contributing
When adding new integration tests:
1. Follow the existing test patterns
2. Clean up test data properly
3. Use descriptive test names
4. Add appropriate assertions
5. Update this README if needed
## Performance Benchmarks
@ -248,3 +324,41 @@ The concurrent load tests provide basic performance benchmarks:
- **Expected Throughput**: > 100 requests/second
These benchmarks help ensure the service can handle reasonable concurrent load.
## Test Architecture
### Go Integration Tests
The integration tests use:
- **testify/suite**: For organized test suites with setup/teardown
- **httptest**: For HTTP server testing
- **testify/assert**: For test assertions
- **testify/require**: For test requirements
### E2E Bash Tests
The bash script provides:
- **Automatic Server Detection**: Waits for server readiness
- **Dynamic Test Data**: Creates and cleans up test resources
- **Comprehensive Error Testing**: Tests various error conditions
- **Robust Error Handling**: Graceful cleanup and clear error messages
## Contributing
When adding new tests:
1. Follow the existing test patterns
2. Clean up test data properly
3. Use descriptive test names
4. Add appropriate assertions
5. Update this documentation if needed
## File Structure
```
test/
├── integration_test.go # Go integration test suite
├── test_helpers.go # Test utilities and mocks
├── mock_repositories.go # Mock implementations for testing
├── e2e_test.sh # Bash end-to-end test script
└── README.md # This comprehensive testing guide

68
test_content_type.sh
View File

@ -1,68 +0,0 @@
#!/bin/bash
# Simple test script to check content-type validation
BASE_URL="${BASE_URL:-http://localhost:8080}"
API_BASE="${BASE_URL}/api"
USER_EMAIL="${USER_EMAIL:-test@example.com}"
USER_ID="${USER_ID:-test-user-123}"
echo "Testing content-type validation..."
# Test 1: POST without Content-Type header (should return 400)
echo "Test 1: POST without Content-Type header"
response=$(curl -s -w "\n%{http_code}" -X POST "$API_BASE/applications" \
-H "X-User-Email: $USER_EMAIL" \
-H "X-User-ID: $USER_ID" \
-d '{
"app_id": "test-content-type-validation",
"app_link": "https://example.com/test-app",
"type": ["static"],
"callback_url": "https://example.com/callback"
}' 2>/dev/null || echo -e "\n000")
status_code=$(echo "$response" | tail -n1)
body=$(echo "$response" | head -n -1)
echo "Status Code: $status_code"
echo "Response Body: $body"
if [[ "$status_code" == "400" ]]; then
echo "✅ PASS: Content-type validation working correctly"
else
echo "❌ FAIL: Expected 400, got $status_code"
fi
echo ""
# Test 2: POST with correct Content-Type header (should work)
echo "Test 2: POST with correct Content-Type header"
response2=$(curl -s -w "\n%{http_code}" -X POST "$API_BASE/applications" \
-H "Content-Type: application/json" \
-H "X-User-Email: $USER_EMAIL" \
-H "X-User-ID: $USER_ID" \
-d '{
"app_id": "test-content-type-validation-2",
"app_link": "https://example.com/test-app",
"type": ["static"],
"callback_url": "https://example.com/callback",
"token_renewal_duration": 604800000000000,
"max_token_duration": 2592000000000000,
"owner": {
"type": "individual",
"name": "Test User",
"owner": "test@example.com"
}
}' 2>/dev/null || echo -e "\n000")
status_code2=$(echo "$response2" | tail -n1)
body2=$(echo "$response2" | head -n -1)
echo "Status Code: $status_code2"
echo "Response Body: $body2" | head -c 200
echo ""
if [[ "$status_code2" == "201" ]]; then
echo "✅ PASS: Request with correct Content-Type works"
else
echo "❌ FAIL: Expected 201, got $status_code2"
fi