Skip to content

Add tests for issue validation workflow #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
f3l1x merged 1 commit into from
Dec 14, 2025
Merged

Add tests for issue validation workflow #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,75 @@ tests/Sniffs/Generic.Files/LineLength/
- Second line: Comment explaining test (e.g., `// Test: lineLimit=120`)
- Use tabs for indentation
- Include code that exercises the specific parameter being tested

## Issue Testing

Issue tests validate that specific GitHub issues are properly handled. These tests are located in `tests/Issue/{number}/`.

### Creating Issue Tests

1. **Create issue directory**: `tests/Issue/{number}/`
2. **Create test files**:
- `good.php` - Code that should pass without errors (regression test)
- `good.ruleset.xml` - Ruleset with relevant sniffs
- Optionally: `bad.php` / `bad.ruleset.xml` for cases that should produce errors
3. **Regenerate snapshots**: `php bin/snapshots --issues` or `php bin/snapshots --issue={number}`
4. **Run tests**: `make tests`

**Example issue test structure:**

```
tests/Issue/19/
├── good.php # Code demonstrating the fix works
├── good.ruleset.xml # Ruleset with relevant sniffs
└── good.snapshot.json # Expected output (0 errors for good.php)
```

**Example good.php:**

```php
<?php declare(strict_types = 1);

// Issue #19: Traits are removed from enum
// @link https://github.com/contributte/qa/issues/19
// Test: Enum with traits should pass without errors

namespace Tests\Issue19;

trait EnumValues
{
// trait implementation
}

enum ImageTransform: int
{
use EnumValues;

case Fit = 1;
}
```

**Example good.ruleset.xml:**

```xml
<?xml version="1.0" encoding="UTF-8"?>
<ruleset name="Test Issue #19 - Enum with traits">
<!-- Issue #19: Traits should not be removed from enum -->
<!-- @link https://github.com/contributte/qa/issues/19 -->
<rule ref="SlevomatCodingStandard.Classes.ClassStructure"/>
<rule ref="SlevomatCodingStandard.Classes.TraitUseDeclaration"/>
</ruleset>
```

**Commands for issue testing:**

```bash
# Generate all issue snapshots
php bin/snapshots --issues

# Generate specific issue snapshot
php bin/snapshots --issue=19

# Test code manually against an issue ruleset
vendor/bin/phpcs --standard=tests/Issue/19/good.ruleset.xml tests/Issue/19/good.php
```
84 changes: 82 additions & 2 deletions bin/snapshots
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@
* php bin/snapshots --set=8.0 # Generate only set 8.0
* php bin/snapshots --sniffs # Generate only sniff snapshots
* php bin/snapshots --sniff=Generic.Arrays.DisallowLongArraySyntax
* php bin/snapshots --issues # Generate only issue snapshots
* php bin/snapshots --issue=19 # Generate only specific issue snapshot
*/

$rootDir = dirname(__DIR__);
Expand All @@ -23,8 +25,10 @@ use Tests\Toolkit\Codesniffer;
// Parse CLI arguments
$filterSet = null;
$filterSniff = null;
$filterIssue = null;
$sniffsOnly = false;
$setsOnly = false;
$issuesOnly = false;

foreach ($argv as $arg) {
if (str_starts_with($arg, '--set=')) {
Expand All @@ -37,20 +41,29 @@ foreach ($argv as $arg) {
$sniffsOnly = true;
}

if (str_starts_with($arg, '--issue=')) {
$filterIssue = substr($arg, 8);
$issuesOnly = true;
}

if ($arg === '--sniffs') {
$sniffsOnly = true;
}

if ($arg === '--sets') {
$setsOnly = true;
}

if ($arg === '--issues') {
$issuesOnly = true;
}
}

// ============================================================================
// Part 1: Generate Set Snapshots
// ============================================================================

if (!$sniffsOnly) {
if (!$sniffsOnly && !$issuesOnly) {
$setsDir = $rootDir . '/tests/Sets';

echo "=== Generating Set Snapshots ===\n\n";
Expand Down Expand Up @@ -126,7 +139,7 @@ if (!$sniffsOnly) {
// Part 2: Generate Sniff Snapshots
// ============================================================================

if (!$setsOnly) {
if (!$setsOnly && !$issuesOnly) {
$sniffsDir = $rootDir . '/tests/Sniffs';

echo "\n=== Generating Sniff Snapshots ===\n\n";
Expand Down Expand Up @@ -196,4 +209,71 @@ if (!$setsOnly) {
echo "\nSniff snapshots: {$generated} generated.\n";
}

// ============================================================================
// Part 3: Generate Issue Snapshots
// ============================================================================

if (!$setsOnly && !$sniffsOnly) {
$issuesDir = $rootDir . '/tests/Issue';

echo "\n=== Generating Issue Snapshots ===\n\n";

$generated = 0;

if (is_dir($issuesDir)) {
// Find all issue directories (e.g., 19)
foreach (Finder::findDirectories('*')->in($issuesDir) as $issueDir) {
$issueName = $issueDir->getBasename();

if ($filterIssue !== null && $filterIssue !== $issueName) {
continue;
}

// Find all PHP test files in the issue directory
foreach (Finder::findFiles('*.php')->in($issueDir->getPathname()) as $phpFile) {
$baseName = $phpFile->getBasename('.php');
$snapshotPath = $issueDir->getPathname() . '/' . $baseName . '.snapshot.json';
$rulesetPath = $issueDir->getPathname() . '/' . $baseName . '.ruleset.xml';

if (!file_exists($rulesetPath)) {
echo "Warning: Ruleset not found: {$rulesetPath}\n";

continue;
}

$process = new Process([
'vendor/bin/phpcs',
'--standard=' . $rulesetPath,
'--report=json',
'-q',
$phpFile->getPathname(),
], $rootDir);
$process->run();

$data = json_decode($process->getOutput(), true);

if ($data === null) {
echo "Error: Failed to parse phpcs output for Issue/{$issueName}/{$baseName}.php\n";

continue;
}

$normalized = Codesniffer::normalize($data);
$json = json_encode($normalized, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES) . "\n";

file_put_contents($snapshotPath, $json);

$errorCount = $normalized['totals']['errors'];
$warningCount = $normalized['totals']['warnings'];
echo "Generated: Issue/{$issueName}/{$baseName}.snapshot.json ({$errorCount} errors, {$warningCount} warnings)\n";
$generated++;
}
}
} else {
echo "No Issue directory found.\n";
}

echo "\nIssue snapshots: {$generated} generated.\n";
}

echo "\nDone!\n";
51 changes: 51 additions & 0 deletions tests/Issue/19/good.php
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<?php declare(strict_types = 1);

// Issue #19: Traits are removed from enum
// @link https://github.com/contributte/qa/issues/19
// Test: Enum with traits should pass without errors

namespace Tests\Issue19;

trait EnumValues
{

/**
* @return array<int, int|string>
*/
public static function values(): array
{
return array_column(self::cases(), 'value');
}

}

trait EnumNames
{

/**
* @return array<int, string>
*/
public static function names(): array
{
return array_column(self::cases(), 'name');
}

}

enum ImageTransform: int
{

use EnumValues;
use EnumNames;

case Fit = 1;

case Exact = 2;

case Fill = 3;

case Stretch = 4;

case Shrink = 5;

}
46 changes: 46 additions & 0 deletions tests/Issue/19/good.ruleset.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<?xml version="1.0" encoding="UTF-8"?>
<ruleset name="Test Issue #19 - Enum with traits">
<!-- Issue #19: Traits should not be removed from enum -->
<!-- @link https://github.com/contributte/qa/issues/19 -->

<!-- Include ClassStructure which was involved in the issue -->
<rule ref="SlevomatCodingStandard.Classes.ClassStructure">
<properties>
<property name="groups" type="array">
<element value="uses"/>
<element value="enum cases"/>
<element value="public constants"/>
<element value="protected constants"/>
<element value="private constants"/>
<element value="public static properties"/>
<element value="public properties"/>
<element value="protected static properties"/>
<element value="protected properties"/>
<element value="private static properties"/>
<element value="private properties"/>
<element value="constructor"/>
<element value="destructor"/>
<element value="static constructors"/>
<element value="public static abstract methods"/>
<element value="public static final methods"/>
<element value="public static methods"/>
<element value="public abstract methods"/>
<element value="public final methods"/>
<element value="public methods"/>
<element value="protected static abstract methods"/>
<element value="protected static final methods"/>
<element value="protected static methods"/>
<element value="protected abstract methods"/>
<element value="protected final methods"/>
<element value="protected methods"/>
<element value="private static methods"/>
<element value="private methods"/>
<element value="magic methods"/>
</property>
</properties>
</rule>

<!-- Other relevant sniffs for enums -->
<rule ref="SlevomatCodingStandard.Classes.TraitUseDeclaration"/>
<rule ref="SlevomatCodingStandard.Classes.TraitUseSpacing"/>
</ruleset>
13 changes: 13 additions & 0 deletions tests/Issue/19/good.snapshot.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"totals": {
"errors": 0,
"warnings": 0
},
"files": {
"good.php": {
"errors": 0,
"warnings": 0,
"messages": []
}
}
}