Skip to content

Commit 48ca452

Browse files
author
numen-bot
committed
feat(quality): polish — OpenAPI spec, API docs, blog post, README, test infrastructure fixes (phpunit.xml APP_BASE_PATH, PipelineRunFactory)
1 parent cbdb4a0 commit 48ca452

File tree

12 files changed

+657
-13
lines changed

12 files changed

+657
-13
lines changed

‎README.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,17 @@ Each stage is a queued job. The pipeline is event-driven. Stages are defined in
3737

3838
## Features
3939

40+
### AI Content Quality Scoring
41+
**New in v0.10.0.** Automated multi-dimensional content quality analysis across five dimensions:
42+
- **Readability** — Flesch-Kincaid metrics, sentence and word complexity
43+
- **SEO** — Keyword density, heading structure, meta optimization
44+
- **Brand Consistency** — LLM-powered brand voice and tone analysis
45+
- **Factual Accuracy** — Cross-referenced claim validation
46+
- **Engagement Prediction** — AI-predicted engagement score
47+
48+
Features: real-time score ring in the editor sidebar, quality dashboard with Chart.js trend visualization, space leaderboard, configurable pipeline quality gates, auto-score on publish, and a `quality.scored` webhook event.
49+
50+
4051
### Content Generation Pipeline
4152
Submit a brief → AI agents generate, illustrate, optimize, and quality-gate content → auto-publish or human review.
4253

‎app/Http/Controllers/Api/ContentQualityController.php‎

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -199,13 +199,28 @@ public function updateConfig(Request $request): ContentQualityConfigResource
199199
$user = $request->user();
200200
$this->authz->authorize($user, 'settings.manage', $validated['space_id']);
201201

202-
$config = ContentQualityConfig::firstOrNew(['space_id' => $validated['space_id']]);
203-
$config->fill(array_filter($validated, fn ($v, $k) => $k !== 'space_id', ARRAY_FILTER_USE_BOTH));
204-
205-
if (! $config->exists) {
206-
$config->space_id = $validated['space_id'];
207-
}
202+
$defaults = [
203+
'dimension_weights' => [
204+
'readability' => 0.25,
205+
'seo' => 0.25,
206+
'brand_consistency' => 0.20,
207+
'factual_accuracy' => 0.15,
208+
'engagement_prediction' => 0.15,
209+
],
210+
'thresholds' => ['poor' => 40, 'fair' => 60, 'good' => 75, 'excellent' => 90],
211+
'enabled_dimensions' => ['readability', 'seo', 'brand_consistency', 'factual_accuracy', 'engagement_prediction'],
212+
'auto_score_on_publish' => true,
213+
'pipeline_gate_enabled' => false,
214+
'pipeline_gate_min_score' => 70.0,
215+
];
216+
217+
$config = ContentQualityConfig::firstOrNew(
218+
['space_id' => $validated['space_id']],
219+
array_merge($defaults, ['space_id' => $validated['space_id']]),
220+
);
208221

222+
$updates = array_filter($validated, fn ($v, $k) => $k !== 'space_id', ARRAY_FILTER_USE_BOTH);
223+
$config->fill($updates);
209224
$config->save();
210225

211226
return new ContentQualityConfigResource($config);

‎app/Listeners/AutoScoreOnPublishListener.php‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ public function handle(ContentPublished $event): void
2121

2222
// Score if: no config (default on) OR config explicitly enables it
2323
if ($config === null || $config->auto_score_on_publish) {
24-
ScoreContentQualityJob::dispatch($content);
24+
ScoreContentQualityJob::dispatch($content->id);
2525
}
2626
}
2727
}

‎app/Models/PipelineRun.php‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
namespace App\Models;
44

55
use Illuminate\Database\Eloquent\Concerns\HasUlids;
6+
use Illuminate\Database\Eloquent\Factories\HasFactory;
67
use Illuminate\Database\Eloquent\Model;
78
use Illuminate\Database\Eloquent\Relations\BelongsTo;
89
use Illuminate\Database\Eloquent\Relations\HasMany;
@@ -28,6 +29,7 @@
2829
*/
2930
class PipelineRun extends Model
3031
{
32+
use HasFactory;
3133
use HasUlids;
3234

3335
protected $fillable = [

‎database/factories/PipelineRunFactory.php‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
<?php
2+
3+
namespace Database\Factories;
4+
5+
use App\Models\ContentPipeline;
6+
use App\Models\PipelineRun;
7+
use Illuminate\Database\Eloquent\Factories\Factory;
8+
9+
class PipelineRunFactory extends Factory
10+
{
11+
protected $model = PipelineRun::class;
12+
13+
public function definition(): array
14+
{
15+
return [
16+
'pipeline_id' => ContentPipeline::factory(),
17+
'content_id' => null,
18+
'content_brief_id' => null,
19+
'status' => 'running',
20+
'current_stage' => 'stage_1',
21+
'stage_results' => [],
22+
'context' => [],
23+
'started_at' => now(),
24+
'completed_at' => null,
25+
];
26+
}
27+
}

‎docs/api/quality-api.md‎

Lines changed: 250 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,250 @@
1+
# Content Quality Scoring API Reference
2+
3+
**Version:** 1.0.0
4+
**Base URL:** `/api/v1/quality`
5+
**Added:** 2026-03-16
6+
7+
All endpoints require Sanctum authentication. Permission `content.view` is required
8+
for read operations; `settings.manage` is required for config updates.
9+
10+
---
11+
12+
## Endpoints
13+
14+
### 1. GET /api/v1/quality/scores
15+
16+
List quality scores for a space, optionally filtered by content item.
17+
18+
**Query parameters:**
19+
20+
| Parameter | Type | Required | Description |
21+
|-----------|------|----------|-------------|
22+
| `space_id` | string (ULID) | ✅ | Space to filter by |
23+
| `content_id` | string (ULID) | ❌ | Filter to specific content item |
24+
| `per_page` | integer (1–100) | ❌ | Scores per page (default: 20) |
25+
26+
**Response:**
27+
```json
28+
{
29+
"data": [
30+
{
31+
"id": "01J...",
32+
"space_id": "01J...",
33+
"content_id": "01J...",
34+
"content_version_id": "01J...",
35+
"overall_score": 82.5,
36+
"dimensions": {
37+
"readability": 88.0,
38+
"seo": 79.0,
39+
"brand": 85.0,
40+
"factual": 91.0,
41+
"engagement": 70.0
42+
},
43+
"scoring_model": "content-quality-v1",
44+
"scoring_duration_ms": 1240,
45+
"scored_at": "2026-03-16T10:00:00Z",
46+
"items": []
47+
}
48+
],
49+
"links": {...},
50+
"meta": {...}
51+
}
52+
```
53+
54+
---
55+
56+
### 2. GET /api/v1/quality/scores/{score}
57+
58+
Get a single quality score with its dimension items.
59+
60+
**Response:** Single `ContentQualityScoreResource` with `items` array.
61+
62+
---
63+
64+
### 3. POST /api/v1/quality/score
65+
66+
Trigger an async quality scoring job for a content item.
67+
68+
**Request body:**
69+
```json
70+
{ "content_id": "01J..." }
71+
```
72+
73+
**Response (202):**
74+
```json
75+
{ "message": "Quality scoring job queued.", "content_id": "01J..." }
76+
```
77+
78+
---
79+
80+
### 4. GET /api/v1/quality/trends
81+
82+
Aggregate daily trend data, leaderboard, and dimension distributions for a space.
83+
84+
**Query parameters:**
85+
86+
| Parameter | Type | Required | Description |
87+
|-----------|------|----------|-------------|
88+
| `space_id` | string (ULID) | ✅ | Space to query |
89+
| `from` | date (YYYY-MM-DD) | ❌ | Start date (default: 30 days ago) |
90+
| `to` | date (YYYY-MM-DD) | ❌ | End date (default: today) |
91+
92+
**Response:**
93+
```json
94+
{
95+
"data": {
96+
"trends": {
97+
"2026-03-15": {
98+
"overall": 81.3,
99+
"readability": 85.2,
100+
"seo": 78.4,
101+
"brand": 82.0,
102+
"factual": 90.1,
103+
"engagement": 71.5,
104+
"total": 12
105+
}
106+
},
107+
"leaderboard": [
108+
{
109+
"score_id": "01J...",
110+
"content_id": "01J...",
111+
"title": "10 Ways to Write Better Content",
112+
"overall_score": 94.5,
113+
"scored_at": "2026-03-16T09:00:00Z"
114+
}
115+
],
116+
"distribution": {
117+
"overall": { "0-10": 0, "10-20": 1, "20-30": 2, ..., "90-100": 8 }
118+
},
119+
"period": { "from": "2026-02-14", "to": "2026-03-16" }
120+
}
121+
}
122+
```
123+
124+
---
125+
126+
### 5. GET /api/v1/quality/config
127+
128+
Get quality configuration for a space. Creates a default config if none exists.
129+
130+
**Query parameters:** `space_id` (required)
131+
132+
**Response:**
133+
```json
134+
{
135+
"data": {
136+
"id": "01J...",
137+
"space_id": "01J...",
138+
"dimension_weights": {
139+
"readability": 0.25,
140+
"seo": 0.25,
141+
"brand_consistency": 0.20,
142+
"factual_accuracy": 0.15,
143+
"engagement_prediction": 0.15
144+
},
145+
"thresholds": { "poor": 40, "fair": 60, "good": 75, "excellent": 90 },
146+
"enabled_dimensions": ["readability", "seo", "brand_consistency", "factual_accuracy", "engagement_prediction"],
147+
"auto_score_on_publish": true,
148+
"pipeline_gate_enabled": false,
149+
"pipeline_gate_min_score": 70.0,
150+
"created_at": "2026-03-16T08:00:00Z",
151+
"updated_at": "2026-03-16T08:00:00Z"
152+
}
153+
}
154+
```
155+
156+
---
157+
158+
### 6. PUT /api/v1/quality/config
159+
160+
Update quality configuration for a space. Requires `settings.manage` permission.
161+
162+
**Request body:**
163+
```json
164+
{
165+
"space_id": "01J...",
166+
"pipeline_gate_enabled": true,
167+
"pipeline_gate_min_score": 75,
168+
"auto_score_on_publish": true,
169+
"enabled_dimensions": ["readability", "seo"],
170+
"dimension_weights": { "readability": 0.5, "seo": 0.5 }
171+
}
172+
```
173+
174+
**Response:** Updated `ContentQualityConfigResource`.
175+
176+
---
177+
178+
## Webhook Event
179+
180+
### `quality.scored`
181+
182+
Fired when a content item is successfully scored.
183+
184+
**Payload:**
185+
```json
186+
{
187+
"id": "01J...",
188+
"event": "quality.scored",
189+
"timestamp": "2026-03-16T10:00:00Z",
190+
"data": {
191+
"score_id": "01J...",
192+
"content_id": "01J...",
193+
"space_id": "01J...",
194+
"overall_score": 82.5,
195+
"readability_score": 88.0,
196+
"seo_score": 79.0,
197+
"brand_score": 85.0,
198+
"factual_score": 91.0,
199+
"engagement_score": 70.0,
200+
"scored_at": "2026-03-16T10:00:00Z"
201+
}
202+
}
203+
```
204+
205+
---
206+
207+
## Pipeline Stage: `quality_gate`
208+
209+
Add a `quality_gate` stage to any pipeline to enforce quality thresholds before publishing.
210+
211+
**Pipeline definition example:**
212+
```json
213+
{
214+
"stages": [
215+
{ "name": "ai_generate", "type": "ai_generate" },
216+
{
217+
"name": "quality_check",
218+
"type": "quality_gate",
219+
"min_score": 75
220+
},
221+
{ "name": "publish", "type": "auto_publish" }
222+
]
223+
}
224+
```
225+
226+
If `min_score` is omitted, the stage uses the space's `pipeline_gate_min_score` config (default: 70).
227+
228+
If the score is below the threshold, the pipeline is paused with status `paused_for_review`.
229+
230+
---
231+
232+
## Scoring Dimensions
233+
234+
| Dimension | Key | Description |
235+
|-----------|-----|-------------|
236+
| Readability | `readability` | Flesch-Kincaid score, sentence length, word complexity |
237+
| SEO | `seo` | Keyword density, meta tags, heading structure |
238+
| Brand Consistency | `brand_consistency` | Tone, voice, and brand guideline adherence (LLM-based) |
239+
| Factual Accuracy | `factual_accuracy` | Fact-check claims against knowledge base (LLM-based) |
240+
| Engagement Prediction | `engagement_prediction` | Predicted engagement based on content patterns (LLM-based) |
241+
242+
## Score Interpretation
243+
244+
| Range | Label | Meaning |
245+
|-------|-------|---------|
246+
| 90–100 | Excellent | Ready to publish, high-quality |
247+
| 75–89 | Good | Publish-ready, minor improvements possible |
248+
| 60–74 | Fair | Consider improvements before publishing |
249+
| 40–59 | Poor | Significant improvements needed |
250+
| 0–39 | Critical | Major revision required |

0 commit comments

Comments
 (0)