Skip to content

Commit b7ceeba

Browse files
The-Great-AeochsAbdealiLoKo
authored andcommitted
Add language translation pipeline example documentation.
1 parent b12d196 commit b7ceeba

File tree

8 files changed

+246
-0
lines changed

8 files changed

+246
-0
lines changed

docs/.nav.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,8 @@ nav:
2020
- Intent Classification Pipeline Example:
2121
- Prompt Registration: "register-and-refine/examples/intent-classification-pipeline-registration/prompt.md"
2222
- Pipeline Registration: "register-and-refine/examples/intent-classification-pipeline-registration/pipeline.md"
23+
- Language Translation Pipeline Example:
24+
- Pipeline Registration: "register-and-refine/examples/language-translation-pipeline-registration/pipeline.md"
2325

2426
- Evaluate and Approve:
2527
- Overview: "evaluate-and-approve/index.md"
67 KB
Loading
83.2 KB
Loading
39.8 KB
Loading
20.4 KB
Loading
47.7 KB
Loading
37.6 KB
Loading
Lines changed: 244 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,244 @@
1+
# Pipeline Registration Guide: English to French Translation
2+
3+
This guide walks you through registering an **English to French Translation Pipeline** on the Corridor platform. This pipeline automatically detects English text and provides high-quality French translations using Gemini 2.0 Flash.
4+
5+
**What This Pipeline Does:**
6+
7+
- Detects if input text is in English
8+
- Translates English text to French with preserved tone and style
9+
- Returns error messages for non-English input
10+
- Tracks API usage costs
11+
12+
If you are new to Pipelines, read [What are Pipelines?](../../inventory-management/pipelines/index.md) to understand how they work.
13+
14+
15+
## Prerequisites
16+
17+
Before registering this pipeline, ensure you have:
18+
19+
-**Registered Gemini 2.0 Flash Model** - Follow the [Model Registration Guide](../../model/) to register the model
20+
21+
-**API Token Configured** - Ensure `GOOGLE_API_TOKEN` is set up in Platform Integrations
22+
23+
**Quick Check:** Navigate to **GenAI Studio → Model Catalog** and verify `gemini_2_0_flash` is available.
24+
25+
If you haven't completed these steps, please do so before proceeding.
26+
27+
28+
## Registration Steps
29+
30+
### Step 1. Navigate to Pipeline Registry
31+
32+
Go to **GenAI Studio → Pipeline Registry** and click the **Create** button.
33+
34+
### Step 2. Fill in Basic Information
35+
36+
![Pipeline Basic Information](english-to-french-pipeline-description.png)
37+
38+
**Basic Information** fields help organize and identify your pipeline:
39+
40+
- **Description:** Clear explanation of what the pipeline does and its workflow
41+
- **Usecase Type:** The primary use case category - select **Translation**
42+
- **Task Type:** Specific task the pipeline performs - select **Generative Responses**
43+
- **Impact of Generated Output:** Scope of the pipeline's usage - select **External Facing**
44+
- **Data Usage:** Whether the pipeline uses additional data sources - leave empty for this pipeline
45+
- **Group:** Category for organizing similar pipelines - select **Example Pipelines**
46+
- **Permissible Purpose:** Approved use cases and business scenarios for this pipeline
47+
48+
**Example Description:**
49+
50+
```
51+
English to French Translation Assistant - Translate English text to French using Gemini 2.0 Flash.
52+
53+
Key Features:
54+
- Automatic English language detection
55+
- High-quality French translations using Gemini 2.0 Flash
56+
- Preserves tone, style, and cultural nuances
57+
- Cost tracking for API usage
58+
59+
Usage:
60+
- Simple translation: "Hello, how are you?"
61+
- Any English text: "The weather is beautiful today."
62+
- Formal or informal: Automatically preserves the tone
63+
64+
Note: This pipeline only translates FROM English TO French.
65+
```
66+
67+
### Step 3. Configure Code Settings
68+
69+
![Pipeline Code Configuration](english-to-french-pipeline-code-config.png)
70+
71+
**Code Settings** define how your pipeline operates and which resources it uses.
72+
73+
**Configuration Fields:**
74+
75+
- **Alias:** `english_to_french_translation`: A Python variable name to reference this pipeline in code
76+
77+
- **Input Type:** Select **Python Function** : This pipeline uses custom Python code for translation logic
78+
79+
- **Agent Provider:** Select **Other** : We're not using a pre-built agent provider for this translation pipeline
80+
81+
- **Pipeline Type:** Select **Chat Based Pipeline** :Enables conversational interface and message history.
82+
83+
- **Context Type:** `dict[str, str]`
84+
85+
- Data type for storing information across conversation turns
86+
- For this pipeline, context stores translation metadata (costs, language detection)
87+
88+
- **Interaction Type:** `TypedDict[{'role': str, 'content': str}]`
89+
90+
- Format for conversation history messages
91+
- Standard chat message format with role (user/assistant) and content
92+
93+
94+
💡 *Note: While this is a single-turn translation, Chat Based Pipeline allows for future enhancements like multi-turn conversations*
95+
96+
97+
### Step 4. Add Resources
98+
99+
![Pipeline Resources](english-to-french-pipeline-resource.png)
100+
101+
**Resources** are the pre-registered components your pipeline will use.
102+
103+
Click **+ Create New** or search for existing resources to add:
104+
105+
**LLMs / Models:** `gemini_2_0_flash` - The foundation model for generating translations
106+
107+
**Prompts (Optional):** `english_to_french_translation` - The translation prompt for generating translations
108+
109+
- Follow the [Prompt Registration Guide](../../intent-classification-pipeline-registration/prompt/) to create a reusable prompt
110+
111+
**Other Resources** (Not required for this pipeline):
112+
113+
- **RAGs:** For retrieving translation dictionaries or context
114+
- **Agents & Sub-Pipelines:** For complex multi-step translation workflows
115+
- **Helper Functions:** For pre/post-processing text
116+
117+
### Step 5. Write Pipeline Scoring Logic
118+
119+
![Pipeline Scoring Logic](english-to-french-pipeline-scoring-logic.png)
120+
121+
**Pipeline Scoring Logic** orchestrates how resources work together to perform the translation.
122+
123+
**Variables Available in the Pipeline:**
124+
125+
- `user_message` - The English text to translate (type: String)
126+
- `history` - Previous conversation messages (type: list[TypedDict[{'role': str, 'content': str}]])
127+
- `context` - Information stored across turns (type: dict[str, str])
128+
129+
**Complete Pipeline Code:**
130+
131+
```python
132+
# Step 1: Generate strict translation prompt
133+
prompt = english_to_french_translation(user_message=user_message)
134+
135+
# Step 2: Get translation from Gemini
136+
result = gemini_2_0_flash(
137+
text=prompt,
138+
temperature=0.3,
139+
system_instruction='None'
140+
)
141+
142+
translated_text = result["response"]
143+
144+
# Step 3: Return result
145+
return {
146+
"output": translated_text
147+
}
148+
```
149+
150+
**What This Code Does:**
151+
152+
- Use the registered Translation Prompt to convert the user's message to a French translation
153+
- Call the registered Gemini 2.0 Flash Model with the translation prompt to generate the translation:
154+
- Return the translation as the output of the pipeline
155+
156+
### Step 6. Add Examples (Optional)
157+
158+
![Pipeline Examples Section](english-to-french-pipeline-examples.png)
159+
160+
Add test examples to validate pipeline behavior:
161+
162+
| Input | Expected Output |
163+
|-------|----------------|
164+
| "Hello, how are you?" | "Bonjour, comment allez-vous ?" |
165+
| "The weather is beautiful today." | "Le temps est magnifique aujourd'hui." |
166+
| "Thank you very much!" | "Merci beaucoup !" |
167+
| "Hola, ¿cómo estás?" (Spanish) | "Error: Input text must be in English. Detected language: Spanish" |
168+
169+
**Note:** Examples help with testing and documenting expected behavior. They also serve as regression tests when updating the pipeline.
170+
171+
### Step 7. Save the Pipeline
172+
173+
Click **Create** to register the pipeline.
174+
175+
The pipeline is now:
176+
177+
- ✅ Available in the Pipeline Registry
178+
- ✅ Ready for simulation and testing
179+
- ✅ Ready for use in downstream applications
180+
181+
182+
## Testing Your Pipeline
183+
184+
After creating the pipeline, test it to verify translation quality and error handling
185+
186+
### Quick Test (During Creation/Editing)
187+
188+
![Test Code](english-to-french-pipeline-test-code.png)
189+
190+
1. While creating or editing the pipeline, scroll to the **Code** section
191+
2. Click **Test Code** in the bottom right corner
192+
3. Enter test inputs to verify logic without saving
193+
194+
**Sample Test Cases:**
195+
196+
```python
197+
# Test Case 1: Simple greeting
198+
user_message = "Hello, how are you?"
199+
# Expected: "Bonjour, comment allez-vous ?"
200+
201+
# Test Case 2: Non-English input (error handling)
202+
user_message = "Hola, ¿cómo estás?"
203+
# Expected: "Error: Input text must be in English. Detected language: Spanish"
204+
```
205+
206+
### Interactive Test (After Saving)
207+
208+
- Navigate to your saved pipeline
209+
- Click **Run****Chat Session** (top right corner)
210+
- Enter sample English messages to test the translation flow
211+
212+
**Verify:**
213+
214+
- Translations are accurate and natural
215+
- Tone and style are preserved (formal/informal)
216+
- Non-English inputs return proper error messages
217+
- Output format is clean (no extra explanations)
218+
219+
**Chat Session Testing Tips:**
220+
221+
- Test both formal and informal language
222+
- Try technical terms and idioms
223+
- Verify cultural nuances are preserved
224+
- Test edge cases (very short/long text, special characters)
225+
226+
## Want to Improve/Extend Your Pipeline? Try These Ideas:
227+
228+
- Auto-detect source language using NLP techiques and see how it performs compared to the current pipeline
229+
- Add translation confidence scores and quality of the translations using evaluation providers
230+
- Extend to support other languages
231+
232+
## Conclusion:
233+
234+
You've successfully learned how to register an English to French Translation Pipeline that:
235+
236+
- ✅ Detects English language automatically
237+
- ✅ Provides high-quality French translations
238+
- ✅ Handles non-English input gracefully
239+
- ✅ Maintains clean, production-ready code with reusable translation prompt and Gemini 2.0 Flash model
240+
241+
## Related Documentation
242+
243+
- [Model Registration Guide](../../model/) - Register foundation models like Gemini 2.0 Flash
244+
- [Prompt Registration Guide](../../intent-classification-pipeline-registration/prompt/) - Create reusable prompts

0 commit comments

Comments
 (0)