From 1c1a1b567e1b360c0abab1614f4142feef5cf84c Mon Sep 17 00:00:00 2001 From: Lorna Howes Date: Tue, 18 Nov 2025 17:19:14 +0000 Subject: [PATCH 1/6] Create 2025-11-18-Welsh-usability-testing-nov25.md --- ...025-11-18-Welsh-usability-testing-nov25.md | 130 ++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 research/2025-11-18-Welsh-usability-testing-nov25.md diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md new file mode 100644 index 000000000..e678354af --- /dev/null +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -0,0 +1,130 @@ +# Usability testing: Welsh forms November 2025 + +## 2025-11-18 / Sprint 29 + +## Aims +**Primary goals** +- Test how well Concept 2 works for users and what we can do to make it work as well as possible. +- Learn more about some specific questions that have come up during the design process. + +**Secondary goals** +- Learn more about how users decide what language to communicate with form fillers in. +- Discover the language that participants use when thinking about creating the Welsh version of a form. + +## Participants +- Civil servants. +- All participants had some experience of using GOV.UK Forms, with varying levels of experience. +- All had experience of publishing Welsh forms or content. + +- 6 participants + +## Methodology +- 60-minute usability testing sessions. + +## Key Headlines +**The overall user experience of building a Welsh version of a form** +- Broadly, participants successfully used the prototype to create the Welsh version. + +- Some participants mentioned the risk of making mistakes while copying and pasting. However, this was attributed as a general risk of the nature of the activity - not something made riskier by the prototype. + +- Some participants made mistakes while copying and pasting. However, all except one spotted their mistake and easily corrected it. + +- They demonstrated they could cope with copying and pasting the question and guidance that were intentionally presented in a different order in the document and the prototype. + +- The need for a Welsh expert to review the final output was mentioned by several. + +- 2 participants experienced challenges caused by the pasted Welsh content boxes behave. One struggles with the scrolling boxes. Another had pasted in carriage returns which pushed the text out of view when the English was only a single line. Didn’t understand what the corner expander icon meant. + +- After using it, participants said it was easy, logical and straightforward. + +- Lots of positive comments on the layout and way it worked. + +**Specific questions about adding the Welsh version** + +**_Can they find the way to add a Welsh form?_** +- 5 of 6 had spotted the new task during an earlier activity. +- The person who hadn’t seen it before clicked on “Create a form”. +- We can probably safely assume that if people have built the English form first they will have spotted the Welsh task in the task list. However, if they haven't spotted it, it is unlikely they will realise it's a possibility to add a Welsh form in this way. + +**_Do people know these fields are editable?_** +- 4 participants easily realised they could click in the fields and edit them. +- One said they were looking for an edit button at first, but did “it didn't take me too long to figure out." +- One didn’t work it out for a long time, and had a very different mental model of what this page was. + +**_How do people manage if the declaration is missing?_** +- 5 out of 6 noticed that the declaration needed to be added. +- All 5 easily understood that they needed to save their work before exiting the Welsh to create a declaration in English. +- Two mentioned the risk of not saving their work. +- Only one said it was “a little bit fiddly” and “a bit aggravating”. + +**_Do people want to be able to edit the English content too?_** +- Only 2 people mentioned the possibility of editing the English from the Welsh page. +- One was really keen, the other mentioned it in passing as something she wondered about. + +**_What do they think they'd do if they didn't have the privacy policy in Welsh?_** +- Generally, people thought it was unlikely the system would allow it to be left blank. +- Only one person raised the possibility of having a Welsh version of the privacy statement. +- Some tested saving it empty to see what happened. Those who saw that the system allows them to save it empty assume that it will default to the English link. +- Three added the English link even though it's not in the doc. +- On the whole, they seems to not be concerned about linking to the English version and didn’t raise the possibility of having a Welsh privacy statement. Two people explicitly said they wouldnt have a Welsh privacy notice. + +**_How do they respond to fields using markdown?_** +**_Do they need preview?_** +- All participants happily copied and pasted the English and Welsh text with markdown. +- Only 2 people mentioned markdown throughout. +- One uses markdown regularly, the other would use regular formatting. +- Inconclusive. + +**_Is the form too long?_** +- The test form wasn’t very long, due to session constraints. +- Nobody complained or lost their work. +- Inconclusive. + +_**Do people need different methods of contact for support in Welsh v English forms? +Does this need to be a separate task?**_ +- Low sample caveat! +- 4 out of 6 participants said they would likely have the same contact details for English and Welsh. +- 2 said they could potentially be different. +- One didn’t know. + +**_Downloading a copy of the English form_** +- When asked to get a copy of their English form, they mostly didn't anticipate that the prototype would offer a download. +- Even if they had mentioned previously that it would be helpful, they didn’t necessarily look for it. +- 4 went to the English form preview. +- Participants generally talked about sharing the preview or copying and pasting the questions into a document. +- They discuss preview as a workable option. +- However, a few do mention the burden of copying and pasting. +- A couple of people spotted it later and thought it would be useful. +- One person expected it would download the Welsh as well and they could send it to the translator for review. + +**Specific questions about which language to communicate with form fillers in** + +**_Do form owners / processors want a way to ask about the form fillers' preferred language for follow up communications?_** +- Wrong users caveat! +- A mixed response. +- Generally a fairly low level of knowledge and understanding of this part of the process. +- On the whole they are aware of the need for the option for Welsh people to be communicated with in the language of their choosing (including having both). +- However, their ways of learning or deciding varied or were unknown. +- They would need a way to determine what language their form fillers prefer. +- None seemed to have a strong expectations for whether or how this would be provided by the platform. + +**_Do form processors have separate email addresses for Welsh forms?_** +- Low sample caveat! +- This could sometimes be the case, but it seems to be a manual triaging task by form processors (few mentioned it). +- 3 said it would be good to have the flexibility to have a different inbox if needed. + +**The design of the Group page** +- “Welsh version: not yet named” was generally understood. +- However, there was a tendency to link it to the status of the Welsh form being in draft . +- It was useful to know there was a Welsh version. +- Whether the name was necessary at all was queried by some. +- One person made the point that form names can be long and Welsh would be even longer. +- Some mentioned that it isn’t useful without other information. +- The other information they mentioned was to link to the Welsh form, and the status of the Welsh form. +- There was mixed comprehension of the status labels. +- 4 were unclear which language version of the form was live or draft. + +## Supporting Evidence +- [Research findings deck](https://docs.google.com/presentation/d/1qQ0hFHpBueber24XyoU2tMLb2sAfLzCiEGbTD12hAt4/edit?slide=id.g32d739b8369_0_4#slide=id.g32d739b8369_0_4) +- [Playback](https://drive.google.com/file/d/1N7a9D1woKhGGDsKEQ6Sa0c4GL_QK-eIQ/view?usp=sharing) +- [Further documentation](https://drive.google.com/drive/folders/1yV0vzujDWVsIDVNaR6282wMvXKEm6aBr) From 736b8c8a7e3c38f19d48cc8705b4f833effacf7e Mon Sep 17 00:00:00 2001 From: Lorna Howes Date: Wed, 19 Nov 2025 09:29:13 +0000 Subject: [PATCH 2/6] Update research/2025-11-18-Welsh-usability-testing-nov25.md Co-authored-by: Christopher Cameron <35372982+christophercameron-ixd@users.noreply.github.com> --- research/2025-11-18-Welsh-usability-testing-nov25.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md index e678354af..1b5792fcb 100644 --- a/research/2025-11-18-Welsh-usability-testing-nov25.md +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -33,7 +33,7 @@ - The need for a Welsh expert to review the final output was mentioned by several. -- 2 participants experienced challenges caused by the pasted Welsh content boxes behave. One struggles with the scrolling boxes. Another had pasted in carriage returns which pushed the text out of view when the English was only a single line. Didn’t understand what the corner expander icon meant. +- 2 participants experienced challenges caused by the pasted Welsh content boxes behaviour. One struggled with the scrolling boxes. Another had pasted in carriage returns which pushed the text out of view when the English was only a single line. Didn’t understand what the corner expander icon meant. - After using it, participants said it was easy, logical and straightforward. From 5eb0592fe963541da7029a4518779f3dc4bed278 Mon Sep 17 00:00:00 2001 From: Lorna Howes Date: Wed, 19 Nov 2025 09:29:28 +0000 Subject: [PATCH 3/6] Update research/2025-11-18-Welsh-usability-testing-nov25.md Co-authored-by: Christopher Cameron <35372982+christophercameron-ixd@users.noreply.github.com> --- research/2025-11-18-Welsh-usability-testing-nov25.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md index 1b5792fcb..9f7ff0bd6 100644 --- a/research/2025-11-18-Welsh-usability-testing-nov25.md +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -48,7 +48,7 @@ **_Do people know these fields are editable?_** - 4 participants easily realised they could click in the fields and edit them. -- One said they were looking for an edit button at first, but did “it didn't take me too long to figure out." +- One said they were looking for an edit button at first, but said “it didn't take me too long to figure out." - One didn’t work it out for a long time, and had a very different mental model of what this page was. **_How do people manage if the declaration is missing?_** From 78d3bfd8394e0c082b342c154e681fcd854dbf77 Mon Sep 17 00:00:00 2001 From: Lorna Howes Date: Wed, 19 Nov 2025 09:29:45 +0000 Subject: [PATCH 4/6] Update research/2025-11-18-Welsh-usability-testing-nov25.md Co-authored-by: Christopher Cameron <35372982+christophercameron-ixd@users.noreply.github.com> --- research/2025-11-18-Welsh-usability-testing-nov25.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md index 9f7ff0bd6..3f1a1083d 100644 --- a/research/2025-11-18-Welsh-usability-testing-nov25.md +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -66,7 +66,7 @@ - Only one person raised the possibility of having a Welsh version of the privacy statement. - Some tested saving it empty to see what happened. Those who saw that the system allows them to save it empty assume that it will default to the English link. - Three added the English link even though it's not in the doc. -- On the whole, they seems to not be concerned about linking to the English version and didn’t raise the possibility of having a Welsh privacy statement. Two people explicitly said they wouldnt have a Welsh privacy notice. +- On the whole, they don’t seem to be concerned about linking to the English version and didn’t raise the possibility of having a Welsh privacy statement. Two people explicitly said they wouldn’t have a Welsh privacy notice. **_How do they respond to fields using markdown?_** **_Do they need preview?_** From f25f678a4508ba838fb6a083a575c61951b1ff9d Mon Sep 17 00:00:00 2001 From: Lorna Howes Date: Wed, 19 Nov 2025 09:29:58 +0000 Subject: [PATCH 5/6] Update research/2025-11-18-Welsh-usability-testing-nov25.md Co-authored-by: Christopher Cameron <35372982+christophercameron-ixd@users.noreply.github.com> --- research/2025-11-18-Welsh-usability-testing-nov25.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md index 3f1a1083d..8b0d73a1c 100644 --- a/research/2025-11-18-Welsh-usability-testing-nov25.md +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -106,7 +106,7 @@ Does this need to be a separate task?**_ - On the whole they are aware of the need for the option for Welsh people to be communicated with in the language of their choosing (including having both). - However, their ways of learning or deciding varied or were unknown. - They would need a way to determine what language their form fillers prefer. -- None seemed to have a strong expectations for whether or how this would be provided by the platform. +- None seemed to have strong expectations for whether or how this would be provided by the platform. **_Do form processors have separate email addresses for Welsh forms?_** - Low sample caveat! From 620ca30a24d351e9c4f89cf1b121c3f8849e31af Mon Sep 17 00:00:00 2001 From: william-castoncook Date: Wed, 19 Nov 2025 09:37:16 +0000 Subject: [PATCH 6/6] Update 2025-11-18-Welsh-usability-testing-nov25.md Removed a space before a full stop. --- research/2025-11-18-Welsh-usability-testing-nov25.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md index 8b0d73a1c..502940554 100644 --- a/research/2025-11-18-Welsh-usability-testing-nov25.md +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -115,7 +115,7 @@ Does this need to be a separate task?**_ **The design of the Group page** - “Welsh version: not yet named” was generally understood. -- However, there was a tendency to link it to the status of the Welsh form being in draft . +- However, there was a tendency to link it to the status of the Welsh form being in draft. - It was useful to know there was a Welsh version. - Whether the name was necessary at all was queried by some. - One person made the point that form names can be long and Welsh would be even longer.