diff --git a/research/2025-11-18-Welsh-usability-testing-nov25.md b/research/2025-11-18-Welsh-usability-testing-nov25.md new file mode 100644 index 000000000..502940554 --- /dev/null +++ b/research/2025-11-18-Welsh-usability-testing-nov25.md @@ -0,0 +1,130 @@ +# Usability testing: Welsh forms November 2025 + +## 2025-11-18 / Sprint 29 + +## Aims +**Primary goals** +- Test how well Concept 2 works for users and what we can do to make it work as well as possible. +- Learn more about some specific questions that have come up during the design process. + +**Secondary goals** +- Learn more about how users decide what language to communicate with form fillers in. +- Discover the language that participants use when thinking about creating the Welsh version of a form. + +## Participants +- Civil servants. +- All participants had some experience of using GOV.UK Forms, with varying levels of experience. +- All had experience of publishing Welsh forms or content. + +- 6 participants + +## Methodology +- 60-minute usability testing sessions. + +## Key Headlines +**The overall user experience of building a Welsh version of a form** +- Broadly, participants successfully used the prototype to create the Welsh version. + +- Some participants mentioned the risk of making mistakes while copying and pasting. However, this was attributed as a general risk of the nature of the activity - not something made riskier by the prototype. + +- Some participants made mistakes while copying and pasting. However, all except one spotted their mistake and easily corrected it. + +- They demonstrated they could cope with copying and pasting the question and guidance that were intentionally presented in a different order in the document and the prototype. + +- The need for a Welsh expert to review the final output was mentioned by several. + +- 2 participants experienced challenges caused by the pasted Welsh content boxes behaviour. One struggled with the scrolling boxes. Another had pasted in carriage returns which pushed the text out of view when the English was only a single line. Didn’t understand what the corner expander icon meant. + +- After using it, participants said it was easy, logical and straightforward. + +- Lots of positive comments on the layout and way it worked. + +**Specific questions about adding the Welsh version** + +**_Can they find the way to add a Welsh form?_** +- 5 of 6 had spotted the new task during an earlier activity. +- The person who hadn’t seen it before clicked on “Create a form”. +- We can probably safely assume that if people have built the English form first they will have spotted the Welsh task in the task list. However, if they haven't spotted it, it is unlikely they will realise it's a possibility to add a Welsh form in this way. + +**_Do people know these fields are editable?_** +- 4 participants easily realised they could click in the fields and edit them. +- One said they were looking for an edit button at first, but said “it didn't take me too long to figure out." +- One didn’t work it out for a long time, and had a very different mental model of what this page was. + +**_How do people manage if the declaration is missing?_** +- 5 out of 6 noticed that the declaration needed to be added. +- All 5 easily understood that they needed to save their work before exiting the Welsh to create a declaration in English. +- Two mentioned the risk of not saving their work. +- Only one said it was “a little bit fiddly” and “a bit aggravating”. + +**_Do people want to be able to edit the English content too?_** +- Only 2 people mentioned the possibility of editing the English from the Welsh page. +- One was really keen, the other mentioned it in passing as something she wondered about. + +**_What do they think they'd do if they didn't have the privacy policy in Welsh?_** +- Generally, people thought it was unlikely the system would allow it to be left blank. +- Only one person raised the possibility of having a Welsh version of the privacy statement. +- Some tested saving it empty to see what happened. Those who saw that the system allows them to save it empty assume that it will default to the English link. +- Three added the English link even though it's not in the doc. +- On the whole, they don’t seem to be concerned about linking to the English version and didn’t raise the possibility of having a Welsh privacy statement. Two people explicitly said they wouldn’t have a Welsh privacy notice. + +**_How do they respond to fields using markdown?_** +**_Do they need preview?_** +- All participants happily copied and pasted the English and Welsh text with markdown. +- Only 2 people mentioned markdown throughout. +- One uses markdown regularly, the other would use regular formatting. +- Inconclusive. + +**_Is the form too long?_** +- The test form wasn’t very long, due to session constraints. +- Nobody complained or lost their work. +- Inconclusive. + +_**Do people need different methods of contact for support in Welsh v English forms? +Does this need to be a separate task?**_ +- Low sample caveat! +- 4 out of 6 participants said they would likely have the same contact details for English and Welsh. +- 2 said they could potentially be different. +- One didn’t know. + +**_Downloading a copy of the English form_** +- When asked to get a copy of their English form, they mostly didn't anticipate that the prototype would offer a download. +- Even if they had mentioned previously that it would be helpful, they didn’t necessarily look for it. +- 4 went to the English form preview. +- Participants generally talked about sharing the preview or copying and pasting the questions into a document. +- They discuss preview as a workable option. +- However, a few do mention the burden of copying and pasting. +- A couple of people spotted it later and thought it would be useful. +- One person expected it would download the Welsh as well and they could send it to the translator for review. + +**Specific questions about which language to communicate with form fillers in** + +**_Do form owners / processors want a way to ask about the form fillers' preferred language for follow up communications?_** +- Wrong users caveat! +- A mixed response. +- Generally a fairly low level of knowledge and understanding of this part of the process. +- On the whole they are aware of the need for the option for Welsh people to be communicated with in the language of their choosing (including having both). +- However, their ways of learning or deciding varied or were unknown. +- They would need a way to determine what language their form fillers prefer. +- None seemed to have strong expectations for whether or how this would be provided by the platform. + +**_Do form processors have separate email addresses for Welsh forms?_** +- Low sample caveat! +- This could sometimes be the case, but it seems to be a manual triaging task by form processors (few mentioned it). +- 3 said it would be good to have the flexibility to have a different inbox if needed. + +**The design of the Group page** +- “Welsh version: not yet named” was generally understood. +- However, there was a tendency to link it to the status of the Welsh form being in draft. +- It was useful to know there was a Welsh version. +- Whether the name was necessary at all was queried by some. +- One person made the point that form names can be long and Welsh would be even longer. +- Some mentioned that it isn’t useful without other information. +- The other information they mentioned was to link to the Welsh form, and the status of the Welsh form. +- There was mixed comprehension of the status labels. +- 4 were unclear which language version of the form was live or draft. + +## Supporting Evidence +- [Research findings deck](https://docs.google.com/presentation/d/1qQ0hFHpBueber24XyoU2tMLb2sAfLzCiEGbTD12hAt4/edit?slide=id.g32d739b8369_0_4#slide=id.g32d739b8369_0_4) +- [Playback](https://drive.google.com/file/d/1N7a9D1woKhGGDsKEQ6Sa0c4GL_QK-eIQ/view?usp=sharing) +- [Further documentation](https://drive.google.com/drive/folders/1yV0vzujDWVsIDVNaR6282wMvXKEm6aBr)