Potential New Obstacle: Communication Clarity

[stijn-dejongh]

Obstacle: Communication Clarity

Description

Lack of clear communication skills limits your ability to get efficient results from LLM-based tools.

Remediation

• Prompt in English
• Pay attention to the words the LLM uses to respond to your requests, and emulate them
• Train an in-house LLM on a non-English corpus of work ( if possible / sufficiently exhaustive )

Rationale

As a non-native speaker, I notice that my level of English is a huge differentiator between my LLM results and those of others.
My reasoning for this observation is:

• LLMs are trained largely on an English-based corpus, and respond best when prompted in English.
• Non-native speakers have difficulty expressing their thoughts / instructions clearly.
• LLMs can usually make sense of instructions with lackluster grammar, but tend to misinterpret punctuation.

[lexler]

Totally agree it's a obstacle, problem is that it's an obstacle in the human interacting with a genie...
I originally intended obstacles to be 'genie problems'. So not sure what to do here. Some antipatterns are covering 'mental models that humans have about genies', like assuming they should remember the whole of their training set.

Feels like it might want its own category, or I'll just need to think about this more. @devill @nitsanavni do you have any thoughts? Or do you have any suggestions @stijn-dejongh?

[lexler]

Regarding "I notice that my level of English is a huge differentiator between my LLM results and those of others", totally agree and I think you can probably mitigate it a lot for yourself by running all your prompts through the AI asking to help with the English bit & tweaking instructions until you see a big improvement. But I'm guessing might have already thought of it? :)

[stijn-dejongh]

@lexler : The issue is threefold, in my opinion:

• The genie pretends to understand what you are asking, even if you made no sense ( lousy grammar or punctuation, phrases that are literal translations of foreign languages. ...)
• The user's ability to clearly communicate intent is significantly impacted by their ability to communicate in a structured / understandable way
• The genie pretends to speak other languages well, but doesn't really

You can always rephrase this as "The genie pretends to understand other languages, but secretly only speaks English."

I have tried prompting in Dutch, and notice the outputs are significantly worse ( misinterpretations, hallucinations, ...)

To your point: I personally think both takes on it would make a helpful pattern to add to your list. Maybe it is a language specific variation on

And indeed: asking an LLM "I want to ask an LLM to do , how should I phrase it?" is a super handy trick.

[devill]

I think what we might have here is more an anti-pattern. I think language understanding is just a small part of the bigger problem around communication clarity in general, and it is closely related to the "Silent Misalignment" and "Perfect recall fallacy" anti-patterns. I propose either the refinement of "Silent Misalignment" to specify lack of communication clarity as one potential cause of it, or introducing this as an anti-pattern rather than an obstacle.

[stijn-dejongh]

I agree with @devill for the most part. This idea can serve as a good "usage example" or "compound problem" linking to some other patterns and anti-patterns.

@lexler : Your suggested approach of doing 'meta-analysis' on your own prompts to try and refine them to fit the preferences of your genie(s) is a valueble one! I feel that definitely deserves an entry in the repository.